{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/soft-actor-critic","slug":"soft-actor-critic","name":"Soft Actor Critic","full_name":"Soft Actor Critic","full_name_withheld":false,"description_markdown":"**Soft Actor Critic**, or **SAC**, is an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy. That is, to succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as [Q-learning methods](https://paperswithcode.com/method/q-learning). [SAC](https://paperswithcode.com/method/sac) combines off-policy updates with a stable stochastic actor-critic formulation.\r\n\r\nThe SAC objective has a number of advantages. First, the policy is incentivized to explore more widely, while giving up on clearly unpromising avenues. Second, the policy can capture multiple modes of near-optimal behavior. In problem settings where multiple actions seem equally attractive, the policy will commit equal probability mass to those actions. Lastly, the authors present evidence that it improves learning speed over state-of-art methods that optimize the conventional RL objective function.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor","paper":"/paper/soft-actor-critic-off-policy-maximum-entropy","first_author":"Tuomas Haarnoja","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/soft-actor-critic-off-policy-maximum-entropy"},"source":{"url":"http://arxiv.org/abs/1801.01290v2","title":"Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Policy Gradient Methods","url":"/methods/category/policy-gradient-methods","pwc_aliases":[]}],"n_papers_tagged":58,"archive_num_papers":58,"papers_newest_first":[{"paper":null,"slug":"moderate-actor-critic-methods-controlling","title":"Moderate Actor-Critic Methods: Controlling Overestimation Bias via Expectile Loss","date":"2025-04-14","arxiv_id":"2504.09929","n_code_links":0,"syntology":null},{"paper":null,"slug":"closing-the-intent-to-reality-gap-via","title":"Closing the Intent-to-Behavior Gap via Fulfillment Priority Logic","date":"2025-03-04","arxiv_id":"2503.05818","n_code_links":0,"syntology":null},{"paper":null,"slug":"il-soar-imitation-learning-with-soft","title":"IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic","date":"2025-02-27","arxiv_id":"2502.19859","n_code_links":0,"syntology":null},{"paper":"/paper/langevin-soft-actor-critic-efficient","slug":"langevin-soft-actor-critic-efficient","title":"Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning","date":"2025-01-29","arxiv_id":"2501.17827","n_code_links":1,"syntology":{"ran":6,"of":11,"n_ran_checked":3,"n_instrument":3,"unverified":5,"pointer_only":0,"phrase":"6 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified","official":{"repos":["hmishfaq/lsac"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":1,"n_ran_no_instrument_failure":3,"n_unverified":5,"ran_from_kinds":["official"]}}},{"paper":"/paper/reinforcement-learning-controlled-adaptive","slug":"reinforcement-learning-controlled-adaptive","title":"Reinforcement Learning Controlled Adaptive PSO for Task Offloading in IIoT Edge Computing","date":"2025-01-25","arxiv_id":"2501.15203","n_code_links":1,"syntology":null},{"paper":null,"slug":"average-reward-reinforcement-learning-for","title":"Average Reward Reinforcement Learning for Wireless Radio Resource Management","date":"2025-01-12","arxiv_id":"2501.06700","n_code_links":0,"syntology":null},{"paper":null,"slug":"learn-2-rage-experiencing-the-emotional","title":"Learn 2 Rage: Experiencing The Emotional Roller Coaster That Is Reinforcement Learning","date":"2024-10-24","arxiv_id":"2410.18462","n_code_links":0,"syntology":null},{"paper":null,"slug":"augmented-lagrangian-based-safe-reinforcement","title":"Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control","date":"2024-10-19","arxiv_id":"2410.15188","n_code_links":0,"syntology":null},{"paper":null,"slug":"solving-the-dynamic-volatility-fitting","title":"Solving The Dynamic Volatility Fitting Problem: A Deep Reinforcement Learning Approach","date":"2024-10-15","arxiv_id":"2410.11789","n_code_links":0,"syntology":null},{"paper":"/paper/deep-attention-driven-reinforcement-learning","slug":"deep-attention-driven-reinforcement-learning","title":"Deep Attention Driven Reinforcement Learning (DAD-RL) for Autonomous Decision-Making in Dynamic Environment","date":"2024-07-12","arxiv_id":"2407.08932","n_code_links":1,"syntology":null},{"paper":null,"slug":"real-time-system-optimal-traffic-routing","title":"Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning?","date":"2024-07-10","arxiv_id":"2407.07364","n_code_links":0,"syntology":null},{"paper":null,"slug":"enhanced-safety-in-autonomous-driving","title":"Enhanced Safety in Autonomous Driving: Integrating Latent State Diffusion Model for End-to-End Navigation","date":"2024-07-08","arxiv_id":"2407.06317","n_code_links":0,"syntology":null},{"paper":"/paper/a-fast-balance-optimization-approach-for","slug":"a-fast-balance-optimization-approach-for","title":"A fast balance optimization approach for charging enhancement of lithium-ion battery packs through deep reinforcement learning","date":"2024-04-24","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"imitation-game-a-model-based-and-imitation","title":"Imitation Game: A Model-based and Imitation Learning Deep Reinforcement Learning Hybrid","date":"2024-04-02","arxiv_id":"2404.01794","n_code_links":0,"syntology":null},{"paper":null,"slug":"tuning-for-the-unknown-revisiting-evaluation","title":"K-percent Evaluation for Lifelong RL","date":"2024-04-02","arxiv_id":"2404.02113","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-for-local-path","title":"Deep Reinforcement Learning for Local Path Following of an Autonomous Formula SAE Vehicle","date":"2024-01-05","arxiv_id":"2401.02903","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamic-fairness-aware-spectrum-auction-for","title":"Dynamic Fairness-Aware Spectrum Auction for Enhanced Licensed Shared Access in 6G Networks","date":"2023-12-20","arxiv_id":"2312.12867","n_code_links":0,"syntology":null},{"paper":null,"slug":"on-designing-multi-uav-aided-wireless-powered","title":"On Designing Multi-UAV aided Wireless Powered Dynamic Communication via Hierarchical Deep Reinforcement Learning","date":"2023-12-13","arxiv_id":"2312.07917","n_code_links":0,"syntology":null},{"paper":null,"slug":"joint-sensing-and-communication-optimization","title":"Joint Sensing and Communication Optimization in Target-Mounted STARS-Assisted Vehicular Networks: A MADRL Approach","date":"2023-11-17","arxiv_id":"2311.10352","n_code_links":0,"syntology":null},{"paper":"/paper/belief-projection-based-reinforcement","slug":"belief-projection-based-reinforcement","title":"Belief Projection-Based Reinforcement Learning for Environments with Delayed Feedback","date":"2023-09-21","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"hybrid-of-representation-learning-and","title":"Hybrid of representation learning and reinforcement learning for dynamic and complex robotic motion planning","date":"2023-09-07","arxiv_id":"2309.03758","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-safe-deep-reinforcement-learning-approach","title":"A Safe Deep Reinforcement Learning Approach for Energy Efficient Federated Learning in Wireless Communication Networks","date":"2023-08-21","arxiv_id":"2308.10664","n_code_links":0,"syntology":null},{"paper":"/paper/hint-assisted-reinforcement-learning-an","slug":"hint-assisted-reinforcement-learning-an","title":"Hint assisted reinforcement learning: an application in radio astronomy","date":"2023-01-10","arxiv_id":"2301.03933","n_code_links":1,"syntology":null},{"paper":null,"slug":"efficient-exploration-in-resource-restricted","title":"Efficient Exploration in Resource-Restricted Reinforcement Learning","date":"2022-12-14","arxiv_id":"2212.06988","n_code_links":0,"syntology":null},{"paper":null,"slug":"rl-based-guidance-in-outpatient-hysteroscopy","title":"RL-Based Guidance in Outpatient Hysteroscopy Training: A Feasibility Study","date":"2022-11-26","arxiv_id":"2211.14541","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-deep-reinforcement-learning-based-charging","title":"A Deep Reinforcement Learning-Based Charging Scheduling Approach with Augmented Lagrangian for Electric Vehicle","date":"2022-09-20","arxiv_id":"2209.09772","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-for-dynamic-4","title":"Plug-and-Play Model-Agnostic Counterfactual Policy Synthesis for Deep Reinforcement Learning based Recommendation","date":"2022-08-10","arxiv_id":"2208.05142","n_code_links":0,"syntology":null},{"paper":"/paper/imitation-learning-by-state-only-distribution","slug":"imitation-learning-by-state-only-distribution","title":"Imitation Learning by State-Only Distribution Matching","date":"2022-02-09","arxiv_id":"2202.04332","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["FeMa42/soil-tdm"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/smart-magnetic-microrobots-learn-to-swim-with","slug":"smart-magnetic-microrobots-learn-to-swim-with","title":"Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning","date":"2022-01-14","arxiv_id":"2201.05599","n_code_links":1,"syntology":null},{"paper":null,"slug":"actor-loss-of-soft-actor-critic-explained","title":"Actor Loss of Soft Actor Critic Explained","date":"2021-12-31","arxiv_id":"2112.15568","n_code_links":0,"syntology":null},{"paper":null,"slug":"hyper-parameter-optimization-based-on-soft","title":"Hyper-parameter optimization based on soft actor critic and hierarchical mixture regularization","date":"2021-12-08","arxiv_id":"2112.04084","n_code_links":0,"syntology":null},{"paper":"/paper/mixture-of-variational-experts-for-continual","slug":"mixture-of-variational-experts-for-continual","title":"Mixture-of-Variational-Experts for Continual Learning","date":"2021-10-25","arxiv_id":"2110.12667","n_code_links":1,"syntology":null},{"paper":"/paper/risk-conditioned-neural-motion-planning","slug":"risk-conditioned-neural-motion-planning","title":"Risk Conditioned Neural Motion Planning","date":"2021-08-04","arxiv_id":"2108.01851","n_code_links":1,"syntology":null},{"paper":"/paper/context-based-soft-actor-critic-for","slug":"context-based-soft-actor-critic-for","title":"Context-Based Soft Actor Critic for Environments with Non-stationary Dynamics","date":"2021-05-07","arxiv_id":"2105.03310","n_code_links":1,"syntology":null},{"paper":"/paper/end-to-end-grasping-policies-for-human-in-the","slug":"end-to-end-grasping-policies-for-human-in-the","title":"End-to-end grasping policies for human-in-the-loop robots via deep reinforcement learning","date":"2021-04-26","arxiv_id":"2104.12842","n_code_links":1,"syntology":null},{"paper":null,"slug":"stay-alive-with-many-options-a-reinforcement","title":"Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm","date":"2021-01-30","arxiv_id":"2102.00168","n_code_links":0,"syntology":null},{"paper":null,"slug":"factored-action-spaces-in-deep-reinforcement","title":"Factored Action Spaces in Deep Reinforcement Learning","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"learning-from-simulation-racing-in-reality","title":"Learning from Simulation, Racing in Reality","date":"2020-11-26","arxiv_id":"2011.13332","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-rl-with-information-constrained-policies","title":"Deep RL With Information Constrained Policies: Generalization in Continuous Control","date":"2020-10-09","arxiv_id":"2010.04646","n_code_links":0,"syntology":null},{"paper":null,"slug":"emaq-expected-max-q-learning-operator-for","title":"EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL","date":"2020-07-21","arxiv_id":"2007.11091","n_code_links":0,"syntology":null},{"paper":null,"slug":"developing-cooperative-policies-for-multi","title":"Developing cooperative policies for multi-stage tasks","date":"2020-07-01","arxiv_id":"2007.00203","n_code_links":0,"syntology":null},{"paper":"/paper/experience-replay-with-likelihood-free","slug":"experience-replay-with-likelihood-free","title":"Experience Replay with Likelihood-free Importance Weights","date":"2020-06-23","arxiv_id":"2006.13169","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":2,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":null}},{"paper":"/paper/generalized-state-dependent-exploration-for","slug":"generalized-state-dependent-exploration-for","title":"Smooth Exploration for Robotic Reinforcement Learning","date":"2020-05-12","arxiv_id":"2005.05719","n_code_links":4,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["DLR-RM/stable-baselines3"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"distributional-soft-actor-critic-for-risk","title":"DSAC: Distributional Soft Actor Critic for Risk-Sensitive Reinforcement Learning","date":"2020-04-30","arxiv_id":"2004.14547","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-with-augmented-data","slug":"reinforcement-learning-with-augmented-data","title":"Reinforcement Learning with Augmented Data","date":"2020-04-30","arxiv_id":"2004.14990","n_code_links":2,"syntology":{"ran":21,"of":24,"n_ran_checked":9,"n_instrument":12,"unverified":3,"pointer_only":21,"phrase":"21 ran (of which 8 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 12 where Syntology's instrument failed) · 3 unverified","official":null}},{"paper":null,"slug":"adaptive-experience-selection-for-policy","title":"Adaptive Experience Selection for Policy Gradient","date":"2020-02-17","arxiv_id":"2002.06946","n_code_links":0,"syntology":null},{"paper":null,"slug":"cooperative-highway-work-zone-merge-control","title":"Cooperative Highway Work Zone Merge Control based on Reinforcement Learning in A Connected and Automated Environment","date":"2020-01-21","arxiv_id":"2001.08581","n_code_links":0,"syntology":null},{"paper":"/paper/discriminator-soft-actor-critic-without","slug":"discriminator-soft-actor-critic-without","title":"Discriminator Soft Actor Critic without Extrinsic Rewards","date":"2020-01-19","arxiv_id":"2001.06808","n_code_links":1,"syntology":null},{"paper":"/paper/slm-lab-a-comprehensive-benchmark-and-modular-1","slug":"slm-lab-a-comprehensive-benchmark-and-modular-1","title":"SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning","date":"2019-12-28","arxiv_id":"1912.12482","n_code_links":1,"syntology":{"ran":10,"of":13,"n_ran_checked":10,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","official":{"repos":["kengz/SLM-Lab"],"state":"official (archive's flag): 10 ran","n_ran":10,"n_constructed":0,"n_ran_no_instrument_failure":10,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"on-policy-reinforcement-learning-with-entropy","title":"Policy Optimization Reinforcement Learning with Entropy Regularization","date":"2019-12-02","arxiv_id":"1912.01557","n_code_links":0,"syntology":null},{"paper":"/paper/better-exploration-with-optimistic-actor-1","slug":"better-exploration-with-optimistic-actor-1","title":"Better Exploration with Optimistic Actor Critic","date":"2019-12-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"data-efficient-co-adaptation-of-morphology","title":"Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning","date":"2019-11-15","arxiv_id":"1911.06832","n_code_links":0,"syntology":null},{"paper":null,"slug":"better-exploration-with-optimistic-actor","title":"Better Exploration with Optimistic Actor-Critic","date":"2019-10-28","arxiv_id":"1910.12807","n_code_links":0,"syntology":null},{"paper":"/paper/towards-simplicity-in-deep-reinforcement-1","slug":"towards-simplicity-in-deep-reinforcement-1","title":"Striving for Simplicity and Performance in Off-Policy DRL: Output Normalization and Non-Uniform Sampling","date":"2019-10-05","arxiv_id":"1910.02208","n_code_links":3,"syntology":null},{"paper":null,"slug":"modified-actor-critics","title":"Modified Actor-Critics","date":"2019-07-02","arxiv_id":"1907.01298","n_code_links":0,"syntology":null},{"paper":"/paper/improving-exploration-in-soft-actor-critic","slug":"improving-exploration-in-soft-actor-critic","title":"Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policies","date":"2019-06-06","arxiv_id":"1906.02771","n_code_links":1,"syntology":{"ran":4,"of":4,"n_ran_checked":2,"n_instrument":2,"unverified":0,"pointer_only":3,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["joeybose/FloRL"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/deep-reinforcement-learning-in-a-handful-of","slug":"deep-reinforcement-learning-in-a-handful-of","title":"Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models","date":"2018-05-30","arxiv_id":"1805.12114","n_code_links":9,"syntology":{"ran":12,"of":19,"n_ran_checked":7,"n_instrument":5,"unverified":7,"pointer_only":17,"phrase":"12 ran (of which 5 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 5 where Syntology's instrument failed) · 7 unverified","official":{"repos":["kchua/handful-of-trials"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/soft-actor-critic-off-policy-maximum-entropy","slug":"soft-actor-critic-off-policy-maximum-entropy","title":"Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor","date":"2018-01-04","arxiv_id":"1801.01290","n_code_links":86,"syntology":{"ran":91,"of":148,"n_ran_checked":73,"n_instrument":18,"unverified":57,"pointer_only":66,"phrase":"91 ran (of which 61 constructed an object rather than computing a result; 73 with no instrument failure: 3 honoured, 1 violated, 69 with no contract checked; 18 where Syntology's instrument failed) · 57 unverified","official":{"repos":["haarnoja/sac"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}}],"papers_shown":58,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":58,"with_a_code_link":22,"where_syntology_ran_a_sample":9,"not_listed_spam_title":0,"listed":58,"listed_where_code_ran":9,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":8,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":8,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":9,"papers_where_code_ran_first_page":"/method/soft-actor-critic/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/soft-actor-critic/papers/ran/1","json":"/method/soft-actor-critic/papers/ran/1.json","rows":9}],"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":32},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":28},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":25},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":22},{"task":"/task/continuous-control","name":"Continuous Control","papers":14},{"task":"/task/continuous-control","name":"continuous-control","papers":12},{"task":"/task/decision-making","name":"Decision Making","papers":6},{"task":"/task/imitation-learning","name":"Imitation Learning","papers":5},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":5},{"task":"/task/efficient-exploration","name":"Efficient Exploration","papers":4},{"task":"/task/q-learning","name":"Q-Learning","papers":4},{"task":"/task/motion-planning","name":"Motion Planning","papers":3},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":2},{"task":"/task/autonomous-racing","name":"Autonomous Racing","papers":2},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":2},{"task":"/task/management","name":"Management","papers":2},{"task":"/task/model-based-reinforcement-learning","name":"Model-based Reinforcement Learning","papers":2},{"task":"/task/mujoco","name":"MuJoCo","papers":2},{"task":"/task/astronomy","name":"Astronomy","papers":1},{"task":"/task/atari-games","name":"Atari Games","papers":1}],"tasks_shown":20,"n_tasks":55,"usage_by_year":[{"year":"2018","papers":2},{"year":"2019","papers":8},{"year":"2020","papers":11},{"year":"2021","papers":8},{"year":"2022","papers":6},{"year":"2023","papers":7},{"year":"2024","papers":10},{"year":"2025","papers":6}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/soft-actor-critic"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Soft%20Actor%20Critic","read_at":"2026-10-05T01:02:12+00:00","query":"Soft Actor Critic","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"soft actor-critic","origin_arxiv_id":"1801.01290"},"resolution":"token_match","resolved_name_differs":true,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"1801.01290","repositories":{"listed":5,"in_graph":86,"list_truncated":true,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"1801.01290","citations":12178},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'Soft Actor Critic'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[{"name":"SOFT","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='SOFT')","origin_arxiv_id":"2110.11945"},{"name":"Soft MoE","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Soft MoE')","origin_arxiv_id":"2308.00951"},{"name":"soft softmax-triplet loss","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='soft softmax-triplet loss')","origin_arxiv_id":"2001.01526"},{"name":"MODEL-AUGMENTED ACTOR-CRITIC","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='MODEL-AUGMENTED ACTOR-CRITIC')","origin_arxiv_id":"2005.08068"},{"name":"Critic Regularized Regression","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Critic Regularized Regression')","origin_arxiv_id":"2006.15134"}],"code_nearby_caption":"Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.","origin_paper":{"arxiv_id":"1801.01290","page":"/paper/soft-actor-critic-off-policy-maximum-entropy","syntology_url":"https://syntology.ai/paper/1801.01290","in_graph_layer":true,"samples":148,"ran":91,"n_ran_checked":73,"n_instrument":18,"unverified":57,"phrase":"91 ran (of which 61 constructed an object rather than computing a result; 73 with no instrument failure: 3 honoured, 1 violated, 69 with no contract checked; 18 where Syntology's instrument failed) · 57 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"for a different name, 'soft actor-critic': repository known, no verified implementation (declared_code; repositories: 5 listed of 86 in the graph, strongest link: official). The lookup matched a different name in Syntology's graph, 'soft actor-critic' (resolution: token_match); this answer is about that name. This answer is about arXiv:1801.01290, the paper the archive credits with this method.","level_legend":{"state":"quoted","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","card_sha256":"479785a84658dd682a11b10237b3620d651d7f87799f5dc7add970aaa83f6d39","note":"Syntology's served verification levels, quoted verbatim from its MCP server card; a row is split where the card separates clauses with a double hyphen, and the separator is not reproduced","levels":[{"level":"V0","quoted":["imports"]},{"level":"V1","quoted":["executes against a synthesized fixture and returns a value of the recorded type"]},{"level":"V2","quoted":["two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs","neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)"]},{"level":"V3","quoted":["paper-derived property tests pass on both of those same two generations."]}],"no_rung":"No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results."}},"syntology_have_state":"loaded","record_sha256":"8be4200b8bb0e7b463ec4928d9b13edd212c9e90a491d3b76263a94f0de88dd8","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}