{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/clipped-double-q-learning","slug":"clipped-double-q-learning","name":"Clipped Double Q-learning","full_name":"Clipped Double Q-learning","full_name_withheld":false,"description_markdown":"**Clipped Double Q-learning** is a variant on [Double Q-learning](https://paperswithcode.com/method/double-q-learning) that upper-bounds the less biased Q estimate $Q\\_{\\theta\\_{2}}$ by the biased estimate $Q\\_{\\theta\\_{1}}$. This is equivalent to taking the minimum of the two estimates, resulting in the following target update:\r\n\r\n$$ y\\_{1} = r + \\gamma\\min\\_{i=1,2}Q\\_{\\theta'\\_{i}}\\left(s', \\pi\\_{\\phi\\_{1}}\\left(s'\\right)\\right) $$\r\n\r\nThe motivation for this extension is that vanilla double [Q-learning](https://paperswithcode.com/method/q-learning) is sometimes ineffective if the target and current networks are too similar, e.g. with a slow-changing policy in an actor-critic framework.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1802.09477v3","title":"Addressing Function Approximation Error in Actor-Critic Methods","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Off-Policy TD Control","url":"/methods/category/off-policy-td-control","pwc_aliases":[]}],"n_papers_tagged":122,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/gradual-transition-from-bellman-optimality","slug":"gradual-transition-from-bellman-optimality","title":"Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning","date":"2025-06-06","arxiv_id":"2506.05968","n_code_links":1,"syntology":{"ran":1,"of":4,"n_ran_checked":1,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","official":{"repos":["motokiomura/annealed-q-learning"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"fasttd3-simple-fast-and-capable-reinforcement","title":"FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control","date":"2025-05-28","arxiv_id":"2505.22642","n_code_links":0,"syntology":null},{"paper":null,"slug":"llm-explorer-a-plug-in-reinforcement-learning","title":"LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models","date":"2025-05-21","arxiv_id":"2505.15293","n_code_links":0,"syntology":null},{"paper":null,"slug":"monte-carlo-beam-search-for-actor-critic","title":"Monte Carlo Beam Search for Actor-Critic Reinforcement Learning in Continuous Control","date":"2025-05-13","arxiv_id":"2505.09029","n_code_links":0,"syntology":null},{"paper":null,"slug":"energy-efficient-rsma-based-leo-satellite","title":"Energy Efficient RSMA-Based LEO Satellite Communications Assisted by UAV-Mounted BD-Active RIS: A DRL Approach","date":"2025-05-07","arxiv_id":"2505.04148","n_code_links":0,"syntology":null},{"paper":null,"slug":"alphagrad-non-linear-gradient-normalization","title":"AlphaGrad: Non-Linear Gradient Normalization Optimizer","date":"2025-04-22","arxiv_id":"2504.16020","n_code_links":0,"syntology":null},{"paper":null,"slug":"motion-control-in-multi-rotor-aerial-robots","title":"Motion Control in Multi-Rotor Aerial Robots Using Deep Reinforcement Learning","date":"2025-02-09","arxiv_id":"2502.05996","n_code_links":0,"syntology":null},{"paper":"/paper/td3-tucker-decomposition-based-dataset","slug":"td3-tucker-decomposition-based-dataset","title":"TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation","date":"2025-02-05","arxiv_id":"2502.02854","n_code_links":1,"syntology":null},{"paper":"/paper/evorl-a-gpu-accelerated-framework-for","slug":"evorl-a-gpu-accelerated-framework-for","title":"EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning","date":"2025-01-25","arxiv_id":"2501.15129","n_code_links":2,"syntology":null},{"paper":null,"slug":"enhancing-uav-path-planning-efficiency","title":"Enhancing UAV Path Planning Efficiency Through Accelerated Learning","date":"2025-01-17","arxiv_id":"2501.10141","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-advantage-based-optimization-method-for","title":"An Advantage-based Optimization Method for Reinforcement Learning in Large Action Space","date":"2024-12-17","arxiv_id":"2412.12605","n_code_links":0,"syntology":null},{"paper":null,"slug":"provably-efficient-action-manipulation-attack","title":"Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning","date":"2024-11-20","arxiv_id":"2411.13116","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-gradients-as-vitamin","slug":"reinforcement-learning-gradients-as-vitamin","title":"Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers","date":"2024-10-31","arxiv_id":"2410.24108","n_code_links":1,"syntology":{"ran":4,"of":7,"n_ran_checked":2,"n_instrument":2,"unverified":3,"pointer_only":7,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","official":{"repos":["kaiyan289/rl_as_vitamin_for_online_decision_transformers"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":"/paper/networkgym-reinforcement-learning","slug":"networkgym-reinforcement-learning","title":"NetworkGym: Reinforcement Learning Environments for Multi-Access Traffic Management in Network Simulation","date":"2024-10-30","arxiv_id":"2411.04138","n_code_links":1,"syntology":{"ran":15,"of":18,"n_ran_checked":7,"n_instrument":8,"unverified":3,"pointer_only":1,"phrase":"15 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 8 where Syntology's instrument failed) · 3 unverified","official":{"repos":["hmomin/networkgym"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":2,"n_ran_no_instrument_failure":5,"n_unverified":3,"ran_from_kinds":["found_in_text","official"]}}},{"paper":"/paper/human-readable-programs-as-actors-of","slug":"human-readable-programs-as-actors-of","title":"Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution","date":"2024-10-29","arxiv_id":"2410.21940","n_code_links":1,"syntology":null},{"paper":null,"slug":"mitigating-suboptimality-of-deterministic","title":"Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions","date":"2024-10-15","arxiv_id":"2410.11833","n_code_links":0,"syntology":null},{"paper":null,"slug":"navigation-in-a-simplified-urban-flow-through","title":"Navigation in a simplified Urban Flow through Deep Reinforcement Learning","date":"2024-09-26","arxiv_id":"2409.17922","n_code_links":0,"syntology":null},{"paper":null,"slug":"simultaneous-training-of-first-and-second","title":"Simultaneous Training of First- and Second-Order Optimizers in Population-Based Reinforcement Learning","date":"2024-08-27","arxiv_id":"2408.15421","n_code_links":0,"syntology":null},{"paper":null,"slug":"optimizing-td3-for-7-dof-robotic-arm-grasping","title":"Optimizing TD3 for 7-DOF Robotic Arm Grasping: Overcoming Suboptimality with Exploration-Enhanced Contrastive Learning","date":"2024-08-26","arxiv_id":"2408.14009","n_code_links":0,"syntology":null},{"paper":null,"slug":"image-based-deep-reinforcement-learning-with","title":"Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks","date":"2024-07-31","arxiv_id":"2407.21338","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-strategies-in","title":"Deep Reinforcement Learning Strategies in Finance: Insights into Asset Holding, Trading Behavior, and Purchase Diversity","date":"2024-06-29","arxiv_id":"2407.09557","n_code_links":0,"syntology":null},{"paper":null,"slug":"performance-comparison-of-deep-rl-algorithms-1","title":"Performance Comparison of Deep RL Algorithms for Mixed Traffic Cooperative Lane-Changing","date":"2024-06-25","arxiv_id":"2407.02521","n_code_links":0,"syntology":null},{"paper":null,"slug":"value-improved-actor-critic-algorithms","title":"Value Improved Actor Critic Algorithms","date":"2024-06-03","arxiv_id":"2406.01423","n_code_links":0,"syntology":null},{"paper":null,"slug":"chaos-based-reinforcement-learning-with-td3","title":"Chaos-based reinforcement learning with TD3","date":"2024-05-15","arxiv_id":"2405.09086","n_code_links":0,"syntology":null},{"paper":null,"slug":"offline-reinforcement-learning-with-8","title":"Offline Reinforcement Learning with Behavioral Supervisor Tuning","date":"2024-04-25","arxiv_id":"2404.16399","n_code_links":0,"syntology":null},{"paper":null,"slug":"rlingua-improving-reinforcement-learning","title":"RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Models","date":"2024-03-11","arxiv_id":"2403.06420","n_code_links":0,"syntology":null},{"paper":null,"slug":"conservative-ddpg-pessimistic-rl-without","title":"Conservative DDPG -- Pessimistic RL without Ensemble","date":"2024-03-08","arxiv_id":"2403.05732","n_code_links":0,"syntology":null},{"paper":null,"slug":"fill-and-spill-deep-reinforcement-learning","title":"Fill-and-Spill: Deep Reinforcement Learning Policy Gradient Methods for Reservoir Operation Decision and Control","date":"2024-03-07","arxiv_id":"2403.04195","n_code_links":0,"syntology":null},{"paper":null,"slug":"exploiting-estimation-bias-in-deep-double-q","title":"Exploiting Estimation Bias in Clipped Double Q-Learning for Continous Control Reinforcement Learning Tasks","date":"2024-02-14","arxiv_id":"2402.09078","n_code_links":0,"syntology":null},{"paper":null,"slug":"textit-minmaxmin-q-learning","title":"MinMaxMin $Q$-learning","date":"2024-02-03","arxiv_id":"2402.05951","n_code_links":0,"syntology":null},{"paper":null,"slug":"textit-sqt-textit-std-q-target","title":"SQT -- std $Q$-target","date":"2024-02-03","arxiv_id":"2402.05950","n_code_links":0,"syntology":null},{"paper":null,"slug":"neural-style-transfer-with-twin-delayed-ddpg","title":"Neural Style Transfer with Twin-Delayed DDPG for Shared Control of Robotic Manipulators","date":"2024-02-01","arxiv_id":"2402.00722","n_code_links":0,"syntology":null},{"paper":"/paper/reconciling-spatial-and-temporal-abstractions","slug":"reconciling-spatial-and-temporal-abstractions","title":"Reconciling Spatial and Temporal Abstractions for Goal Representation","date":"2024-01-18","arxiv_id":"2401.09870","n_code_links":1,"syntology":{"ran":5,"of":11,"n_ran_checked":3,"n_instrument":2,"unverified":6,"pointer_only":11,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 6 unverified","official":{"repos":["cosynus-lix/STAR"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":6,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"sum-throughput-maximization-in-multi-bd","title":"Sum Throughput Maximization in Multi-BD Symbiotic Radio NOMA Network Assisted by Active-STAR-RIS","date":"2024-01-16","arxiv_id":"2401.08301","n_code_links":0,"syntology":null},{"paper":"/paper/an-experimental-evaluation-of-deep","slug":"an-experimental-evaluation-of-deep","title":"An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control","date":"2024-01-11","arxiv_id":"2401.05737","n_code_links":1,"syntology":null},{"paper":null,"slug":"adaptive-kalman-based-hybrid-car-following","title":"Adaptive Kalman-based hybrid car following strategy using TD3 and CACC","date":"2023-12-26","arxiv_id":"2312.15993","n_code_links":0,"syntology":null},{"paper":null,"slug":"multiagent-copilot-approach-for-shared","title":"Multiagent Copilot Approach for Shared Autonomy between Human EEG and TD3 Deep Reinforcement Learning","date":"2023-12-22","arxiv_id":"2312.14458","n_code_links":0,"syntology":null},{"paper":null,"slug":"6g-fresnel-spot-beamfocusing-using-large","title":"6G Fresnel Spot Beamfocusing using Large-Scale Metasurfaces: A Distributed DRL-Based Approach","date":"2023-11-18","arxiv_id":"2311.11109","n_code_links":0,"syntology":null},{"paper":null,"slug":"mitigating-estimation-errors-by-twin-td","title":"Mitigating Estimation Errors by Twin TD-Regularized Actor and Critic for Deep Reinforcement Learning","date":"2023-11-07","arxiv_id":"2311.03711","n_code_links":0,"syntology":null},{"paper":null,"slug":"decoupled-actor-critic","title":"On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economics","date":"2023-10-30","arxiv_id":"2310.19527","n_code_links":0,"syntology":null},{"paper":"/paper/hierarchical-framework-for-interpretable-and","slug":"hierarchical-framework-for-interpretable-and","title":"Hierarchical Framework for Interpretable and Probabilistic Model-Based Safe Reinforcement Learning","date":"2023-10-28","arxiv_id":"2310.18811","n_code_links":1,"syntology":null},{"paper":"/paper/dsac-t-distributional-soft-actor-critic-with","slug":"dsac-t-distributional-soft-actor-critic-with","title":"Distributional Soft Actor-Critic with Three Refinements","date":"2023-10-09","arxiv_id":"2310.05858","n_code_links":2,"syntology":{"ran":5,"of":5,"n_ran_checked":4,"n_instrument":1,"unverified":0,"pointer_only":5,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["jingliang-duan/dsac-t","jingliang-duan/dsac-v2"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"a-long-n-step-surrogate-stage-reward-for-deep","title":"A Long $N$-step Surrogate Stage Reward for Deep Reinforcement Learning","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/double-gumbel-q-learning","slug":"double-gumbel-q-learning","title":"Double Gumbel Q-Learning","date":"2023-09-21","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"taylor-td-learning","title":"Taylor TD-learning","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"caching-at-stars-the-next-generation-edge","title":"Caching-at-STARS: the Next Generation Edge Caching","date":"2023-08-01","arxiv_id":"2308.00562","n_code_links":0,"syntology":null},{"paper":null,"slug":"evolutionary-strategy-guided-reinforcement","title":"Evolutionary Strategy Guided Reinforcement Learning via MultiBuffer Communication","date":"2023-06-20","arxiv_id":"2306.11535","n_code_links":0,"syntology":null},{"paper":"/paper/backproptools-a-fast-portable-deep","slug":"backproptools-a-fast-portable-deep","title":"RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control","date":"2023-06-06","arxiv_id":"2306.03530","n_code_links":1,"syntology":null},{"paper":"/paper/for-sale-state-action-representation-learning-1","slug":"for-sale-state-action-representation-learning-1","title":"For SALE: State-Action Representation Learning for Deep Reinforcement Learning","date":"2023-06-04","arxiv_id":"2306.02451","n_code_links":2,"syntology":{"ran":3,"of":3,"n_ran_checked":1,"n_instrument":2,"unverified":0,"pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["sfujim/td7"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["found_in_text","official"]}}},{"paper":"/paper/relu-to-the-rescue-improve-your-on-policy","slug":"relu-to-the-rescue-improve-your-on-policy","title":"ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages","date":"2023-06-02","arxiv_id":"2306.01460","n_code_links":1,"syntology":null},{"paper":"/paper/temporally-layered-architecture-for-efficient","slug":"temporally-layered-architecture-for-efficient","title":"Optimizing Attention and Cognitive Control Costs Using Temporally-Layered Architectures","date":"2023-05-30","arxiv_id":"2305.18701","n_code_links":1,"syntology":null},{"paper":null,"slug":"understanding-reinforcement-learning","title":"Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization","date":"2023-03-31","arxiv_id":"2304.00026","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-strategy-oriented-bayesian-soft-actor","title":"A Strategy-Oriented Bayesian Soft Actor-Critic Model","date":"2023-03-07","arxiv_id":"2303.04193","n_code_links":0,"syntology":null},{"paper":"/paper/swim-a-general-purpose-high-performing-and","slug":"swim-a-general-purpose-high-performing-and","title":"Swim: A General-Purpose, High-Performing, and Efficient Activation Function for Locomotion Control Tasks","date":"2023-03-05","arxiv_id":"2303.02640","n_code_links":1,"syntology":null},{"paper":null,"slug":"the-ladder-in-chaos-a-simple-and-effective","title":"The Ladder in Chaos: A Simple and Effective Improvement to General DRL Algorithms by Policy Path Trimming and Boosting","date":"2023-03-02","arxiv_id":"2303.01391","n_code_links":0,"syntology":null},{"paper":"/paper/automatic-noise-filtering-with-dynamic-sparse","slug":"automatic-noise-filtering-with-dynamic-sparse","title":"Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning","date":"2023-02-13","arxiv_id":"2302.06548","n_code_links":1,"syntology":{"ran":4,"of":4,"n_ran_checked":4,"n_instrument":0,"unverified":0,"pointer_only":1,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["bramgrooten/automatic-noise-filtering"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"ris-assisted-jamming-rejection-and-path","title":"RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework","date":"2023-02-10","arxiv_id":"2302.04994","n_code_links":0,"syntology":null},{"paper":null,"slug":"automated-deep-reinforcement-learning-for","title":"Automated deep reinforcement learning for real-time scheduling strategy of multi-energy system integrated with post-carbon and direct-air carbon captured system","date":"2023-01-18","arxiv_id":"2301.07768","n_code_links":0,"syntology":null},{"paper":null,"slug":"hierarchical-deep-q-learning-based-handover","title":"Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity","date":"2023-01-13","arxiv_id":"2301.05391","n_code_links":0,"syntology":null},{"paper":null,"slug":"actor-director-critic-a-novel-deep","title":"Actor-Director-Critic: A Novel Deep Reinforcement Learning Framework","date":"2023-01-10","arxiv_id":"2301.03887","n_code_links":0,"syntology":null},{"paper":"/paper/extreme-q-learning-maxent-rl-without-entropy","slug":"extreme-q-learning-maxent-rl-without-entropy","title":"Extreme Q-Learning: MaxEnt RL without Entropy","date":"2023-01-05","arxiv_id":"2301.02328","n_code_links":4,"syntology":{"ran":8,"of":13,"n_ran_checked":5,"n_instrument":3,"unverified":5,"pointer_only":7,"phrase":"8 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified","official":null}},{"paper":null,"slug":"off-policy-deep-reinforcement-learning-1","title":"Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks","date":"2022-12-11","arxiv_id":"2212.05572","n_code_links":0,"syntology":null},{"paper":"/paper/td3-with-reverse-kl-regularizer-for-offline","slug":"td3-with-reverse-kl-regularizer-for-offline","title":"TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets","date":"2022-12-05","arxiv_id":"2212.02125","n_code_links":1,"syntology":null},{"paper":null,"slug":"algorithmic-trading-using-continuous-action","title":"Algorithmic Trading Using Continuous Action Space Deep Reinforcement Learning","date":"2022-10-07","arxiv_id":"2210.03469","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-novel-entropy-maximizing-td3-based","title":"A Novel Entropy-Maximizing TD3-based Reinforcement Learning for Automatic PID Tuning","date":"2022-10-05","arxiv_id":"2210.02381","n_code_links":0,"syntology":null},{"paper":"/paper/partial-observability-during-drl-for-robot","slug":"partial-observability-during-drl-for-robot","title":"Experimental Study on The Effect of Multi-step Deep Reinforcement Learning in POMDPs","date":"2022-09-12","arxiv_id":"2209.04999","n_code_links":1,"syntology":null},{"paper":"/paper/bsac-bayesian-strategy-network-based-soft","slug":"bsac-bayesian-strategy-network-based-soft","title":"Bayesian Soft Actor-Critic: A Directed Acyclic Strategy Graph Based Deep Reinforcement Learning","date":"2022-08-11","arxiv_id":"2208.06033","n_code_links":2,"syntology":null},{"paper":"/paper/performance-comparison-of-deep-rl-algorithms","slug":"performance-comparison-of-deep-rl-algorithms","title":"Performance Comparison of Deep RL Algorithms for Energy Systems Optimal Scheduling","date":"2022-08-01","arxiv_id":"2208.00728","n_code_links":1,"syntology":null},{"paper":null,"slug":"multi-agent-deep-reinforcement-learning-2","title":"Multi-Agent Deep Reinforcement Learning-Driven Mitigation of Adverse Effects of Cyber-Attacks on Electric Vehicle Charging Station","date":"2022-07-14","arxiv_id":"2207.07041","n_code_links":0,"syntology":null},{"paper":null,"slug":"equivariant-reinforcement-learning-for","title":"Equivariant Reinforcement Learning for Quadrotor UAV","date":"2022-06-02","arxiv_id":"2206.01233","n_code_links":0,"syntology":null},{"paper":null,"slug":"non-parametric-stochastic-policy-gradient","title":"Non-Parametric Stochastic Policy Gradient with Strategic Retreat for Non-Stationary Environment","date":"2022-03-24","arxiv_id":"2203.14905","n_code_links":0,"syntology":null},{"paper":"/paper/action-candidate-driven-clipped-double-q","slug":"action-candidate-driven-clipped-double-q","title":"Action Candidate Driven Clipped Double Q-learning for Discrete and Continuous Action Tasks","date":"2022-03-22","arxiv_id":"2203.11526","n_code_links":1,"syntology":null},{"paper":"/paper/adaptively-calibrated-critic-estimates-for","slug":"adaptively-calibrated-critic-estimates-for","title":"Adaptively Calibrated Critic Estimates for Deep Reinforcement Learning","date":"2021-11-24","arxiv_id":"2111.12673","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["nicolinho/acc"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"improving-learning-from-demonstrations-by-1","title":"Improving Learning from Demonstrations by Learning from Experience","date":"2021-11-16","arxiv_id":"2111.08156","n_code_links":0,"syntology":null},{"paper":"/paper/recurrent-off-policy-baselines-for-memory","slug":"recurrent-off-policy-baselines-for-memory","title":"Recurrent Off-policy Baselines for Memory-based Continuous Control","date":"2021-10-25","arxiv_id":"2110.12628","n_code_links":1,"syntology":null},{"paper":null,"slug":"parallel-actors-and-learners-a-framework-for","title":"Parallel Actors and Learners: A Framework for Generating Scalable RL Implementations","date":"2021-10-03","arxiv_id":"2110.01101","n_code_links":0,"syntology":null},{"paper":"/paper/explanation-aware-experience-replay-in-rule","slug":"explanation-aware-experience-replay-in-rule","title":"Explanation-Aware Experience Replay in Rule-Dense Environments","date":"2021-09-29","arxiv_id":"2109.14711","n_code_links":1,"syntology":null},{"paper":null,"slug":"faster-reinforcement-learning-with-value","title":"Faster Reinforcement Learning with Value Target Lower Bounding","date":"2021-09-29","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"meta-attention-for-off-policy-actor-critic","title":"Meta Attention For Off-Policy Actor-Critic","date":"2021-09-29","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"robust-and-data-efficient-q-learning-by","title":"Robust and Data-efficient Q-learning by Composite Value-estimation","date":"2021-09-29","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"spp-rl-state-planning-policy-reinforcement","title":"SPP-RL: State Planning Policy Reinforcement Learning","date":"2021-09-29","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/parameter-free-deterministic-reduction-of-the","slug":"parameter-free-deterministic-reduction-of-the","title":"Parameter-free Reduction of the Estimation Bias in Deep Reinforcement Learning for Deterministic Policy Gradients","date":"2021-09-24","arxiv_id":"2109.11788","n_code_links":1,"syntology":null},{"paper":"/paper/estimation-error-correction-in-deep","slug":"estimation-error-correction-in-deep","title":"Estimation Error Correction in Deep Reinforcement Learning for Deterministic Actor-Critic Methods","date":"2021-09-22","arxiv_id":"2109.10736","n_code_links":1,"syntology":null},{"paper":null,"slug":"ader-adapting-between-exploration-and","title":"ADER:Adapting between Exploration and Robustness for Actor-Critic Methods","date":"2021-09-08","arxiv_id":"2109.03443","n_code_links":0,"syntology":null},{"paper":null,"slug":"wad-a-deep-reinforcement-learning-agent-for","title":"WAD: A Deep Reinforcement Learning Agent for Urban Autonomous Driving","date":"2021-08-27","arxiv_id":"2108.12134","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-for-continuous","title":"Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study","date":"2021-08-05","arxiv_id":"2108.02665","n_code_links":0,"syntology":null},{"paper":null,"slug":"value-based-reinforcement-learning-for","title":"Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings","date":"2021-07-28","arxiv_id":"2107.13356","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-continuous-control-with-double","slug":"efficient-continuous-control-with-double","title":"Efficient Continuous Control with Double Actors and Regularized Critics","date":"2021-06-06","arxiv_id":"2106.03050","n_code_links":1,"syntology":null},{"paper":"/paper/action-candidate-based-clipped-double-q","slug":"action-candidate-based-clipped-double-q","title":"Action Candidate Based Clipped Double Q-learning for Discrete and Continuous Action Tasks","date":"2021-05-03","arxiv_id":"2105.00704","n_code_links":1,"syntology":null},{"paper":"/paper/learning-to-play-soccer-from-scratch-sample","slug":"learning-to-play-soccer-from-scratch-sample","title":"Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competition","date":"2021-03-09","arxiv_id":"2103.05174","n_code_links":1,"syntology":null},{"paper":null,"slug":"application-of-twin-delayed-deep","title":"Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control","date":"2021-02-25","arxiv_id":"2102.13012","n_code_links":0,"syntology":null},{"paper":"/paper/memory-based-deep-reinforcement-learning-for-1","slug":"memory-based-deep-reinforcement-learning-for-1","title":"Memory-based Deep Reinforcement Learning for POMDPs","date":"2021-02-24","arxiv_id":"2102.12344","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":1,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["LinghengMeng/LSTM-TD3"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/offcon-3-what-is-state-of-the-art-anyway","slug":"offcon-3-what-is-state-of-the-art-anyway","title":"OffCon$^3$: What is state of the art anyway?","date":"2021-01-27","arxiv_id":"2101.11331","n_code_links":1,"syntology":null},{"paper":null,"slug":"gst-group-sparse-training-for-accelerating","title":"GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning","date":"2021-01-24","arxiv_id":"2101.09650","n_code_links":0,"syntology":null},{"paper":"/paper/learning-synthetic-environments-for","slug":"learning-synthetic-environments-for","title":"Learning Synthetic Environments for Reinforcement Learning with Evolution Strategies","date":"2021-01-24","arxiv_id":"2101.09721","n_code_links":1,"syntology":null},{"paper":null,"slug":"pgps-coupling-policy-gradient-with-population","title":"PGPS : Coupling Policy Gradient with Population-based Search","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/policy-gradient-rl-algorithms-as-directed","slug":"policy-gradient-rl-algorithms-as-directed","title":"Policy Gradient RL Algorithms as Directed Acyclic Graphs","date":"2020-12-14","arxiv_id":"2012.07763","n_code_links":1,"syntology":null},{"paper":null,"slug":"ranking-items-in-large-scale-item-search","title":"Ranking Items in Large-Scale Item Search Engines with Reinforcement Learning","date":"2020-12-14","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"virtual-autonomous-driving-with-reinforcement","title":"Virtual Autonomous Driving with Reinforcement Learning","date":"2020-12-14","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"opac-opportunistic-actor-critic","title":"OPAC: Opportunistic Actor-Critic","date":"2020-12-11","arxiv_id":"2012.06555","n_code_links":0,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":122,"with_a_code_link":49,"where_syntology_ran_a_sample":18,"not_listed_spam_title":0,"listed":122,"listed_where_code_ran":18,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":17,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":17,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":18,"papers_where_code_ran_first_page":"/method/clipped-double-q-learning/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/clipped-double-q-learning/papers/2","json":"/method/clipped-double-q-learning/papers/2.json","rows":22},{"order":"ran","page":1,"of":1,"url":"/method/clipped-double-q-learning/papers/ran/1","json":"/method/clipped-double-q-learning/papers/ran/1.json","rows":18}],"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":70},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":56},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":55},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":46},{"task":"/task/continuous-control","name":"Continuous Control","papers":32},{"task":"/task/continuous-control","name":"continuous-control","papers":31},{"task":"/task/mujoco","name":"MuJoCo","papers":27},{"task":"/task/q-learning","name":"Q-Learning","papers":17},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":10},{"task":"/task/decision-making","name":"Decision Making","papers":8},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":6},{"task":"/task/offline-rl","name":"Offline RL","papers":5},{"task":"/task/benchmarking","name":"Benchmarking","papers":4},{"task":"/task/meta-learning","name":"Meta-Learning","papers":4},{"task":"/task/policy-gradient-methods","name":"Policy Gradient Methods","papers":4},{"task":"/task/d4rl","name":"D4RL","papers":3},{"task":null,"name":"GPU","papers":3},{"task":"/task/scheduling","name":"Scheduling","papers":3},{"task":"/task/atari-games","name":"Atari Games","papers":2},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":2}],"tasks_shown":20,"n_tasks":69,"usage_by_year":[{"year":"2018","papers":1},{"year":"2019","papers":6},{"year":"2020","papers":19},{"year":"2021","papers":24},{"year":"2022","papers":11},{"year":"2023","papers":26},{"year":"2024","papers":25},{"year":"2025","papers":10}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/clipped-double-q-learning"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Clipped%20Double%20Q-learning","read_at":"2026-10-05T00:53:45+00:00","query":"Clipped Double Q-learning","kind":"method","have":"unknown_name","tier":null,"verified_implementation":null,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":null,"resolution":null,"resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":null,"archive_paper_arxiv_id":null,"repositories":null,"retrieve_with":null,"name_competition":{"verdict":"none","may_be_wrong_sense":false,"bearers_found":0,"served":null,"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing in this corpus is named 'Clipped Double Q-learning' -- no method node, no paper announcing it in its title, no expanded form declaring it as an acronym."},"name_competition_text":null,"near_matches":["Action Candidate Based Clipped Double Q-learning","DOUBLE","Double-P","Semi-Clipped","clipped-SSTM","Double Ratchet","double-scoring","double-loop MGDA"],"code_nearby":[],"code_nearby_caption":null,"origin_paper":null,"line":"name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"325bb78dce094e3116e3fa6f45b2354fb9ba3ab62d9ec3e83640a8f0acde91cb","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}