{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/double-q-learning","slug":"double-q-learning","name":"Double Q-learning","full_name":"Double Q-learning","full_name_withheld":false,"description_markdown":"**Double Q-learning** is an off-policy reinforcement learning algorithm that utilises double estimation to counteract overestimation problems with traditional Q-learning. \r\n\r\nThe max operator in standard [Q-learning](https://paperswithcode.com/method/q-learning) and [DQN](https://paperswithcode.com/method/dqn) uses the same values both to select and to evaluate an action. This makes it more likely to select overestimated values, resulting in overoptimistic value estimates. To prevent this, we can decouple the selection from the evaluation, which is the idea behind Double Q-learning:\r\n\r\n$$ Y^{Q}\\_{t} = R\\_{t+1} + \\gamma{Q}\\left(S\\_{t+1}, \\arg\\max\\_{a}Q\\left(S\\_{t+1}, a; \\mathbb{\\theta}\\_{t}\\right);\\mathbb{\\theta}\\_{t}\\right) $$\r\n\r\nThe Double Q-learning error can then be written as:\r\n\r\n$$ Y^{DoubleQ}\\_{t} = R\\_{t+1} + \\gamma{Q}\\left(S\\_{t+1}, \\arg\\max\\_{a}Q\\left(S\\_{t+1}, a; \\mathbb{\\theta}\\_{t}\\right);\\mathbb{\\theta}^{'}\\_{t}\\right) $$\r\n\r\nHere the selection of the action in the $\\arg\\max$ is still due to the online weights $\\theta\\_{t}$. But we use a second set of weights $\\mathbb{\\theta}^{'}\\_{t}$ to fairly evaluate the value of this policy.\r\n\r\nSource: [Deep Reinforcement Learning with Double Q-learning](https://paperswithcode.com/paper/deep-reinforcement-learning-with-double-q)","description_state":"present","introduced_year":null,"introduced_by":{"title":"Double Q-learning","paper":"/paper/double-q-learning","first_author":"Hado V. Hasselt","n_authors":1,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/double-q-learning"},"source":{"url":"http://papers.nips.cc/paper/3964-double-q-learning","title":"Double Q-learning","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Off-Policy TD Control","url":"/methods/category/off-policy-td-control","pwc_aliases":[]}],"n_papers_tagged":112,"archive_num_papers":112,"papers_newest_first":[{"paper":null,"slug":"reinforcement-learning-based-policy","title":"Reinforcement Learning-Based Policy Optimisation For Heterogeneous Radio Access","date":"2025-06-18","arxiv_id":"2506.15273","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamic-operating-system-scheduling-using","title":"Dynamic Operating System Scheduling Using Double DQN: A Reinforcement Learning Approach to Task Optimization","date":"2025-03-31","arxiv_id":"2503.23659","n_code_links":0,"syntology":null},{"paper":null,"slug":"distribution-free-uncertainty-quantification-2","title":"Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework","date":"2024-12-17","arxiv_id":"2412.12597","n_code_links":0,"syntology":null},{"paper":"/paper/beyond-the-rainbow-high-performance-deep","slug":"beyond-the-rainbow-high-performance-deep","title":"Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC","date":"2024-11-06","arxiv_id":"2411.03820","n_code_links":3,"syntology":{"ran":16,"of":25,"n_ran_checked":12,"n_instrument":4,"unverified":9,"pointer_only":21,"phrase":"16 ran (of which 10 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 4 where Syntology's instrument failed) · 9 unverified","official":{"repos":["viptankz/btr"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["listed","official"]}}},{"paper":null,"slug":"bootstrapping-expectiles-in-reinforcement","title":"Bootstrapping Expectiles in Reinforcement Learning","date":"2024-06-06","arxiv_id":"2406.04081","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-new-view-on-planning-in-online","title":"A New View on Planning in Online Reinforcement Learning","date":"2024-06-03","arxiv_id":"2406.01562","n_code_links":0,"syntology":null},{"paper":null,"slug":"active-search-and-coverage-using-point-cloud","title":"Active search and coverage using point-cloud reinforcement learning","date":"2023-12-18","arxiv_id":"2312.11410","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-sparse-reward-goal-conditioned","slug":"efficient-sparse-reward-goal-conditioned","title":"Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization","date":"2023-12-10","arxiv_id":"2312.05787","n_code_links":1,"syntology":null},{"paper":null,"slug":"data-efficient-deep-reinforcement-learning-2","title":"Data-efficient Deep Reinforcement Learning for Vehicle Trajectory Control","date":"2023-11-30","arxiv_id":"2311.18393","n_code_links":0,"syntology":null},{"paper":null,"slug":"advancing-algorithmic-trading-a-multi","title":"Advancing Algorithmic Trading: A Multi-Technique Enhancement of Deep Q-Network Models","date":"2023-11-09","arxiv_id":"2311.05743","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-for-the-heat","title":"Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets","date":"2023-09-25","arxiv_id":"2309.13955","n_code_links":0,"syntology":null},{"paper":null,"slug":"ai-driven-patient-monitoring-with-multi-agent","title":"Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions","date":"2023-09-20","arxiv_id":"2309.10980","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-for-artificial","title":"Deep Reinforcement Learning for Artificial Upwelling Energy Management","date":"2023-08-20","arxiv_id":"2308.10199","n_code_links":0,"syntology":null},{"paper":null,"slug":"interpretable-and-secure-trajectory","title":"Interpretable and Secure Trajectory Optimization for UAV-Assisted Communication","date":"2023-07-05","arxiv_id":"2307.02002","n_code_links":0,"syntology":null},{"paper":null,"slug":"optimizing-credit-limit-adjustments-under","title":"Optimizing Credit Limit Adjustments Under Adversarial Goals Using Reinforcement Learning","date":"2023-06-27","arxiv_id":"2306.15585","n_code_links":0,"syntology":null},{"paper":null,"slug":"vanishing-bias-heuristic-guided-reinforcement","title":"Vanishing Bias Heuristic-guided Reinforcement Learning Algorithm","date":"2023-06-17","arxiv_id":"2306.10216","n_code_links":0,"syntology":null},{"paper":null,"slug":"2305-14656","title":"RSRM: Reinforcement Symbolic Regression Machine","date":"2023-05-24","arxiv_id":"2305.14656","n_code_links":0,"syntology":null},{"paper":"/paper/extracting-diagnosis-pathways-from-electronic","slug":"extracting-diagnosis-pathways-from-electronic","title":"Extracting Diagnosis Pathways from Electronic Health Records Using Deep Reinforcement Learning","date":"2023-05-10","arxiv_id":"2305.06295","n_code_links":1,"syntology":null},{"paper":"/paper/train-a-real-world-local-path-planner-in-one","slug":"train-a-real-world-local-path-planner-in-one","title":"Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversity","date":"2023-05-07","arxiv_id":"2305.04180","n_code_links":1,"syntology":null},{"paper":null,"slug":"smoothed-q-learning","title":"Smoothed Q-learning","date":"2023-03-15","arxiv_id":"2303.08631","n_code_links":0,"syntology":null},{"paper":null,"slug":"double-a3c-deep-reinforcement-learning-on","title":"Double A3C: Deep Reinforcement Learning on OpenAI Gym Games","date":"2023-03-04","arxiv_id":"2303.02271","n_code_links":0,"syntology":null},{"paper":null,"slug":"asynchronous-deep-double-duelling-q-learning","title":"Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets","date":"2023-01-20","arxiv_id":"2301.08688","n_code_links":0,"syntology":null},{"paper":null,"slug":"hierarchical-deep-q-learning-based-handover","title":"Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity","date":"2023-01-13","arxiv_id":"2301.05391","n_code_links":0,"syntology":null},{"paper":null,"slug":"neighboring-state-based-rl-exploration","title":"Neighboring state-based RL Exploration","date":"2022-12-21","arxiv_id":"2212.10712","n_code_links":0,"syntology":null},{"paper":null,"slug":"elastic-step-dqn-a-novel-multi-step-algorithm","title":"Elastic Step DQN: A novel multi-step algorithm to alleviate overestimation in Deep QNetworks","date":"2022-10-07","arxiv_id":"2210.03325","n_code_links":0,"syntology":null},{"paper":"/paper/m-2-dqn-a-robust-method-for-accelerating-deep","slug":"m-2-dqn-a-robust-method-for-accelerating-deep","title":"M$^2$DQN: A Robust Method for Accelerating Deep Q-learning Network","date":"2022-09-16","arxiv_id":"2209.07809","n_code_links":1,"syntology":null},{"paper":"/paper/reducing-variance-in-temporal-difference","slug":"reducing-variance-in-temporal-difference","title":"Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks","date":"2022-09-16","arxiv_id":"2209.07670","n_code_links":1,"syntology":null},{"paper":null,"slug":"double-q-learning-for-citizen-relocation","title":"Double Q-Learning for Citizen Relocation During Natural Hazards","date":"2022-09-08","arxiv_id":"2209.03800","n_code_links":0,"syntology":null},{"paper":"/paper/deep-reinforcement-learning-with-swin","slug":"deep-reinforcement-learning-with-swin","title":"Deep Reinforcement Learning with Swin Transformers","date":"2022-06-30","arxiv_id":"2206.15269","n_code_links":1,"syntology":null},{"paper":"/paper/dna-proximal-policy-optimization-with-a-dual","slug":"dna-proximal-policy-optimization-with-a-dual","title":"DNA: Proximal Policy Optimization with a Dual Network Architecture","date":"2022-06-20","arxiv_id":"2206.10027","n_code_links":1,"syntology":null},{"paper":null,"slug":"goal-space-planning-with-subgoal-models","title":"Goal-Space Planning with Subgoal Models","date":"2022-06-06","arxiv_id":"2206.02902","n_code_links":0,"syntology":null},{"paper":"/paper/simultaneous-double-q-learning-with","slug":"simultaneous-double-q-learning-with","title":"Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods","date":"2022-05-08","arxiv_id":"2205.03819","n_code_links":1,"syntology":null},{"paper":null,"slug":"optimizing-the-long-term-behaviour-of-deep","title":"Optimizing the Long-Term Behaviour of Deep Reinforcement Learning for Pushing and Grasping","date":"2022-04-07","arxiv_id":"2204.03487","n_code_links":0,"syntology":null},{"paper":"/paper/action-candidate-driven-clipped-double-q","slug":"action-candidate-driven-clipped-double-q","title":"Action Candidate Driven Clipped Double Q-learning for Discrete and Continuous Action Tasks","date":"2022-03-22","arxiv_id":"2203.11526","n_code_links":1,"syntology":null},{"paper":null,"slug":"random-ensemble-reinforcement-learning-for","title":"Random Ensemble Reinforcement Learning for Traffic Signal Control","date":"2022-03-10","arxiv_id":"2203.05961","n_code_links":0,"syntology":null},{"paper":"/paper/skrl-modular-and-flexible-library-for","slug":"skrl-modular-and-flexible-library-for","title":"skrl: Modular and Flexible Library for Reinforcement Learning","date":"2022-02-08","arxiv_id":"2202.03825","n_code_links":1,"syntology":null},{"paper":null,"slug":"an-improved-reinforcement-learning-algorithm","title":"An Improved Reinforcement Learning Algorithm for Learning to Branch","date":"2022-01-17","arxiv_id":"2201.06213","n_code_links":0,"syntology":null},{"paper":"/paper/intelligent-traffic-light-via-policy-based","slug":"intelligent-traffic-light-via-policy-based","title":"Intelligent Traffic Light via Policy-based Deep Reinforcement Learning","date":"2021-12-27","arxiv_id":"2112.13817","n_code_links":1,"syntology":null},{"paper":null,"slug":"faster-non-asymptotic-convergence-for-double","title":"Faster Non-asymptotic Convergence for Double Q-learning","date":"2021-12-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"aggressive-q-learning-with-ensembles-1","title":"Aggressive Q-Learning with Ensembles: Achieving Both High Sample Efficiency and High Asymptotic Performance","date":"2021-11-17","arxiv_id":"2111.09159","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-study-of-first-passage-time-minimization","title":"A study of first-passage time minimization via Q-learning in heated gridworlds","date":"2021-10-05","arxiv_id":"2110.02129","n_code_links":0,"syntology":null},{"paper":"/paper/dropout-q-functions-for-doubly-efficient","slug":"dropout-q-functions-for-doubly-efficient","title":"Dropout Q-Functions for Doubly Efficient Reinforcement Learning","date":"2021-10-05","arxiv_id":"2110.02034","n_code_links":2,"syntology":{"ran":6,"of":8,"n_ran_checked":6,"n_instrument":0,"unverified":2,"pointer_only":0,"phrase":"6 ran (of which 3 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","official":{"repos":["TakuyaHiraoka/Dropout-Q-Functions-for-Doubly-Efficient-Reinforcement-Learning"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["found_in_text","official"]}}},{"paper":"/paper/on-the-estimation-bias-in-double-q-learning-1","slug":"on-the-estimation-bias-in-double-q-learning-1","title":"On the Estimation Bias in Double Q-Learning","date":"2021-09-29","arxiv_id":"2109.14419","n_code_links":1,"syntology":{"ran":4,"of":6,"n_ran_checked":1,"n_instrument":3,"unverified":2,"pointer_only":0,"phrase":"4 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","official":{"repos":["stilwell-git/doubly-bounded-q-learning"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/parameter-free-deterministic-reduction-of-the","slug":"parameter-free-deterministic-reduction-of-the","title":"Parameter-free Reduction of the Estimation Bias in Deep Reinforcement Learning for Deterministic Policy Gradients","date":"2021-09-24","arxiv_id":"2109.11788","n_code_links":1,"syntology":null},{"paper":"/paper/estimation-error-correction-in-deep","slug":"estimation-error-correction-in-deep","title":"Estimation Error Correction in Deep Reinforcement Learning for Deterministic Actor-Critic Methods","date":"2021-09-22","arxiv_id":"2109.10736","n_code_links":1,"syntology":null},{"paper":"/paper/deep-reinforcement-learning-at-the-edge-of","slug":"deep-reinforcement-learning-at-the-edge-of","title":"Deep Reinforcement Learning at the Edge of the Statistical Precipice","date":"2021-08-30","arxiv_id":"2108.13264","n_code_links":3,"syntology":{"ran":5,"of":5,"n_ran_checked":5,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 4 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["google-research/rliable"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"modified-double-dqn-addressing-stability","title":"Modified Double DQN: addressing stability","date":"2021-08-09","arxiv_id":"2108.04115","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-improved-algorithm-of-robot-path-planning","title":"An Improved Algorithm of Robot Path Planning in Complex Environment Based on Double DQN","date":"2021-07-23","arxiv_id":"2107.11245","n_code_links":0,"syntology":null},{"paper":"/paper/a-reinforcement-learning-environment-for-2","slug":"a-reinforcement-learning-environment-for-2","title":"A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis","date":"2021-07-15","arxiv_id":"2107.07373","n_code_links":1,"syntology":null},{"paper":null,"slug":"transfer-learning-in-multi-agent","title":"Transfer Learning in Multi-Agent Reinforcement Learning with Double Q-Networks for Distributed Resource Sharing in V2X Communication","date":"2021-07-13","arxiv_id":"2107.06195","n_code_links":0,"syntology":null},{"paper":null,"slug":"expert-q-learning-deep-q-learning-with-state","title":"Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples","date":"2021-06-28","arxiv_id":"2106.14642","n_code_links":0,"syntology":null},{"paper":null,"slug":"learning-to-optimize-industry-scale-dynamic","title":"Learning to Optimize Industry-Scale Dynamic Pickup and Delivery Problems","date":"2021-05-27","arxiv_id":"2105.12899","n_code_links":0,"syntology":null},{"paper":"/paper/action-candidate-based-clipped-double-q","slug":"action-candidate-based-clipped-double-q","title":"Action Candidate Based Clipped Double Q-learning for Discrete and Continuous Action Tasks","date":"2021-05-03","arxiv_id":"2105.00704","n_code_links":1,"syntology":null},{"paper":"/paper/independent-reinforcement-learning-for-weakly","slug":"independent-reinforcement-learning-for-weakly","title":"Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem","date":"2021-04-22","arxiv_id":"2104.10917","n_code_links":1,"syntology":null},{"paper":"/paper/a-coevolutionairy-approach-to-deep-multi","slug":"a-coevolutionairy-approach-to-deep-multi","title":"A coevolutionary approach to deep multi-agent reinforcement learning","date":"2021-04-12","arxiv_id":"2104.05610","n_code_links":1,"syntology":null},{"paper":null,"slug":"variational-quantum-compiling-with-double-q","title":"Variational quantum compiling with double Q-learning","date":"2021-03-22","arxiv_id":"2103.11611","n_code_links":0,"syntology":null},{"paper":null,"slug":"greedy-multi-step-off-policy-reinforcement-1","title":"Greedy-Step Off-Policy Reinforcement Learning","date":"2021-02-23","arxiv_id":"2102.11717","n_code_links":0,"syntology":null},{"paper":"/paper/recurrent-rational-networks","slug":"recurrent-rational-networks","title":"Adaptive Rational Activations to Boost Deep Reinforcement Learning","date":"2021-02-18","arxiv_id":"2102.09407","n_code_links":4,"syntology":{"ran":6,"of":6,"n_ran_checked":0,"n_instrument":6,"unverified":0,"pointer_only":3,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified","official":{"repos":["ml-research/rational_activations","ml-research/rational_rl","ml-research/rational_sl","k4ntz/activation-functions"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/randomized-ensembled-double-q-learning-1","slug":"randomized-ensembled-double-q-learning-1","title":"Randomized Ensembled Double Q-Learning: Learning Fast Without a Model","date":"2021-01-15","arxiv_id":"2101.05982","n_code_links":6,"syntology":{"ran":4,"of":5,"n_ran_checked":4,"n_instrument":0,"unverified":1,"pointer_only":3,"phrase":"4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 4 samples that ran constructed an object rather than computing a result","official":{"repos":["watchernyu/REDQ"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"double-q-learning-new-analysis-and-sharper","title":"Double Q-learning: New Analysis and Sharper Finite-time Bound","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"weighted-bellman-backups-for-improved-signal","title":"Weighted Bellman Backups for Improved Signal-to-Noise in Q-Updates","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"a-state-representation-dueling-network-for","title":"A State Representation Dueling Network for Deep Reinforcement Learning","date":"2020-12-24","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"mobile-robots-autonomous-exploration-with","title":"Mobile Robots Autonomous Exploration with Reinforcement Learning","date":"2020-12-14","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/resolving-implicit-coordination-in-multi","slug":"resolving-implicit-coordination-in-multi","title":"Resolving Implicit Coordination in Multi-Agent Deep Reinforcement Learning with Deep Q-Networks & Game Theory","date":"2020-12-08","arxiv_id":"2012.09136","n_code_links":1,"syntology":null},{"paper":null,"slug":"self-correcting-q-learning","title":"Self-correcting Q-Learning","date":"2020-12-02","arxiv_id":"2012.01100","n_code_links":0,"syntology":null},{"paper":null,"slug":"energy-and-service-priority-aware-trajectory","title":"Energy and Service-priority aware Trajectory Design for UAV-BSs using Double Q-Learning","date":"2020-10-26","arxiv_id":"2010.13346","n_code_links":0,"syntology":null},{"paper":null,"slug":"finite-time-analysis-for-double-q-learning","title":"Finite-Time Analysis for Double Q-learning","date":"2020-09-29","arxiv_id":"2009.14257","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-new-approach-for-tactical-decision-making","title":"A New Approach for Tactical Decision Making in Lane Changing: Sample Efficient Deep Q Learning with a Safety Feedback Reward","date":"2020-09-24","arxiv_id":"2009.11905","n_code_links":0,"syntology":null},{"paper":null,"slug":"chrome-dino-run-using-reinforcement-learning","title":"Chrome Dino Run using Reinforcement Learning","date":"2020-08-15","arxiv_id":"2008.06799","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-with-quantum","slug":"reinforcement-learning-with-quantum","title":"Reinforcement Learning with Quantum Variational Circuits","date":"2020-08-15","arxiv_id":"2008.07524","n_code_links":2,"syntology":null},{"paper":"/paper/qplex-duplex-dueling-multi-agent-q-learning","slug":"qplex-duplex-dueling-multi-agent-q-learning","title":"QPLEX: Duplex Dueling Multi-Agent Q-Learning","date":"2020-08-03","arxiv_id":"2008.01062","n_code_links":6,"syntology":{"ran":4,"of":4,"n_ran_checked":2,"n_instrument":2,"unverified":0,"pointer_only":1,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["wjh720/QPLEX"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/provably-efficient-double-q-learning","slug":"provably-efficient-double-q-learning","title":"The Mean-Squared Error of Double Q-Learning","date":"2020-07-09","arxiv_id":"2007.05034","n_code_links":1,"syntology":{"ran":2,"of":3,"n_ran_checked":2,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["wentaoweng/The-Mean-Squared-Error-of-Double-Q-Learning"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/sunrise-a-simple-unified-framework-for","slug":"sunrise-a-simple-unified-framework-for","title":"SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning","date":"2020-07-09","arxiv_id":"2007.04938","n_code_links":1,"syntology":null},{"paper":null,"slug":"regularly-updated-deterministic-policy","title":"Regularly Updated Deterministic Policy Gradient Algorithm","date":"2020-07-01","arxiv_id":"2007.00169","n_code_links":0,"syntology":null},{"paper":null,"slug":"noise-overestimation-and-exploration-in-deep","title":"Some approaches used to overcome overestimation in Deep Reinforcement Learning algorithms","date":"2020-06-25","arxiv_id":"2006.14167","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-control-for-radar","title":"Deep Reinforcement Learning Control for Radar Detection and Tracking in Congested Spectral Environments","date":"2020-06-23","arxiv_id":"2006.13173","n_code_links":0,"syntology":null},{"paper":null,"slug":"parameterized-mdps-and-reinforcement-learning","title":"Parameterized MDPs and Reinforcement Learning Problems -- A Maximum Entropy Principle Based Framework","date":"2020-06-17","arxiv_id":"2006.09646","n_code_links":0,"syntology":null},{"paper":"/paper/learning-heuristic-selection-with-dynamic","slug":"learning-heuristic-selection-with-dynamic","title":"Learning Heuristic Selection with Dynamic Algorithm Configuration","date":"2020-06-15","arxiv_id":"2006.08246","n_code_links":1,"syntology":null},{"paper":null,"slug":"decorrelated-double-q-learning","title":"Decorrelated Double Q-learning","date":"2020-06-12","arxiv_id":"2006.06956","n_code_links":0,"syntology":null},{"paper":null,"slug":"privacy-cost-management-in-smart-meters-an","title":"Privacy-Cost Management in Smart Meters with Mutual Information-Based Reinforcement Learning","date":"2020-06-10","arxiv_id":"2006.06106","n_code_links":0,"syntology":null},{"paper":null,"slug":"balancing-a-cartpole-system-with","title":"Balancing a CartPole System with Reinforcement Learning -- A Tutorial","date":"2020-06-08","arxiv_id":"2006.04938","n_code_links":0,"syntology":null},{"paper":"/paper/acme-a-research-framework-for-distributed","slug":"acme-a-research-framework-for-distributed","title":"Acme: A Research Framework for Distributed Reinforcement Learning","date":"2020-06-01","arxiv_id":"2006.00979","n_code_links":5,"syntology":null},{"paper":"/paper/dynamic-algorithm-configuration-foundation-of","slug":"dynamic-algorithm-configuration-foundation-of","title":"Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework","date":"2020-06-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"basal-glucose-control-in-type-1-diabetes","title":"Basal Glucose Control in Type 1 Diabetes using Deep Reinforcement Learning: An In Silico Validation","date":"2020-05-18","arxiv_id":"2005.09059","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-double-q-learning-approach-for-navigation","title":"A Double Q-Learning Approach for Navigation of Aerial Vehicles with Connectivity Constraint","date":"2020-02-24","arxiv_id":"2002.10563","n_code_links":0,"syntology":null},{"paper":null,"slug":"disentangling-controllable-object-through","title":"Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning","date":"2020-02-21","arxiv_id":"2002.09136","n_code_links":0,"syntology":null},{"paper":null,"slug":"fast-reinforcement-learning-for-anti-jamming","title":"Fast Reinforcement Learning for Anti-jamming Communications","date":"2020-02-13","arxiv_id":"2002.05364","n_code_links":0,"syntology":null},{"paper":null,"slug":"regret-bounds-for-discounted-mdps","title":"Regret Bounds for Discounted MDPs","date":"2020-02-12","arxiv_id":"2002.05138","n_code_links":0,"syntology":null},{"paper":"/paper/slm-lab-a-comprehensive-benchmark-and-modular-1","slug":"slm-lab-a-comprehensive-benchmark-and-modular-1","title":"SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning","date":"2019-12-28","arxiv_id":"1912.12482","n_code_links":1,"syntology":{"ran":10,"of":13,"n_ran_checked":10,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","official":{"repos":["kengz/SLM-Lab"],"state":"official (archive's flag): 10 ran","n_ran":10,"n_constructed":0,"n_ran_no_instrument_failure":10,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"exploiting-the-potential-of-deep","title":"Exploiting the potential of deep reinforcement learning for classification tasks in high-dimensional and unstructured data","date":"2019-12-20","arxiv_id":"1912.09595","n_code_links":0,"syntology":null},{"paper":"/paper/task-oriented-language-grounding-for-language","slug":"task-oriented-language-grounding-for-language","title":"Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order","date":"2019-10-27","arxiv_id":"1910.12354","n_code_links":1,"syntology":null},{"paper":null,"slug":"reverse-experience-replay","title":"Reverse Experience Replay","date":"2019-10-19","arxiv_id":"1910.08780","n_code_links":0,"syntology":null},{"paper":null,"slug":"to-combine-or-not-to-combine-a-rainbow-deep","title":"To Combine or Not To Combine? A Rainbow Deep Reinforcement Learning Agent for Dialog Policies","date":"2019-09-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/performing-deep-recurrent-double-q-learning","slug":"performing-deep-recurrent-double-q-learning","title":"Performing Deep Recurrent Double Q-Learning for Atari Games","date":"2019-08-16","arxiv_id":"1908.06040","n_code_links":2,"syntology":null},{"paper":null,"slug":"large-scale-traffic-signal-control-using-a","title":"Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning","date":"2019-08-10","arxiv_id":"1908.03761","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-models-of-human","slug":"reinforcement-learning-models-of-human","title":"A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry","date":"2019-06-21","arxiv_id":"1906.11286","n_code_links":1,"syntology":null},{"paper":null,"slug":"generative-adversarial-imagination-for-sample","title":"Generative Adversarial Imagination for Sample Efficient Deep Reinforcement Learning","date":"2019-04-30","arxiv_id":"1904.13255","n_code_links":0,"syntology":null},{"paper":null,"slug":"double-deep-q-learning-for-optimal-execution","title":"Double Deep Q-Learning for Optimal Execution","date":"2018-12-17","arxiv_id":"1812.06600","n_code_links":0,"syntology":null},{"paper":"/paper/macro-action-selection-with-deep","slug":"macro-action-selection-with-deep","title":"Macro action selection with deep reinforcement learning in StarCraft","date":"2018-12-02","arxiv_id":"1812.00336","n_code_links":1,"syntology":null},{"paper":"/paper/revisiting-the-softmax-bellman-operator","slug":"revisiting-the-softmax-bellman-operator","title":"Revisiting the Softmax Bellman Operator: New Benefits and New Perspective","date":"2018-12-02","arxiv_id":"1812.00456","n_code_links":2,"syntology":{"ran":0,"of":1,"n_ran_checked":0,"n_instrument":0,"unverified":1,"pointer_only":1,"phrase":"0 ran · 1 unverified","official":{"repos":["zhao-song/Softmax-DQN"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":112,"with_a_code_link":45,"where_syntology_ran_a_sample":15,"not_listed_spam_title":0,"listed":112,"listed_where_code_ran":15,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":13,"every_run_a_failure_of_syntologys_instrument":2,"listed_with_a_run_with_no_instrument_failure":13,"listed_every_run_a_failure_of_syntologys_instrument":2,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":15,"papers_where_code_ran_first_page":"/method/double-q-learning/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/double-q-learning/papers/2","json":"/method/double-q-learning/papers/2.json","rows":12},{"order":"ran","page":1,"of":1,"url":"/method/double-q-learning/papers/ran/1","json":"/method/double-q-learning/papers/ran/1.json","rows":15}],"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":72},{"task":"/task/q-learning","name":"Q-Learning","papers":63},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":61},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":58},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":47},{"task":"/task/atari-games","name":"Atari Games","papers":18},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":10},{"task":"/task/decision-making","name":"Decision Making","papers":9},{"task":"/task/continuous-control","name":"Continuous Control","papers":7},{"task":"/task/continuous-control","name":"continuous-control","papers":6},{"task":"/task/mujoco","name":"MuJoCo","papers":5},{"task":"/task/management","name":"Management","papers":4},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":4},{"task":"/task/scheduling","name":"Scheduling","papers":4},{"task":"/task/efficient-exploration","name":"Efficient Exploration","papers":3},{"task":"/task/general-reinforcement-learning","name":"General Reinforcement Learning","papers":3},{"task":"/task/traffic-signal-control","name":"Traffic Signal Control","papers":3},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":2},{"task":"/task/diversity","name":"Diversity","papers":2},{"task":"/task/ensemble-learning","name":"Ensemble Learning","papers":2}],"tasks_shown":20,"n_tasks":67,"usage_by_year":[{"year":"2010","papers":1},{"year":"2015","papers":2},{"year":"2016","papers":2},{"year":"2017","papers":2},{"year":"2018","papers":8},{"year":"2019","papers":9},{"year":"2020","papers":27},{"year":"2021","papers":24},{"year":"2022","papers":14},{"year":"2023","papers":17},{"year":"2024","papers":4},{"year":"2025","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/double-q-learning"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Double%20Q-learning","read_at":"2026-10-05T00:55:14+00:00","query":"Double Q-learning","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"DOUBLE","origin_arxiv_id":"2601.05524"},"resolution":"token_match","resolved_name_differs":true,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":1,"in_graph":1,"list_truncated":false,"strongest_link":"listed_only"},"retrieve_with":null,"name_competition":{"verdict":"contested","may_be_wrong_sense":false,"bearers_found":2,"served":{"arxiv_id":"2601.05524","citations":17},"leading_other_bearer":{"arxiv_id":"openreview_MwxaStJXK6v","title":"Double Q-learning: New Analysis and Sharper Finite-time Bound","citations":null,"named_by":["title_prefix"],"bearer_name":"Double Q-learning: New Analysis and Sharper Finite-time Bound","ask_with":"have('openreview_MwxaStJXK6v')"},"n_other_bearers_listed":1,"note":"1 other thing(s) here are also called 'Double Q-learning', and none of them is cited more than the paper this answer is about. They are listed in `also_named` -- check `served.arxiv_id` is the paper you meant."},"name_competition_text":null,"near_matches":null,"code_nearby":[{"name":"Double-Restart Q-UCB","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='Double-Restart Q-UCB')","origin_arxiv_id":"2010.03161"},{"name":"Periodic Double Oracle (PDO)","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Periodic Double Oracle (PDO)')","origin_arxiv_id":"2304.10498"},{"name":"Randomized Ensembled Double Q-Learning (REDQ)","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Randomized Ensembled Double Q-Learning (REDQ)')","origin_arxiv_id":"2101.05982"}],"code_nearby_caption":"Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.","origin_paper":{"arxiv_id":"2601.05524","page":"/paper/arxiv-2601-05524","syntology_url":"https://syntology.ai/paper/2601.05524","in_graph_layer":true,"samples":0},"line":"for a different name, 'DOUBLE': repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: listed only). The lookup matched a different name in Syntology's graph, 'DOUBLE' (resolution: token_match); this answer is about that name. The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2601.05524) is about this method is not checked.","level_legend":{"state":"quoted","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","card_sha256":"479785a84658dd682a11b10237b3620d651d7f87799f5dc7add970aaa83f6d39","note":"Syntology's served verification levels, quoted verbatim from its MCP server card; a row is split where the card separates clauses with a double hyphen, and the separator is not reproduced","levels":[{"level":"V0","quoted":["imports"]},{"level":"V1","quoted":["executes against a synthesized fixture and returns a value of the recorded type"]},{"level":"V2","quoted":["two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs","neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)"]},{"level":"V3","quoted":["paper-derived property tests pass on both of those same two generations."]}],"no_rung":"No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results."}},"syntology_have_state":"loaded","record_sha256":"a6715de2abf249a7d3cdf160db09676e1962127750c7e69911a8810492883fc3","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}