{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/double-dqn","slug":"double-dqn","name":"Double DQN","full_name":"Double DQN","full_name_withheld":false,"description_markdown":"A **Double Deep Q-Network**, or **Double DQN** utilises [Double Q-learning](https://paperswithcode.com/method/double-q-learning) to reduce overestimation by decomposing the max operation in the target into action selection and action evaluation. We evaluate the greedy policy according to the online network, but we use the target network to estimate its value.  The update is the same as for [DQN](https://paperswithcode.com/method/dqn), but replacing the target $Y^{DQN}\\_{t}$ with:\r\n\r\n$$ Y^{DoubleDQN}\\_{t} = R\\_{t+1}+\\gamma{Q}\\left(S\\_{t+1}, \\arg\\max\\_{a}Q\\left(S\\_{t+1}, a; \\theta\\_{t}\\right);\\theta\\_{t}^{-}\\right) $$\r\n\r\nCompared to the original formulation of Double [Q-Learning](https://paperswithcode.com/method/q-learning), in Double DQN the weights of the second network $\\theta^{'}\\_{t}$ are replaced with the weights of the target network $\\theta\\_{t}^{-}$ for the evaluation of the current greedy policy.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1509.06461v3","title":"Deep Reinforcement Learning with Double Q-learning","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Q-Learning Networks","url":"/methods/category/q-learning-networks","pwc_aliases":["q-learning"]}],"n_papers_tagged":45,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"slug":"dynamic-operating-system-scheduling-using","title":"Dynamic Operating System Scheduling Using Double DQN: A Reinforcement Learning Approach to Task Optimization","date":"2025-03-31","arxiv_id":"2503.23659","n_code_links":0,"syntology":null},{"paper":null,"slug":"distribution-free-uncertainty-quantification-2","title":"Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework","date":"2024-12-17","arxiv_id":"2412.12597","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-new-view-on-planning-in-online","title":"A New View on Planning in Online Reinforcement Learning","date":"2024-06-03","arxiv_id":"2406.01562","n_code_links":0,"syntology":null},{"paper":null,"slug":"advancing-algorithmic-trading-a-multi","title":"Advancing Algorithmic Trading: A Multi-Technique Enhancement of Deep Q-Network Models","date":"2023-11-09","arxiv_id":"2311.05743","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-for-the-heat","title":"Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets","date":"2023-09-25","arxiv_id":"2309.13955","n_code_links":0,"syntology":null},{"paper":null,"slug":"ai-driven-patient-monitoring-with-multi-agent","title":"Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions","date":"2023-09-20","arxiv_id":"2309.10980","n_code_links":0,"syntology":null},{"paper":null,"slug":"interpretable-and-secure-trajectory","title":"Interpretable and Secure Trajectory Optimization for UAV-Assisted Communication","date":"2023-07-05","arxiv_id":"2307.02002","n_code_links":0,"syntology":null},{"paper":null,"slug":"vanishing-bias-heuristic-guided-reinforcement","title":"Vanishing Bias Heuristic-guided Reinforcement Learning Algorithm","date":"2023-06-17","arxiv_id":"2306.10216","n_code_links":0,"syntology":null},{"paper":"/paper/extracting-diagnosis-pathways-from-electronic","slug":"extracting-diagnosis-pathways-from-electronic","title":"Extracting Diagnosis Pathways from Electronic Health Records Using Deep Reinforcement Learning","date":"2023-05-10","arxiv_id":"2305.06295","n_code_links":1,"syntology":null},{"paper":"/paper/train-a-real-world-local-path-planner-in-one","slug":"train-a-real-world-local-path-planner-in-one","title":"Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversity","date":"2023-05-07","arxiv_id":"2305.04180","n_code_links":1,"syntology":null},{"paper":null,"slug":"neighboring-state-based-rl-exploration","title":"Neighboring state-based RL Exploration","date":"2022-12-21","arxiv_id":"2212.10712","n_code_links":0,"syntology":null},{"paper":null,"slug":"elastic-step-dqn-a-novel-multi-step-algorithm","title":"Elastic Step DQN: A novel multi-step algorithm to alleviate overestimation in Deep QNetworks","date":"2022-10-07","arxiv_id":"2210.03325","n_code_links":0,"syntology":null},{"paper":"/paper/m-2-dqn-a-robust-method-for-accelerating-deep","slug":"m-2-dqn-a-robust-method-for-accelerating-deep","title":"M$^2$DQN: A Robust Method for Accelerating Deep Q-learning Network","date":"2022-09-16","arxiv_id":"2209.07809","n_code_links":1,"syntology":null},{"paper":"/paper/deep-reinforcement-learning-with-swin","slug":"deep-reinforcement-learning-with-swin","title":"Deep Reinforcement Learning with Swin Transformers","date":"2022-06-30","arxiv_id":"2206.15269","n_code_links":1,"syntology":null},{"paper":null,"slug":"goal-space-planning-with-subgoal-models","title":"Goal-Space Planning with Subgoal Models","date":"2022-06-06","arxiv_id":"2206.02902","n_code_links":0,"syntology":null},{"paper":null,"slug":"random-ensemble-reinforcement-learning-for","title":"Random Ensemble Reinforcement Learning for Traffic Signal Control","date":"2022-03-10","arxiv_id":"2203.05961","n_code_links":0,"syntology":null},{"paper":"/paper/skrl-modular-and-flexible-library-for","slug":"skrl-modular-and-flexible-library-for","title":"skrl: Modular and Flexible Library for Reinforcement Learning","date":"2022-02-08","arxiv_id":"2202.03825","n_code_links":1,"syntology":null},{"paper":null,"slug":"an-improved-reinforcement-learning-algorithm","title":"An Improved Reinforcement Learning Algorithm for Learning to Branch","date":"2022-01-17","arxiv_id":"2201.06213","n_code_links":0,"syntology":null},{"paper":"/paper/intelligent-traffic-light-via-policy-based","slug":"intelligent-traffic-light-via-policy-based","title":"Intelligent Traffic Light via Policy-based Deep Reinforcement Learning","date":"2021-12-27","arxiv_id":"2112.13817","n_code_links":1,"syntology":null},{"paper":null,"slug":"modified-double-dqn-addressing-stability","title":"Modified Double DQN: addressing stability","date":"2021-08-09","arxiv_id":"2108.04115","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-improved-algorithm-of-robot-path-planning","title":"An Improved Algorithm of Robot Path Planning in Complex Environment Based on Double DQN","date":"2021-07-23","arxiv_id":"2107.11245","n_code_links":0,"syntology":null},{"paper":"/paper/a-reinforcement-learning-environment-for-2","slug":"a-reinforcement-learning-environment-for-2","title":"A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis","date":"2021-07-15","arxiv_id":"2107.07373","n_code_links":1,"syntology":null},{"paper":null,"slug":"learning-to-optimize-industry-scale-dynamic","title":"Learning to Optimize Industry-Scale Dynamic Pickup and Delivery Problems","date":"2021-05-27","arxiv_id":"2105.12899","n_code_links":0,"syntology":null},{"paper":"/paper/independent-reinforcement-learning-for-weakly","slug":"independent-reinforcement-learning-for-weakly","title":"Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem","date":"2021-04-22","arxiv_id":"2104.10917","n_code_links":1,"syntology":null},{"paper":null,"slug":"greedy-multi-step-off-policy-reinforcement-1","title":"Greedy-Step Off-Policy Reinforcement Learning","date":"2021-02-23","arxiv_id":"2102.11717","n_code_links":0,"syntology":null},{"paper":"/paper/recurrent-rational-networks","slug":"recurrent-rational-networks","title":"Adaptive Rational Activations to Boost Deep Reinforcement Learning","date":"2021-02-18","arxiv_id":"2102.09407","n_code_links":4,"syntology":{"ran":6,"of":6,"n_ran_checked":0,"n_instrument":6,"unverified":0,"pointer_only":3,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified","official":{"repos":["ml-research/rational_activations","ml-research/rational_rl","ml-research/rational_sl","k4ntz/activation-functions"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"mobile-robots-autonomous-exploration-with","title":"Mobile Robots Autonomous Exploration with Reinforcement Learning","date":"2020-12-14","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"self-correcting-q-learning","title":"Self-correcting Q-Learning","date":"2020-12-02","arxiv_id":"2012.01100","n_code_links":0,"syntology":null},{"paper":null,"slug":"chrome-dino-run-using-reinforcement-learning","title":"Chrome Dino Run using Reinforcement Learning","date":"2020-08-15","arxiv_id":"2008.06799","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-with-quantum","slug":"reinforcement-learning-with-quantum","title":"Reinforcement Learning with Quantum Variational Circuits","date":"2020-08-15","arxiv_id":"2008.07524","n_code_links":2,"syntology":null},{"paper":null,"slug":"noise-overestimation-and-exploration-in-deep","title":"Some approaches used to overcome overestimation in Deep Reinforcement Learning algorithms","date":"2020-06-25","arxiv_id":"2006.14167","n_code_links":0,"syntology":null},{"paper":"/paper/learning-heuristic-selection-with-dynamic","slug":"learning-heuristic-selection-with-dynamic","title":"Learning Heuristic Selection with Dynamic Algorithm Configuration","date":"2020-06-15","arxiv_id":"2006.08246","n_code_links":1,"syntology":null},{"paper":null,"slug":"balancing-a-cartpole-system-with","title":"Balancing a CartPole System with Reinforcement Learning -- A Tutorial","date":"2020-06-08","arxiv_id":"2006.04938","n_code_links":0,"syntology":null},{"paper":"/paper/dynamic-algorithm-configuration-foundation-of","slug":"dynamic-algorithm-configuration-foundation-of","title":"Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework","date":"2020-06-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"disentangling-controllable-object-through","title":"Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning","date":"2020-02-21","arxiv_id":"2002.09136","n_code_links":0,"syntology":null},{"paper":null,"slug":"fast-reinforcement-learning-for-anti-jamming","title":"Fast Reinforcement Learning for Anti-jamming Communications","date":"2020-02-13","arxiv_id":"2002.05364","n_code_links":0,"syntology":null},{"paper":"/paper/slm-lab-a-comprehensive-benchmark-and-modular-1","slug":"slm-lab-a-comprehensive-benchmark-and-modular-1","title":"SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning","date":"2019-12-28","arxiv_id":"1912.12482","n_code_links":1,"syntology":{"ran":10,"of":13,"n_ran_checked":10,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","official":{"repos":["kengz/SLM-Lab"],"state":"official (archive's flag): 10 ran","n_ran":10,"n_constructed":0,"n_ran_no_instrument_failure":10,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"exploiting-the-potential-of-deep","title":"Exploiting the potential of deep reinforcement learning for classification tasks in high-dimensional and unstructured data","date":"2019-12-20","arxiv_id":"1912.09595","n_code_links":0,"syntology":null},{"paper":"/paper/task-oriented-language-grounding-for-language","slug":"task-oriented-language-grounding-for-language","title":"Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order","date":"2019-10-27","arxiv_id":"1910.12354","n_code_links":1,"syntology":null},{"paper":null,"slug":"reverse-experience-replay","title":"Reverse Experience Replay","date":"2019-10-19","arxiv_id":"1910.08780","n_code_links":0,"syntology":null},{"paper":null,"slug":"double-deep-q-learning-for-optimal-execution","title":"Double Deep Q-Learning for Optimal Execution","date":"2018-12-17","arxiv_id":"1812.06600","n_code_links":0,"syntology":null},{"paper":null,"slug":"weighted-double-deep-multiagent-reinforcement","title":"Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments","date":"2018-02-23","arxiv_id":"1802.08534","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-exploration-through-bayesian-deep-q","slug":"efficient-exploration-through-bayesian-deep-q","title":"Efficient Exploration through Bayesian Deep Q-Networks","date":"2018-02-13","arxiv_id":"1802.04412","n_code_links":1,"syntology":null},{"paper":null,"slug":"faster-deep-q-learning-using-neural-episodic","title":"Faster Deep Q-learning using Neural Episodic Control","date":"2018-01-06","arxiv_id":"1801.01968","n_code_links":0,"syntology":null},{"paper":"/paper/deep-reinforcement-learning-with-double-q","slug":"deep-reinforcement-learning-with-double-q","title":"Deep Reinforcement Learning with Double Q-learning","date":"2015-09-22","arxiv_id":"1509.06461","n_code_links":97,"syntology":{"ran":56,"of":106,"n_ran_checked":55,"n_instrument":1,"unverified":50,"pointer_only":57,"phrase":"56 ran (of which 38 constructed an object rather than computing a result; 55 with no instrument failure: 0 honoured, 0 violated, 55 with no contract checked; 1 where Syntology's instrument failed) · 50 unverified","official":null}}],"papers_shown":45,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":45,"with_a_code_link":16,"where_syntology_ran_a_sample":3,"not_listed_spam_title":0,"listed":45,"listed_where_code_ran":3,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":2,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":2,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":3,"papers_where_code_ran_first_page":"/method/double-dqn/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/double-dqn/papers/ran/1","json":"/method/double-dqn/papers/ran/1.json","rows":3}],"tasks":[{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":27},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":26},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":25},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":16},{"task":"/task/q-learning","name":"Q-Learning","papers":15},{"task":"/task/atari-games","name":"Atari Games","papers":8},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":5},{"task":"/task/decision-making","name":"Decision Making","papers":3},{"task":"/task/general-reinforcement-learning","name":"General Reinforcement Learning","papers":2},{"task":"/task/model-based-reinforcement-learning","name":"Model-based Reinforcement Learning","papers":2},{"task":"/task/scheduling","name":"Scheduling","papers":2},{"task":"/task/traffic-signal-control","name":"Traffic Signal Control","papers":2},{"task":"/task/algorithmic-trading","name":"Algorithmic Trading","papers":1},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":1},{"task":"/task/cloud-computing","name":"Cloud Computing","papers":1},{"task":"/task/collision-avoidance","name":"Collision Avoidance","papers":1},{"task":"/task/combinatorial-optimization","name":"Combinatorial Optimization","papers":1},{"task":"/task/conformal-prediction","name":"Conformal Prediction","papers":1},{"task":"/task/diversity","name":"Diversity","papers":1},{"task":"/task/efficient-exploration","name":"Efficient Exploration","papers":1}],"tasks_shown":20,"n_tasks":43,"usage_by_year":[{"year":"2015","papers":1},{"year":"2018","papers":4},{"year":"2019","papers":4},{"year":"2020","papers":10},{"year":"2021","papers":8},{"year":"2022","papers":8},{"year":"2023","papers":7},{"year":"2024","papers":2},{"year":"2025","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/double-dqn"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Double%20DQN","read_at":"2026-10-05T00:55:14+00:00","query":"Double DQN","kind":"method","have":"unknown_name","tier":null,"verified_implementation":null,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":null,"resolution":null,"resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":null,"archive_paper_arxiv_id":null,"repositories":null,"retrieve_with":null,"name_competition":{"verdict":"none","may_be_wrong_sense":false,"bearers_found":0,"served":null,"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing in this corpus is named 'Double DQN' -- no method node, no paper announcing it in its title, no expanded form declaring it as an acronym."},"name_competition_text":null,"near_matches":["Double Q-learning adaptation to DQN","automated framework for VITE circuit design using Double Deep-Q Networks (DDQN)","DOUBLE","Double-P","Spectral DQN","Double Ratchet","double-scoring","equivariant DQN"],"code_nearby":[],"code_nearby_caption":null,"origin_paper":null,"line":"name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"7a4bbdd62865490b4e899a07ae4c946681c5d6392f3bc8198b7b0689da055199","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}