Methods › Reinforcement Learning › Policy Gradient Methods › TD3
Twin Delayed Deep Deterministic
TD3
Introduced by Scott Fujimoto et al. in Addressing Function Approximation Error in Actor-Critic Methods
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
TD3 builds on the DDPG algorithm for reinforcement learning, with a couple of modifications aimed at tackling overestimation bias with the value function. In particular, it utilises clipped double Q-learning, delayed update of target and policy networks, and target policy smoothing (which is similar to a SARSA based update; a safer update, as they provide higher value to actions resistant to perturbations).
In Syntology about another paper, arXiv:2502.02854: repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: official). The lookup matched a paper by its title, arXiv:2502.02854, not a method entry. About another paper: this answer is about arXiv:2502.02854, while the archive credits arXiv:1802.09477.
Origin paper of this answer, arXiv:2502.02854 (its page on this site · its page on Syntology).
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 117 · with a code link: 46 · where Syntology ran a sample: 18 (17 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (18 of 117 tagged: 17 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 117, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning 6 Jun 2025 · 1 repository · arXiv:2506.05968Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples)
-
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control 28 May 2025 · 0 repositories · arXiv:2505.22642
-
LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models 21 May 2025 · 0 repositories · arXiv:2505.15293
-
Monte Carlo Beam Search for Actor-Critic Reinforcement Learning in Continuous Control 13 May 2025 · 0 repositories · arXiv:2505.09029
-
Energy Efficient RSMA-Based LEO Satellite Communications Assisted by UAV-Mounted BD-Active RIS: A DRL Approach 7 May 2025 · 0 repositories · arXiv:2505.04148
-
AlphaGrad: Non-Linear Gradient Normalization Optimizer 22 Apr 2025 · 0 repositories · arXiv:2504.16020
-
Motion Control in Multi-Rotor Aerial Robots Using Deep Reinforcement Learning 9 Feb 2025 · 0 repositories · arXiv:2502.05996
-
TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation 5 Feb 2025 · 1 repository · arXiv:2502.02854
-
EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning 25 Jan 2025 · 2 repositories · arXiv:2501.15129
-
Enhancing UAV Path Planning Efficiency Through Accelerated Learning 17 Jan 2025 · 0 repositories · arXiv:2501.10141
-
An Advantage-based Optimization Method for Reinforcement Learning in Large Action Space 17 Dec 2024 · 0 repositories · arXiv:2412.12605
-
Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning 20 Nov 2024 · 0 repositories · arXiv:2411.13116
-
Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers 31 Oct 2024 · 1 repository · arXiv:2410.24108Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
NetworkGym: Reinforcement Learning Environments for Multi-Access Traffic Management in Network Simulation 30 Oct 2024 · 1 repository · arXiv:2411.04138Syntology official (archive's flag): 6 ran · 15 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 8 where Syntology's instrument failed) · 3 unverified (of 18 harvested samples) · 1 pointer-only (licence)
-
Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution 29 Oct 2024 · 1 repository · arXiv:2410.21940
-
Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions 15 Oct 2024 · 0 repositories · arXiv:2410.11833
-
Navigation in a simplified Urban Flow through Deep Reinforcement Learning 26 Sep 2024 · 0 repositories · arXiv:2409.17922
-
Simultaneous Training of First- and Second-Order Optimizers in Population-Based Reinforcement Learning 27 Aug 2024 · 0 repositories · arXiv:2408.15421
-
Optimizing TD3 for 7-DOF Robotic Arm Grasping: Overcoming Suboptimality with Exploration-Enhanced Contrastive Learning 26 Aug 2024 · 0 repositories · arXiv:2408.14009
-
Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks 31 Jul 2024 · 0 repositories · arXiv:2407.21338
-
Deep Reinforcement Learning Strategies in Finance: Insights into Asset Holding, Trading Behavior, and Purchase Diversity 29 Jun 2024 · 0 repositories · arXiv:2407.09557
-
Performance Comparison of Deep RL Algorithms for Mixed Traffic Cooperative Lane-Changing 25 Jun 2024 · 0 repositories · arXiv:2407.02521
-
Value Improved Actor Critic Algorithms 3 Jun 2024 · 0 repositories · arXiv:2406.01423
-
Chaos-based reinforcement learning with TD3 15 May 2024 · 0 repositories · arXiv:2405.09086
-
Offline Reinforcement Learning with Behavioral Supervisor Tuning 25 Apr 2024 · 0 repositories · arXiv:2404.16399
-
RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Models 11 Mar 2024 · 0 repositories · arXiv:2403.06420
-
Conservative DDPG -- Pessimistic RL without Ensemble 8 Mar 2024 · 0 repositories · arXiv:2403.05732
-
Fill-and-Spill: Deep Reinforcement Learning Policy Gradient Methods for Reservoir Operation Decision and Control 7 Mar 2024 · 0 repositories · arXiv:2403.04195
-
Snapshot Reinforcement Learning: Leveraging Prior Trajectories for Efficiency 1 Mar 2024 · 1 repository · arXiv:2403.00673
-
Exploiting Estimation Bias in Clipped Double Q-Learning for Continous Control Reinforcement Learning Tasks 14 Feb 2024 · 0 repositories · arXiv:2402.09078
-
MinMaxMin Q-learning 3 Feb 2024 · 0 repositories · arXiv:2402.05951
-
SQT -- std Q-target 3 Feb 2024 · 0 repositories · arXiv:2402.05950
-
Neural Style Transfer with Twin-Delayed DDPG for Shared Control of Robotic Manipulators 1 Feb 2024 · 0 repositories · arXiv:2402.00722
-
Reconciling Spatial and Temporal Abstractions for Goal Representation 18 Jan 2024 · 1 repository · arXiv:2401.09870Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 6 unverified (of 11 harvested samples) · 11 pointer-only (licence)
-
Sum Throughput Maximization in Multi-BD Symbiotic Radio NOMA Network Assisted by Active-STAR-RIS 16 Jan 2024 · 0 repositories · arXiv:2401.08301
-
An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control 11 Jan 2024 · 1 repository · arXiv:2401.05737
-
Adaptive Kalman-based hybrid car following strategy using TD3 and CACC 26 Dec 2023 · 0 repositories · arXiv:2312.15993
-
Multiagent Copilot Approach for Shared Autonomy between Human EEG and TD3 Deep Reinforcement Learning 22 Dec 2023 · 0 repositories · arXiv:2312.14458
-
6G Fresnel Spot Beamfocusing using Large-Scale Metasurfaces: A Distributed DRL-Based Approach 18 Nov 2023 · 0 repositories · arXiv:2311.11109
-
Mitigating Estimation Errors by Twin TD-Regularized Actor and Critic for Deep Reinforcement Learning 7 Nov 2023 · 0 repositories · arXiv:2311.03711
-
On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economics 30 Oct 2023 · 0 repositories · arXiv:2310.19527
-
Hierarchical Framework for Interpretable and Probabilistic Model-Based Safe Reinforcement Learning 28 Oct 2023 · 1 repository · arXiv:2310.18811
-
Distributional Soft Actor-Critic with Three Refinements 9 Oct 2023 · 2 repositories · arXiv:2310.05858Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
A Long N-step Surrogate Stage Reward for Deep Reinforcement Learning 21 Sep 2023 · 0 repositories
-
Double Gumbel Q-Learning 21 Sep 2023 · 1 repository
-
Taylor TD-learning 21 Sep 2023 · 0 repositories
-
Caching-at-STARS: the Next Generation Edge Caching 1 Aug 2023 · 0 repositories · arXiv:2308.00562
-
Evolutionary Strategy Guided Reinforcement Learning via MultiBuffer Communication 20 Jun 2023 · 0 repositories · arXiv:2306.11535
-
RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control 6 Jun 2023 · 1 repository · arXiv:2306.03530
-
For SALE: State-Action Representation Learning for Deep Reinforcement Learning 4 Jun 2023 · 2 repositories · arXiv:2306.02451Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 1 pointer-only (licence)
-
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages 2 Jun 2023 · 1 repository · arXiv:2306.01460
-
Optimizing Attention and Cognitive Control Costs Using Temporally-Layered Architectures 30 May 2023 · 1 repository · arXiv:2305.18701
-
Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization 31 Mar 2023 · 0 repositories · arXiv:2304.00026
-
A Strategy-Oriented Bayesian Soft Actor-Critic Model 7 Mar 2023 · 0 repositories · arXiv:2303.04193
-
Swim: A General-Purpose, High-Performing, and Efficient Activation Function for Locomotion Control Tasks 5 Mar 2023 · 1 repository · arXiv:2303.02640
-
The Ladder in Chaos: A Simple and Effective Improvement to General DRL Algorithms by Policy Path Trimming and Boosting 2 Mar 2023 · 0 repositories · arXiv:2303.01391
-
Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning 13 Feb 2023 · 1 repository · arXiv:2302.06548Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 1 pointer-only (licence)
-
RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework 10 Feb 2023 · 0 repositories · arXiv:2302.04994
-
Automated deep reinforcement learning for real-time scheduling strategy of multi-energy system integrated with post-carbon and direct-air carbon captured system 18 Jan 2023 · 0 repositories · arXiv:2301.07768
-
Actor-Director-Critic: A Novel Deep Reinforcement Learning Framework 10 Jan 2023 · 0 repositories · arXiv:2301.03887
-
Extreme Q-Learning: MaxEnt RL without Entropy 5 Jan 2023 · 4 repositories · arXiv:2301.02328Syntology 8 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified (of 13 harvested samples) · 7 pointer-only (licence)
-
Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks 11 Dec 2022 · 0 repositories · arXiv:2212.05572
-
TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets 5 Dec 2022 · 1 repository · arXiv:2212.02125
-
Algorithmic Trading Using Continuous Action Space Deep Reinforcement Learning 7 Oct 2022 · 0 repositories · arXiv:2210.03469
-
A Novel Entropy-Maximizing TD3-based Reinforcement Learning for Automatic PID Tuning 5 Oct 2022 · 0 repositories · arXiv:2210.02381
-
Experimental Study on The Effect of Multi-step Deep Reinforcement Learning in POMDPs 12 Sep 2022 · 1 repository · arXiv:2209.04999
-
Bayesian Soft Actor-Critic: A Directed Acyclic Strategy Graph Based Deep Reinforcement Learning 11 Aug 2022 · 2 repositories · arXiv:2208.06033
-
Performance Comparison of Deep RL Algorithms for Energy Systems Optimal Scheduling 1 Aug 2022 · 1 repository · arXiv:2208.00728
-
Multi-Agent Deep Reinforcement Learning-Driven Mitigation of Adverse Effects of Cyber-Attacks on Electric Vehicle Charging Station 14 Jul 2022 · 0 repositories · arXiv:2207.07041
-
Equivariant Reinforcement Learning for Quadrotor UAV 2 Jun 2022 · 0 repositories · arXiv:2206.01233
-
Non-Parametric Stochastic Policy Gradient with Strategic Retreat for Non-Stationary Environment 24 Mar 2022 · 0 repositories · arXiv:2203.14905
-
Adaptively Calibrated Critic Estimates for Deep Reinforcement Learning 24 Nov 2021 · 1 repository · arXiv:2111.12673Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Improving Learning from Demonstrations by Learning from Experience 16 Nov 2021 · 0 repositories · arXiv:2111.08156
-
Recurrent Off-policy Baselines for Memory-based Continuous Control 25 Oct 2021 · 1 repository · arXiv:2110.12628
-
Parallel Actors and Learners: A Framework for Generating Scalable RL Implementations 3 Oct 2021 · 0 repositories · arXiv:2110.01101
-
Explanation-Aware Experience Replay in Rule-Dense Environments 29 Sep 2021 · 1 repository · arXiv:2109.14711
-
Faster Reinforcement Learning with Value Target Lower Bounding 29 Sep 2021 · 0 repositories
-
Meta Attention For Off-Policy Actor-Critic 29 Sep 2021 · 0 repositories
-
Robust and Data-efficient Q-learning by Composite Value-estimation 29 Sep 2021 · 0 repositories
-
SPP-RL: State Planning Policy Reinforcement Learning 29 Sep 2021 · 0 repositories
-
ADER:Adapting between Exploration and Robustness for Actor-Critic Methods 8 Sep 2021 · 0 repositories · arXiv:2109.03443
-
WAD: A Deep Reinforcement Learning Agent for Urban Autonomous Driving 27 Aug 2021 · 0 repositories · arXiv:2108.12134
-
Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study 5 Aug 2021 · 0 repositories · arXiv:2108.02665
-
Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings 28 Jul 2021 · 0 repositories · arXiv:2107.13356
-
Efficient Continuous Control with Double Actors and Regularized Critics 6 Jun 2021 · 1 repository · arXiv:2106.03050
-
Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competition 9 Mar 2021 · 1 repository · arXiv:2103.05174
-
Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control 25 Feb 2021 · 0 repositories · arXiv:2102.13012
-
Memory-based Deep Reinforcement Learning for POMDPs 24 Feb 2021 · 1 repository · arXiv:2102.12344Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
OffCon³: What is state of the art anyway? 27 Jan 2021 · 1 repository · arXiv:2101.11331
-
GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning 24 Jan 2021 · 0 repositories · arXiv:2101.09650
-
Learning Synthetic Environments for Reinforcement Learning with Evolution Strategies 24 Jan 2021 · 1 repository · arXiv:2101.09721
-
PGPS : Coupling Policy Gradient with Population-based Search 1 Jan 2021 · 0 repositories
-
Policy Gradient RL Algorithms as Directed Acyclic Graphs 14 Dec 2020 · 1 repository · arXiv:2012.07763
-
Ranking Items in Large-Scale Item Search Engines with Reinforcement Learning 14 Dec 2020 · 0 repositories
-
Virtual Autonomous Driving with Reinforcement Learning 14 Dec 2020 · 0 repositories
-
OPAC: Opportunistic Actor-Critic 11 Dec 2020 · 0 repositories · arXiv:2012.06555
-
Efficient Reservoir Management through Deep Reinforcement Learning 7 Dec 2020 · 0 repositories · arXiv:2012.03822
-
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance 19 Nov 2020 · 6 repositories · arXiv:2011.09607
-
Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking 15 Nov 2020 · 1 repository · arXiv:2011.07537Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)
-
RealAnt: An Open-Source Low-Cost Quadruped for Education and Research in Real-World Reinforcement Learning 5 Nov 2020 · 1 repository · arXiv:2011.03085
Tasks archive 2025-07-28
20 shown of 69 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections