Methods › Reinforcement Learning › Policy Gradient Methods › A3C
A3C
Introduced by Volodymyr Mnih et al. in Asynchronous Methods for Deep Reinforcement Learning
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
A3C, Asynchronous Advantage Actor Critic, is a policy gradient algorithm in reinforcement learning that maintains a policy π(aₜ|sₜ; θ) and an estimate of the value function V(sₜ; θᵥ). It operates in the forward view and uses a mix of n-step returns to update both the policy and the value-function. The policy and the value function are updated after every tₘₐₓ actions or when a terminal state is reached. The update performed by the algorithm can be seen as ∇_(θ’)logπ(aₜ|sₜ; θ’)A(sₜ, aₜ; θ, θᵥ) where A(sₜ, aₜ; θ, θᵥ) is an estimate of the advantage function given by:
∑ᵏ⁻¹ᵢ₌₀γⁱrₜ₊ᵢ + γᵏV(sₜ₊ₖ; θᵥ) - V(sₜ; θᵥ)
where k can vary from state to state and is upper-bounded by tₘₐₓ.
The critics in A3C learn the value function while multiple actors are trained in parallel and get synced with global parameters every so often. The gradients are accumulated as part of training for stability - this is like parallelized stochastic gradient descent.
Note that while the parameters θ of the policy and θᵥ of the value function are shown as being separate for generality, we always share some of the parameters in practice. We typically use a convolutional neural network that has one softmax output for the policy π(aₜ|sₜ; θ) and one linear output for the value function V(sₜ; θᵥ), with all non-output layers shared.
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 57 · with a code link: 20 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 57 tagged: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 57 of 57, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Detecting and Mitigating Reward Hacking in Reinforcement Learning Systems: A Comprehensive Empirical Study 8 Jul 2025 · 0 repositories · arXiv:2507.05619
-
Energy Efficient RSMA-Based LEO Satellite Communications Assisted by UAV-Mounted BD-Active RIS: A DRL Approach 7 May 2025 · 0 repositories · arXiv:2505.04148
-
Intelligent Task Scheduling for Microservices via A3C-Based Reinforcement Learning 1 May 2025 · 0 repositories · arXiv:2505.00299
-
Demand-Aware Beam Hopping and Power Allocation for Load Balancing in Digital Twin empowered LEO Satellite Networks 29 Oct 2024 · 0 repositories · arXiv:2411.08896
-
Survival of the Fittest: Evolutionary Adaptation of Policies for Environmental Shifts 22 Oct 2024 · 0 repositories · arXiv:2410.19852
-
Physical Informed-Inspired Deep Reinforcement Learning Based Bi-Level Programming for Microgrid Scheduling 15 Oct 2024 · 0 repositories · arXiv:2410.11932
-
Criticality and Safety Margins for Reinforcement Learning 26 Sep 2024 · 0 repositories · arXiv:2409.18289
-
Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN 22 Jul 2024 · 0 repositories · arXiv:2407.15656
-
A Deep Reinforcement Learning Approach for Trading Optimization in the Forex Market with Multi-Agent Asynchronous Distribution 30 May 2024 · 0 repositories · arXiv:2405.19982
-
Sum Throughput Maximization in Multi-BD Symbiotic Radio NOMA Network Assisted by Active-STAR-RIS 16 Jan 2024 · 0 repositories · arXiv:2401.08301
-
Learning Actions and Control of Focus of Attention with a Log-Polar-like Sensor 22 Sep 2023 · 0 repositories · arXiv:2309.12634
-
Safety Margins for Reinforcement Learning 25 Jul 2023 · 0 repositories · arXiv:2307.13642
-
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages 2 Jun 2023 · 1 repository · arXiv:2306.01460
-
Double A3C: Deep Reinforcement Learning on OpenAI Gym Games 4 Mar 2023 · 0 repositories · arXiv:2303.02271
-
Reinforcement Learning for Molecular Dynamics Optimization: A Stochastic Pontryagin Maximum Principle Approach 6 Dec 2022 · 1 repository · arXiv:2212.03320
-
Point Cloud Scene Completion with Joint Color and Semantic Estimation from Single RGB-D Image 12 Oct 2022 · 0 repositories · arXiv:2210.05891
-
Comparing Deep Reinforcement Learning Algorithms in Two-Echelon Supply Chains 20 Apr 2022 · 1 repository · arXiv:2204.09603
-
RL-CoSeg : A Novel Image Co-Segmentation Algorithm with Deep Reinforcement Learning 12 Apr 2022 · 0 repositories · arXiv:2204.05951
-
Learning Reward Machines: A Study in Partially Observable Reinforcement Learning 17 Dec 2021 · 0 repositories · arXiv:2112.09477
-
Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning 6 Mar 2021 · 0 repositories · arXiv:2103.04067
-
A review of motion planning algorithms for intelligent robotics 4 Feb 2021 · 0 repositories · arXiv:2102.02376
-
Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup 31 Dec 2020 · 0 repositories · arXiv:2012.15511
-
Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments using A3C learning and Residual Recurrent Neural Networks 1 Sep 2020 · 1 repository · arXiv:2009.02186
-
Lagrangian Duality in Reinforcement Learning 20 Jul 2020 · 0 repositories · arXiv:2007.09998
-
PFPN: Continuous Control of Physically Simulated Characters using Particle Filtering Policy Network 16 Mar 2020 · 1 repository · arXiv:2003.06959
-
Explore and Exploit with Heterotic Line Bundle Models 10 Mar 2020 · 1 repository · arXiv:2003.04817
-
Fully Asynchronous Policy Evaluation in Distributed Reinforcement Learning over Networks 1 Mar 2020 · 0 repositories · arXiv:2003.00433
-
A Visual Communication Map for Multi-Agent Deep Reinforcement Learning 27 Feb 2020 · 0 repositories · arXiv:2002.11882
-
Intelligent Roundabout Insertion using Deep Reinforcement Learning 3 Jan 2020 · 0 repositories · arXiv:2001.00786
-
Intelligent Coordination among Multiple Traffic Intersections Using Multi-Agent Reinforcement Learning 9 Dec 2019 · 0 repositories · arXiv:1912.03851
-
Adversary A3C for Robust Reinforcement Learning 1 Dec 2019 · 0 repositories · arXiv:1912.00330
-
Learning Reward Machines for Partially Observable Reinforcement Learning 1 Dec 2019 · 1 repository
-
VUSFA:Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation 18 Aug 2019 · 2 repositories · arXiv:1908.06376
-
Incremental Reinforcement Learning --- a New Continuous Reinforcement Learning Frame Based on Stochastic Differential Equation methods 8 Aug 2019 · 0 repositories · arXiv:1908.02974
-
Terminal Prediction as an Auxiliary Task for Deep Reinforcement Learning 24 Jul 2019 · 0 repositories · arXiv:1907.10827
-
Agent Modeling as Auxiliary Task for Deep Reinforcement Learning 22 Jul 2019 · 0 repositories · arXiv:1907.09597
-
Jointly Pre-training with Supervised, Autoencoder, and Value Losses for Deep Reinforcement Learning 3 Apr 2019 · 1 repository · arXiv:1904.02206
-
Unsupervised Traffic Accident Detection in First-Person Videos 2 Mar 2019 · 2 repositories · arXiv:1903.00618Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
Combinational Q-Learning for Dou Di Zhu 24 Jan 2019 · 1 repository · arXiv:1901.08925
-
Using Monte Carlo Tree Search as a Demonstrator within Asynchronous Deep RL 30 Nov 2018 · 0 repositories · arXiv:1812.00045
-
Single-Agent Policy Tree Search With Guarantees 27 Nov 2018 · 1 repository · arXiv:1811.10928
-
Gradient Band-based Adversarial Training for Generalized Attack Immunity of A3C Path Finding 18 Jul 2018 · 0 repositories · arXiv:1807.06752
-
Crawling in Rogue's dungeons with (partitioned) A3C 23 Apr 2018 · 1 repository · arXiv:1804.08685
-
A Brandom-ian view of Reinforcement Learning towards strong-AI 7 Mar 2018 · 0 repositories · arXiv:1803.02912
-
Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning 18 Dec 2017 · 12 repositories · arXiv:1712.06567Syntology 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 4 pointer-only (licence)
-
Natural Value Approximators: Learning when to Trust Past Estimates 1 Dec 2017 · 0 repositories
-
Teaching a Machine to Read Maps with Deep Reinforcement Learning 20 Nov 2017 · 1 repository · arXiv:1711.07479
-
Improving Search through A3C Reinforcement Learning based Conversational Agent 17 Sep 2017 · 0 repositories · arXiv:1709.05638
-
DARLA: Improving Zero-Shot Transfer in Reinforcement Learning 26 Jul 2017 · 1 repository · arXiv:1707.08475
-
Noisy Networks for Exploration 30 Jun 2017 · 15 repositories · arXiv:1706.10295Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
Learning to Factor Policies and Action-Value Functions: Factored Action Space Representations for Deep Reinforcement learning 20 May 2017 · 0 repositories · arXiv:1705.07269
-
Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning 18 May 2017 · 1 repository · arXiv:1705.06769
-
Equivalence Between Policy Gradients and Soft Q-Learning 21 Apr 2017 · 0 repositories · arXiv:1704.06440
-
The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning 15 Apr 2017 · 0 repositories · arXiv:1704.04651
-
Tactics of Adversarial Attack on Deep Reinforcement Learning Agents 8 Mar 2017 · 0 repositories · arXiv:1703.06748
-
Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU 18 Nov 2016 · 3 repositories · arXiv:1611.06256
-
Asynchronous Methods for Deep Reinforcement Learning 4 Feb 2016 · 70 repositories · arXiv:1602.01783Syntology 60 ran (of which 20 constructed an object rather than computing a result; 50 with no instrument failure: 2 honoured, 1 violated, 47 with no contract checked; 10 where Syntology's instrument failed) · 35 unverified (of 95 harvested samples) · 20 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 61 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections