Methods › Reinforcement Learning › Eligibility Traces › Eligibility Trace
Eligibility Trace
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
An Eligibility Trace is a memory vector zₜ ∈ℝᵈ that parallels the long-term weight vector wₜ ∈ℝᵈ. The idea is that when a component of wₜ participates in producing an estimated value, the corresponding component of zₜ is bumped up and then begins to fade away. Learning will then occur in that component of wₜ if a nonzero TD error occurs before the trade falls back to zero. The trace-decay parameter λ∈[0, 1] determines the rate at which the trace falls.
Intuitively, they tackle the credit assignment problem by capturing both a frequency heuristic - states that are visited more often deserve more credit - and a recency heuristic - states that are visited more recently deserve more credit.
E₀(s) = 0 Eₜ(s) = γλEₜ₋₁(s) + 1(Sₜ = s)
Source: Sutton and Barto, Reinforcement Learning, 2nd Edition
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: Selective Eligibility Traces (S-trace); TRACE; TRACE; TRACE; TRACE; TRACE; TRACE; Trace.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 11 · with a code link: 2 · where Syntology ran a sample: 1 (1 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (1 of 11 tagged: 1 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 11 of 11, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Noise-based reward-modulated learning 31 Mar 2025 · 0 repositories · arXiv:2503.23972
-
Predecessor Features 1 Jun 2022 · 0 repositories · arXiv:2206.00303
-
META-Learning Eligibility Traces for More Sample Efficient Temporal Difference Learning 16 Jun 2020 · 1 repository · arXiv:2006.08906
-
Efficient Use of heuristics for accelerating XCS-based Policy Learning in Markov Games 26 May 2020 · 0 repositories · arXiv:2005.12553
-
Gradient Q(σ, λ): A Unified Algorithm with Function Approximation for Reinforcement Learning 6 Sep 2019 · 0 repositories · arXiv:1909.02877
-
Gap-Increasing Policy Evaluation for Efficient and Noise-Tolerant Reinforcement Learning 18 Jun 2019 · 0 repositories · arXiv:1906.07586
-
META-Learning State-based Eligibility Traces for More Sample-Efficient Policy Evaluation 25 Apr 2019 · 2 repositories · arXiv:1904.11439Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 11 unverified (of 14 harvested samples)
-
Metatrace Actor-Critic: Online Step-size Tuning by Meta-gradient Descent for Reinforcement Learning Control 10 May 2018 · 0 repositories · arXiv:1805.04514
-
A Unified Approach for Multi-step Temporal-Difference Learning with Eligibility Traces in Reinforcement Learning 9 Feb 2018 · 0 repositories · arXiv:1802.03171
-
A forward model at Purkinje cell synapses facilitates cerebellar anticipatory control 1 Dec 2016 · 0 repositories
-
Q(λ) with Off-Policy Corrections 16 Feb 2016 · 0 repositories · arXiv:1602.04951
Tasks archive 2025-07-28
7 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Reinforcement Learning | 6 |
| Reinforcement Learning (RL) | 6 |
| Meta-Learning | 3 |
| reinforcement-learning | 2 |
| Atari Games | 1 |
| Q-Learning | 1 |
| counterfactual | 1 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections