{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/experience-replay","slug":"experience-replay","name":"Experience Replay","full_name":"Experience Replay","full_name_withheld":false,"description_markdown":"**Experience Replay** is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\\_{t} = \\left(s\\_{t}, a\\_{t}, r\\_{t}, s\\_{t+1}\\right)$ in a data-set $D = e\\_{1}, \\cdots, e\\_{N}$ , pooled over many episodes into a replay memory. We then usually sample the memory randomly for a minibatch of experience, and use this to learn off-policy, as with Deep Q-Networks. This tackles the problem of autocorrelation leading to unstable training, by making the problem more like a supervised learning problem.\r\n\r\nImage Credit: [Hands-On Reinforcement Learning with Python, Sudharsan Ravichandiran](https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781788836524)","description_state":"present","introduced_year":1993,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Replay Memory","url":"/methods/category/replay-memory","pwc_aliases":[]}],"n_papers_tagged":865,"archive_num_papers":865,"papers_newest_first":[{"paper":null,"slug":"turning-sand-to-gold-recycling-data-to-bridge","title":"Turning Sand to Gold: Recycling Data to Bridge On-Policy and Off-Policy Learning via Causal Bound","date":"2025-07-15","arxiv_id":"2507.11269","n_code_links":0,"syntology":null},{"paper":"/paper/deep-reinforcement-learning-with-gradient-1","slug":"deep-reinforcement-learning-with-gradient-1","title":"Deep Reinforcement Learning with Gradient Eligibility Traces","date":"2025-07-12","arxiv_id":"2507.09087","n_code_links":1,"syntology":null},{"paper":null,"slug":"multi-objective-reinforcement-learning-for-6","title":"Multi-Objective Reinforcement Learning for Cognitive Radar Resource Management","date":"2025-06-25","arxiv_id":"2506.20853","n_code_links":0,"syntology":null},{"paper":"/paper/cawr-corruption-averse-advantage-weighted","slug":"cawr-corruption-averse-advantage-weighted","title":"CAWR: Corruption-Averse Advantage-Weighted Regression for Robust Policy Optimization","date":"2025-06-18","arxiv_id":"2506.15654","n_code_links":1,"syntology":null},{"paper":null,"slug":"reliable-critics-monotonic-improvement-and","title":"Reliable Critics: Monotonic Improvement and Convergence Guarantees for Reinforcement Learning","date":"2025-06-08","arxiv_id":"2506.07134","n_code_links":0,"syntology":null},{"paper":null,"slug":"contextual-experience-replay-for-self","title":"Contextual Experience Replay for Self-Improvement of Language Agents","date":"2025-06-07","arxiv_id":"2506.06698","n_code_links":0,"syntology":null},{"paper":"/paper/gradual-transition-from-bellman-optimality","slug":"gradual-transition-from-bellman-optimality","title":"Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning","date":"2025-06-06","arxiv_id":"2506.05968","n_code_links":1,"syntology":{"ran":1,"of":4,"n_ran_checked":1,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","official":{"repos":["motokiomura/annealed-q-learning"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"a-novel-deep-reinforcement-learning-method","title":"A Novel Deep Reinforcement Learning Method for Computation Offloading in Multi-User Mobile Edge Computing with Decentralization","date":"2025-06-03","arxiv_id":"2506.02458","n_code_links":0,"syntology":null},{"paper":null,"slug":"fasttd3-simple-fast-and-capable-reinforcement","title":"FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control","date":"2025-05-28","arxiv_id":"2505.22642","n_code_links":0,"syntology":null},{"paper":null,"slug":"analyzing-mitigation-strategies-for","title":"Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models","date":"2025-05-23","arxiv_id":"2505.17496","n_code_links":0,"syntology":null},{"paper":null,"slug":"llm-explorer-a-plug-in-reinforcement-learning","title":"LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models","date":"2025-05-21","arxiv_id":"2505.15293","n_code_links":0,"syntology":null},{"paper":"/paper/lifelongagentbench-evaluating-llm-agents-as","slug":"lifelongagentbench-evaluating-llm-agents-as","title":"LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners","date":"2025-05-17","arxiv_id":"2505.11942","n_code_links":1,"syntology":null},{"paper":null,"slug":"unveiling-the-black-box-a-multi-layer","title":"Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agents","date":"2025-05-16","arxiv_id":"2505.11708","n_code_links":0,"syntology":null},{"paper":null,"slug":"electric-bus-charging-schedules-relying-on","title":"Electric Bus Charging Schedules Relying on Real Data-Driven Targets Based on Hierarchical Deep Reinforcement Learning","date":"2025-05-15","arxiv_id":"2505.10262","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-based-4","title":"Deep reinforcement learning-based longitudinal control strategy for automated vehicles at signalised intersections","date":"2025-05-13","arxiv_id":"2505.08896","n_code_links":0,"syntology":null},{"paper":null,"slug":"gradmix-gradient-based-selective-mixup-for","title":"GradMix: Gradient-based Selective Mixup for Robust Data Augmentation in Class-Incremental Learning","date":"2025-05-13","arxiv_id":"2505.08528","n_code_links":0,"syntology":null},{"paper":null,"slug":"monte-carlo-beam-search-for-actor-critic","title":"Monte Carlo Beam Search for Actor-Critic Reinforcement Learning in Continuous Control","date":"2025-05-13","arxiv_id":"2505.09029","n_code_links":0,"syntology":null},{"paper":"/paper/online-learning-based-adaptive-beam-switching","slug":"online-learning-based-adaptive-beam-switching","title":"Online Learning-based Adaptive Beam Switching for 6G Networks: Enhancing Efficiency and Resilience","date":"2025-05-12","arxiv_id":"2505.08032","n_code_links":1,"syntology":null},{"paper":null,"slug":"energy-efficient-rsma-based-leo-satellite","title":"Energy Efficient RSMA-Based LEO Satellite Communications Assisted by UAV-Mounted BD-Active RIS: A DRL Approach","date":"2025-05-07","arxiv_id":"2505.04148","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-goal-oriented-reinforcement-learning-based","title":"A Goal-Oriented Reinforcement Learning-Based Path Planning Algorithm for Modular Self-Reconfigurable Satellites","date":"2025-05-04","arxiv_id":"2505.01966","n_code_links":0,"syntology":null},{"paper":null,"slug":"approximation-to-deep-q-network-by-stochastic","title":"Approximation to Deep Q-Network by Stochastic Delay Differential Equations","date":"2025-05-01","arxiv_id":"2505.00382","n_code_links":0,"syntology":null},{"paper":null,"slug":"alphagrad-non-linear-gradient-normalization","title":"AlphaGrad: Non-Linear Gradient Normalization Optimizer","date":"2025-04-22","arxiv_id":"2504.16020","n_code_links":0,"syntology":null},{"paper":null,"slug":"next-future-sample-efficient-policy-learning","title":"Next-Future: Sample-Efficient Policy Learning for Robotic-Arm Tasks","date":"2025-04-15","arxiv_id":"2504.11247","n_code_links":0,"syntology":null},{"paper":null,"slug":"moderate-actor-critic-methods-controlling","title":"Moderate Actor-Critic Methods: Controlling Overestimation Bias via Expectile Loss","date":"2025-04-14","arxiv_id":"2504.09929","n_code_links":0,"syntology":null},{"paper":null,"slug":"graph-based-deep-reinforcement-learning-aided","title":"Graph Based Deep Reinforcement Learning Aided by Transformers for Multi-Agent Cooperation","date":"2025-04-11","arxiv_id":"2504.08195","n_code_links":0,"syntology":null},{"paper":"/paper/proxy-anchor-and-evt-driven-continual","slug":"proxy-anchor-and-evt-driven-continual","title":"Proxy-Anchor and EVT-Driven Continual Learning Method for Generalized Category Discovery","date":"2025-04-11","arxiv_id":"2504.08550","n_code_links":1,"syntology":null},{"paper":null,"slug":"neuron-level-balance-between-stability-and","title":"Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning","date":"2025-04-09","arxiv_id":"2504.08000","n_code_links":0,"syntology":null},{"paper":null,"slug":"think-small-act-big-primitive-prompt-learning","title":"Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulation","date":"2025-04-01","arxiv_id":"2504.00420","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamic-operating-system-scheduling-using","title":"Dynamic Operating System Scheduling Using Double DQN: A Reinforcement Learning Approach to Task Optimization","date":"2025-03-31","arxiv_id":"2503.23659","n_code_links":0,"syntology":null},{"paper":"/paper/pareto-continual-learning-preference","slug":"pareto-continual-learning-preference","title":"Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off","date":"2025-03-30","arxiv_id":"2503.23390","n_code_links":1,"syntology":null},{"paper":"/paper/world-model-agents-with-change-based","slug":"world-model-agents-with-change-based","title":"World Model Agents with Change-Based Intrinsic Motivation","date":"2025-03-26","arxiv_id":"2503.21047","n_code_links":1,"syntology":null},{"paper":null,"slug":"experience-replay-addresses-loss-of","title":"Experience Replay Addresses Loss of Plasticity in Continual Learning","date":"2025-03-25","arxiv_id":"2503.20018","n_code_links":0,"syntology":null},{"paper":"/paper/trajectory-balance-with-asynchrony-decoupling","slug":"trajectory-balance-with-asynchrony-decoupling","title":"Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training","date":"2025-03-24","arxiv_id":"2503.18929","n_code_links":1,"syntology":{"ran":7,"of":10,"n_ran_checked":7,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","official":null}},{"paper":null,"slug":"h2-marl-multi-agent-reinforcement-learning","title":"H2-MARL: Multi-Agent Reinforcement Learning for Pareto Optimality in Hospital Capacity Strain and Human Mobility during Epidemic","date":"2025-03-13","arxiv_id":"2503.10907","n_code_links":0,"syntology":null},{"paper":null,"slug":"intelligent-joint-security-and-delay","title":"Intelligent Joint Security and Delay Determinacy Performance Guarantee Strategy in RIS-Assisted IIoT Communication Systems","date":"2025-03-11","arxiv_id":"2503.08086","n_code_links":0,"syntology":null},{"paper":null,"slug":"per-dpp-sampling-framework-and-its","title":"PER-DPP Sampling Framework and Its Application in Path Planning","date":"2025-03-10","arxiv_id":"2503.07411","n_code_links":0,"syntology":null},{"paper":null,"slug":"fully-decentralized-maddpg-with-networked","title":"Fully-Decentralized MADDPG with Networked Agents","date":"2025-03-09","arxiv_id":"2503.06747","n_code_links":0,"syntology":null},{"paper":null,"slug":"closing-the-intent-to-reality-gap-via","title":"Closing the Intent-to-Behavior Gap via Fulfillment Priority Logic","date":"2025-03-04","arxiv_id":"2503.05818","n_code_links":0,"syntology":null},{"paper":null,"slug":"il-soar-imitation-learning-with-soft","title":"IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic","date":"2025-02-27","arxiv_id":"2502.19859","n_code_links":0,"syntology":null},{"paper":null,"slug":"pvbf-a-framework-for-mitigating-parameter","title":"PVBF: A Framework for Mitigating Parameter Variation Imbalance in Online Continual Learning","date":"2025-02-25","arxiv_id":"2502.17794","n_code_links":0,"syntology":null},{"paper":null,"slug":"secura-sigmoid-enhanced-cur-decomposition","title":"SECURA: Sigmoid-Enhanced CUR Decomposition with Uninterrupted Retention and Low-Rank Adaptation in Large Language Models","date":"2025-02-25","arxiv_id":"2502.18168","n_code_links":0,"syntology":null},{"paper":null,"slug":"theoretical-barriers-in-bellman-based","title":"Theoretical Barriers in Bellman-Based Reinforcement Learning","date":"2025-02-17","arxiv_id":"2502.11968","n_code_links":0,"syntology":null},{"paper":null,"slug":"motion-control-in-multi-rotor-aerial-robots","title":"Motion Control in Multi-Rotor Aerial Robots Using Deep Reinforcement Learning","date":"2025-02-09","arxiv_id":"2502.05996","n_code_links":0,"syntology":null},{"paper":"/paper/td3-tucker-decomposition-based-dataset","slug":"td3-tucker-decomposition-based-dataset","title":"TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation","date":"2025-02-05","arxiv_id":"2502.02854","n_code_links":1,"syntology":null},{"paper":null,"slug":"actor-critic-with-experience-replay-based","title":"Actor Critic with Experience Replay-based automatic treatment planning for prostate cancer intensity modulated radiotherapy","date":"2025-02-01","arxiv_id":"2502.00346","n_code_links":0,"syntology":null},{"paper":null,"slug":"reward-prediction-error-prioritisation-in","title":"Reward Prediction Error Prioritisation in Experience Replay: The RPE-PER Method","date":"2025-01-30","arxiv_id":"2501.18093","n_code_links":0,"syntology":null},{"paper":"/paper/langevin-soft-actor-critic-efficient","slug":"langevin-soft-actor-critic-efficient","title":"Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning","date":"2025-01-29","arxiv_id":"2501.17827","n_code_links":1,"syntology":{"ran":6,"of":11,"n_ran_checked":3,"n_instrument":3,"unverified":5,"pointer_only":0,"phrase":"6 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified","official":{"repos":["hmishfaq/lsac"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":1,"n_ran_no_instrument_failure":3,"n_unverified":5,"ran_from_kinds":["official"]}}},{"paper":"/paper/evorl-a-gpu-accelerated-framework-for","slug":"evorl-a-gpu-accelerated-framework-for","title":"EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning","date":"2025-01-25","arxiv_id":"2501.15129","n_code_links":2,"syntology":null},{"paper":"/paper/reinforcement-learning-controlled-adaptive","slug":"reinforcement-learning-controlled-adaptive","title":"Reinforcement Learning Controlled Adaptive PSO for Task Offloading in IIoT Edge Computing","date":"2025-01-25","arxiv_id":"2501.15203","n_code_links":1,"syntology":null},{"paper":"/paper/clofai-a-dataset-of-real-and-fake-image","slug":"clofai-a-dataset-of-real-and-fake-image","title":"CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learning","date":"2025-01-19","arxiv_id":"2501.11140","n_code_links":1,"syntology":null},{"paper":null,"slug":"federated-deep-reinforcement-learning-for-1","title":"Federated Deep Reinforcement Learning for Energy Efficient Multi-Functional RIS-Assisted Low-Earth Orbit Networks","date":"2025-01-19","arxiv_id":"2501.11079","n_code_links":0,"syntology":null},{"paper":null,"slug":"enhancing-uav-path-planning-efficiency","title":"Enhancing UAV Path Planning Efficiency Through Accelerated Learning","date":"2025-01-17","arxiv_id":"2501.10141","n_code_links":0,"syntology":null},{"paper":null,"slug":"autoloop-fast-visual-slam-fine-tuning-through","title":"AutoLoop: Fast Visual SLAM Fine-tuning through Agentic Curriculum Learning","date":"2025-01-15","arxiv_id":"2501.09160","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamic-portfolio-optimization-via-augmented","title":"Dynamic Portfolio Optimization via Augmented DDPG with Quantum Price Levels-Based Trading Strategy","date":"2025-01-15","arxiv_id":"2501.08528","n_code_links":0,"syntology":null},{"paper":null,"slug":"continual-learning-with-embedding-layer","title":"Continual Learning with Embedding Layer Surgery and Task-wise Beam Search using Whisper","date":"2025-01-14","arxiv_id":"2501.07875","n_code_links":0,"syntology":null},{"paper":null,"slug":"average-reward-reinforcement-learning-for","title":"Average Reward Reinforcement Learning for Wireless Radio Resource Management","date":"2025-01-12","arxiv_id":"2501.06700","n_code_links":0,"syntology":null},{"paper":null,"slug":"sale-based-offline-reinforcement-learning","title":"SALE-Based Offline Reinforcement Learning with Ensemble Q-Networks","date":"2025-01-07","arxiv_id":"2501.03676","n_code_links":0,"syntology":null},{"paper":null,"slug":"the-artificial-scientist-in-transit-machine","title":"The Artificial Scientist -- in-transit Machine Learning of Plasma Simulations","date":"2025-01-06","arxiv_id":"2501.03383","n_code_links":0,"syntology":null},{"paper":null,"slug":"protein-structure-prediction-in-the-3d-hp","title":"Protein Structure Prediction in the 3D HP Model Using Deep Reinforcement Learning","date":"2024-12-29","arxiv_id":"2412.20329","n_code_links":0,"syntology":null},{"paper":null,"slug":"generalized-back-stepping-experience-replay","title":"Generalized Back-Stepping Experience Replay in Sparse-Reward Environments","date":"2024-12-20","arxiv_id":"2412.15525","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-advantage-based-optimization-method-for","title":"An Advantage-based Optimization Method for Reinforcement Learning in Large Action Space","date":"2024-12-17","arxiv_id":"2412.12605","n_code_links":0,"syntology":null},{"paper":null,"slug":"distribution-free-uncertainty-quantification-2","title":"Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework","date":"2024-12-17","arxiv_id":"2412.12597","n_code_links":0,"syntology":null},{"paper":null,"slug":"integrated-sensing-and-communications-for-low","title":"Integrated Sensing and Communications for Low-Altitude Economy: A Deep Reinforcement Learning Approach","date":"2024-12-05","arxiv_id":"2412.04074","n_code_links":0,"syntology":null},{"paper":null,"slug":"oner-online-experience-replay-for-incremental","title":"ONER: Online Experience Replay for Incremental Anomaly Detection","date":"2024-12-05","arxiv_id":"2412.03907","n_code_links":0,"syntology":null},{"paper":null,"slug":"continual-learning-of-personalized-generative","title":"Continual Learning of Personalized Generative Face Models with Experience Replay","date":"2024-12-03","arxiv_id":"2412.02627","n_code_links":0,"syntology":null},{"paper":null,"slug":"dependency-aware-cav-task-scheduling-via","title":"Dependency-Aware CAV Task Scheduling via Diffusion-Based Reinforcement Learning","date":"2024-11-27","arxiv_id":"2411.18230","n_code_links":0,"syntology":null},{"paper":null,"slug":"broad-critic-deep-actor-reinforcement","title":"Broad Critic Deep Actor Reinforcement Learning for Continuous Control","date":"2024-11-24","arxiv_id":"2411.15806","n_code_links":0,"syntology":null},{"paper":null,"slug":"provably-efficient-action-manipulation-attack","title":"Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning","date":"2024-11-20","arxiv_id":"2411.13116","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamics-of-resource-allocation-in-o-rans-an","title":"Dynamics of Resource Allocation in O-RANs: An In-depth Exploration of On-Policy and Off-Policy Deep Reinforcement Learning for Real-Time Applications","date":"2024-11-17","arxiv_id":"2412.01839","n_code_links":0,"syntology":null},{"paper":null,"slug":"quantum-policy-gradient-in-reproducing-kernel","title":"Quantum Policy Gradient in Reproducing Kernel Hilbert Space","date":"2024-11-11","arxiv_id":"2411.06650","n_code_links":0,"syntology":null},{"paper":null,"slug":"autonomous-decision-making-for-uav","title":"Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning","date":"2024-11-05","arxiv_id":"2411.02983","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamic-weight-adjusting-deep-q-networks-for","title":"Dynamic Weight Adjusting Deep Q-Networks for Real-Time Environmental Adaptation","date":"2024-11-04","arxiv_id":"2411.02559","n_code_links":0,"syntology":null},{"paper":null,"slug":"maximum-entropy-hindsight-experience-replay","title":"Maximum Entropy Hindsight Experience Replay","date":"2024-10-31","arxiv_id":"2410.24016","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-gradients-as-vitamin","slug":"reinforcement-learning-gradients-as-vitamin","title":"Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers","date":"2024-10-31","arxiv_id":"2410.24108","n_code_links":1,"syntology":{"ran":4,"of":7,"n_ran_checked":2,"n_instrument":2,"unverified":3,"pointer_only":7,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","official":{"repos":["kaiyan289/rl_as_vitamin_for_online_decision_transformers"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":"/paper/networkgym-reinforcement-learning","slug":"networkgym-reinforcement-learning","title":"NetworkGym: Reinforcement Learning Environments for Multi-Access Traffic Management in Network Simulation","date":"2024-10-30","arxiv_id":"2411.04138","n_code_links":1,"syntology":{"ran":15,"of":18,"n_ran_checked":7,"n_instrument":8,"unverified":3,"pointer_only":1,"phrase":"15 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 8 where Syntology's instrument failed) · 3 unverified","official":{"repos":["hmomin/networkgym"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":2,"n_ran_no_instrument_failure":5,"n_unverified":3,"ran_from_kinds":["found_in_text","official"]}}},{"paper":null,"slug":"hindsight-experience-replay-accelerates","title":"Hindsight Experience Replay Accelerates Proximal Policy Optimization","date":"2024-10-29","arxiv_id":"2410.22524","n_code_links":0,"syntology":null},{"paper":"/paper/human-readable-programs-as-actors-of","slug":"human-readable-programs-as-actors-of","title":"Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution","date":"2024-10-29","arxiv_id":"2410.21940","n_code_links":1,"syntology":null},{"paper":null,"slug":"efficient-diversity-based-experience-replay","title":"Efficient Diversity-based Experience Replay for Deep Reinforcement Learning","date":"2024-10-27","arxiv_id":"2410.20487","n_code_links":0,"syntology":null},{"paper":null,"slug":"learn-2-rage-experiencing-the-emotional","title":"Learn 2 Rage: Experiencing The Emotional Roller Coaster That Is Reinforcement Learning","date":"2024-10-24","arxiv_id":"2410.18462","n_code_links":0,"syntology":null},{"paper":null,"slug":"augmented-lagrangian-based-safe-reinforcement","title":"Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control","date":"2024-10-19","arxiv_id":"2410.15188","n_code_links":0,"syntology":null},{"paper":null,"slug":"enhancing-llm-agents-for-code-generation-with","title":"Enhancing LLM Agents for Code Generation with Possibility and Pass-rate Prioritized Experience Replay","date":"2024-10-16","arxiv_id":"2410.12236","n_code_links":0,"syntology":null},{"paper":null,"slug":"sac-glam-improving-online-rl-for-llm-agents","title":"SAC-GLAM: Improving Online RL for LLM agents with Soft Actor-Critic and Hindsight Relabeling","date":"2024-10-16","arxiv_id":"2410.12481","n_code_links":0,"syntology":null},{"paper":null,"slug":"mitigating-suboptimality-of-deterministic","title":"Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions","date":"2024-10-15","arxiv_id":"2410.11833","n_code_links":0,"syntology":null},{"paper":null,"slug":"physical-informed-inspired-deep-reinforcement","title":"Physical Informed-Inspired Deep Reinforcement Learning Based Bi-Level Programming for Microgrid Scheduling","date":"2024-10-15","arxiv_id":"2410.11932","n_code_links":0,"syntology":null},{"paper":null,"slug":"solving-the-dynamic-volatility-fitting","title":"Solving The Dynamic Volatility Fitting Problem: A Deep Reinforcement Learning Approach","date":"2024-10-15","arxiv_id":"2410.11789","n_code_links":0,"syntology":null},{"paper":null,"slug":"etgl-ddpg-a-deep-deterministic-policy","title":"ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control","date":"2024-10-07","arxiv_id":"2410.05225","n_code_links":0,"syntology":null},{"paper":null,"slug":"task-unaware-lifelong-robot-learning-with","title":"Task-free Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation","date":"2024-10-03","arxiv_id":"2410.02995","n_code_links":0,"syntology":null},{"paper":null,"slug":"ssl-nbv-a-self-supervised-learning-based-next","title":"SSL-NBV: A Self-Supervised-Learning-Based Next-Best-View algorithm for Efficient 3D Plant Reconstruction by a Robot","date":"2024-10-02","arxiv_id":"2410.14790","n_code_links":0,"syntology":null},{"paper":"/paper/forgetting-ignorance-or-myopia-revisiting-key","slug":"forgetting-ignorance-or-myopia-revisiting-key","title":"Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning","date":"2024-09-28","arxiv_id":"2409.19245","n_code_links":1,"syntology":{"ran":5,"of":5,"n_ran_checked":1,"n_instrument":4,"unverified":0,"pointer_only":5,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","official":{"repos":["wxr99/Forgetting-Ignorance-or-Myopia-Revisiting-Key-Challenges-in-Online-Continual-Learning"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"robust-deep-reinforcement-learning-for-volt","title":"Robust Deep Reinforcement Learning for Volt-VAR Optimization in Active Distribution System under Uncertainty","date":"2024-09-27","arxiv_id":"2409.18937","n_code_links":0,"syntology":null},{"paper":null,"slug":"navigation-in-a-simplified-urban-flow-through","title":"Navigation in a simplified Urban Flow through Deep Reinforcement Learning","date":"2024-09-26","arxiv_id":"2409.17922","n_code_links":0,"syntology":null},{"paper":null,"slug":"cabage-data-free-model-extraction-using-class","title":"CaBaGe: Data-Free Model Extraction using ClAss BAlanced Generator Ensemble","date":"2024-09-16","arxiv_id":"2409.10643","n_code_links":0,"syntology":null},{"paper":"/paper/learning-discrete-world-models-for-heuristic","slug":"learning-discrete-world-models-for-heuristic","title":"Learning Discrete World Models for Heuristic Search","date":"2024-09-14","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"dynamic-demand-management-for-parcel-lockers","title":"Dynamic Demand Management for Parcel Lockers","date":"2024-09-08","arxiv_id":"2409.05061","n_code_links":0,"syntology":null},{"paper":null,"slug":"granular-ball-representation-learning-for","title":"Granular-ball Representation Learning for Deep CNN on Learning with Label Noise","date":"2024-09-05","arxiv_id":"2409.03254","n_code_links":0,"syntology":null},{"paper":"/paper/a-tighter-convergence-proof-of-reverse","slug":"a-tighter-convergence-proof-of-reverse","title":"A Tighter Convergence Proof of Reverse Experience Replay","date":"2024-08-30","arxiv_id":"2408.16999","n_code_links":1,"syntology":null},{"paper":null,"slug":"simultaneous-training-of-first-and-second","title":"Simultaneous Training of First- and Second-Order Optimizers in Population-Based Reinforcement Learning","date":"2024-08-27","arxiv_id":"2408.15421","n_code_links":0,"syntology":null},{"paper":"/paper/may-the-forgetting-be-with-you-alternate","slug":"may-the-forgetting-be-with-you-alternate","title":"May the Forgetting Be with You: Alternate Replay for Learning with Noisy Labels","date":"2024-08-26","arxiv_id":"2408.14284","n_code_links":1,"syntology":null},{"paper":null,"slug":"optimizing-td3-for-7-dof-robotic-arm-grasping","title":"Optimizing TD3 for 7-DOF Robotic Arm Grasping: Overcoming Suboptimality with Exploration-Enhanced Contrastive Learning","date":"2024-08-26","arxiv_id":"2408.14009","n_code_links":0,"syntology":null},{"paper":"/paper/sigmarl-a-sample-efficient-and-generalizable","slug":"sigmarl-a-sample-efficient-and-generalizable","title":"SigmaRL: A Sample-Efficient and Generalizable Multi-Agent Reinforcement Learning Framework for Motion Planning","date":"2024-08-14","arxiv_id":"2408.07644","n_code_links":4,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":865,"with_a_code_link":317,"where_syntology_ran_a_sample":94,"not_listed_spam_title":0,"listed":865,"listed_where_code_ran":94,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":83,"every_run_a_failure_of_syntologys_instrument":11,"listed_with_a_run_with_no_instrument_failure":83,"listed_every_run_a_failure_of_syntologys_instrument":11,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":94,"papers_where_code_ran_first_page":"/method/experience-replay/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":9,"url":"/method/experience-replay/papers/2","json":"/method/experience-replay/papers/2.json","rows":100},{"order":"archive","page":3,"of":9,"url":"/method/experience-replay/papers/3","json":"/method/experience-replay/papers/3.json","rows":100},{"order":"archive","page":4,"of":9,"url":"/method/experience-replay/papers/4","json":"/method/experience-replay/papers/4.json","rows":100},{"order":"archive","page":5,"of":9,"url":"/method/experience-replay/papers/5","json":"/method/experience-replay/papers/5.json","rows":100},{"order":"archive","page":6,"of":9,"url":"/method/experience-replay/papers/6","json":"/method/experience-replay/papers/6.json","rows":100},{"order":"archive","page":7,"of":9,"url":"/method/experience-replay/papers/7","json":"/method/experience-replay/papers/7.json","rows":100},{"order":"archive","page":8,"of":9,"url":"/method/experience-replay/papers/8","json":"/method/experience-replay/papers/8.json","rows":100},{"order":"archive","page":9,"of":9,"url":"/method/experience-replay/papers/9","json":"/method/experience-replay/papers/9.json","rows":65},{"order":"ran","page":1,"of":1,"url":"/method/experience-replay/papers/ran/1","json":"/method/experience-replay/papers/ran/1.json","rows":94}],"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":440},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":375},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":371},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":296},{"task":"/task/continuous-control","name":"Continuous Control","papers":99},{"task":"/task/q-learning","name":"Q-Learning","papers":94},{"task":"/task/continuous-control","name":"continuous-control","papers":93},{"task":"/task/continual-learning","name":"Continual Learning","papers":90},{"task":"/task/mujoco","name":"MuJoCo","papers":54},{"task":"/task/decision-making","name":"Decision Making","papers":49},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":43},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":42},{"task":"/task/atari-games","name":"Atari Games","papers":34},{"task":"/task/management","name":"Management","papers":25},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":22},{"task":"/task/imitation-learning","name":"Imitation Learning","papers":21},{"task":"/task/incremental-learning","name":"Incremental Learning","papers":21},{"task":"/task/scheduling","name":"Scheduling","papers":20},{"task":"/task/diversity","name":"Diversity","papers":17},{"task":"/task/multi-goal-reinforcement-learning","name":"Multi-Goal Reinforcement Learning","papers":17}],"tasks_shown":20,"n_tasks":289,"usage_by_year":[{"year":"2013","papers":1},{"year":"2015","papers":3},{"year":"2016","papers":5},{"year":"2017","papers":17},{"year":"2018","papers":57},{"year":"2019","papers":86},{"year":"2020","papers":129},{"year":"2021","papers":145},{"year":"2022","papers":113},{"year":"2023","papers":134},{"year":"2024","papers":117},{"year":"2025","papers":58}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/experience-replay"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Experience%20Replay","read_at":"2026-10-05T00:55:46+00:00","query":"Experience Replay","kind":"method","have":"no_code","tier":"paper_only","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"experience replay","origin_arxiv_id":"2108.12081"},"resolution":"case_insensitive","resolved_name_differs":false,"absence_asserted":"for_this_exact_name_only","absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":null,"in_graph":0,"list_truncated":null,"strongest_link":null},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"2108.12081","citations":59},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'Experience Replay'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[{"name":"Curious Replay","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Curious Replay')","origin_arxiv_id":"2306.15934"},{"name":"Dark Experience Replay","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Dark Experience Replay')","origin_arxiv_id":"2004.07211"},{"name":"a simple approach that learns a reward model from online experience","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='a simple approach that learns a reward model from online experience')","origin_arxiv_id":"2311.05067"}],"code_nearby_caption":"Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.","origin_paper":{"arxiv_id":"2108.12081","page":null,"syntology_url":"https://syntology.ai/paper/2108.12081","in_graph_layer":false,"samples":0},"line":"no code under this name (no_code; paper_only). The lookup matched 'experience replay' ignoring case. The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2108.12081) is about this method is not checked; it does not say whether this method has code.","level_legend":{"state":"quoted","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","card_sha256":"479785a84658dd682a11b10237b3620d651d7f87799f5dc7add970aaa83f6d39","note":"Syntology's served verification levels, quoted verbatim from its MCP server card; a row is split where the card separates clauses with a double hyphen, and the separator is not reproduced","levels":[{"level":"V0","quoted":["imports"]},{"level":"V1","quoted":["executes against a synthesized fixture and returns a value of the recorded type"]},{"level":"V2","quoted":["two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs","neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)"]},{"level":"V3","quoted":["paper-derived property tests pass on both of those same two generations."]}],"no_rung":"No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results."}},"syntology_have_state":"loaded","record_sha256":"90a2ce37c2b20a9496fe3faf0e3a5e5b00ee562880aed2724a655437021e9a9f","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}