Browse State-of-the-Art › Montezuma's Revenge
Montezuma's Revenge
31 papers with code · 1 benchmark · 1 dataset archive 2025-07-28
Montezuma's Revenge is an ATARI 2600 Benchmark game that is known to be difficult to perform on for reinforcement learning algorithms. Solutions typically employ algorithms that incentivise environment exploration in different ways.
For the state-of-the art tables, please consult the parent Atari Games task.
( Image credit: Q-map )
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| Atari 2600 Montezuma's Revenge (3 rows) | Flare | Reinforcement Learning with Latent Flow | code | Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
1 dataset whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 61 · with a code link: 31 · where Syntology ran a sample: 12 (8 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (12 of 61 tagged: 8 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 61 of 61, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
6 Oct 2017 34 repositories listed Syntology 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 1 pointer-only (licence)The deep reinforcement learning community has made several independent improvements to the DQN algorithm.
-
30 Oct 2018 22 repositories listed Syntology community repositories only · 31 ran (of which 12 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 7 where Syntology's instrument failed) · 12 unverified (of 43 harvested samples) · 16 pointer-only (licence)In particular we establish state of the art performance on Montezuma's Revenge, a game famously difficult for deep reinforcement learning methods.
-
20 Apr 2016 4 repositories listedLearning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms.
-
30 Jan 2019 3 repositories listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 4 pointer-only (licence)Go-Explore can also harness human-provided domain knowledge and, when augmented with it, scores a mean of over 650k points on Montezuma's Revenge.
-
5 Jun 2023 2 repositories listedThis results in an algorithm which sets a new state of the art across 16 tasks from the MiniHack suite used in prior work, and also performs robustly on Habitat and Montezuma's Revenge.
-
6 Jan 2021 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Temporal information is essential to learning effective policies with Reinforcement Learning (RL).
-
27 Apr 2020 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 4 pointer-only (licence)The promise of reinforcement learning is to solve complex sequential decision problems autonomously by specifying a high-level reward function only.
-
10 Mar 2020 2 repositories listedIn this paper, we focus on improving the effectiveness of finding unknown states and propose action balance exploration, which balances the frequency of selecting each action at a given state and can be treated as an…
-
6 Oct 2018 2 repositories listedBeing able to reach any desired location in the environment can be a valuable asset for an agent.
-
16 May 2025 1 repository listedLearning how the world works is central to building AI agents that can adapt to complex environments.
-
29 May 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 3 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified (of 11 harvested samples) · 11 pointer-only (licence)Continual learning with deep neural networks presents challenges distinct from both the fixed-dataset and convex continual learning regimes.
-
5 Feb 2024 1 repository listedFine-tuning is a widespread technique that allows practitioners to transfer pre-trained capabilities, as recently showcased by the successful applications of foundation models.
-
5 Jun 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)We propose a new method for count-based exploration in high-dimensional state spaces.
-
14 Nov 2022 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)However, on easy exploration tasks, the agent gets distracted by intrinsic rewards and performs unnecessary exploration even when sufficient task (also called extrinsic) reward is available.
-
13 Oct 2022 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)We consider a hybrid reinforcement learning setting (Hybrid RL), in which an agent has access to an offline dataset and the ability to collect experience via real-world online interaction.
-
31 Aug 2022 1 repository listedIn this paper, we introduce Latent Go-Explore (LGE), a simple and general approach based on the Go-Explore paradigm for exploration in reinforcement learning (RL).
-
16 Feb 2022 1 repository listedInspired by the great success of unsupervised learning in Computer Vision and Natural Language Processing, the Reinforcement Learning community has recently started to focus more on unsupervised discovery of skills.
-
1 Dec 2021 1 repository listedWe analyze NovelD thoroughly in MiniGrid and found that empirically it helps the agent explore the environment more uniformly with a focus on exploring beyond the boundary.
-
12 Jul 2020 1 repository listedModel-based reinforcement learning (RL) is appealing because (i) it enables planning and thus more strategic exploration, and (ii) by decoupling dynamics from rewards, it enables fast transfer to new reward functions.
-
19 Dec 2019 1 repository listedThe former manifests itself through the use of value function, while the latter is powered by a tree search planner.
-
22 Nov 2019 1 repository listedThis paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving…
-
25 Sep 2019 1 repository listedNotably, our method performs well in environments with sparse rewards where standard TD(1) backups fail.
-
18 May 2019 1 repository listedOur work is a simple extension of the paper "Exploration by Random Network Distillation".
-
5 Mar 2019 1 repository listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the agent intermediate rewards for progress towards the goal.
-
11 Oct 2018 1 repository listedHowever, many of the state of the art deep reinforcement learning algorithms, that rely on epsilon-greedy, fail on these environments.
-
29 May 2018 1 repository listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)One successful method of guiding exploration in these domains is to imitate trajectories provided by a human demonstrator.
-
18 May 2017 1 repository listedWe highlight the advantage of our approach in one of the hardest games -- Montezuma's revenge -- for which the ability to handle sparse rewards is key.
-
18 Apr 2017 1 repository listedWe introduce the first deep reinforcement learning agent that learns to beat Atari games with the aid of natural language instructions.
-
3 Mar 2017 1 repository listed Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)This pseudo-count was used to generate an exploration bonus for a DQN agent and combined with a mixed Monte Carlo update was sufficient to achieve state of the art on the Atari 2600 game Montezuma's Revenge.
-
18 Jul 2016 1 repository listedThis paper introduces a novel method for learning how to play the most difficult Atari 2600 games from the Arcade Learning Environment using deep reinforcement learning.
-
6 Jun 2016 1 repository listed
-
Action-Dependent Optimality-Preserving Reward Shaping19 May 2025 0 repositories listed
-
Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning20 Jun 2023 0 repositories listed
-
Sample Efficient Deep Reinforcement Learning via Local Planning29 Jan 2023 0 repositories listed
-
Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments18 Nov 2022 0 repositories listed
-
Paused Agent Replay Refresh26 Sep 2022 0 repositories listed
-
GAN-based Intrinsic Exploration For Sample Efficient Reinforcement Learning28 Jun 2022 0 repositories listed
-
Parametrically Retargetable Decision-Makers Tend To Seek Power27 Jun 2022 0 repositories listed
-
Understanding and Preventing Capacity Loss in Reinforcement Learning20 Apr 2022 0 repositories listed
-
Generative Adversarial Exploration for Reinforcement Learning27 Jan 2022 0 repositories listed
-
Exploration by Random Network Distillation17 Jan 2022 0 repositories listed
-
Creativity of AI: Hierarchical Planning Model Learning for Facilitating Deep Reinforcement Learning18 Dec 2021 0 repositories listed
-
Entropic Desired Dynamics for Intrinsic Control1 Dec 2021 0 repositories listed
-
On Bonus-Based Exploration Methods in the Arcade Learning Environment22 Sep 2021 0 repositories listed
-
Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations4 Feb 2020 0 repositories listed
-
MIME: Mutual Information Minimisation Exploration16 Jan 2020 0 repositories listed
-
On Bonus Based Exploration Methods In The Arcade Learning Environment1 Jan 2020 0 repositories listed
-
Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment6 Aug 2019 0 repositories listed
-
Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards24 Jul 2019 0 repositories listed
-
Learning and Exploiting Multiple Subgoals for Fast Exploration in Hierarchical Reinforcement Learning13 May 2019 0 repositories listed
-
Escape Room: A Configurable Testbed for Hierarchical Reinforcement Learning22 Dec 2018 0 repositories listed
-
Learning Montezuma's Revenge from a Single Demonstration8 Dec 2018 0 repositories listed
-
5 Nov 2018 0 repositories listed
-
Learning Representations in Model-Free Hierarchical Reinforcement Learning23 Oct 2018 0 repositories listed
-
Deep Curiosity Search: Intra-Life Exploration Can Improve Performance on Challenging Deep Reinforcement Learning Problems1 Jun 2018 0 repositories listed
-
Observe and Look Further: Achieving Consistent Performance on Atari29 May 2018 0 repositories listed
-
Hierarchical Imitation and Reinforcement Learning1 Mar 2018 0 repositories listed
-
Learning High-level Representations from Demonstrations19 Feb 2018 0 repositories listed
-
Exploration in Feature Space for Reinforcement Learning5 Oct 2017 0 repositories listed
-
Deep Abstract Q-Networks2 Oct 2017 0 repositories listed
-
Micro-Objective Learning : Accelerating Deep Reinforcement Learning through the Discovery of Continuous Subgoals11 Mar 2017 0 repositories listed
Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections