Methods › Reinforcement Learning › Heuristic Search Algorithms › Monte-Carlo Tree Search
Monte-Carlo Tree Search
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Monte-Carlo Tree Search is a planning algorithm that accumulates value estimates obtained from Monte Carlo simulations in order to successively direct simulations towards more highly-rewarded trajectories. We execute MCTS after encountering each new state to select an agent's action for that state: it is executed again to select the action for the next state. Each execution is an iterative process that simulates many trajectories starting from the current state to the terminal state. The core idea is to successively focus multiple simulations starting at the current state by extending the initial portions of trajectories that have received high evaluations from earlier simulations.
Source: Sutton and Barto, Reinforcement Learning (2nd Edition)
Image Credit: Chaslot et al
In Syntology for a different name, 'Monte Carlo tree search': repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: official). The lookup matched a different name in Syntology's graph, 'Monte Carlo tree search' (resolution: token_match); this answer is about that name. The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2010.12069) is about this method is not checked.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- SPO-tree · shares_name_tokens (shares name tokens with this name; a different method) · level V2 · retrieve with
get_reference_implementation(name='SPO-tree') - SP-SEARCH · shares_name_tokens (shares name tokens with this name; a different method) · level V2 · retrieve with
get_reference_implementation(name='SP-SEARCH') - Tree of Thoughts · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Tree of Thoughts') - back-search algorithm · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='back-search algorithm') - δ-Conservative Search · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='δ-Conservative Search') - local search strategy · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='local search strategy')
Origin paper of this answer, arXiv:2010.12069 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 166 · with a code link: 62 · where Syntology ran a sample: 12 (10 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (12 of 166 tagged: 10 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 166, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents 17 Jun 2025 · 0 repositories · arXiv:2506.21582
-
Calibrated Value-Aware Model Learning with Stochastic Environment Models 28 May 2025 · 0 repositories · arXiv:2505.22772
-
Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planning 21 May 2025 · 0 repositories · arXiv:2505.15782
-
Adaptive Stress Testing Black-Box LLM Planners 8 May 2025 · 0 repositories · arXiv:2505.05665
-
Adaptive Branch-and-Bound Tree Exploration for Neural Network Verification 2 May 2025 · 1 repository · arXiv:2505.00963
-
Trans-Zero: Self-Play Incentivizes Large Language Models for Multilingual Translation Without Parallel Data 20 Apr 2025 · 1 repository · arXiv:2504.14669
-
Neural-Guided Equation Discovery 21 Mar 2025 · 1 repository · arXiv:2503.16953
-
A Neural Symbolic Model for Space Physics 11 Mar 2025 · 1 repository · arXiv:2503.07994
-
OptionZero: Planning with Learned Options 23 Feb 2025 · 1 repository · arXiv:2502.16634Syntology official (archive's flag): 7 ran · 7 ran (of which 7 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 7 samples that ran constructed an object rather than computing a result (of 8 harvested samples) · 8 pointer-only (licence)
-
Boost, Disentangle, and Customize: A Robust System2-to-System1 Pipeline for Code Generation 18 Feb 2025 · 0 repositories · arXiv:2502.12492
-
Reinforcement Learning in Strategy-Based and Atari Games: A Review of Google DeepMinds Innovations 14 Feb 2025 · 0 repositories · arXiv:2502.10303
-
Evaluating World Models with LLM for Decision Making 13 Nov 2024 · 0 repositories · arXiv:2411.08794
-
Evaluating Robustness of Reinforcement Learning Algorithms for Autonomous Shipping 7 Nov 2024 · 0 repositories · arXiv:2411.04915
-
Interpreting the Learned Model in MuZero Planning 7 Nov 2024 · 0 repositories · arXiv:2411.04580
-
Human-aligned Chess with a Bit of Search 4 Oct 2024 · 1 repository · arXiv:2410.03893
-
Zero-Shot Multi-Hop Question Answering via Monte-Carlo Tree Search with Large Language Models 28 Sep 2024 · 0 repositories · arXiv:2409.19382
-
An Efficient and Generalizable Symbolic Regression Method for Time Series Analysis 6 Sep 2024 · 0 repositories · arXiv:2409.03986
-
DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search 15 Aug 2024 · 2 repositories · arXiv:2408.08152Syntology official (archive's flag): 6 ran · 10 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 0 violated, 4 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 10 harvested samples)
-
Combining AI Control Systems and Human Decision Support via Robustness and Criticality 3 Jul 2024 · 0 repositories · arXiv:2407.03210
-
Enhancements for Real-Time Monte-Carlo Tree Search in General Video Game Playing 3 Jul 2024 · 0 repositories · arXiv:2407.03049
-
Towards a Characterisation of Monte-Carlo Tree Search Performance in Different Games 13 Jun 2024 · 0 repositories · arXiv:2406.09242
-
Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstraction 2 Jun 2024 · 1 repository · arXiv:2406.00614
-
Global Rewards in Restless Multi-Armed Bandits 2 Jun 2024 · 0 repositories · arXiv:2406.00738
-
Extreme Value Monte Carlo Tree Search 28 May 2024 · 0 repositories · arXiv:2405.18248
-
PyTAG: Tabletop Games for Multi-Agent Reinforcement Learning 28 May 2024 · 1 repository · arXiv:2405.18123
-
Efficient Multi-agent Reinforcement Learning by Planning 20 May 2024 · 1 repository · arXiv:2405.11778
-
Value Augmented Sampling for Language Model Alignment and Personalization 10 May 2024 · 1 repository · arXiv:2405.06639Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
ReZero: Boosting MCTS-based Algorithms by Backward-view and Entire-buffer Reanalyze 25 Apr 2024 · 1 repository · arXiv:2404.16364
-
Tree Search-Based Policy Optimization under Stochastic Execution Delay 8 Apr 2024 · 1 repository · arXiv:2404.05440Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples) · 8 pointer-only (licence)
-
Can Large Language Models Play Games? A Case Study of A Self-Play Approach 8 Mar 2024 · 0 repositories · arXiv:2403.05632
-
Retrieval Augmented Thought Process for Private Data Handling in Healthcare 12 Feb 2024 · 0 repositories · arXiv:2402.07812
-
The Strain of Success: A Predictive Model for Injury Risk Mitigation and Team Success in Soccer 7 Feb 2024 · 1 repository · arXiv:2402.04898
-
Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chess 30 Jan 2024 · 1 repository · arXiv:2401.16852
-
Learning Backdoors for Mixed Integer Linear Programs with Contrastive Learning 19 Jan 2024 · 0 repositories · arXiv:2401.10467
-
Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding 21 Nov 2023 · 0 repositories · arXiv:2311.12572
-
MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Games 17 Oct 2023 · 1 repository · arXiv:2310.11305Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)
-
Alpha Elimination: Using Deep Reinforcement Learning to Reduce Fill-In during Sparse Matrix Decomposition 15 Oct 2023 · 0 repositories · arXiv:2310.09852
-
Accelerating Monte Carlo Tree Search with Probability Tree State Abstraction 10 Oct 2023 · 0 repositories · arXiv:2310.06513
-
Rollout Heuristics for Online Stochastic Contingent Planning 3 Oct 2023 · 0 repositories · arXiv:2310.02345
-
Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding 26 Sep 2023 · 0 repositories · arXiv:2309.15028
-
POMDP Planning for Object Search in Partially Unknown Environment 21 Sep 2023 · 0 repositories
-
[Re] On Explainability of Graph Neural Networks via Subgraph Explorations 21 Sep 2023 · 0 repositories
-
Self-Predictive Universal AI 21 Sep 2023 · 0 repositories
-
Monte-Carlo tree search with uncertainty propagation via optimal transport 19 Sep 2023 · 0 repositories · arXiv:2309.10737
-
Chain-of-Thought Reasoning is a Policy Improvement Operator 15 Sep 2023 · 0 repositories · arXiv:2309.08589
-
Efficient and Explainable Graph Neural Architecture Search via Monte-Carlo Tree Search 30 Aug 2023 · 1 repository · arXiv:2308.15734
-
JiangJun: Mastering Xiangqi by Tackling Non-Transitivity in Two-Player Zero-Sum Games 9 Aug 2023 · 0 repositories · arXiv:2308.04719
-
AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning 7 Aug 2023 · 1 repository · arXiv:2308.03526Syntology 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
-
Monte-Carlo Tree Search for Multi-Agent Pathfinding: Preliminary Results 25 Jul 2023 · 0 repositories · arXiv:2307.13453
-
On-Robot Bayesian Reinforcement Learning for POMDPs 22 Jul 2023 · 0 repositories · arXiv:2307.11954
-
Reinforcement Learning and Data-Generation for Syntax-Guided Synthesis 13 Jul 2023 · 0 repositories · arXiv:2307.09564
-
λ-models: Effective Decision-Aware Reinforcement Learning with Latent Models 30 Jun 2023 · 0 repositories · arXiv:2306.17366
-
Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments 9 Jun 2023 · 1 repository · arXiv:2306.06067
-
Habits of Mind: Reusing Action Sequences for Efficient Planning 8 Jun 2023 · 0 repositories · arXiv:2306.05298
-
What model does MuZero learn? 1 Jun 2023 · 0 repositories · arXiv:2306.00840
-
Know your Enemy: Investigating Monte-Carlo Tree Search with Opponent Models in Pommerman 22 May 2023 · 1 repository · arXiv:2305.13206
-
Model Predictive Control with Self-supervised Representation Learning 14 Apr 2023 · 1 repository · arXiv:2304.07219
-
Proof Number Based Monte-Carlo Tree Search 16 Mar 2023 · 1 repository · arXiv:2303.09449
-
A Parallel Monte-Carlo Tree Search-Based Metaheuristic For Optimal Fleet Composition Considering Vehicle Routing Using Branch & Bound 6 Mar 2023 · 0 repositories · arXiv:2303.03156
-
Deconstructing deep active inference 2 Mar 2023 · 0 repositories · arXiv:2303.01618
-
Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search 22 Feb 2023 · 0 repositories · arXiv:2302.11223
-
Equivariant MuZero 9 Feb 2023 · 0 repositories · arXiv:2302.04798
-
Monte-Carlo Tree-Search for Leveraging Performance of Blackbox Job-Shop Scheduling Heuristics 14 Dec 2022 · 0 repositories · arXiv:2212.07543
-
Self-Improving Safety Performance of Reinforcement Learning Based Driving with Black-Box Verification Algorithms 29 Oct 2022 · 2 repositories · arXiv:2210.16575
-
Spending Thinking Time Wisely: Accelerating MCTS with Virtual Expansions 23 Oct 2022 · 0 repositories · arXiv:2210.12628
-
Epistemic Monte Carlo Tree Search 21 Oct 2022 · 0 repositories · arXiv:2210.13455
-
Efficient Offline Policy Optimization with a Learned Model 12 Oct 2022 · 1 repository · arXiv:2210.05980
-
Human-level Atari 200x faster 15 Sep 2022 · 1 repository · arXiv:2209.07550Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)
-
Transformers are Sample-Efficient World Models 1 Sep 2022 · 2 repositories · arXiv:2209.00588Syntology official (archive's flag): 17 ran · 17 ran (of which 13 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 0 violated, 15 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified (of 26 harvested samples) · 26 pointer-only (licence)
-
An AlphaZero-Inspired Approach to Solving Search Problems 2 Jul 2022 · 0 repositories · arXiv:2207.00919
-
Automated Quantum Circuit Design with Nested Monte Carlo Tree Search 1 Jul 2022 · 1 repository · arXiv:2207.00132
-
Multi-Modal and Multi-Factor Branching Time Active Inference 24 Jun 2022 · 0 repositories · arXiv:2206.12503
-
Deep Reinforcement Learning for Exact Combinatorial Optimization: Learning to Branch 14 Jun 2022 · 0 repositories · arXiv:2206.06965
-
Combining Monte-Carlo Tree Search with Proof-Number Search 8 Jun 2022 · 0 repositories · arXiv:2206.03965
-
Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization 3 Jun 2022 · 1 repository · arXiv:2206.01409
-
Policy Gradient Algorithms with Monte Carlo Tree Learning for Non-Markov Decision Processes 2 Jun 2022 · 0 repositories · arXiv:2206.01011
-
Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control 27 May 2022 · 0 repositories · arXiv:2205.13836
-
Impartial Games: A Challenge for Reinforcement Learning 25 May 2022 · 1 repository · arXiv:2205.12787
-
Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods 25 Apr 2022 · 1 repository · arXiv:2204.11464
-
Reward Reports for Reinforcement Learning 22 Apr 2022 · 1 repository · arXiv:2204.10817
-
Machine Learning for CUDA+MPI Design Rules 4 Mar 2022 · 0 repositories · arXiv:2203.02530
-
BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs 17 Feb 2022 · 0 repositories · arXiv:2202.08884
-
MuZero with Self-competition for Rate Control in VP9 Video Compression 14 Feb 2022 · 0 repositories · arXiv:2202.06626
-
Split Moves for Monte-Carlo Tree Search 14 Dec 2021 · 2 repositories · arXiv:2112.07761
-
Generate plane quad mesh with neural networks and tree search 15 Nov 2021 · 1 repository · arXiv:2111.07613
-
Optimised Playout Implementations for the Ludii General Game System 4 Nov 2021 · 0 repositories · arXiv:2111.02839
-
Procedural Generalization by Planning with Self-Supervised World Models 2 Nov 2021 · 0 repositories · arXiv:2111.01587
-
Mastering Atari Games with Limited Data 30 Oct 2021 · 3 repositories · arXiv:2111.00210Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
Planning in Stochastic Environments with a Learned Model 29 Sep 2021 · 2 repositories
-
Policy improvement by planning with Gumbel 29 Sep 2021 · 2 repositories
-
Optimisation of MCTS Player for The Lord of the Rings: The Card Game 24 Sep 2021 · 0 repositories · arXiv:2109.12001
-
Proper Value Equivalence 18 Jun 2021 · 1 repository · arXiv:2106.10316
-
Vector Quantized Models for Planning 8 Jun 2021 · 0 repositories · arXiv:2106.04615
-
Learning and Planning in Complex Action Spaces 13 Apr 2021 · 1 repository · arXiv:2104.06303
-
Online and Offline Reinforcement Learning by Planning with a Learned Model 13 Apr 2021 · 2 repositories · arXiv:2104.06294
-
Machine Translation Decoding beyond Beam Search 12 Apr 2021 · 0 repositories · arXiv:2104.05336
-
Neural-Driven Search-Based Paraphrase Generation 1 Apr 2021 · 0 repositories
-
Accelerating Online Reinforcement Learning via Model-Based Meta-Learning 15 Mar 2021 · 0 repositories
-
Monte Carlo Scene Search for 3D Scene Understanding 14 Mar 2021 · 2 repositories · arXiv:2103.07969
-
Extended Tree Search for Robot Task and Motion Planning 9 Mar 2021 · 1 repository · arXiv:2103.05456
Tasks archive 2025-07-28
20 shown of 111 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections