{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/monte-carlo-tree-search","slug":"monte-carlo-tree-search","name":"Monte-Carlo Tree Search","full_name":"Monte-Carlo Tree Search","full_name_withheld":false,"description_markdown":"**Monte-Carlo Tree Search** is a planning algorithm that accumulates value estimates obtained from Monte Carlo simulations in order to successively direct simulations towards more highly-rewarded trajectories. We execute MCTS after encountering each new state to select an agent's action for that state: it is executed again to select the action for the next state. Each execution is an iterative process that simulates many trajectories starting from the current state to the terminal state. The core idea is to successively focus multiple simulations starting at the current state by extending the initial portions of trajectories that have received high evaluations from earlier simulations.\r\n\r\nSource: Sutton and Barto, Reinforcement Learning (2nd Edition)\r\n\r\nImage Credit: [Chaslot et al](https://www.aaai.org/Papers/AIIDE/2008/AIIDE08-036.pdf)","description_state":"present","introduced_year":2006,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Heuristic Search Algorithms","url":"/methods/category/heuristic-search-algorithms","pwc_aliases":[]}],"n_papers_tagged":166,"archive_num_papers":166,"papers_newest_first":[{"paper":null,"slug":"videe-visual-and-interactive-decomposition","title":"VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents","date":"2025-06-17","arxiv_id":"2506.21582","n_code_links":0,"syntology":null},{"paper":null,"slug":"calibrated-value-aware-model-learning-with","title":"Calibrated Value-Aware Model Learning with Stochastic Environment Models","date":"2025-05-28","arxiv_id":"2505.22772","n_code_links":0,"syntology":null},{"paper":null,"slug":"solving-general-utility-markov-decision","title":"Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planning","date":"2025-05-21","arxiv_id":"2505.15782","n_code_links":0,"syntology":null},{"paper":null,"slug":"adaptive-stress-testing-black-box-llm","title":"Adaptive Stress Testing Black-Box LLM Planners","date":"2025-05-08","arxiv_id":"2505.05665","n_code_links":0,"syntology":null},{"paper":"/paper/adaptive-branch-and-bound-tree-exploration","slug":"adaptive-branch-and-bound-tree-exploration","title":"Adaptive Branch-and-Bound Tree Exploration for Neural Network Verification","date":"2025-05-02","arxiv_id":"2505.00963","n_code_links":1,"syntology":null},{"paper":"/paper/trans-zero-self-play-incentivizes-large","slug":"trans-zero-self-play-incentivizes-large","title":"Trans-Zero: Self-Play Incentivizes Large Language Models for Multilingual Translation Without Parallel Data","date":"2025-04-20","arxiv_id":"2504.14669","n_code_links":1,"syntology":null},{"paper":"/paper/neural-guided-equation-discovery","slug":"neural-guided-equation-discovery","title":"Neural-Guided Equation Discovery","date":"2025-03-21","arxiv_id":"2503.16953","n_code_links":1,"syntology":null},{"paper":"/paper/a-neural-symbolic-model-for-space-physics","slug":"a-neural-symbolic-model-for-space-physics","title":"A Neural Symbolic Model for Space Physics","date":"2025-03-11","arxiv_id":"2503.07994","n_code_links":1,"syntology":null},{"paper":"/paper/optionzero-planning-with-learned-options","slug":"optionzero-planning-with-learned-options","title":"OptionZero: Planning with Learned Options","date":"2025-02-23","arxiv_id":"2502.16634","n_code_links":1,"syntology":{"ran":7,"of":8,"n_ran_checked":7,"n_instrument":0,"unverified":1,"pointer_only":8,"phrase":"7 ran (of which 7 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 7 samples that ran constructed an object rather than computing a result","official":{"repos":["rlglab/optionzero"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":7,"n_ran_no_instrument_failure":7,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"boost-disentangle-and-customize-a-robust","title":"Boost, Disentangle, and Customize: A Robust System2-to-System1 Pipeline for Code Generation","date":"2025-02-18","arxiv_id":"2502.12492","n_code_links":0,"syntology":null},{"paper":null,"slug":"reinforcement-learning-in-strategy-based-and","title":"Reinforcement Learning in Strategy-Based and Atari Games: A Review of Google DeepMinds Innovations","date":"2025-02-14","arxiv_id":"2502.10303","n_code_links":0,"syntology":null},{"paper":null,"slug":"evaluating-world-models-with-llm-for-decision","title":"Evaluating World Models with LLM for Decision Making","date":"2024-11-13","arxiv_id":"2411.08794","n_code_links":0,"syntology":null},{"paper":null,"slug":"evaluating-robustness-of-reinforcement","title":"Evaluating Robustness of Reinforcement Learning Algorithms for Autonomous Shipping","date":"2024-11-07","arxiv_id":"2411.04915","n_code_links":0,"syntology":null},{"paper":null,"slug":"interpreting-the-learned-model-in-muzero","title":"Interpreting the Learned Model in MuZero Planning","date":"2024-11-07","arxiv_id":"2411.04580","n_code_links":0,"syntology":null},{"paper":"/paper/human-aligned-chess-with-a-bit-of-search","slug":"human-aligned-chess-with-a-bit-of-search","title":"Human-aligned Chess with a Bit of Search","date":"2024-10-04","arxiv_id":"2410.03893","n_code_links":1,"syntology":null},{"paper":null,"slug":"zero-shot-multi-hop-question-answering-via","title":"Zero-Shot Multi-Hop Question Answering via Monte-Carlo Tree Search with Large Language Models","date":"2024-09-28","arxiv_id":"2409.19382","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-efficient-and-generalizable-symbolic","title":"An Efficient and Generalizable Symbolic Regression Method for Time Series Analysis","date":"2024-09-06","arxiv_id":"2409.03986","n_code_links":0,"syntology":null},{"paper":"/paper/deepseek-prover-v1-5-harnessing-proof","slug":"deepseek-prover-v1-5-harnessing-proof","title":"DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search","date":"2024-08-15","arxiv_id":"2408.08152","n_code_links":2,"syntology":{"ran":10,"of":10,"n_ran_checked":6,"n_instrument":4,"unverified":0,"pointer_only":0,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 0 violated, 4 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","official":{"repos":["deepseek-ai/deepseek-prover-v1.5"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"paper":null,"slug":"combining-ai-control-systems-and-human","title":"Combining AI Control Systems and Human Decision Support via Robustness and Criticality","date":"2024-07-03","arxiv_id":"2407.03210","n_code_links":0,"syntology":null},{"paper":null,"slug":"enhancements-for-real-time-monte-carlo-tree","title":"Enhancements for Real-Time Monte-Carlo Tree Search in General Video Game Playing","date":"2024-07-03","arxiv_id":"2407.03049","n_code_links":0,"syntology":null},{"paper":null,"slug":"towards-a-characterisation-of-monte-carlo","title":"Towards a Characterisation of Monte-Carlo Tree Search Performance in Different Games","date":"2024-06-13","arxiv_id":"2406.09242","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-monte-carlo-tree-search-via-on-the","slug":"efficient-monte-carlo-tree-search-via-on-the","title":"Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstraction","date":"2024-06-02","arxiv_id":"2406.00614","n_code_links":1,"syntology":null},{"paper":null,"slug":"global-rewards-in-restless-multi-armed","title":"Global Rewards in Restless Multi-Armed Bandits","date":"2024-06-02","arxiv_id":"2406.00738","n_code_links":0,"syntology":null},{"paper":null,"slug":"extreme-value-monte-carlo-tree-search","title":"Extreme Value Monte Carlo Tree Search","date":"2024-05-28","arxiv_id":"2405.18248","n_code_links":0,"syntology":null},{"paper":"/paper/pytag-tabletop-games-for-multi-agent","slug":"pytag-tabletop-games-for-multi-agent","title":"PyTAG: Tabletop Games for Multi-Agent Reinforcement Learning","date":"2024-05-28","arxiv_id":"2405.18123","n_code_links":1,"syntology":null},{"paper":"/paper/efficient-multi-agent-reinforcement-learning","slug":"efficient-multi-agent-reinforcement-learning","title":"Efficient Multi-agent Reinforcement Learning by Planning","date":"2024-05-20","arxiv_id":"2405.11778","n_code_links":1,"syntology":null},{"paper":"/paper/value-augmented-sampling-for-language-model","slug":"value-augmented-sampling-for-language-model","title":"Value Augmented Sampling for Language Model Alignment and Personalization","date":"2024-05-10","arxiv_id":"2405.06639","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["idanshen/Value-Augmented-Sampling"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/rezero-boosting-mcts-based-algorithms-by-just","slug":"rezero-boosting-mcts-based-algorithms-by-just","title":"ReZero: Boosting MCTS-based Algorithms by Backward-view and Entire-buffer Reanalyze","date":"2024-04-25","arxiv_id":"2404.16364","n_code_links":1,"syntology":null},{"paper":"/paper/tree-search-based-policy-optimization-under","slug":"tree-search-based-policy-optimization-under","title":"Tree Search-Based Policy Optimization under Stochastic Execution Delay","date":"2024-04-08","arxiv_id":"2404.05440","n_code_links":1,"syntology":{"ran":8,"of":8,"n_ran_checked":2,"n_instrument":6,"unverified":0,"pointer_only":8,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified","official":{"repos":["davidva1/delayed-ez"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"can-large-language-models-play-games-a-case","title":"Can Large Language Models Play Games? A Case Study of A Self-Play Approach","date":"2024-03-08","arxiv_id":"2403.05632","n_code_links":0,"syntology":null},{"paper":null,"slug":"retrieval-augmented-thought-process-as","title":"Retrieval Augmented Thought Process for Private Data Handling in Healthcare","date":"2024-02-12","arxiv_id":"2402.07812","n_code_links":0,"syntology":null},{"paper":"/paper/the-strain-of-success-a-predictive-model-for","slug":"the-strain-of-success-a-predictive-model-for","title":"The Strain of Success: A Predictive Model for Injury Risk Mitigation and Team Success in Soccer","date":"2024-02-07","arxiv_id":"2402.04898","n_code_links":1,"syntology":null},{"paper":"/paper/checkmating-one-by-using-many-combining","slug":"checkmating-one-by-using-many-combining","title":"Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chess","date":"2024-01-30","arxiv_id":"2401.16852","n_code_links":1,"syntology":null},{"paper":null,"slug":"learning-backdoors-for-mixed-integer-programs","title":"Learning Backdoors for Mixed Integer Linear Programs with Contrastive Learning","date":"2024-01-19","arxiv_id":"2401.10467","n_code_links":0,"syntology":null},{"paper":null,"slug":"scheduling-distributed-flexible-assembly","title":"Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding","date":"2023-11-21","arxiv_id":"2311.12572","n_code_links":0,"syntology":null},{"paper":"/paper/minizero-comparative-analysis-of-alphazero","slug":"minizero-comparative-analysis-of-alphazero","title":"MiniZero: Comparative Analysis of AlphaZero and MuZero on Go, Othello, and Atari Games","date":"2023-10-17","arxiv_id":"2310.11305","n_code_links":1,"syntology":{"ran":6,"of":7,"n_ran_checked":6,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["rlglab/minizero"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"alpha-elimination-using-deep-reinforcement","title":"Alpha Elimination: Using Deep Reinforcement Learning to Reduce Fill-In during Sparse Matrix Decomposition","date":"2023-10-15","arxiv_id":"2310.09852","n_code_links":0,"syntology":null},{"paper":null,"slug":"accelerating-monte-carlo-tree-search-with","title":"Accelerating Monte Carlo Tree Search with Probability Tree State Abstraction","date":"2023-10-10","arxiv_id":"2310.06513","n_code_links":0,"syntology":null},{"paper":null,"slug":"rollout-heuristics-for-online-stochastic","title":"Rollout Heuristics for Online Stochastic Contingent Planning","date":"2023-10-03","arxiv_id":"2310.02345","n_code_links":0,"syntology":null},{"paper":null,"slug":"making-ppo-even-better-value-guided-monte","title":"Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding","date":"2023-09-26","arxiv_id":"2309.15028","n_code_links":0,"syntology":null},{"paper":null,"slug":"pomdp-planning-for-object-search-in-partially","title":"POMDP Planning for Object Search in Partially Unknown Environment","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"re-on-explainability-of-graph-neural-networks","title":"[Re] On Explainability of Graph Neural Networks via Subgraph Explorations","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"self-predictive-universal-ai","title":"Self-Predictive Universal AI","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"monte-carlo-tree-search-with-uncertainty","title":"Monte-Carlo tree search with uncertainty propagation via optimal transport","date":"2023-09-19","arxiv_id":"2309.10737","n_code_links":0,"syntology":null},{"paper":null,"slug":"chain-of-thought-reasoning-is-a-policy","title":"Chain-of-Thought Reasoning is a Policy Improvement Operator","date":"2023-09-15","arxiv_id":"2309.08589","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-and-explainable-graph-neural","slug":"efficient-and-explainable-graph-neural","title":"Efficient and Explainable Graph Neural Architecture Search via Monte-Carlo Tree Search","date":"2023-08-30","arxiv_id":"2308.15734","n_code_links":1,"syntology":null},{"paper":null,"slug":"jiangjun-mastering-xiangqi-by-tackling-non","title":"JiangJun: Mastering Xiangqi by Tackling Non-Transitivity in Two-Player Zero-Sum Games","date":"2023-08-09","arxiv_id":"2308.04719","n_code_links":0,"syntology":null},{"paper":"/paper/alphastar-unplugged-large-scale-offline","slug":"alphastar-unplugged-large-scale-offline","title":"AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning","date":"2023-08-07","arxiv_id":"2308.03526","n_code_links":1,"syntology":{"ran":4,"of":4,"n_ran_checked":4,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":null}},{"paper":null,"slug":"monte-carlo-tree-search-for-multi-agent","title":"Monte-Carlo Tree Search for Multi-Agent Pathfinding: Preliminary Results","date":"2023-07-25","arxiv_id":"2307.13453","n_code_links":0,"syntology":null},{"paper":null,"slug":"on-robot-bayesian-reinforcement-learning-for","title":"On-Robot Bayesian Reinforcement Learning for POMDPs","date":"2023-07-22","arxiv_id":"2307.11954","n_code_links":0,"syntology":null},{"paper":null,"slug":"reinforcement-learning-for-syntax-guided","title":"Reinforcement Learning and Data-Generation for Syntax-Guided Synthesis","date":"2023-07-13","arxiv_id":"2307.09564","n_code_links":0,"syntology":null},{"paper":null,"slug":"l-ac-learning-latent-decision-aware-models","title":"$λ$-models: Effective Decision-Aware Reinforcement Learning with Latent Models","date":"2023-06-30","arxiv_id":"2306.17366","n_code_links":0,"syntology":null},{"paper":"/paper/combining-a-meta-policy-and-monte-carlo","slug":"combining-a-meta-policy-and-monte-carlo","title":"Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments","date":"2023-06-09","arxiv_id":"2306.06067","n_code_links":1,"syntology":null},{"paper":null,"slug":"habits-of-mind-reusing-action-sequences-for","title":"Habits of Mind: Reusing Action Sequences for Efficient Planning","date":"2023-06-08","arxiv_id":"2306.05298","n_code_links":0,"syntology":null},{"paper":null,"slug":"what-model-does-muzero-learn","title":"What model does MuZero learn?","date":"2023-06-01","arxiv_id":"2306.00840","n_code_links":0,"syntology":null},{"paper":"/paper/know-your-enemy-investigating-monte-carlo","slug":"know-your-enemy-investigating-monte-carlo","title":"Know your Enemy: Investigating Monte-Carlo Tree Search with Opponent Models in Pommerman","date":"2023-05-22","arxiv_id":"2305.13206","n_code_links":1,"syntology":null},{"paper":"/paper/model-predictive-control-with-self-supervised","slug":"model-predictive-control-with-self-supervised","title":"Model Predictive Control with Self-supervised Representation Learning","date":"2023-04-14","arxiv_id":"2304.07219","n_code_links":1,"syntology":null},{"paper":"/paper/proof-number-based-monte-carlo-tree-search","slug":"proof-number-based-monte-carlo-tree-search","title":"Proof Number Based Monte-Carlo Tree Search","date":"2023-03-16","arxiv_id":"2303.09449","n_code_links":1,"syntology":null},{"paper":null,"slug":"a-parallel-monte-carlo-tree-search-based","title":"A Parallel Monte-Carlo Tree Search-Based Metaheuristic For Optimal Fleet Composition Considering Vehicle Routing Using Branch & Bound","date":"2023-03-06","arxiv_id":"2303.03156","n_code_links":0,"syntology":null},{"paper":null,"slug":"deconstructing-deep-active-inference","title":"Deconstructing deep active inference","date":"2023-03-02","arxiv_id":"2303.01618","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-generative-symbolic-regression-with","title":"Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search","date":"2023-02-22","arxiv_id":"2302.11223","n_code_links":0,"syntology":null},{"paper":null,"slug":"equivariant-muzero","title":"Equivariant MuZero","date":"2023-02-09","arxiv_id":"2302.04798","n_code_links":0,"syntology":null},{"paper":null,"slug":"monte-carlo-tree-search-for-leveraging","title":"Monte-Carlo Tree-Search for Leveraging Performance of Blackbox Job-Shop Scheduling Heuristics","date":"2022-12-14","arxiv_id":"2212.07543","n_code_links":0,"syntology":null},{"paper":"/paper/self-improving-safety-performance-of","slug":"self-improving-safety-performance-of","title":"Self-Improving Safety Performance of Reinforcement Learning Based Driving with Black-Box Verification Algorithms","date":"2022-10-29","arxiv_id":"2210.16575","n_code_links":2,"syntology":null},{"paper":null,"slug":"spending-thinking-time-wisely-accelerating-1","title":"Spending Thinking Time Wisely: Accelerating MCTS with Virtual Expansions","date":"2022-10-23","arxiv_id":"2210.12628","n_code_links":0,"syntology":null},{"paper":null,"slug":"planning-with-uncertainty-deep-exploration-in","title":"Epistemic Monte Carlo Tree Search","date":"2022-10-21","arxiv_id":"2210.13455","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-offline-policy-optimization-with-a","slug":"efficient-offline-policy-optimization-with-a","title":"Efficient Offline Policy Optimization with a Learned Model","date":"2022-10-12","arxiv_id":"2210.05980","n_code_links":1,"syntology":null},{"paper":"/paper/human-level-atari-200x-faster","slug":"human-level-atari-200x-faster","title":"Human-level Atari 200x faster","date":"2022-09-15","arxiv_id":"2209.07550","n_code_links":1,"syntology":{"ran":1,"of":3,"n_ran_checked":0,"n_instrument":1,"unverified":2,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified","official":null}},{"paper":"/paper/transformers-are-sample-efficient-world","slug":"transformers-are-sample-efficient-world","title":"Transformers are Sample-Efficient World Models","date":"2022-09-01","arxiv_id":"2209.00588","n_code_links":2,"syntology":{"ran":17,"of":26,"n_ran_checked":16,"n_instrument":1,"unverified":9,"pointer_only":26,"phrase":"17 ran (of which 13 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 0 violated, 15 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified","official":{"repos":["eloialonso/iris"],"state":"official (archive's flag): 17 ran","n_ran":17,"n_constructed":13,"n_ran_no_instrument_failure":16,"n_unverified":9,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"an-alphazero-inspired-approach-to-solving","title":"An AlphaZero-Inspired Approach to Solving Search Problems","date":"2022-07-02","arxiv_id":"2207.00919","n_code_links":0,"syntology":null},{"paper":"/paper/automated-quantum-circuit-design-with-nested","slug":"automated-quantum-circuit-design-with-nested","title":"Automated Quantum Circuit Design with Nested Monte Carlo Tree Search","date":"2022-07-01","arxiv_id":"2207.00132","n_code_links":1,"syntology":null},{"paper":null,"slug":"multi-modal-and-multi-factor-branching-time","title":"Multi-Modal and Multi-Factor Branching Time Active Inference","date":"2022-06-24","arxiv_id":"2206.12503","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-reinforcement-learning-for-exact","title":"Deep Reinforcement Learning for Exact Combinatorial Optimization: Learning to Branch","date":"2022-06-14","arxiv_id":"2206.06965","n_code_links":0,"syntology":null},{"paper":null,"slug":"combining-monte-carlo-tree-search-with-proof","title":"Combining Monte-Carlo Tree Search with Proof-Number Search","date":"2022-06-08","arxiv_id":"2206.03965","n_code_links":0,"syntology":null},{"paper":"/paper/hybrid-models-for-mixed-variables-in-bayesian","slug":"hybrid-models-for-mixed-variables-in-bayesian","title":"Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization","date":"2022-06-03","arxiv_id":"2206.01409","n_code_links":1,"syntology":null},{"paper":null,"slug":"policy-gradient-algorithms-with-monte-carlo","title":"Policy Gradient Algorithms with Monte Carlo Tree Learning for Non-Markov Decision Processes","date":"2022-06-02","arxiv_id":"2206.01011","n_code_links":0,"syntology":null},{"paper":null,"slug":"feudal-multi-agent-reinforcement-learning","title":"Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control","date":"2022-05-27","arxiv_id":"2205.13836","n_code_links":0,"syntology":null},{"paper":"/paper/impartial-games-a-challenge-for-reinforcement","slug":"impartial-games-a-challenge-for-reinforcement","title":"Impartial Games: A Challenge for Reinforcement Learning","date":"2022-05-25","arxiv_id":"2205.12787","n_code_links":1,"syntology":null},{"paper":"/paper/towards-evaluating-adaptivity-of-model-based","slug":"towards-evaluating-adaptivity-of-model-based","title":"Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods","date":"2022-04-25","arxiv_id":"2204.11464","n_code_links":1,"syntology":null},{"paper":"/paper/reward-reports-for-reinforcement-learning","slug":"reward-reports-for-reinforcement-learning","title":"Reward Reports for Reinforcement Learning","date":"2022-04-22","arxiv_id":"2204.10817","n_code_links":1,"syntology":null},{"paper":null,"slug":"machine-learning-for-cuda-mpi-design-rules","title":"Machine Learning for CUDA+MPI Design Rules","date":"2022-03-04","arxiv_id":"2203.02530","n_code_links":0,"syntology":null},{"paper":null,"slug":"baddr-bayes-adaptive-deep-dropout-rl-for","title":"BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs","date":"2022-02-17","arxiv_id":"2202.08884","n_code_links":0,"syntology":null},{"paper":null,"slug":"muzero-with-self-competition-for-rate-control","title":"MuZero with Self-competition for Rate Control in VP9 Video Compression","date":"2022-02-14","arxiv_id":"2202.06626","n_code_links":0,"syntology":null},{"paper":"/paper/split-moves-for-monte-carlo-tree-search","slug":"split-moves-for-monte-carlo-tree-search","title":"Split Moves for Monte-Carlo Tree Search","date":"2021-12-14","arxiv_id":"2112.07761","n_code_links":2,"syntology":null},{"paper":"/paper/generate-plane-quad-mesh-with-neural-networks","slug":"generate-plane-quad-mesh-with-neural-networks","title":"Generate plane quad mesh with neural networks and tree search","date":"2021-11-15","arxiv_id":"2111.07613","n_code_links":1,"syntology":null},{"paper":null,"slug":"optimised-playout-implementations-for-the","title":"Optimised Playout Implementations for the Ludii General Game System","date":"2021-11-04","arxiv_id":"2111.02839","n_code_links":0,"syntology":null},{"paper":"/paper/procedural-generalization-by-planning-with-1","slug":"procedural-generalization-by-planning-with-1","title":"Procedural Generalization by Planning with Self-Supervised World Models","date":"2021-11-02","arxiv_id":"2111.01587","n_code_links":0,"syntology":null},{"paper":"/paper/mastering-atari-games-with-limited-data","slug":"mastering-atari-games-with-limited-data","title":"Mastering Atari Games with Limited Data","date":"2021-10-30","arxiv_id":"2111.00210","n_code_links":3,"syntology":{"ran":3,"of":3,"n_ran_checked":1,"n_instrument":2,"unverified":0,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["werner-duvaud/muzero-general","yewr/efficientzero"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/planning-in-stochastic-environments-with-a","slug":"planning-in-stochastic-environments-with-a","title":"Planning in Stochastic Environments with a Learned Model","date":"2021-09-29","arxiv_id":null,"n_code_links":2,"syntology":null},{"paper":"/paper/policy-improvement-by-planning-with-gumbel","slug":"policy-improvement-by-planning-with-gumbel","title":"Policy improvement by planning with Gumbel","date":"2021-09-29","arxiv_id":null,"n_code_links":2,"syntology":null},{"paper":null,"slug":"optimisation-of-mcts-player-for-the-lord-of","title":"Optimisation of MCTS Player for The Lord of the Rings: The Card Game","date":"2021-09-24","arxiv_id":"2109.12001","n_code_links":0,"syntology":null},{"paper":"/paper/proper-value-equivalence","slug":"proper-value-equivalence","title":"Proper Value Equivalence","date":"2021-06-18","arxiv_id":"2106.10316","n_code_links":1,"syntology":null},{"paper":null,"slug":"vector-quantized-models-for-planning","title":"Vector Quantized Models for Planning","date":"2021-06-08","arxiv_id":"2106.04615","n_code_links":0,"syntology":null},{"paper":"/paper/learning-and-planning-in-complex-action","slug":"learning-and-planning-in-complex-action","title":"Learning and Planning in Complex Action Spaces","date":"2021-04-13","arxiv_id":"2104.06303","n_code_links":1,"syntology":null},{"paper":"/paper/online-and-offline-reinforcement-learning-by","slug":"online-and-offline-reinforcement-learning-by","title":"Online and Offline Reinforcement Learning by Planning with a Learned Model","date":"2021-04-13","arxiv_id":"2104.06294","n_code_links":2,"syntology":null},{"paper":null,"slug":"machine-translation-decoding-beyond-beam","title":"Machine Translation Decoding beyond Beam Search","date":"2021-04-12","arxiv_id":"2104.05336","n_code_links":0,"syntology":null},{"paper":null,"slug":"neural-driven-search-based-paraphrase","title":"Neural-Driven Search-Based Paraphrase Generation","date":"2021-04-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"accelerating-online-reinforcement-learning-1","title":"Accelerating Online Reinforcement Learning via Model-Based Meta-Learning","date":"2021-03-15","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/monte-carlo-scene-search-for-3d-scene","slug":"monte-carlo-scene-search-for-3d-scene","title":"Monte Carlo Scene Search for 3D Scene Understanding","date":"2021-03-14","arxiv_id":"2103.07969","n_code_links":2,"syntology":null},{"paper":"/paper/extended-task-and-motion-planning-of-long","slug":"extended-task-and-motion-planning-of-long","title":"Extended Tree Search for Robot Task and Motion Planning","date":"2021-03-09","arxiv_id":"2103.05456","n_code_links":1,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":166,"with_a_code_link":62,"where_syntology_ran_a_sample":12,"not_listed_spam_title":0,"listed":166,"listed_where_code_ran":12,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":10,"every_run_a_failure_of_syntologys_instrument":2,"listed_with_a_run_with_no_instrument_failure":10,"listed_every_run_a_failure_of_syntologys_instrument":2,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":12,"papers_where_code_ran_first_page":"/method/monte-carlo-tree-search/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/monte-carlo-tree-search/papers/2","json":"/method/monte-carlo-tree-search/papers/2.json","rows":66},{"order":"ran","page":1,"of":1,"url":"/method/monte-carlo-tree-search/papers/ran/1","json":"/method/monte-carlo-tree-search/papers/ran/1.json","rows":12}],"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":47},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":45},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":44},{"task":"/task/model-based-reinforcement-learning","name":"Model-based Reinforcement Learning","papers":21},{"task":"/task/decision-making","name":"Decision Making","papers":20},{"task":"/task/board-games","name":"Board Games","papers":16},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":14},{"task":"/task/atari-games","name":"Atari Games","papers":12},{"task":"/task/continuous-control","name":"Continuous Control","papers":6},{"task":"/task/game-of-go","name":"Game of Go","papers":6},{"task":"/task/continuous-control","name":"continuous-control","papers":5},{"task":"/task/model","name":"model","papers":5},{"task":"/task/sequential-decision-making","name":"Sequential Decision Making","papers":4},{"task":"/task/atari-games-100k","name":"Atari Games 100k","papers":3},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":3},{"task":"/task/bayesian-optimization","name":"Bayesian Optimization","papers":3},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":3},{"task":"/task/general-reinforcement-learning","name":"General Reinforcement Learning","papers":3},{"task":"/task/heuristic-search","name":"Heuristic Search","papers":3},{"task":"/task/machine-translation","name":"Machine Translation","papers":3}],"tasks_shown":20,"n_tasks":111,"usage_by_year":[{"year":"2009","papers":1},{"year":"2010","papers":2},{"year":"2013","papers":2},{"year":"2014","papers":1},{"year":"2015","papers":1},{"year":"2017","papers":2},{"year":"2018","papers":8},{"year":"2019","papers":13},{"year":"2020","papers":29},{"year":"2021","papers":24},{"year":"2022","papers":21},{"year":"2023","papers":28},{"year":"2024","papers":23},{"year":"2025","papers":11}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/monte-carlo-tree-search"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Monte-Carlo%20Tree%20Search","read_at":"2026-10-05T00:59:18+00:00","query":"Monte-Carlo Tree Search","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"Monte Carlo tree search","origin_arxiv_id":"2010.12069"},"resolution":"token_match","resolved_name_differs":true,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":1,"in_graph":1,"list_truncated":false,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"2010.12069","citations":6},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'Monte-Carlo Tree Search'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[{"name":"SPO-tree","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='SPO-tree')","origin_arxiv_id":"2505.23564"},{"name":"SP-SEARCH","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='SP-SEARCH')","origin_arxiv_id":"2209.10492"},{"name":"Tree of Thoughts","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Tree of Thoughts')","origin_arxiv_id":"2305.10601"},{"name":"back-search algorithm","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='back-search algorithm')","origin_arxiv_id":"2006.06649"},{"name":"δ-Conservative Search","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='δ-Conservative Search')","origin_arxiv_id":"2410.04461"},{"name":"local search strategy","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='local search strategy')","origin_arxiv_id":"2201.05666"}],"code_nearby_caption":"Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.","origin_paper":{"arxiv_id":"2010.12069","page":"/paper/improving-policy-constrained-kidney-exchange","syntology_url":"https://syntology.ai/paper/2010.12069","in_graph_layer":true,"samples":1,"ran":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"for a different name, 'Monte Carlo tree search': repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: official). The lookup matched a different name in Syntology's graph, 'Monte Carlo tree search' (resolution: token_match); this answer is about that name. The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2010.12069) is about this method is not checked.","level_legend":{"state":"quoted","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","card_sha256":"479785a84658dd682a11b10237b3620d651d7f87799f5dc7add970aaa83f6d39","note":"Syntology's served verification levels, quoted verbatim from its MCP server card; a row is split where the card separates clauses with a double hyphen, and the separator is not reproduced","levels":[{"level":"V0","quoted":["imports"]},{"level":"V1","quoted":["executes against a synthesized fixture and returns a value of the recorded type"]},{"level":"V2","quoted":["two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs","neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)"]},{"level":"V3","quoted":["paper-derived property tests pass on both of those same two generations."]}],"no_rung":"No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results."}},"syntology_have_state":"loaded","record_sha256":"b1e74b7563cae09e1bb7bcf0839d9d72751796b75aa6eac51244c6a8addba0a8","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}