{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/task/off-policy-evaluation","name":"Off-policy evaluation","slug":"off-policy-evaluation","description_markdown":"Off-policy Evaluation (OPE), or offline evaluation in general, evaluates the performance of hypothetical policies leveraging only offline log data. It is particularly useful in applications where the online interaction involves high stakes and expensive setting such as precision medicine and recommender systems.","categories":[{"name":"Computer Code","url":"/area/computer-code"},{"name":"Computer Vision","url":"/area/computer-vision"},{"name":"Knowledge Base","url":"/area/knowledge-base"},{"name":"Methodology","url":"/area/methodology"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"derived"},"counts":{"papers_tagged":265,"papers_with_code":102,"benchmarks":0,"benchmark_tables_in_archive":0,"benchmark_tables_shown":0,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":0,"subtasks":0,"parent_tasks":1},"benchmarks":[],"datasets":[],"subtasks":[],"parent_tasks":[{"url":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)"}],"papers":{"order":"repositories listed in the archive (most first), then date (newest first), then slug; the archive holds no stars","population":"every paper the archive tags with this task (distinct papers; spam titles not listed). 'of' counts the papers listed; until 2026-09-25 it counted only the papers with a code link, and that number is of_with_code","page":1,"pages_in_order":3,"rows_per_page":100,"shown":100,"of":265,"of_with_code":102,"tagged_in_all":265,"counts":{"archive_papers_tagged":265,"with_a_code_link":102,"where_syntology_ran_a_sample":40,"not_listed_spam_title":0,"listed":265,"listed_where_code_ran":40,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":30,"every_run_a_failure_of_syntologys_instrument":10,"listed_with_a_run_with_no_instrument_failure":30,"listed_every_run_a_failure_of_syntologys_instrument":10,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"items":[{"url":"/paper/a-large-scale-open-dataset-for-bandit","slug":"a-large-scale-open-dataset-for-bandit","title":"Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation","date":"2020-08-17","arxiv_id":"2008.07146","repositories_listed":4,"syntology":{"n":3,"n_ran":3,"n_constructed":0,"n_ran_checked":2,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/a-large-scale-open-dataset-for-bandit#ran","syntology_url":"https://syntology.ai/paper/2008.07146","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.07146"}},"official":{"repos":["st-tech/zr-obp"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/off-policy-evaluation-for-large-action-spaces","slug":"off-policy-evaluation-for-large-action-spaces","title":"Off-Policy Evaluation for Large Action Spaces via Embeddings","date":"2022-02-13","arxiv_id":"2202.06317","repositories_listed":3,"syntology":{"n":3,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/off-policy-evaluation-for-large-action-spaces#ran","syntology_url":"https://syntology.ai/paper/2202.06317","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.06317"}},"official":{"repos":["st-tech/zr-obp","usaito/icml2022-mips"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/benchmarks-for-deep-off-policy-evaluation-1","slug":"benchmarks-for-deep-off-policy-evaluation-1","title":"Benchmarks for Deep Off-Policy Evaluation","date":"2021-03-30","arxiv_id":"2103.16596","repositories_listed":3,"syntology":{"n":3,"n_ran":3,"n_constructed":0,"n_ran_checked":0,"n_instrument":3,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":3,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/benchmarks-for-deep-off-policy-evaluation-1#ran","syntology_url":"https://syntology.ai/paper/2103.16596","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.16596"}},"official":{"repos":["google-research/deep_ope"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/optimal-and-adaptive-off-policy-evaluation-in","slug":"optimal-and-adaptive-off-policy-evaluation-in","title":"Optimal and Adaptive Off-policy Evaluation in Contextual Bandits","date":"2016-12-04","arxiv_id":"1612.01205","repositories_listed":3,"syntology":null},{"url":"/paper/balanced-off-policy-evaluation-for","slug":"balanced-off-policy-evaluation-for","title":"Balanced Off-Policy Evaluation for Personalized Pricing","date":"2023-02-24","arxiv_id":"2302.12736","repositories_listed":2,"syntology":null},{"url":"/paper/doubly-robust-off-policy-evaluation-for","slug":"doubly-robust-off-policy-evaluation-for","title":"Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model","date":"2022-02-03","arxiv_id":"2202.01562","repositories_listed":2,"syntology":null},{"url":"/paper/evaluating-the-robustness-of-off-policy","slug":"evaluating-the-robustness-of-off-policy","title":"Evaluating the Robustness of Off-Policy Evaluation","date":"2021-08-31","arxiv_id":"2108.13703","repositories_listed":2,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/evaluating-the-robustness-of-off-policy#ran","syntology_url":"https://syntology.ai/paper/2108.13703","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.13703"}},"official":{"repos":["st-tech/zr-obp","sony/pyieoe"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/robust-generalization-despite-distribution","slug":"robust-generalization-despite-distribution","title":"Robust Generalization despite Distribution Shift via Minimum Discriminating Information","date":"2021-06-08","arxiv_id":"2106.04443","repositories_listed":2,"syntology":null},{"url":"/paper/confident-off-policy-evaluation-and-selection","slug":"confident-off-policy-evaluation-and-selection","title":"Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting","date":"2020-06-18","arxiv_id":"2006.10460","repositories_listed":2,"syntology":{"n":5,"n_ran":4,"n_constructed":0,"n_ran_checked":4,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/confident-off-policy-evaluation-and-selection#ran","syntology_url":"https://syntology.ai/paper/2006.10460","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.10460"}},"official":{"repos":["deepmind/offpolicy_selection_eslb"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/intrinsically-efficient-stable-and-bounded","slug":"intrinsically-efficient-stable-and-bounded","title":"Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning","date":"2019-06-09","arxiv_id":"1906.03735","repositories_listed":2,"syntology":null},{"url":"/paper/dolce-decomposing-off-policy-evaluation","slug":"dolce-decomposing-off-policy-evaluation","title":"DOLCE: Decomposing Off-Policy Evaluation/Learning into Lagged and Current Effects","date":"2025-05-02","arxiv_id":"2505.00961","repositories_listed":1,"syntology":null},{"url":"/paper/trajectory-world-models-for-heterogeneous","slug":"trajectory-world-models-for-heterogeneous","title":"Trajectory World Models for Heterogeneous Environments","date":"2025-02-03","arxiv_id":"2502.01366","repositories_listed":1,"syntology":null},{"url":"/paper/two-way-deconfounder-for-off-policy","slug":"two-way-deconfounder-for-off-policy","title":"Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning","date":"2024-12-08","arxiv_id":"2412.05783","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_constructed":0,"n_ran_checked":3,"n_instrument":0,"n_unverified":1,"n_honours":3,"n_violates":0,"n_no_contract":0,"n_pointer_only":4,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/two-way-deconfounder-for-off-policy#ran","syntology_url":"https://syntology.ai/paper/2412.05783","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.05783"}},"official":{"repos":["fsmiu/Two-way-Deconfounder"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/minimum-empirical-divergence-for-sub-gaussian","slug":"minimum-empirical-divergence-for-sub-gaussian","title":"Minimum Empirical Divergence for Sub-Gaussian Linear Bandits","date":"2024-10-31","arxiv_id":"2411.00229","repositories_listed":1,"syntology":null},{"url":"/paper/abstract-reward-processes-leveraging-state","slug":"abstract-reward-processes-leveraging-state","title":"Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluation","date":"2024-10-03","arxiv_id":"2410.02172","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":1,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":1,"phrase":"1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result","sample_list":"/paper/abstract-reward-processes-leveraging-state#ran","syntology_url":"https://syntology.ai/paper/2410.02172","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.02172"}},"official":{"repos":["shreyasc-13/star"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/causal-deepsets-for-off-policy-evaluation","slug":"causal-deepsets-for-off-policy-evaluation","title":"Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferences","date":"2024-07-25","arxiv_id":"2407.17910","repositories_listed":1,"syntology":null},{"url":"/paper/forward-and-backward-state-abstractions-for","slug":"forward-and-backward-state-abstractions-for","title":"Off-policy Evaluation with Deeply-abstracted States","date":"2024-06-27","arxiv_id":"2406.19531","repositories_listed":1,"syntology":null},{"url":"/paper/kernel-metric-learning-for-in-sample-off","slug":"kernel-metric-learning-for-in-sample-off","title":"Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies","date":"2024-05-29","arxiv_id":"2405.18792","repositories_listed":1,"syntology":{"n":8,"n_ran":5,"n_constructed":2,"n_ran_checked":2,"n_instrument":3,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":1,"phrase":"5 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/kernel-metric-learning-for-in-sample-off#ran","syntology_url":"https://syntology.ai/paper/2405.18792","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18792"}},"official":{"repos":["haanvid/kmifqe"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":2,"n_ran_no_instrument_failure":2,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/cross-validated-off-policy-evaluation","slug":"cross-validated-off-policy-evaluation","title":"Cross-Validated Off-Policy Evaluation","date":"2024-05-24","arxiv_id":"2405.15332","repositories_listed":1,"syntology":null},{"url":"/paper/logarithmic-smoothing-for-pessimistic-off","slug":"logarithmic-smoothing-for-pessimistic-off","title":"Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning","date":"2024-05-23","arxiv_id":"2405.14335","repositories_listed":1,"syntology":{"n":2,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":2,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/logarithmic-smoothing-for-pessimistic-off#ran","syntology_url":"https://syntology.ai/paper/2405.14335","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14335"}},"official":{"repos":["otmhi/offpolicy_ls"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/long-term-off-policy-evaluation-and-learning","slug":"long-term-off-policy-evaluation-and-learning","title":"Long-term Off-Policy Evaluation and Learning","date":"2024-04-24","arxiv_id":"2404.15691","repositories_listed":1,"syntology":null},{"url":"/paper/predictive-performance-comparison-of-decision","slug":"predictive-performance-comparison-of-decision","title":"Predictive Performance Comparison of Decision Policies Under Confounding","date":"2024-04-01","arxiv_id":"2404.00848","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_constructed":0,"n_ran_checked":0,"n_instrument":3,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":4,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/predictive-performance-comparison-of-decision#ran","syntology_url":"https://syntology.ai/paper/2404.00848","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.00848"}},"official":{"repos":["lguerdan/icml24_predictive_performance_comparison_dps"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/efficient-and-sharp-off-policy-evaluation-in","slug":"efficient-and-sharp-off-policy-evaluation-in","title":"Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes","date":"2024-03-29","arxiv_id":"2404.00099","repositories_listed":1,"syntology":{"n":3,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":3,"phrase":"0 ran · 3 unverified","sample_list":"/paper/efficient-and-sharp-off-policy-evaluation-in#ran","syntology_url":"https://syntology.ai/paper/2404.00099","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.00099"}},"official":{"repos":["causalml/adversarial-ope"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":3,"ran_from_kinds":[]}}},{"url":"/paper/off-policy-evaluation-of-slate-bandit","slug":"off-policy-evaluation-of-slate-bandit","title":"Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstraction","date":"2024-02-03","arxiv_id":"2402.02171","repositories_listed":1,"syntology":null},{"url":"/paper/distributional-off-policy-evaluation-with","slug":"distributional-off-policy-evaluation-with","title":"Distributional Off-policy Evaluation with Bellman Residual Minimization","date":"2024-02-02","arxiv_id":"2402.01900","repositories_listed":1,"syntology":null},{"url":"/paper/debiased-machine-learning-and-network","slug":"debiased-machine-learning-and-network","title":"RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions","date":"2023-12-11","arxiv_id":"2312.06403","repositories_listed":1,"syntology":null},{"url":"/paper/marginal-density-ratio-for-off-policy-1","slug":"marginal-density-ratio-for-off-policy-1","title":"Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits","date":"2023-12-03","arxiv_id":"2312.01457","repositories_listed":1,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":1,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/marginal-density-ratio-for-off-policy-1#ran","syntology_url":"https://syntology.ai/paper/2312.01457","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.01457"}},"official":{"repos":["faaizt/mr-ope"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/scope-rl-a-python-library-for-offline","slug":"scope-rl-a-python-library-for-offline","title":"SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation","date":"2023-11-30","arxiv_id":"2311.18206","repositories_listed":1,"syntology":null},{"url":"/paper/towards-assessing-and-benchmarking-risk","slug":"towards-assessing-and-benchmarking-risk","title":"Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluation","date":"2023-11-30","arxiv_id":"2311.18207","repositories_listed":1,"syntology":null},{"url":"/paper/when-is-off-policy-evaluation-useful-a-data","slug":"when-is-off-policy-evaluation-useful-a-data","title":"When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspective","date":"2023-11-23","arxiv_id":"2311.14110","repositories_listed":1,"syntology":{"n":6,"n_ran":5,"n_constructed":0,"n_ran_checked":5,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":6,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/when-is-off-policy-evaluation-useful-a-data#ran","syntology_url":"https://syntology.ai/paper/2311.14110","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.14110"}},"official":{"repos":["holarissun/Data-Centric-OPE"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/robust-offline-policy-evaluation-and","slug":"robust-offline-policy-evaluation-and","title":"Robust Offline Reinforcement learning with Heavy-Tailed Rewards","date":"2023-10-28","arxiv_id":"2310.18715","repositories_listed":1,"syntology":{"n":4,"n_ran":4,"n_constructed":0,"n_ran_checked":3,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":3,"n_pointer_only":4,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/robust-offline-policy-evaluation-and#ran","syntology_url":"https://syntology.ai/paper/2310.18715","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.18715"}},"official":{"repos":["mamba413/room"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/state-action-similarity-based-representations-1","slug":"state-action-similarity-based-representations-1","title":"State-Action Similarity-Based Representations for Off-Policy Evaluation","date":"2023-10-27","arxiv_id":"2310.18409","repositories_listed":1,"syntology":{"n":9,"n_ran":6,"n_constructed":3,"n_ran_checked":6,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":1,"n_no_contract":5,"n_pointer_only":9,"phrase":"6 ran (of which 3 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/state-action-similarity-based-representations-1#ran","syntology_url":"https://syntology.ai/paper/2310.18409","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.18409"}},"official":{"repos":["badger-rl/rope"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":3,"n_ran_no_instrument_failure":6,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/counterfactual-augmented-importance-sampling-1","slug":"counterfactual-augmented-importance-sampling-1","title":"Counterfactual-Augmented Importance Sampling for Semi-Offline Policy Evaluation","date":"2023-10-26","arxiv_id":"2310.17146","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/counterfactual-augmented-importance-sampling-1#ran","syntology_url":"https://syntology.ai/paper/2310.17146","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.17146"}},"official":{"repos":["mld3/counterfactualannot-semiope"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/distributional-off-policy-evaluation-for","slug":"distributional-off-policy-evaluation-for","title":"Distributional Off-Policy Evaluation for Slate Recommendations","date":"2023-08-27","arxiv_id":"2308.14165","repositories_listed":1,"syntology":null},{"url":"/paper/doubly-robust-estimator-for-off-policy","slug":"doubly-robust-estimator-for-off-policy","title":"Doubly Robust Estimator for Off-Policy Evaluation with Large Action Spaces","date":"2023-08-07","arxiv_id":"2308.03443","repositories_listed":1,"syntology":null},{"url":"/paper/on-normalised-discounted-cumulative-gain-as","slug":"on-normalised-discounted-cumulative-gain-as","title":"On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-$n$ Recommendation","date":"2023-07-27","arxiv_id":"2307.15053","repositories_listed":1,"syntology":null},{"url":"/paper/hindsight-dice-stable-credit-assignment-for","slug":"hindsight-dice-stable-credit-assignment-for","title":"Hindsight-DICE: Stable Credit Assignment for Deep Reinforcement Learning","date":"2023-07-21","arxiv_id":"2307.11897","repositories_listed":1,"syntology":null},{"url":"/paper/leveraging-factored-action-spaces-for-off","slug":"leveraging-factored-action-spaces-for-off","title":"Leveraging Factored Action Spaces for Off-Policy Evaluation","date":"2023-07-13","arxiv_id":"2307.07014","repositories_listed":1,"syntology":{"n":3,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":3,"phrase":"0 ran · 3 unverified","sample_list":"/paper/leveraging-factored-action-spaces-for-off#ran","syntology_url":"https://syntology.ai/paper/2307.07014","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.07014"}},"official":{"repos":["ai4ai-lab/factored-action-spaces-for-ope"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":3,"ran_from_kinds":[]}}},{"url":"/paper/off-policy-evaluation-of-ranking-policies","slug":"off-policy-evaluation-of-ranking-policies","title":"Off-Policy Evaluation of Ranking Policies under Diverse User Behavior","date":"2023-06-26","arxiv_id":"2306.15098","repositories_listed":1,"syntology":null},{"url":"/paper/off-policy-evaluation-in-doubly-inhomogeneous","slug":"off-policy-evaluation-in-doubly-inhomogeneous","title":"Off-policy Evaluation in Doubly Inhomogeneous Environments","date":"2023-06-14","arxiv_id":"2306.08719","repositories_listed":1,"syntology":null},{"url":"/paper/k-nearest-neighbor-resampling-for-off-policy","slug":"k-nearest-neighbor-resampling-for-off-policy","title":"$K$-Nearest-Neighbor Resampling for Off-Policy Evaluation in Stochastic Control","date":"2023-06-07","arxiv_id":"2306.04836","repositories_listed":1,"syntology":null},{"url":"/paper/counterfactual-evaluation-of-peer-review-1","slug":"counterfactual-evaluation-of-peer-review-1","title":"Counterfactual Evaluation of Peer-Review Assignment Policies","date":"2023-05-27","arxiv_id":"2305.17339","repositories_listed":1,"syntology":null},{"url":"/paper/human-choice-prediction-in-non-cooperative","slug":"human-choice-prediction-in-non-cooperative","title":"Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation","date":"2023-05-17","arxiv_id":"2305.10361","repositories_listed":1,"syntology":null},{"url":"/paper/learning-action-embeddings-for-off-policy","slug":"learning-action-embeddings-for-off-policy","title":"Learning Action Embeddings for Off-Policy Evaluation","date":"2023-05-06","arxiv_id":"2305.03954","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/learning-action-embeddings-for-off-policy#ran","syntology_url":"https://syntology.ai/paper/2305.03954","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.03954"}},"official":{"repos":["amazon-science/ope-learn-action-embeddings"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/hallucinated-adversarial-control-for","slug":"hallucinated-adversarial-control-for","title":"Hallucinated Adversarial Control for Conservative Offline Policy Evaluation","date":"2023-03-02","arxiv_id":"2303.01076","repositories_listed":1,"syntology":null},{"url":"/paper/post-episodic-reinforcement-learning","slug":"post-episodic-reinforcement-learning","title":"Post Reinforcement Learning Inference","date":"2023-02-17","arxiv_id":"2302.08854","repositories_listed":1,"syntology":null},{"url":"/paper/variational-latent-branching-model-for-off","slug":"variational-latent-branching-model-for-off","title":"Variational Latent Branching Model for Off-Policy Evaluation","date":"2023-01-28","arxiv_id":"2301.12056","repositories_listed":1,"syntology":null},{"url":"/paper/off-policy-evaluation-for-action-dependent","slug":"off-policy-evaluation-for-action-dependent","title":"Off-Policy Evaluation for Action-Dependent Non-Stationary Environments","date":"2023-01-24","arxiv_id":"2301.10330","repositories_listed":1,"syntology":{"n":4,"n_ran":4,"n_constructed":4,"n_ran_checked":4,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 4 samples that ran constructed an object rather than computing a result","sample_list":"/paper/off-policy-evaluation-for-action-dependent#ran","syntology_url":"https://syntology.ai/paper/2301.10330","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.10330"}},"official":{"repos":["yashchandak/activens"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":4,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/offline-policy-evaluation-with-out-of-sample","slug":"offline-policy-evaluation-with-out-of-sample","title":"Off-Policy Evaluation with Out-of-Sample Guarantees","date":"2023-01-20","arxiv_id":"2301.08649","repositories_listed":1,"syntology":{"n":2,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"0 ran · 2 unverified","sample_list":"/paper/offline-policy-evaluation-with-out-of-sample#ran","syntology_url":"https://syntology.ai/paper/2301.08649","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.08649"}},"official":{"repos":["sofiaek/off-policy-evaluation"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":2,"ran_from_kinds":[]}}},{"url":"/paper/doubly-robust-kernel-statistics-for-testing","slug":"doubly-robust-kernel-statistics-for-testing","title":"Doubly Robust Kernel Statistics for Testing Distributional Treatment Effects","date":"2022-12-09","arxiv_id":"2212.04922","repositories_listed":1,"syntology":null},{"url":"/paper/low-variance-off-policy-evaluation-with-state","slug":"low-variance-off-policy-evaluation-with-state","title":"Low Variance Off-policy Evaluation with State-based Importance Sampling","date":"2022-12-07","arxiv_id":"2212.03932","repositories_listed":1,"syntology":{"n":7,"n_ran":4,"n_constructed":0,"n_ran_checked":4,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/low-variance-off-policy-evaluation-with-state#ran","syntology_url":"https://syntology.ai/paper/2212.03932","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.03932"}},"official":{"repos":["bossdm/importancesampling"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/policy-adaptive-estimator-selection-for-off","slug":"policy-adaptive-estimator-selection-for-off","title":"Policy-Adaptive Estimator Selection for Off-Policy Evaluation","date":"2022-11-25","arxiv_id":"2211.13904","repositories_listed":1,"syntology":null},{"url":"/paper/counterfactual-learning-with-multioutput-deep","slug":"counterfactual-learning-with-multioutput-deep","title":"Counterfactual Learning with Multioutput Deep Kernels","date":"2022-11-20","arxiv_id":"2211.11119","repositories_listed":1,"syntology":null},{"url":"/paper/local-metric-learning-for-off-policy","slug":"local-metric-learning-for-off-policy","title":"Local Metric Learning for Off-Policy Evaluation in Contextual Bandits with Continuous Actions","date":"2022-10-24","arxiv_id":"2210.13373","repositories_listed":1,"syntology":{"n":11,"n_ran":6,"n_constructed":0,"n_ran_checked":5,"n_instrument":1,"n_unverified":5,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified","sample_list":"/paper/local-metric-learning-for-off-policy#ran","syntology_url":"https://syntology.ai/paper/2210.13373","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.13373"}},"official":{"repos":["haanvid/kmis"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":5,"ran_from_kinds":["official"]}}},{"url":"/paper/anytime-valid-off-policy-inference-for","slug":"anytime-valid-off-policy-inference-for","title":"Anytime-valid off-policy inference for contextual bandits","date":"2022-10-19","arxiv_id":"2210.10768","repositories_listed":1,"syntology":null},{"url":"/paper/a-policy-guided-imitation-approach-for","slug":"a-policy-guided-imitation-approach-for","title":"A Policy-Guided Imitation Approach for Offline Reinforcement Learning","date":"2022-10-15","arxiv_id":"2210.08323","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":1,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result","sample_list":"/paper/a-policy-guided-imitation-approach-for#ran","syntology_url":"https://syntology.ai/paper/2210.08323","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.08323"}},"official":{"repos":["ryanxhr/por"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/on-the-reuse-bias-in-off-policy-reinforcement","slug":"on-the-reuse-bias-in-off-policy-reinforcement","title":"On the Reuse Bias in Off-Policy Reinforcement Learning","date":"2022-09-15","arxiv_id":"2209.07074","repositories_listed":1,"syntology":null},{"url":"/paper/future-dependent-value-based-off-policy-1","slug":"future-dependent-value-based-off-policy-1","title":"Future-Dependent Value-Based Off-Policy Evaluation in POMDPs","date":"2022-07-26","arxiv_id":"2207.13081","repositories_listed":1,"syntology":{"n":8,"n_ran":3,"n_constructed":0,"n_ran_checked":2,"n_instrument":1,"n_unverified":5,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified","sample_list":"/paper/future-dependent-value-based-off-policy-1#ran","syntology_url":"https://syntology.ai/paper/2207.13081","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.13081"}},"official":{"repos":["aiueola/neurips2023-future-dependent-ope"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":5,"ran_from_kinds":["official"]}}},{"url":"/paper/conformal-off-policy-prediction","slug":"conformal-off-policy-prediction","title":"Conformal Off-policy Prediction","date":"2022-06-14","arxiv_id":"2206.06711","repositories_listed":1,"syntology":null},{"url":"/paper/coptidice-offline-constrained-reinforcement-1","slug":"coptidice-offline-constrained-reinforcement-1","title":"COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation","date":"2022-04-19","arxiv_id":"2204.08957","repositories_listed":1,"syntology":{"n":7,"n_ran":4,"n_constructed":0,"n_ran_checked":4,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/coptidice-offline-constrained-reinforcement-1#ran","syntology_url":"https://syntology.ai/paper/2204.08957","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.08957"}},"official":{"repos":["deepmind/constrained_optidice"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/model-free-and-model-based-policy-evaluation","slug":"model-free-and-model-based-policy-evaluation","title":"Model-Free and Model-Based Policy Evaluation when Causality is Uncertain","date":"2022-04-02","arxiv_id":"2204.00956","repositories_listed":1,"syntology":{"n":5,"n_ran":3,"n_constructed":0,"n_ran_checked":0,"n_instrument":3,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":5,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/model-free-and-model-based-policy-evaluation#ran","syntology_url":"https://syntology.ai/paper/2204.00956","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.00956"}},"official":null}},{"url":"/paper/a-multi-agent-reinforcement-learning","slug":"a-multi-agent-reinforcement-learning","title":"A Multi-Agent Reinforcement Learning Framework for Off-Policy Evaluation in Two-sided Markets","date":"2022-02-21","arxiv_id":"2202.10574","repositories_listed":1,"syntology":null},{"url":"/paper/doubly-robust-distributionally-robust-off","slug":"doubly-robust-distributionally-robust-off","title":"Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning","date":"2022-02-19","arxiv_id":"2202.09667","repositories_listed":1,"syntology":{"n":5,"n_ran":4,"n_constructed":1,"n_ran_checked":3,"n_instrument":1,"n_unverified":1,"n_honours":1,"n_violates":1,"n_no_contract":1,"n_pointer_only":0,"phrase":"4 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/doubly-robust-distributionally-robust-off#ran","syntology_url":"https://syntology.ai/paper/2202.09667","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.09667"}},"official":{"repos":["causalml/doubly-robust-dropel"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":1,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/off-policy-evaluation-using-information","slug":"off-policy-evaluation-using-information","title":"Off-Policy Evaluation Using Information Borrowing and Context-Based Switching","date":"2021-12-18","arxiv_id":"2112.09865","repositories_listed":1,"syntology":null},{"url":"/paper/bcorle-lambda-an-offline-reinforcement","slug":"bcorle-lambda-an-offline-reinforcement","title":"BCORLE($\\lambda$): An Offline Reinforcement Learning and Evaluation Framework for Coupons Allocation in E-commerce Market","date":"2021-12-01","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/subgaussian-and-differentiable-importance","slug":"subgaussian-and-differentiable-importance","title":"Subgaussian and Differentiable Importance Sampling for Off-Policy Evaluation and Learning","date":"2021-12-01","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/a-minimax-learning-approach-to-off-policy","slug":"a-minimax-learning-approach-to-off-policy","title":"A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes","date":"2021-11-12","arxiv_id":"2111.06784","repositories_listed":1,"syntology":null},{"url":"/paper/sope-spectrum-of-off-policy-estimators","slug":"sope-spectrum-of-off-policy-estimators","title":"SOPE: Spectrum of Off-Policy Estimators","date":"2021-11-06","arxiv_id":"2111.03936","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/sope-spectrum-of-off-policy-estimators#ran","syntology_url":"https://syntology.ai/paper/2111.03936","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.03936"}},"official":{"repos":["pearl-utexas/sope"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/proximal-reinforcement-learning-efficient-off","slug":"proximal-reinforcement-learning-efficient-off","title":"Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processes","date":"2021-10-28","arxiv_id":"2110.15332","repositories_listed":1,"syntology":{"n":2,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":2,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/proximal-reinforcement-learning-efficient-off#ran","syntology_url":"https://syntology.ai/paper/2110.15332","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.15332"}},"official":{"repos":["causalml/proximalrl"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/towards-hyperparameter-free-policy-selection","slug":"towards-hyperparameter-free-policy-selection","title":"Towards Hyperparameter-free Policy Selection for Offline Reinforcement Learning","date":"2021-10-26","arxiv_id":"2110.14000","repositories_listed":1,"syntology":{"n":2,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":2,"phrase":"0 ran · 2 unverified","sample_list":"/paper/towards-hyperparameter-free-policy-selection#ran","syntology_url":"https://syntology.ai/paper/2110.14000","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.14000"}},"official":{"repos":["jasonzhang929/BVFT_empirical_experiments"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":2,"ran_from_kinds":[]}}},{"url":"/paper/state-relevance-for-off-policy-evaluation","slug":"state-relevance-for-off-policy-evaluation","title":"State Relevance for Off-Policy Evaluation","date":"2021-09-13","arxiv_id":"2109.06310","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_constructed":0,"n_ran_checked":2,"n_instrument":1,"n_unverified":1,"n_honours":2,"n_violates":0,"n_no_contract":0,"n_pointer_only":4,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/state-relevance-for-off-policy-evaluation#ran","syntology_url":"https://syntology.ai/paper/2109.06310","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.06310"}},"official":{"repos":["dtak/osiris"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/model-selection-for-offline-reinforcement","slug":"model-selection-for-offline-reinforcement","title":"Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings","date":"2021-07-23","arxiv_id":"2107.11003","repositories_listed":1,"syntology":null},{"url":"/paper/supervised-off-policy-ranking","slug":"supervised-off-policy-ranking","title":"Supervised Off-Policy Ranking","date":"2021-07-03","arxiv_id":"2107.01360","repositories_listed":1,"syntology":{"n":2,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/supervised-off-policy-ranking#ran","syntology_url":"https://syntology.ai/paper/2107.01360","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.01360"}},"official":{"repos":["SOPR-T/SOPR-T"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/unifying-gradient-estimators-for-meta","slug":"unifying-gradient-estimators-for-meta","title":"Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation","date":"2021-06-24","arxiv_id":"2106.13125","repositories_listed":1,"syntology":{"n":3,"n_ran":3,"n_constructed":0,"n_ran_checked":0,"n_instrument":3,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":3,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/unifying-gradient-estimators-for-meta#ran","syntology_url":"https://syntology.ai/paper/2106.13125","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.13125"}},"official":{"repos":["robintyh1/neurips2021-meta-gradient-offpolicy-evaluation"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/active-offline-policy-selection","slug":"active-offline-policy-selection","title":"Active Offline Policy Selection","date":"2021-06-18","arxiv_id":"2106.10251","repositories_listed":1,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/active-offline-policy-selection#ran","syntology_url":"https://syntology.ai/paper/2106.10251","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.10251"}},"official":{"repos":["deepmind/active_ops"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/offline-rl-without-off-policy-evaluation","slug":"offline-rl-without-off-policy-evaluation","title":"Offline RL Without Off-Policy Evaluation","date":"2021-06-16","arxiv_id":"2106.08909","repositories_listed":1,"syntology":null},{"url":"/paper/control-variates-for-slate-off-policy","slug":"control-variates-for-slate-off-policy","title":"Control Variates for Slate Off-Policy Evaluation","date":"2021-06-15","arxiv_id":"2106.07914","repositories_listed":1,"syntology":null},{"url":"/paper/a-deep-reinforcement-learning-approach-to-4","slug":"a-deep-reinforcement-learning-approach-to-4","title":"A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representation","date":"2021-06-12","arxiv_id":"2106.06854","repositories_listed":1,"syntology":{"n":4,"n_ran":4,"n_constructed":4,"n_ran_checked":4,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 4 samples that ran constructed an object rather than computing a result","sample_list":"/paper/a-deep-reinforcement-learning-approach-to-4#ran","syntology_url":"https://syntology.ai/paper/2106.06854","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.06854"}},"official":{"repos":["sfujim/SR-DICE"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":4,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/deep-proxy-causal-learning-and-its","slug":"deep-proxy-causal-learning-and-its","title":"Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation","date":"2021-06-07","arxiv_id":"2106.03907","repositories_listed":1,"syntology":null},{"url":"/paper/off-policy-evaluation-via-adaptive-weighting","slug":"off-policy-evaluation-via-adaptive-weighting","title":"Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Bandits","date":"2021-06-03","arxiv_id":"2106.02029","repositories_listed":1,"syntology":{"n":25,"n_ran":13,"n_constructed":0,"n_ran_checked":13,"n_instrument":0,"n_unverified":12,"n_honours":0,"n_violates":1,"n_no_contract":12,"n_pointer_only":1,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 1 violated, 12 with no contract checked; 0 where Syntology's instrument failed) · 12 unverified","sample_list":"/paper/off-policy-evaluation-via-adaptive-weighting#ran","syntology_url":"https://syntology.ai/paper/2106.02029","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.02029"}},"official":{"repos":["gsbDBI/contextual_bandits_evaluation"],"state":"official (archive's flag): 13 ran","n_ran":13,"n_constructed":0,"n_ran_no_instrument_failure":13,"n_unverified":12,"ran_from_kinds":["official"]}}},{"url":"/paper/deeply-debiased-off-policy-interval","slug":"deeply-debiased-off-policy-interval","title":"Deeply-Debiased Off-Policy Interval Estimation","date":"2021-05-10","arxiv_id":"2105.04646","repositories_listed":1,"syntology":{"n":9,"n_ran":4,"n_constructed":0,"n_ran_checked":4,"n_instrument":0,"n_unverified":5,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":2,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified","sample_list":"/paper/deeply-debiased-off-policy-interval#ran","syntology_url":"https://syntology.ai/paper/2105.04646","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.04646"}},"official":{"repos":["RunzheStat/D2OPE"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":5,"ran_from_kinds":["official"]}}},{"url":"/paper/universal-off-policy-evaluation","slug":"universal-off-policy-evaluation","title":"Universal Off-Policy Evaluation","date":"2021-04-26","arxiv_id":"2104.12820","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":1,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result","sample_list":"/paper/universal-off-policy-evaluation#ran","syntology_url":"https://syntology.ai/paper/2104.12820","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.12820"}},"official":{"repos":["yashchandak/UnO"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/towards-automatic-evaluation-of-dialog","slug":"towards-automatic-evaluation-of-dialog","title":"Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach","date":"2021-02-20","arxiv_id":"2102.10242","repositories_listed":1,"syntology":null},{"url":"/paper/harnessing-distribution-ratio-estimators-for","slug":"harnessing-distribution-ratio-estimators-for","title":"Harnessing Distribution Ratio Estimators for Learning Agents with Quality and Diversity","date":"2020-11-05","arxiv_id":"2011.02614","repositories_listed":1,"syntology":null},{"url":"/paper/deep-jump-q-evaluation-for-offline-policy-1","slug":"deep-jump-q-evaluation-for-offline-policy-1","title":"Deep Jump Learning for Off-Policy Evaluation in Continuous Treatment Settings","date":"2020-10-29","arxiv_id":"2010.15963","repositories_listed":1,"syntology":null},{"url":"/paper/optimal-off-policy-evaluation-from-multiple","slug":"optimal-off-policy-evaluation-from-multiple","title":"Optimal Off-Policy Evaluation from Multiple Logging Policies","date":"2020-10-21","arxiv_id":"2010.11002","repositories_listed":1,"syntology":null},{"url":"/paper/counterfactual-evaluation-of-slate","slug":"counterfactual-evaluation-of-slate","title":"Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions","date":"2020-07-25","arxiv_id":"2007.12986","repositories_listed":1,"syntology":null},{"url":"/paper/strictly-batch-imitation-learning-by-energy","slug":"strictly-batch-imitation-learning-by-energy","title":"Strictly Batch Imitation Learning by Energy-based Distribution Matching","date":"2020-06-25","arxiv_id":"2006.14154","repositories_listed":1,"syntology":{"n":17,"n_ran":16,"n_constructed":5,"n_ran_checked":14,"n_instrument":2,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":14,"n_pointer_only":1,"phrase":"16 ran (of which 5 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/strictly-batch-imitation-learning-by-energy#ran","syntology_url":"https://syntology.ai/paper/2006.14154","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.14154"}},"official":{"repos":["vanderschaarlab/mlforhealthlabpub"],"state":"official: no sample here; runs from other or unrecorded repositories","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["found_in_text"]}}},{"url":"/paper/batch-stationary-distribution-estimation","slug":"batch-stationary-distribution-estimation","title":"Batch Stationary Distribution Estimation","date":"2020-03-02","arxiv_id":"2003.00722","repositories_listed":1,"syntology":null},{"url":"/paper/off-policy-evaluation-and-learning-for","slug":"off-policy-evaluation-and-learning-for","title":"Off-Policy Evaluation and Learning for External Validity under a Covariate Shift","date":"2020-02-26","arxiv_id":"2002.11642","repositories_listed":1,"syntology":{"n":4,"n_ran":2,"n_constructed":0,"n_ran_checked":0,"n_instrument":2,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/off-policy-evaluation-and-learning-for#ran","syntology_url":"https://syntology.ai/paper/2002.11642","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2002.11642"}},"official":{"repos":["MasaKat0/OPE_CS"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/adaptive-estimator-selection-for-off-policy","slug":"adaptive-estimator-selection-for-off-policy","title":"Adaptive Estimator Selection for Off-Policy Evaluation","date":"2020-02-18","arxiv_id":"2002.07729","repositories_listed":1,"syntology":null},{"url":"/paper/safe-exploration-for-optimizing-contextual","slug":"safe-exploration-for-optimizing-contextual","title":"Safe Exploration for Optimizing Contextual Bandits","date":"2020-02-02","arxiv_id":"2002.00467","repositories_listed":1,"syntology":null},{"url":"/paper/from-importance-sampling-to-doubly-robust","slug":"from-importance-sampling-to-doubly-robust","title":"From Importance Sampling to Doubly Robust Policy Gradient","date":"2019-10-20","arxiv_id":"1910.09066","repositories_listed":1,"syntology":{"n":1,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"0 ran · 1 unverified","sample_list":"/paper/from-importance-sampling-to-doubly-robust#ran","syntology_url":"https://syntology.ai/paper/1910.09066","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.09066"}},"official":null}},{"url":"/paper/double-reinforcement-learning-for-efficient","slug":"double-reinforcement-learning-for-efficient","title":"Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes","date":"2019-08-22","arxiv_id":"1908.08526","repositories_listed":1,"syntology":null},{"url":"/paper/doubly-robust-off-policy-evaluation-with","slug":"doubly-robust-off-policy-evaluation-with","title":"Doubly robust off-policy evaluation with shrinkage","date":"2019-07-22","arxiv_id":"1907.09623","repositories_listed":1,"syntology":null},{"url":"/paper/balanced-off-policy-evaluation-general-action","slug":"balanced-off-policy-evaluation-general-action","title":"Balanced off-policy evaluation in general action spaces","date":"2019-06-09","arxiv_id":"1906.03694","repositories_listed":1,"syntology":null},{"url":"/paper/semi-parametric-efficient-policy-learning","slug":"semi-parametric-efficient-policy-learning","title":"Semi-Parametric Efficient Policy Learning with Continuous Actions","date":"2019-05-24","arxiv_id":"1905.10116","repositories_listed":1,"syntology":null},{"url":"/paper/counterfactual-off-policy-evaluation-with","slug":"counterfactual-off-policy-evaluation-with","title":"Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models","date":"2019-05-14","arxiv_id":"1905.05824","repositories_listed":1,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":0,"n_instrument":2,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/counterfactual-off-policy-evaluation-with#ran","syntology_url":"https://syntology.ai/paper/1905.05824","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.05824"}},"official":{"repos":["clinicalml/gumbel-max-scm"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/importance-sampling-policy-evaluation-with-an","slug":"importance-sampling-policy-evaluation-with-an","title":"Importance Sampling Policy Evaluation with an Estimated Behavior Policy","date":"2018-06-04","arxiv_id":"1806.01347","repositories_listed":1,"syntology":{"n":11,"n_ran":10,"n_constructed":0,"n_ran_checked":10,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":10,"n_pointer_only":0,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/importance-sampling-policy-evaluation-with-an#ran","syntology_url":"https://syntology.ai/paper/1806.01347","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1806.01347"}},"official":{"repos":["LARG/regression-importance-sampling"],"state":"official (archive's flag): 10 ran","n_ran":10,"n_constructed":0,"n_ran_no_instrument_failure":10,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/counterfactual-mean-embedding-a-kernel-method","slug":"counterfactual-mean-embedding-a-kernel-method","title":"Counterfactual Mean Embeddings","date":"2018-05-22","arxiv_id":"1805.08845","repositories_listed":1,"syntology":null}],"syntology_records":43,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"key_notes":{"n_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"since_archive_judged":{"papers":0,"papers_placed":0,"judged_by":{"model":"a model","models_as_recorded":[],"from":"lb_rows_run_verified.jsonl: each record's `model` field (the extractor's call that wrote decisions[])"},"note":"a model's judgement from the paper's own tables that they report on this leaderboard, not a result: no value from a paper that was not placed is printed or carried here; papers counts each paper once over these leaderboards","backfill":"8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read)","backfill_complete":false,"leaderboards":[]},"papers_where_code_ran":40,"papers_where_code_ran_first_page":"/task/off-policy-evaluation/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not run it on this task or check it against the task's benchmarks.","pages":[{"order":"archive","page":2,"of":3,"url":"/task/off-policy-evaluation/papers/2","json":"/task/off-policy-evaluation/papers/2.json","rows":100},{"order":"archive","page":3,"of":3,"url":"/task/off-policy-evaluation/papers/3","json":"/task/off-policy-evaluation/papers/3.json","rows":65},{"order":"ran","page":1,"of":1,"url":"/task/off-policy-evaluation/papers/ran/1","json":"/task/off-policy-evaluation/papers/ran/1.json","rows":40}],"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-10-05T00:36:11+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"row_route":"each syntology object on a benchmark's first row and on a paper carries sample_list (this site's paper page, #ran), syntology_url (syntology.ai's page for the paper, only when that route probed live) and mcp (the agent call that lists the paper's harvested samples; not a verdict on the paper)","developers":"https://syntology.ai/developers"},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"},"record_sha256":"5d0ad76fa66d6343762f8d20e2ccd4c12fb4aaa49023f92ef16e6232b927682b","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}