{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/task/distributional-reinforcement-learning","name":"Distributional Reinforcement Learning","slug":"distributional-reinforcement-learning","description_markdown":"Value distribution is the distribution of the random return received by a reinforcement learning agent.  it been used for a specific purpose such as implementing risk-aware behaviour. \r\n\r\nWe have random return Z whose expectation is the value Q. This random return is also described by a recursive equation, but one of a distributional nature","categories":[{"name":"Methodology","url":"/area/methodology"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"derived"},"counts":{"papers_tagged":137,"papers_with_code":43,"benchmarks":0,"benchmark_tables_in_archive":0,"benchmark_tables_shown":0,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":0,"subtasks":0,"parent_tasks":0},"benchmarks":[],"datasets":[],"subtasks":[],"parent_tasks":[],"papers":{"order":"repositories listed in the archive (most first), then date (newest first), then slug; the archive holds no stars","population":"every paper the archive tags with this task (distinct papers; spam titles not listed). 'of' counts the papers listed; until 2026-09-25 it counted only the papers with a code link, and that number is of_with_code","page":1,"pages_in_order":2,"rows_per_page":100,"shown":100,"of":137,"of_with_code":43,"tagged_in_all":137,"counts":{"archive_papers_tagged":137,"with_a_code_link":43,"where_syntology_ran_a_sample":16,"not_listed_spam_title":0,"listed":137,"listed_where_code_ran":16,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":12,"every_run_a_failure_of_syntologys_instrument":4,"listed_with_a_run_with_no_instrument_failure":12,"listed_every_run_a_failure_of_syntologys_instrument":4,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"items":[{"url":"/paper/implicit-quantile-networks-for-distributional","slug":"implicit-quantile-networks-for-distributional","title":"Implicit Quantile Networks for Distributional Reinforcement Learning","date":"2018-06-14","arxiv_id":"1806.06923","repositories_listed":19,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/implicit-quantile-networks-for-distributional#ran","syntology_url":"https://syntology.ai/paper/1806.06923","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1806.06923"}},"official":null}},{"url":"/paper/distributional-reinforcement-learning-with-1","slug":"distributional-reinforcement-learning-with-1","title":"Distributional Reinforcement Learning with Quantile Regression","date":"2017-10-27","arxiv_id":"1710.10044","repositories_listed":17,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":0,"n_instrument":2,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/distributional-reinforcement-learning-with-1#ran","syntology_url":"https://syntology.ai/paper/1710.10044","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1710.10044"}},"official":null}},{"url":"/paper/fully-parameterized-quantile-function-for","slug":"fully-parameterized-quantile-function-for","title":"Fully Parameterized Quantile Function for Distributional Reinforcement Learning","date":"2019-11-05","arxiv_id":"1911.02140","repositories_listed":6,"syntology":{"n":10,"n_ran":6,"n_constructed":0,"n_ran_checked":5,"n_instrument":1,"n_unverified":4,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/fully-parameterized-quantile-function-for#ran","syntology_url":"https://syntology.ai/paper/1911.02140","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1911.02140"}},"official":null}},{"url":"/paper/implicit-distributional-reinforcement","slug":"implicit-distributional-reinforcement","title":"Implicit Distributional Reinforcement Learning","date":"2020-07-13","arxiv_id":"2007.06159","repositories_listed":3,"syntology":{"n":6,"n_ran":4,"n_constructed":4,"n_ran_checked":4,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 4 samples that ran constructed an object rather than computing a result","sample_list":"/paper/implicit-distributional-reinforcement#ran","syntology_url":"https://syntology.ai/paper/2007.06159","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.06159"}},"official":{"repos":["zhougroup/IDAC"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":4,"n_ran_no_instrument_failure":4,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/quota-the-quantile-option-architecture-for","slug":"quota-the-quantile-option-architecture-for","title":"QUOTA: The Quantile Option Architecture for Reinforcement Learning","date":"2018-11-05","arxiv_id":"1811.02073","repositories_listed":3,"syntology":{"n":3,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/quota-the-quantile-option-architecture-for#ran","syntology_url":"https://syntology.ai/paper/1811.02073","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1811.02073"}},"official":{"repos":["ShangtongZhang/DeepRL"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/distributional-reinforcement-learning-for-4","slug":"distributional-reinforcement-learning-for-4","title":"Distributional Reinforcement Learning for Multi-Dimensional Reward Functions","date":"2021-10-26","arxiv_id":"2110.13578","repositories_listed":2,"syntology":null},{"url":"/paper/estimating-risk-and-uncertainty-in-deep","slug":"estimating-risk-and-uncertainty-in-deep","title":"Estimating Risk and Uncertainty in Deep Reinforcement Learning","date":"2019-05-23","arxiv_id":"1905.09638","repositories_listed":2,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/estimating-risk-and-uncertainty-in-deep#ran","syntology_url":"https://syntology.ai/paper/1905.09638","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.09638"}},"official":{"repos":["IndustAI/risk-and-uncertainty"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/addq-adaptive-distributional-double-q","slug":"addq-adaptive-distributional-double-q","title":"ADDQ: Adaptive Distributional Double Q-Learning","date":"2025-06-24","arxiv_id":"2506.19478","repositories_listed":1,"syntology":null},{"url":"/paper/rize-regularized-imitation-learning-via","slug":"rize-regularized-imitation-learning-via","title":"RIZE: Regularized Imitation Learning via Distributional Reinforcement Learning","date":"2025-02-27","arxiv_id":"2502.20089","repositories_listed":1,"syntology":null},{"url":"/paper/tackling-uncertainties-in-multi-agent","slug":"tackling-uncertainties-in-multi-agent","title":"Tackling Uncertainties in Multi-Agent Reinforcement Learning through Integration of Agent Termination Dynamics","date":"2025-01-21","arxiv_id":"2501.12061","repositories_listed":1,"syntology":null},{"url":"/paper/beyond-cvar-leveraging-static-spectral-risk","slug":"beyond-cvar-leveraging-static-spectral-risk","title":"Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learning","date":"2025-01-03","arxiv_id":"2501.02087","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_constructed":1,"n_ran_checked":2,"n_instrument":1,"n_unverified":1,"n_honours":1,"n_violates":0,"n_no_contract":1,"n_pointer_only":4,"phrase":"3 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/beyond-cvar-leveraging-static-spectral-risk#ran","syntology_url":"https://syntology.ai/paper/2501.02087","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.02087"}},"official":{"repos":["mehrdadmoghimi/qrsrm"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official","unlocated"]}}},{"url":"/paper/ex-drl-hedging-against-heavy-losses-with","slug":"ex-drl-hedging-against-heavy-losses-with","title":"EX-DRL: Hedging Against Heavy Losses with EXtreme Distributional Reinforcement Learning","date":"2024-08-22","arxiv_id":"2408.12446","repositories_listed":1,"syntology":null},{"url":"/paper/ctd4-a-deep-continuous-distributional-actor","slug":"ctd4-a-deep-continuous-distributional-actor","title":"CTD4 -- A Deep Continuous Distributional Actor-Critic Agent with a Kalman Fusion of Multiple Critics","date":"2024-05-04","arxiv_id":"2405.02576","repositories_listed":1,"syntology":null},{"url":"/paper/a-distributional-analogue-to-the-successor","slug":"a-distributional-analogue-to-the-successor","title":"A Distributional Analogue to the Successor Representation","date":"2024-02-13","arxiv_id":"2402.08530","repositories_listed":1,"syntology":{"n":4,"n_ran":2,"n_constructed":0,"n_ran_checked":0,"n_instrument":2,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/a-distributional-analogue-to-the-successor#ran","syntology_url":"https://syntology.ai/paper/2402.08530","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.08530"}},"official":{"repos":["jessefarebro/distributional-sr"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/echoes-of-socratic-doubt-embracing","slug":"echoes-of-socratic-doubt-embracing","title":"Echoes of Socratic Doubt: Embracing Uncertainty in Calibrated Evidential Reinforcement Learning","date":"2024-02-11","arxiv_id":"2402.07107","repositories_listed":1,"syntology":null},{"url":"/paper/distributional-off-policy-evaluation-with","slug":"distributional-off-policy-evaluation-with","title":"Distributional Off-policy Evaluation with Bellman Residual Minimization","date":"2024-02-02","arxiv_id":"2402.01900","repositories_listed":1,"syntology":null},{"url":"/paper/a-robust-quantile-huber-loss-with","slug":"a-robust-quantile-huber-loss-with","title":"A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning","date":"2024-01-04","arxiv_id":"2401.02325","repositories_listed":1,"syntology":null},{"url":"/paper/distributional-bellman-operators-over-mean","slug":"distributional-bellman-operators-over-mean","title":"Distributional Bellman Operators over Mean Embeddings","date":"2023-12-09","arxiv_id":"2312.07358","repositories_listed":1,"syntology":null},{"url":"/paper/estimation-and-inference-in-distributional","slug":"estimation-and-inference-in-distributional","title":"Estimation and Inference in Distributional Reinforcement Learning","date":"2023-09-29","arxiv_id":"2309.17262","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_constructed":0,"n_ran_checked":2,"n_instrument":1,"n_unverified":1,"n_honours":0,"n_violates":2,"n_no_contract":0,"n_pointer_only":4,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/estimation-and-inference-in-distributional#ran","syntology_url":"https://syntology.ai/paper/2309.17262","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.17262"}},"official":{"repos":["zhangliangyu32/estimationandinferencedistributionalrl"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/value-distributional-model-based","slug":"value-distributional-model-based","title":"Value-Distributional Model-Based Reinforcement Learning","date":"2023-08-12","arxiv_id":"2308.06590","repositories_listed":1,"syntology":null},{"url":"/paper/variance-control-for-distributional","slug":"variance-control-for-distributional","title":"Variance Control for Distributional Reinforcement Learning","date":"2023-07-30","arxiv_id":"2307.16152","repositories_listed":1,"syntology":null},{"url":"/paper/distributional-model-equivalence-for-risk","slug":"distributional-model-equivalence-for-risk","title":"Distributional Model Equivalence for Risk-Sensitive Reinforcement Learning","date":"2023-07-04","arxiv_id":"2307.01708","repositories_listed":1,"syntology":null},{"url":"/paper/the-benefits-of-being-distributional-small-1","slug":"the-benefits-of-being-distributional-small-1","title":"The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learning","date":"2023-05-25","arxiv_id":"2305.15703","repositories_listed":1,"syntology":{"n":1,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":1,"phrase":"0 ran · 1 unverified","sample_list":"/paper/the-benefits-of-being-distributional-small-1#ran","syntology_url":"https://syntology.ai/paper/2305.15703","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.15703"}},"official":{"repos":["kevinzhou497/distcb"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"url":"/paper/trustworthy-reinforcement-learning-for","slug":"trustworthy-reinforcement-learning-for","title":"Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control","date":"2023-02-22","arxiv_id":"2302.11694","repositories_listed":1,"syntology":null},{"url":"/paper/distributional-constrained-reinforcement","slug":"distributional-constrained-reinforcement","title":"Distributional constrained reinforcement learning for supply chain optimization","date":"2023-02-03","arxiv_id":"2302.01727","repositories_listed":1,"syntology":null},{"url":"/paper/efficient-trust-region-based-safe","slug":"efficient-trust-region-based-safe","title":"Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraints","date":"2023-01-26","arxiv_id":"2301.10923","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_constructed":0,"n_ran_checked":0,"n_instrument":3,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":3,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/efficient-trust-region-based-safe#ran","syntology_url":"https://syntology.ai/paper/2301.10923","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.10923"}},"official":{"repos":["rllab-snu/safe-distributional-actor-critic"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/risk-sensitive-policy-with-distributional","slug":"risk-sensitive-policy-with-distributional","title":"Risk-Sensitive Policy with Distributional Reinforcement Learning","date":"2022-12-30","arxiv_id":"2212.14743","repositories_listed":1,"syntology":null},{"url":"/paper/intelligent-resource-allocation-in-joint","slug":"intelligent-resource-allocation-in-joint","title":"Intelligent Resource Allocation in Joint Radar-Communication With Graph Neural Networks","date":"2022-10-17","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/ign-implicit-generative-networks","slug":"ign-implicit-generative-networks","title":"IGN : Implicit Generative Networks","date":"2022-06-13","arxiv_id":"2206.05860","repositories_listed":1,"syntology":null},{"url":"/paper/gamma-and-vega-hedging-using-deep","slug":"gamma-and-vega-hedging-using-deep","title":"Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning","date":"2022-05-10","arxiv_id":"2205.05614","repositories_listed":1,"syntology":null},{"url":"/paper/adaptive-risk-tendency-nano-drone-navigation","slug":"adaptive-risk-tendency-nano-drone-navigation","title":"Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning","date":"2022-03-28","arxiv_id":"2203.14749","repositories_listed":1,"syntology":null},{"url":"/paper/distributional-reinforcement-learning-via","slug":"distributional-reinforcement-learning-via","title":"Distributional Reinforcement Learning with Regularized Wasserstein Loss","date":"2022-02-01","arxiv_id":"2202.00769","repositories_listed":1,"syntology":null},{"url":"/paper/conjugated-discrete-distributions-for","slug":"conjugated-discrete-distributions-for","title":"Conjugated Discrete Distributions for Distributional Reinforcement Learning","date":"2021-12-14","arxiv_id":"2112.07424","repositories_listed":1,"syntology":null},{"url":"/paper/two-steps-to-risk-sensitivity","slug":"two-steps-to-risk-sensitivity","title":"Two steps to risk sensitivity","date":"2021-11-12","arxiv_id":"2111.06803","repositories_listed":1,"syntology":null},{"url":"/paper/a-cramer-distance-perspective-on-non-crossing","slug":"a-cramer-distance-perspective-on-non-crossing","title":"A Cramér Distance perspective on Quantile Regression based Distributional Reinforcement Learning","date":"2021-10-01","arxiv_id":"2110.00535","repositories_listed":1,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/a-cramer-distance-perspective-on-non-crossing#ran","syntology_url":"https://syntology.ai/paper/2110.00535","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.00535"}},"official":{"repos":["alherit/cr-dqn"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/exploring-the-robustness-of-distributional","slug":"exploring-the-robustness-of-distributional","title":"Exploring the Training Robustness of Distributional Reinforcement Learning against Noisy State Observations","date":"2021-09-17","arxiv_id":"2109.08776","repositories_listed":1,"syntology":null},{"url":"/paper/conservative-offline-distributional","slug":"conservative-offline-distributional","title":"Conservative Offline Distributional Reinforcement Learning","date":"2021-07-12","arxiv_id":"2107.06106","repositories_listed":1,"syntology":{"n":5,"n_ran":4,"n_constructed":2,"n_ran_checked":3,"n_instrument":1,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":3,"n_pointer_only":5,"phrase":"4 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/conservative-offline-distributional#ran","syntology_url":"https://syntology.ai/paper/2107.06106","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.06106"}},"official":{"repos":["JasonMa2016/CODAC"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":2,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/distributional-reinforcement-learning-with-4","slug":"distributional-reinforcement-learning-with-4","title":"Distributional Reinforcement Learning with Unconstrained Monotonic Neural Networks","date":"2021-06-06","arxiv_id":"2106.03228","repositories_listed":1,"syntology":null},{"url":"/paper/unifying-cardiovascular-modelling-with-deep","slug":"unifying-cardiovascular-modelling-with-deep","title":"Unifying Cardiovascular Modelling with Deep Reinforcement Learning for Uncertainty Aware Control of Sepsis Treatment","date":"2021-01-21","arxiv_id":"2101.08477","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/unifying-cardiovascular-modelling-with-deep#ran","syntology_url":"https://syntology.ai/paper/2101.08477","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2101.08477"}},"official":{"repos":["thxsxth/POMDP_RLSepsis"],"state":"official: no sample here; runs from other or unrecorded repositories","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["unlocated"]}}},{"url":"/paper/distributional-reinforcement-learning-with-3","slug":"distributional-reinforcement-learning-with-3","title":"Distributional Reinforcement Learning via Moment Matching","date":"2020-07-24","arxiv_id":"2007.12354","repositories_listed":1,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":0,"n_honours":2,"n_violates":0,"n_no_contract":0,"n_pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/distributional-reinforcement-learning-with-3#ran","syntology_url":"https://syntology.ai/paper/2007.12354","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.12354"}},"official":{"repos":["thanhnguyentang/mmdrl"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/distributional-reinforcement-learning-for-1","slug":"distributional-reinforcement-learning-for-1","title":"Distributional Reinforcement Learning for Energy-Based Sequential Models","date":"2019-12-18","arxiv_id":"1912.08517","repositories_listed":1,"syntology":{"n":7,"n_ran":5,"n_constructed":0,"n_ran_checked":5,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/distributional-reinforcement-learning-for-1#ran","syntology_url":"https://syntology.ai/paper/1912.08517","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1912.08517"}},"official":{"repos":["parshakova/GAMS-for-Data-Efficient-Learning"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/information-directed-exploration-for-deep","slug":"information-directed-exploration-for-deep","title":"Information-Directed Exploration for Deep Reinforcement Learning","date":"2018-12-18","arxiv_id":"1812.07544","repositories_listed":1,"syntology":null},{"url":"/paper/gan-q-learning","slug":"gan-q-learning","title":"GAN Q-learning","date":"2018-05-13","arxiv_id":"1805.04874","repositories_listed":1,"syntology":null},{"url":null,"slug":"distributional-reinforcement-learning-on-path","title":"Distributional Reinforcement Learning on Path-dependent Options","date":"2025-07-16","arxiv_id":"2507.12657","repositories_listed":0,"syntology":null},{"url":null,"slug":"second-order-bounds-for-01-valued-regression","title":"Second-Order Bounds for [0,1]-Valued Regression via Betting Loss","date":"2025-07-16","arxiv_id":"2507.12584","repositories_listed":0,"syntology":null},{"url":null,"slug":"ctrls-chain-of-thought-reasoning-via-latent","title":"CTRLS: Chain-of-Thought Reasoning via Latent State-Transition","date":"2025-07-10","arxiv_id":"2507.08182","repositories_listed":0,"syntology":null},{"url":null,"slug":"a-point-based-algorithm-for-distributional","title":"A Point-Based Algorithm for Distributional Reinforcement Learning in Partially Observable Domains","date":"2025-05-10","arxiv_id":"2505.06518","repositories_listed":0,"syntology":null},{"url":null,"slug":"flow-models-for-unbounded-and-geometry-aware","title":"Flow Models for Unbounded and Geometry-Aware Distributional Reinforcement Learning","date":"2025-05-07","arxiv_id":"2505.04310","repositories_listed":0,"syntology":null},{"url":null,"slug":"deep-distributional-learning-with-non","title":"Deep Distributional Learning with Non-crossing Quantile Network","date":"2025-04-11","arxiv_id":"2504.08215","repositories_listed":0,"syntology":null},{"url":null,"slug":"offline-and-distributional-reinforcement-1","title":"Offline and Distributional Reinforcement Learning for Wireless Communications","date":"2025-04-04","arxiv_id":"2504.03804","repositories_listed":0,"syntology":null},{"url":null,"slug":"adaptive-nesterov-accelerated-distributional","title":"Adaptive Nesterov Accelerated Distributional Deep Hedging for Efficient Volatility Risk Management","date":"2025-02-25","arxiv_id":"2502.17777","repositories_listed":0,"syntology":null},{"url":null,"slug":"finite-sample-analysis-of-distributional-td","title":"A Finite Sample Analysis of Distributional TD Learning with Linear Function Approximation","date":"2025-02-20","arxiv_id":"2502.14172","repositories_listed":0,"syntology":null},{"url":null,"slug":"robust-probabilistic-model-checking-with","title":"Robust Probabilistic Model Checking with Continuous Reward Domains","date":"2025-02-06","arxiv_id":"2502.04530","repositories_listed":0,"syntology":null},{"url":null,"slug":"risk-averse-policies-for-natural-gas-futures","title":"Risk-averse policies for natural gas futures trading using distributional reinforcement learning","date":"2025-01-08","arxiv_id":"2501.04421","repositories_listed":0,"syntology":null},{"url":null,"slug":"hedging-and-pricing-structured-products","title":"Hedging and Pricing Structured Products Featuring Multiple Underlying Assets","date":"2024-11-02","arxiv_id":"2411.01121","repositories_listed":0,"syntology":null},{"url":"/paper/action-gaps-and-advantages-in-continuous-time","slug":"action-gaps-and-advantages-in-continuous-time","title":"Action Gaps and Advantages in Continuous-Time Distributional Reinforcement Learning","date":"2024-10-14","arxiv_id":"2410.11022","repositories_listed":0,"syntology":{"n":3,"n_ran":3,"n_constructed":0,"n_ran_checked":2,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/action-gaps-and-advantages-in-continuous-time#ran","syntology_url":"https://syntology.ai/paper/2410.11022","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.11022"}},"official":null}},{"url":null,"slug":"bellman-diffusion-generative-modeling-as","title":"Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Space","date":"2024-10-02","arxiv_id":"2410.01796","repositories_listed":0,"syntology":null},{"url":null,"slug":"offline-and-distributional-reinforcement","title":"Offline and Distributional Reinforcement Learning for Radio Resource Management","date":"2024-09-25","arxiv_id":"2409.16764","repositories_listed":0,"syntology":null},{"url":null,"slug":"foundations-of-multivariate-distributional","title":"Foundations of Multivariate Distributional Reinforcement Learning","date":"2024-08-31","arxiv_id":"2409.00328","repositories_listed":0,"syntology":null},{"url":null,"slug":"2407-21260","title":"Bellman Unbiasedness: Toward Provably Efficient Distributional Reinforcement Learning with General Value Function Approximation","date":"2024-07-31","arxiv_id":"2407.21260","repositories_listed":0,"syntology":null},{"url":null,"slug":"on-policy-evaluation-algorithms-in","title":"On Policy Evaluation Algorithms in Distributional Reinforcement Learning","date":"2024-07-19","arxiv_id":"2407.14175","repositories_listed":0,"syntology":null},{"url":null,"slug":"pg-rainbow-using-distributional-reinforcement","title":"PG-Rainbow: Using Distributional Reinforcement Learning in Policy Gradient Methods","date":"2024-07-18","arxiv_id":"2407.13146","repositories_listed":0,"syntology":null},{"url":null,"slug":"policy-gradient-methods-for-risk-sensitive","title":"Policy Gradient Methods for Risk-Sensitive Distributional Reinforcement Learning with Provable Convergence","date":"2024-05-23","arxiv_id":"2405.14749","repositories_listed":0,"syntology":null},{"url":null,"slug":"statistical-efficiency-of-distributional","title":"Statistical Efficiency of Distributional Temporal Difference Learning and Freedman's Inequality in Hilbert Spaces","date":"2024-03-09","arxiv_id":"2403.05811","repositories_listed":0,"syntology":null},{"url":null,"slug":"provable-risk-sensitive-distributional","title":"Provable Risk-Sensitive Distributional Reinforcement Learning with General Function Approximation","date":"2024-02-28","arxiv_id":"2402.18159","repositories_listed":0,"syntology":null},{"url":null,"slug":"uncertainty-aware-transient-stability","title":"Uncertainty-Aware Transient Stability-Constrained Preventive Redispatch: A Distributional Reinforcement Learning Approach","date":"2024-02-14","arxiv_id":"2402.09263","repositories_listed":0,"syntology":null},{"url":null,"slug":"near-minimax-optimal-distributional","title":"Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model","date":"2024-02-12","arxiv_id":"2402.07598","repositories_listed":0,"syntology":null},{"url":null,"slug":"more-benefits-of-being-distributional-second","title":"More Benefits of Being Distributional: Second-Order Bounds for Reinforcement Learning","date":"2024-02-11","arxiv_id":"2402.07198","repositories_listed":0,"syntology":null},{"url":null,"slug":"distributional-reinforcement-learning-based","title":"Distributional Reinforcement Learning-based Energy Arbitrage Strategies in Imbalance Settlement Mechanism","date":"2023-12-23","arxiv_id":"2401.00015","repositories_listed":0,"syntology":null},{"url":null,"slug":"noise-distribution-decomposition-based-multi","title":"Noise Distribution Decomposition based Multi-Agent Distributional Reinforcement Learning","date":"2023-12-12","arxiv_id":"2312.07025","repositories_listed":0,"syntology":null},{"url":null,"slug":"an-introduction-to-reinforcement-learning-for","title":"An introduction to reinforcement learning for neuroscience","date":"2023-11-13","arxiv_id":"2311.07315","repositories_listed":0,"syntology":null},{"url":null,"slug":"beyond-average-return-in-markov-decision","title":"Beyond Average Return in Markov Decision Processes","date":"2023-10-31","arxiv_id":"2310.20266","repositories_listed":0,"syntology":null},{"url":null,"slug":"pitfall-of-optimism-distributional","title":"Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion","date":"2023-10-25","arxiv_id":"2310.16546","repositories_listed":0,"syntology":null},{"url":null,"slug":"distributional-reinforcement-learning-with-7","title":"Distributional Reinforcement Learning with Online Risk-awareness Adaption","date":"2023-10-08","arxiv_id":"2310.05179","repositories_listed":0,"syntology":null},{"url":null,"slug":"learning-risk-aware-quadrupedal-locomotion","title":"Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning","date":"2023-09-25","arxiv_id":"2309.14246","repositories_listed":0,"syntology":null},{"url":null,"slug":"deep-reinforcement-learning-for-artificial","title":"Deep Reinforcement Learning for Artificial Upwelling Energy Management","date":"2023-08-20","arxiv_id":"2308.10199","repositories_listed":0,"syntology":null},{"url":null,"slug":"cramer-type-distances-for-learning-gaussian","title":"Cramer Type Distances for Learning Gaussian Mixture Models by Gradient Descent","date":"2023-07-13","arxiv_id":"2307.06753","repositories_listed":0,"syntology":null},{"url":null,"slug":"is-risk-sensitive-reinforcement-learning","title":"Is Risk-Sensitive Reinforcement Learning Properly Resolved?","date":"2023-07-02","arxiv_id":"2307.00547","repositories_listed":0,"syntology":null},{"url":null,"slug":"diverse-projection-ensembles-for","title":"Diverse Projection Ensembles for Distributional Reinforcement Learning","date":"2023-06-12","arxiv_id":"2306.07124","repositories_listed":0,"syntology":null},{"url":null,"slug":"pacer-a-fully-push-forward-based","title":"PACER: A Fully Push-forward-based Distributional Reinforcement Learning Algorithm","date":"2023-06-11","arxiv_id":"2306.06637","repositories_listed":0,"syntology":null},{"url":null,"slug":"improving-the-generalizability-and-robustness","title":"Improving the generalizability and robustness of large-scale traffic signal control","date":"2023-06-02","arxiv_id":"2306.01925","repositories_listed":0,"syntology":null},{"url":null,"slug":"the-statistical-benefits-of-quantile-temporal","title":"The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimation","date":"2023-05-28","arxiv_id":"2305.18388","repositories_listed":0,"syntology":null},{"url":null,"slug":"distributional-reinforcement-learning-with-6","title":"Distributional Reinforcement Learning with Dual Expectile-Quantile Regression","date":"2023-05-26","arxiv_id":"2305.16877","repositories_listed":0,"syntology":null},{"url":null,"slug":"one-step-distributional-reinforcement","title":"One-Step Distributional Reinforcement Learning","date":"2023-04-27","arxiv_id":"2304.14421","repositories_listed":0,"syntology":null},{"url":null,"slug":"policy-evaluation-in-distributional-lqr","title":"Policy Evaluation in Distributional LQR","date":"2023-03-23","arxiv_id":"2303.13657","repositories_listed":0,"syntology":null},{"url":null,"slug":"toward-risk-based-optimistic-exploration-for","title":"Toward Risk-based Optimistic Exploration for Cooperative Multi-Agent Reinforcement Learning","date":"2023-03-03","arxiv_id":"2303.01768","repositories_listed":0,"syntology":null},{"url":null,"slug":"multi-compartment-neuron-and-population","title":"Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learning","date":"2023-01-18","arxiv_id":"2301.07275","repositories_listed":0,"syntology":null},{"url":null,"slug":"an-analysis-of-quantile-temporal-difference","title":"An Analysis of Quantile Temporal-Difference Learning","date":"2023-01-11","arxiv_id":"2301.04462","repositories_listed":0,"syntology":null},{"url":null,"slug":"invariance-to-quantile-selection-in","title":"Invariance to Quantile Selection in Distributional Continuous Control","date":"2022-12-29","arxiv_id":"2212.14262","repositories_listed":0,"syntology":null},{"url":null,"slug":"bridging-distributional-and-risk-sensitive","title":"Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Bounds","date":"2022-10-25","arxiv_id":"2210.14051","repositories_listed":0,"syntology":null},{"url":null,"slug":"how-does-value-distribution-in-distributional","title":"How Does Return Distribution in Distributional Reinforcement Learning Help Optimization?","date":"2022-09-29","arxiv_id":"2209.14513","repositories_listed":0,"syntology":null},{"url":null,"slug":"normality-guided-distributional-reinforcement","title":"Normality-Guided Distributional Reinforcement Learning for Continuous Control","date":"2022-08-28","arxiv_id":"2208.13125","repositories_listed":0,"syntology":null},{"url":null,"slug":"the-nature-of-temporal-difference-errors-in","title":"The Nature of Temporal Difference Errors in Multi-step Distributional Reinforcement Learning","date":"2022-07-15","arxiv_id":"2207.07570","repositories_listed":0,"syntology":null},{"url":null,"slug":"risk-perspective-exploration-in","title":"Risk Perspective Exploration in Distributional Reinforcement Learning","date":"2022-06-28","arxiv_id":"2206.14170","repositories_listed":0,"syntology":null},{"url":null,"slug":"robust-reinforcement-learning-with","title":"Robust Reinforcement Learning with Distributional Risk-averse formulation","date":"2022-06-14","arxiv_id":"2206.06841","repositories_listed":0,"syntology":null},{"url":null,"slug":"gldqn-explicitly-parameterized-quantile","title":"A Simulation Environment and Reinforcement Learning Method for Waste Reduction","date":"2022-05-30","arxiv_id":"2205.15455","repositories_listed":0,"syntology":null},{"url":null,"slug":"interpretable-stochastic-model-predictive","title":"Interpretable Stochastic Model Predictive Control using Distributional Reinforced Estimation for Quadrotor Tracking Systems","date":"2022-05-14","arxiv_id":"2205.07150","repositories_listed":0,"syntology":null},{"url":null,"slug":"distributional-reinforcement-learning-for-5","title":"Distributional Reinforcement Learning for Scheduling of Chemical Production Processes","date":"2022-03-01","arxiv_id":"2203.00636","repositories_listed":0,"syntology":null},{"url":null,"slug":"exploration-with-multi-sample-target-values","title":"Exploration with Multi-Sample Target Values for Distributional Reinforcement Learning","date":"2022-02-06","arxiv_id":"2202.02693","repositories_listed":0,"syntology":null},{"url":null,"slug":"on-solutions-of-the-distributional-bellman","title":"On solutions of the distributional Bellman equation","date":"2022-01-31","arxiv_id":"2202.00081","repositories_listed":0,"syntology":null}],"syntology_records":17,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"key_notes":{"n_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"since_archive_judged":{"papers":0,"papers_placed":0,"judged_by":{"model":"a model","models_as_recorded":[],"from":"lb_rows_run_verified.jsonl: each record's `model` field (the extractor's call that wrote decisions[])"},"note":"a model's judgement from the paper's own tables that they report on this leaderboard, not a result: no value from a paper that was not placed is printed or carried here; papers counts each paper once over these leaderboards","backfill":"8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read)","backfill_complete":false,"leaderboards":[]},"papers_where_code_ran":16,"papers_where_code_ran_first_page":"/task/distributional-reinforcement-learning/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not run it on this task or check it against the task's benchmarks.","pages":[{"order":"archive","page":2,"of":2,"url":"/task/distributional-reinforcement-learning/papers/2","json":"/task/distributional-reinforcement-learning/papers/2.json","rows":37},{"order":"ran","page":1,"of":1,"url":"/task/distributional-reinforcement-learning/papers/ran/1","json":"/task/distributional-reinforcement-learning/papers/ran/1.json","rows":16}],"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-10-05T00:36:11+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"row_route":"each syntology object on a benchmark's first row and on a paper carries sample_list (this site's paper page, #ran), syntology_url (syntology.ai's page for the paper, only when that route probed live) and mcp (the agent call that lists the paper's harvested samples; not a verdict on the paper)","developers":"https://syntology.ai/developers"},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"},"record_sha256":"3313d78dd730522dc9e8d8886437cf31258fc2b2b3642e67e8fd0725ec9750a2","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}