{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/aixijs-a-software-demo-for-general","title":"AIXIjs: A Software Demo for General Reinforcement Learning","arxiv_id":"1705.07615","date":"2017-05-22","proceeding":null,"authors":["John Aslanides"],"abstract":"Reinforcement learning is a general and powerful framework with which to\nstudy and implement artificial intelligence. Recent advances in deep learning\nhave enabled RL algorithms to achieve impressive performance in restricted\ndomains such as playing Atari video games (Mnih et al., 2015) and, recently,\nthe board game Go (Silver et al., 2016). However, we are still far from\nconstructing a generally intelligent agent. Many of the obstacles and open\nquestions are conceptual: What does it mean to be intelligent? How does one\nexplore and learn optimally in general, unknown environments? What, in fact,\ndoes it mean to be optimal in the general sense? The universal Bayesian agent\nAIXI (Hutter, 2005) is a model of a maximally intelligent agent, and plays a\ncentral role in the sub-field of general reinforcement learning (GRL).\nRecently, AIXI has been shown to be flawed in important ways; it doesn't\nexplore enough to be asymptotically optimal (Orseau, 2010), and it can perform\npoorly with certain priors (Leike and Hutter, 2015). Several variants of AIXI\nhave been proposed to attempt to address these shortfalls: among them are\nentropy-seeking agents (Orseau, 2011), knowledge-seeking agents (Orseau et al.,\n2013), Bayes with bursts of exploration (Lattimore, 2013), MDL agents (Leike,\n2016a), Thompson sampling (Leike et al., 2016), and optimism (Sunehag and\nHutter, 2015). We present AIXIjs, a JavaScript implementation of these GRL\nagents. This implementation is accompanied by a framework for running\nexperiments against various environments, similar to OpenAI Gym (Brockman et\nal., 2016), and a suite of interactive demos that explore different properties\nof the agents, similar to REINFORCEjs (Karpathy, 2015). We use AIXIjs to\npresent numerous experiments illustrating fundamental properties of, and\ndifferences between, these agents.","url_abs":"http://arxiv.org/abs/1705.07615v1","url_pdf":"http://arxiv.org/pdf/1705.07615v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"aixijs-a-software-demo-for-general","repo_url":"https://github.com/aslanides/aixijs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"general-reinforcement-learning","task_name":"General Reinforcement Learning"},{"task_slug":"openai-gym","task_name":"OpenAI Gym"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"thompson-sampling","task_name":"Thompson Sampling"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"mdl","method_name":"MDL"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"aslanides/aixijs","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"c56fa9f92cd24cb05efc62cb954b1e5c37e40ffc874824d2915703d7c47dd680","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}