{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/jointly-pre-training-with-supervised","title":"Jointly Pre-training with Supervised, Autoencoder, and Value Losses for Deep Reinforcement Learning","arxiv_id":"1904.02206","date":"2019-04-03","proceeding":null,"authors":["Gabriel V. de la Cruz Jr.","Yunshu Du","Matthew E. Taylor"],"abstract":"Deep Reinforcement Learning (DRL) algorithms are known to be data\ninefficient. One reason is that a DRL agent learns both the feature and the\npolicy tabula rasa. Integrating prior knowledge into DRL algorithms is one way\nto improve learning efficiency since it helps to build helpful representations.\nIn this work, we consider incorporating human knowledge to accelerate the\nasynchronous advantage actor-critic (A3C) algorithm by pre-training a small\namount of non-expert human demonstrations. We leverage the supervised\nautoencoder framework and propose a novel pre-training strategy that jointly\ntrains a weighted supervised classification loss, an unsupervised\nreconstruction loss, and an expected return loss. The resulting pre-trained\nmodel learns more useful features compared to independently training in\nsupervised or unsupervised fashion. Our pre-training method drastically\nimproved the learning performance of the A3C agent in Atari games of Pong and\nMsPacman, exceeding the performance of the state-of-the-art algorithms at a\nmuch smaller number of game interactions. Our method is light-weight and easy\nto implement in a single machine. For reproducibility, our code is available at\ngithub.com/gabrieledcjr/DeepRL/tree/A3C-ALA2019","url_abs":"http://arxiv.org/abs/1904.02206v1","url_pdf":"http://arxiv.org/pdf/1904.02206v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"jointly-pre-training-with-supervised","repo_url":"https://github.com/gabrieledcjr/DeepRL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"a3c","method_name":"A3C"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"gabrieledcjr/DeepRL","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"c0fdc394d7ff2994f8850f8ec3c8b5e53f383ee87f918163dbb33b54d3c56e4a","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}