{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/exploration-by-self-supervised-exploitation","title":"Self-supervised network distillation: an effective approach to exploration in sparse reward environments","arxiv_id":"2302.11563","date":"2023-02-22","proceeding":null,"authors":["Matej Pecháč","Michal Chovanec","Igor Farkaš"],"abstract":"Reinforcement learning can solve decision-making problems and train an agent to behave in an environment according to a predesigned reward function. However, such an approach becomes very problematic if the reward is too sparse and so the agent does not come across the reward during the environmental exploration. The solution to such a problem may be to equip the agent with an intrinsic motivation that will provide informed exploration during which the agent is likely to also encounter external reward. Novelty detection is one of the promising branches of intrinsic motivation research. We present Self-supervised Network Distillation (SND), a class of intrinsic motivation algorithms based on the distillation error as a novelty indicator, where the predictor model and the target model are both trained. We adapted three existing self-supervised methods for this purpose and experimentally tested them on a set of ten environments that are considered difficult to explore. The results show that our approach achieves faster growth and higher external reward for the same training time compared to the baseline models, which implies improved exploration in a very sparse reward environment. In addition, the analytical methods we applied provide valuable explanatory insights into our proposed models.","url_abs":"https://arxiv.org/abs/2302.11563v4","url_pdf":"https://arxiv.org/pdf/2302.11563v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploration-by-self-supervised-exploitation","repo_url":"https://github.com/iskandor/snd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"exploration-by-self-supervised-exploitation","repo_url":"https://github.com/michalnand/reinforcement_learning","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"novelty-detection","task_name":"Novelty Detection"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/atari-games-on-atari-2600-gravitar","task":"Atari Games","dataset":"Atari 2600 Gravitar","model":"SND-VIC","rank_in_archive_order":5,"of":53,"metrics":{"Score":"6712"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gravitar","task":"Atari Games","dataset":"Atari 2600 Gravitar","model":"SND-STD","rank_in_archive_order":10,"of":53,"metrics":{"Score":"4643"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gravitar","task":"Atari Games","dataset":"Atari 2600 Gravitar","model":"SND-V","rank_in_archive_order":14,"of":53,"metrics":{"Score":"2741"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-montezumas-revenge","task":"Atari Games","dataset":"Atari 2600 Montezuma's Revenge","model":"SND-V","rank_in_archive_order":3,"of":50,"metrics":{"Score":"21565"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-montezumas-revenge","task":"Atari Games","dataset":"Atari 2600 Montezuma's Revenge","model":"SND-VIC","rank_in_archive_order":6,"of":50,"metrics":{"Score":"7838"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-montezumas-revenge","task":"Atari Games","dataset":"Atari 2600 Montezuma's Revenge","model":"SND-STD","rank_in_archive_order":7,"of":50,"metrics":{"Score":"7212"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-pitfall","task":"Atari Games","dataset":"Atari 2600 Pitfall!","model":"SND-V","rank_in_archive_order":17,"of":23,"metrics":{"Score":"0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-pitfall","task":"Atari Games","dataset":"Atari 2600 Pitfall!","model":"SND-VIC","rank_in_archive_order":18,"of":23,"metrics":{"Score":"0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-private-eye","task":"Atari Games","dataset":"Atari 2600 Private Eye","model":"SND-VIC","rank_in_archive_order":3,"of":52,"metrics":{"Score":"17313"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-private-eye","task":"Atari Games","dataset":"Atari 2600 Private Eye","model":"SND-STD","rank_in_archive_order":9,"of":52,"metrics":{"Score":"15089"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-private-eye","task":"Atari Games","dataset":"Atari 2600 Private Eye","model":"SND-V","rank_in_archive_order":15,"of":52,"metrics":{"Score":"4213"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-solaris","task":"Atari Games","dataset":"Atari 2600 Solaris","model":"SND-STD","rank_in_archive_order":3,"of":23,"metrics":{"Score":"12460"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-solaris","task":"Atari Games","dataset":"Atari 2600 Solaris","model":"SND-VIC","rank_in_archive_order":4,"of":23,"metrics":{"Score":"11865"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-solaris","task":"Atari Games","dataset":"Atari 2600 Solaris","model":"SND-V","rank_in_archive_order":5,"of":23,"metrics":{"Score":"11582"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-venture","task":"Atari Games","dataset":"Atari 2600 Venture","model":"SND-VIC","rank_in_archive_order":3,"of":55,"metrics":{"Score":"2188"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-venture","task":"Atari Games","dataset":"Atari 2600 Venture","model":"SND-STD","rank_in_archive_order":4,"of":55,"metrics":{"Score":"2138"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-venture","task":"Atari Games","dataset":"Atari 2600 Venture","model":"SND-V","rank_in_archive_order":11,"of":55,"metrics":{"Score":"1787"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"iskandor/snd","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"michalnand/reinforcement_learning","kind":"named_in_paper","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["1810.12894"],"harvested_for_calls":[{"arxiv_id":"1810.12894","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"1810.12894"},"call":"get_harvested_code_for_paper(arxiv_id=\"1810.12894\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"395fd4200ad50127d482e43bea325350f38cbee9abb8af2b9c276275bdeeaf00","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}