{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/towards-redundancy-free-sub-networks-in","title":"Towards Redundancy-Free Sub-networks in Continual Learning","arxiv_id":"2312.00840","date":"2023-12-01","proceeding":null,"authors":["Cheng Chen","Jingkuan Song","Lianli Gao","Heng Tao Shen"],"abstract":"Catastrophic Forgetting (CF) is a prominent issue in continual learning. Parameter isolation addresses this challenge by masking a sub-network for each task to mitigate interference with old tasks. However, these sub-networks are constructed relying on weight magnitude, which does not necessarily correspond to the importance of weights, resulting in maintaining unimportant weights and constructing redundant sub-networks. To overcome this limitation, inspired by information bottleneck, which removes redundancy between adjacent network layers, we propose \\textbf{\\underline{I}nformation \\underline{B}ottleneck \\underline{M}asked sub-network (IBM)} to eliminate redundancy within sub-networks. Specifically, IBM accumulates valuable information into essential weights to construct redundancy-free sub-networks, not only effectively mitigating CF by freezing the sub-networks but also facilitating new tasks training through the transfer of valuable knowledge. Additionally, IBM decomposes hidden representations to automate the construction process and make it flexible. Extensive experiments demonstrate that IBM consistently outperforms state-of-the-art methods. Notably, IBM surpasses the state-of-the-art parameter isolation method with a 70\\% reduction in the number of parameters within sub-networks and an 80\\% decrease in training time.","url_abs":"https://arxiv.org/abs/2312.00840v2","url_pdf":"https://arxiv.org/pdf/2312.00840v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-redundancy-free-sub-networks-in","repo_url":"https://github.com/zackschen/IBM-Net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/continual-learning-on-cifar-100-alexnet-300","task":"Continual Learning","dataset":"CIFAR-100 AlexNet - 300 Epoch","model":"IBM","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"82.69"},"uses_additional_data":false},{"leaderboard":"/sota/continual-learning-on-cifar-100-resnet-18-300","task":"Continual Learning","dataset":"CIFAR-100 ResNet-18 - 300 Epochs","model":"IBM","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"88.15"},"uses_additional_data":false},{"leaderboard":"/sota/continual-learning-on-miniimagenet-resnet-18","task":"Continual Learning","dataset":"MiniImageNet ResNet-18 - 300 Epochs","model":"IBM","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"53.9"},"uses_additional_data":false},{"leaderboard":"/sota/continual-learning-on-tinyimagenet-resnet-18","task":"Continual Learning","dataset":"TinyImageNet ResNet-18 - 300 Epochs","model":"IBM","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"52.38"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.00840","atlas_url":"https://app.syntology.ai/?focus=2312.00840","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2312.00840"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2312.00840\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2312.00840"},"call":"get_concepts_for_paper(arxiv_id=\"2312.00840\")"}],"by_repo":[{"repo":"zackschen/IBM-Net","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"d6a613c6cda5c5e9cd182d8d029b6cbc70f4a11712540c17d5e5db9d9d5bcb25","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}