{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/data-upcycling-knowledge-distillation-for","title":"Data Upcycling Knowledge Distillation for Image Super-Resolution","arxiv_id":"2309.14162","date":"2023-09-25","proceeding":null,"authors":["Yun Zhang","Wei Li","Simiao Li","Hanting Chen","Zhijun Tu","Wenjia Wang","BingYi Jing","Shaohui Lin","Jie Hu"],"abstract":"Knowledge distillation (KD) compresses deep neural networks by transferring task-related knowledge from cumbersome pre-trained teacher models to compact student models. However, current KD methods for super-resolution (SR) networks overlook the nature of SR task that the outputs of the teacher model are noisy approximations to the ground-truth distribution of high-quality images (GT), which shades the teacher model's knowledge to result in limited KD effects. To utilize the teacher model beyond the GT upper-bound, we present the Data Upcycling Knowledge Distillation (DUKD), to transfer the teacher model's knowledge to the student model through the upcycled in-domain data derived from training data. Besides, we impose label consistency regularization to KD for SR by the paired invertible augmentations to improve the student model's performance and robustness. Comprehensive experiments demonstrate that the DUKD method significantly outperforms previous arts on several SR tasks.","url_abs":"https://arxiv.org/abs/2309.14162v4","url_pdf":"https://arxiv.org/pdf/2309.14162v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-upcycling-knowledge-distillation-for","repo_url":"https://github.com/yun224/dukd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-super-resolution","task_name":"Image Super-Resolution"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"super-resolution","task_name":"Super-Resolution"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-super-resolution-on-set14-4x-upscaling","task":"Image Super-Resolution","dataset":"Set14 - 4x upscaling","model":"DUKD","rank_in_archive_order":50,"of":104,"metrics":{"PSNR":"28.80","SSIM":"0.7866"},"uses_additional_data":false},{"leaderboard":"/sota/image-super-resolution-on-urban100-4x","task":"Image Super-Resolution","dataset":"Urban100 - 4x upscaling","model":"RCAN (DUKD)","rank_in_archive_order":32,"of":65,"metrics":{"PSNR":"26.62","SSIM":"0.802"},"uses_additional_data":false},{"leaderboard":"/sota/image-super-resolution-on-urban100-4x","task":"Image Super-Resolution","dataset":"Urban100 - 4x upscaling","model":"EDSR (DUKD)","rank_in_archive_order":37,"of":65,"metrics":{"PSNR":"26.45","SSIM":"0.7963"},"uses_additional_data":false},{"leaderboard":"/sota/image-super-resolution-on-urban100-4x","task":"Image Super-Resolution","dataset":"Urban100 - 4x upscaling","model":"SwinIR (DUKD)","rank_in_archive_order":38,"of":65,"metrics":{"PSNR":"26.43","SSIM":"0.7972"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.14162","atlas_url":"https://app.syntology.ai/?focus=2309.14162","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2309.14162"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2309.14162\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2309.14162"},"call":"get_concepts_for_paper(arxiv_id=\"2309.14162\")"}],"by_repo":[{"repo":"yun224/dukd","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"315104c4bee345c5ff65c0726688c5f6e1e9434f1133e39831f2dda670a762e7","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}