{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/beyond-masked-and-unmasked-discrete-diffusion","title":"Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking","arxiv_id":"2505.18495","date":"2025-05-24","proceeding":null,"authors":["Chen-Hao Chao","Wei-Fang Sun","Hanwen Liang","Chun-Yi Lee","Rahul G. Krishnan"],"abstract":"Masked diffusion models (MDM) are powerful generative models for discrete data that generate samples by progressively unmasking tokens in a sequence. Each token can take one of two states: masked or unmasked. We observe that token sequences often remain unchanged between consecutive sampling steps; consequently, the model repeatedly processes identical inputs, leading to redundant computation. To address this inefficiency, we propose the Partial masking scheme (Prime), which augments MDM by allowing tokens to take intermediate states interpolated between the masked and unmasked states. This design enables the model to make predictions based on partially observed token information, and facilitates a fine-grained denoising process. We derive a variational training objective and introduce a simple architectural design to accommodate intermediate-state inputs. Our method demonstrates superior performance across a diverse set of generative modeling tasks. On text data, it achieves a perplexity of 15.36 on OpenWebText, outperforming previous MDM (21.52), autoregressive models (17.54), and their hybrid variants (17.58), without relying on an autoregressive formulation. On image data, it attains competitive FID scores of 3.26 on CIFAR-10 and 6.98 on ImageNet-32, comparable to leading continuous generative models.","url_abs":"https://arxiv.org/abs/2505.18495v1","url_pdf":"https://arxiv.org/pdf/2505.18495v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-cifar-10","task":"Image Generation","dataset":"CIFAR-10","model":"MDM-Prime","rank_in_archive_order":29,"of":78,"metrics":{"FID":"3.26","IS":"9.67"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-cifar-10","task":"Image Generation","dataset":"CIFAR-10","model":"MDM","rank_in_archive_order":33,"of":78,"metrics":{"FID":"4.66","IS":"9.09"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-32x32","task":"Image Generation","dataset":"ImageNet 32x32","model":"MDM-Prime","rank_in_archive_order":6,"of":35,"metrics":{"FID":"6.98","Inception score":"11.65"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-32x32","task":"Image Generation","dataset":"ImageNet 32x32","model":"MDM","rank_in_archive_order":7,"of":35,"metrics":{"FID":"7.91","Inception score":"11.60"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-openwebtext","task":"Language Modelling","dataset":"OpenWebText","model":"MDLM-Prime","rank_in_archive_order":1,"of":12,"metrics":{"eval_perplexity":"15.36","parameters":"131M"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.18495","atlas_url":"https://app.syntology.ai/?focus=2505.18495","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2505.18495"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2505.18495\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2505.18495"},"call":"get_concepts_for_paper(arxiv_id=\"2505.18495\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"8bb7f98cd75f6cb08373bd0a4a91752b785ba23505dc8f58195827d717e79810","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}