{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/language-modeling-with-sparse-product-of","title":"Language Modeling with Sparse Product of Sememe Experts","arxiv_id":"1810.12387","date":"2018-10-29","proceeding":"EMNLP 2018 10","authors":["Yihong Gu","Jun Yan","Hao Zhu","Zhiyuan Liu","Ruobing Xie","Maosong Sun","Fen Lin","Leyu Lin"],"abstract":"Most language modeling methods rely on large-scale data to statistically\nlearn the sequential patterns of words. In this paper, we argue that words are\natomic language units but not necessarily atomic semantic units. Inspired by\nHowNet, we use sememes, the minimum semantic units in human languages, to\nrepresent the implicit semantics behind words for language modeling, named\nSememe-Driven Language Model (SDLM). More specifically, to predict the next\nword, SDLM first estimates the sememe distribution gave textual context.\nAfterward, it regards each sememe as a distinct semantic expert, and these\nexperts jointly identify the most probable senses and the corresponding word.\nIn this way, SDLM enables language models to work beyond word-level\nmanipulation to fine-grained sememe-level semantics and offers us more powerful\ntools to fine-tune language models and improve the interpretability as well as\nthe robustness of language models. Experiments on language modeling and the\ndownstream application of headline gener- ation demonstrate the significant\neffect of SDLM. Source code and data used in the experiments can be accessed at\nhttps:// github.com/thunlp/SDLM-pytorch.","url_abs":"http://arxiv.org/abs/1810.12387v1","url_pdf":"http://arxiv.org/pdf/1810.12387v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"language-modeling-with-sparse-product-of","repo_url":"https://github.com/thunlp/SDLM-pytorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1810.12387","atlas_url":"https://app.syntology.ai/?focus=1810.12387","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1810.12387"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1810.12387\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1810.12387"},"call":"get_concepts_for_paper(arxiv_id=\"1810.12387\")"}],"by_repo":[{"repo":"thunlp/SDLM-pytorch","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"55ce244e510c5274c78c4639ada96bc948eda190f0b24fa9000fdb9c4cddeb62","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}