{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/subword-encoding-in-lattice-lstm-for-chinese","title":"Subword Encoding in Lattice LSTM for Chinese Word Segmentation","arxiv_id":"1810.12594","date":"2018-10-30","proceeding":"NAACL 2019 6","authors":["Jie Yang","Yue Zhang","Shuailong Liang"],"abstract":"We investigate a lattice LSTM network for Chinese word segmentation (CWS) to\nutilize words or subwords. It integrates the character sequence features with\nall subsequences information matched from a lexicon. The matched subsequences\nserve as information shortcut tunnels which link their start and end characters\ndirectly. Gated units are used to control the contribution of multiple input\nlinks. Through formula derivation and comparison, we show that the lattice LSTM\nis an extension of the standard LSTM with the ability to take multiple inputs.\nPrevious lattice LSTM model takes word embeddings as the lexicon input, we\nprove that subword encoding can give the comparable performance and has the\nbenefit of not relying on any external segmentor. The contribution of lattice\nLSTM comes from both lexicon and pretrained embeddings information, we find\nthat the lexicon information contributes more than the pretrained embeddings\ninformation through controlled experiments. Our experiments show that the\nlattice structure with subword encoding gives competitive or better results\nwith previous state-of-the-art methods on four segmentation benchmarks.\nDetailed analyses are conducted to compare the performance of word encoding and\nsubword encoding in lattice LSTM. We also investigate the performance of\nlattice LSTM structure under different circumstances and when this model works\nor fails.","url_abs":"http://arxiv.org/abs/1810.12594v1","url_pdf":"http://arxiv.org/pdf/1810.12594v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"subword-encoding-in-lattice-lstm-for-chinese","repo_url":"https://github.com/jiesutd/SubwordEncoding-CWS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"chinese-word-segmentation","task_name":"Chinese Word Segmentation"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1810.12594","atlas_url":"https://app.syntology.ai/?focus=1810.12594","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1810.12594"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1810.12594\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1810.12594"},"call":"get_concepts_for_paper(arxiv_id=\"1810.12594\")"}],"by_repo":[{"repo":"jiesutd/SubwordEncoding-CWS","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"6740a7f98e27c5af4d533df93f5489371f2336909a14a93d06fc2a1e76b1514c","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}