{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/urdu-word-segmentation-using-conditional","title":"Urdu Word Segmentation using Conditional Random Fields (CRFs)","arxiv_id":"1806.05432","date":"2018-06-14","proceeding":"COLING 2018 8","authors":["Haris Bin Zia","Agha Ali Raza","Awais Athar"],"abstract":"State-of-the-art Natural Language Processing algorithms rely heavily on\nefficient word segmentation. Urdu is amongst languages for which word\nsegmentation is a complex task as it exhibits space omission as well as space\ninsertion issues. This is partly due to the Arabic script which although\ncursive in nature, consists of characters that have inherent joining and\nnon-joining attributes regardless of word boundary. This paper presents a word\nsegmentation system for Urdu which uses a Conditional Random Field sequence\nmodeler with orthographic, linguistic and morphological features. Our proposed\nmodel automatically learns to predict white space as word boundary as well as\nZero Width Non-Joiner (ZWNJ) as sub-word boundary. Using a manually annotated\ncorpus, our model achieves F1 score of 0.97 for word boundary identification\nand 0.85 for sub-word boundary identification tasks. We have made our code and\ncorpus publicly available to make our results reproducible.","url_abs":"http://arxiv.org/abs/1806.05432v1","url_pdf":"http://arxiv.org/pdf/1806.05432v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"urdu-word-segmentation-using-conditional","repo_url":"https://github.com/harisbinzia/Urdu-Word-Segmentation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"segmentation","task_name":"Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1806.05432","atlas_url":"https://app.syntology.ai/?focus=1806.05432","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1806.05432"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1806.05432\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1806.05432"},"call":"get_concepts_for_paper(arxiv_id=\"1806.05432\")"}],"by_repo":[{"repo":"harisbinzia/Urdu-Word-Segmentation","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"3741a586ad9888b872e4de5a862272e878dedc3964fdb9ac00af71aa0b374478","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}