{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/generate-filter-and-rank-grammaticality","title":"Generate, Filter, and Rank: Grammaticality Classification for Production-Ready NLG Systems","arxiv_id":"1904.03279","date":"2019-04-05","proceeding":"NAACL 2019 6","authors":["Ashwini Challa","Kartikeya Upasani","Anusha Balakrishnan","Rajen Subba"],"abstract":"Neural approaches to Natural Language Generation (NLG) have been promising\nfor goal-oriented dialogue. One of the challenges of productionizing these\napproaches, however, is the ability to control response quality, and ensure\nthat generated responses are acceptable. We propose the use of a generate,\nfilter, and rank framework, in which candidate responses are first filtered to\neliminate unacceptable responses, and then ranked to select the best response.\nWhile acceptability includes grammatical correctness and semantic correctness,\nwe focus only on grammaticality classification in this paper, and show that\nexisting datasets for grammatical error correction don't correctly capture the\ndistribution of errors that data-driven generators are likely to make. We\nrelease a grammatical classification and semantic correctness classification\ndataset for the weather domain that consists of responses generated by 3\ndata-driven NLG systems. We then explore two supervised learning approaches\n(CNNs and GBDTs) for classifying grammaticality. Our experiments show that\ngrammaticality classification is very sensitive to the distribution of errors\nin the data, and that these distributions vary significantly with both the\nsource of the response as well as the domain. We show that it's possible to\nachieve high precision with reasonable recall on our dataset.","url_abs":"http://arxiv.org/abs/1904.03279v2","url_pdf":"http://arxiv.org/pdf/1904.03279v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generate-filter-and-rank-grammaticality","repo_url":"https://github.com/facebookresearch/momi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"grammatical-error-correction","task_name":"Grammatical Error Correction"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"facebookresearch/momi","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"44c04f62515142d681b0e6abc5372fb9d4108176d9f763cdaff8df61983cc113","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}