{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/improving-grammatical-error-correction-via","title":"Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data","arxiv_id":"1903.00138","date":"2019-03-01","proceeding":"NAACL 2019 6","authors":["Wei Zhao","Liang Wang","Kewei Shen","Ruoyu Jia","Jingming Liu"],"abstract":"Neural machine translation systems have become state-of-the-art approaches for Grammatical Error Correction (GEC) task. In this paper, we propose a copy-augmented architecture for the GEC task by copying the unchanged words from the source sentence to the target sentence. Since the GEC suffers from not having enough labeled training data to achieve high accuracy. We pre-train the copy-augmented architecture with a denoising auto-encoder using the unlabeled One Billion Benchmark and make comparisons between the fully pre-trained model and a partially pre-trained model. It is the first time copying words from the source context and fully pre-training a sequence to sequence model are experimented on the GEC task. Moreover, We add token-level and sentence-level multi-task learning for the GEC task. The evaluation results on the CoNLL-2014 test set show that our approach outperforms all recently published state-of-the-art results by a large margin. The code and pre-trained models are released at https://github.com/zhawe01/fairseq-gec.","url_abs":"https://arxiv.org/abs/1903.00138v3","url_pdf":"https://arxiv.org/pdf/1903.00138v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-grammatical-error-correction-via","repo_url":"https://github.com/zhawe01/fairseq-gec","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"improving-grammatical-error-correction-via","repo_url":"https://github.com/raghavmalawat/presentationmastery","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"improving-grammatical-error-correction-via","repo_url":"https://github.com/s-ankur/fairseq-gec","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"improving-grammatical-error-correction-via","repo_url":"https://github.com/soyoung97/fairseq-gec-korean","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"improving-grammatical-error-correction-via","repo_url":"https://github.com/youichiro/transformer-copy","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"improving-grammatical-error-correction-via","repo_url":"https://github.com/yuantiku/fairseq-gec","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"grammatical-error-correction","task_name":"Grammatical Error Correction"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/grammatical-error-correction-on-conll-2014","task":"Grammatical Error Correction","dataset":"CoNLL-2014 Shared Task","model":"Copy-augmented Model (4 Ensemble +Denoising Autoencoder)","rank_in_archive_order":18,"of":23,"metrics":{"F0.5":"61.15","Precision":"71.57","Recall":"38.65"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-correction-on-jfleg","task":"Grammatical Error Correction","dataset":"JFLEG","model":"Copy-augmented Model (4 Ensemble +Denoising Autoencoder)","rank_in_archive_order":4,"of":6,"metrics":{"GLEU":"61.0"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1903.00138","atlas_url":"https://app.syntology.ai/?focus=1903.00138","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1903.00138"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1903.00138\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1903.00138"},"call":"get_concepts_for_paper(arxiv_id=\"1903.00138\")"}],"by_repo":[{"repo":"zhawe01/fairseq-gec","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"raghavmalawat/presentationmastery","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"s-ankur/fairseq-gec","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"soyoung97/fairseq-gec-korean","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"youichiro/transformer-copy","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"yuantiku/fairseq-gec","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"8835eb69b28787abd657873108ddbf66f6a9032df306460ae2cc0a4df2e1e52e","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}