{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/levenshtein-transformer","slug":"levenshtein-transformer","name":"Levenshtein Transformer","full_name":"Levenshtein Transformer","full_name_withheld":false,"description_markdown":"The **Levenshtein Transformer** (LevT) is a type of [transformer](https://paperswithcode.com/method/transformer) that aims to address the lack of flexibility of previous decoding models. Notably, in previous frameworks, the length of generated sequences is either fixed or monotonically increased as the decoding proceeds. The authors argue this is incompatible with human-level intelligence where humans can revise, replace, revoke or delete any part of their generated text. Hence, LevT is proposed to bridge this gap by breaking the in-so-far standardized decoding mechanism and replacing it with two basic operations — insertion and deletion.\r\n\r\nLevT is trained using imitation learning. The resulted model contains two policies and they are executed in an alternate manner. The authors argue that with this model decoding becomes more flexible. For example, when the decoder is given an empty token, it falls back to a normal sequence generation model. On the other hand, the decoder acts as a refinement model when the initial state is a low-quality generated sequence.\r\n\r\nOne crucial component in LevT framework is the learning algorithm. The authors leverage the characteristics of insertion and deletion — they are complementary but also adversarial. The algorithm they propose is called “dual policy learning”. The idea is that when training one policy (insertion or deletion), we use the output from its adversary at the previous iteration as input. An expert policy, on the other hand, is drawn to provide a correction signal.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Levenshtein Transformer","paper":"/paper/levenshtein-transformer","first_author":"Jiatao Gu","n_authors":3,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/levenshtein-transformer"},"source":{"url":"https://arxiv.org/abs/1905.11006v2","title":"Levenshtein Transformer","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/pytorch/fairseq/blob/28876638114948711fd4bd4e350fdd6809013f1e/fairseq/models/nat/levenshtein_transformer.py#L34","code_snippet_url_on_a_code_host":true,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Autoregressive Transformers","url":"/methods/category/autoregressive-transformers","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]}],"n_papers_tagged":12,"archive_num_papers":12,"papers_newest_first":[{"paper":null,"slug":"reinforcement-learning-for-edit-based-non","title":"Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translation","date":"2024-05-02","arxiv_id":"2405.01280","n_code_links":0,"syntology":null},{"paper":"/paper/towards-example-based-nmt-with-multi","slug":"towards-example-based-nmt-with-multi","title":"Towards Example-Based NMT with Multi-Levenshtein Transformers","date":"2023-10-13","arxiv_id":"2310.08967","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["maxwell1447/fairseq"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/non-autoregressive-machine-translation-with-2","slug":"non-autoregressive-machine-translation-with-2","title":"Integrating Translation Memories into Non-Autoregressive Machine Translation","date":"2022-10-12","arxiv_id":"2210.06020","n_code_links":1,"syntology":null},{"paper":"/paper/levenshtein-ocr","slug":"levenshtein-ocr","title":"Levenshtein OCR","date":"2022-09-08","arxiv_id":"2209.03594","n_code_links":2,"syntology":null},{"paper":null,"slug":"the-jhu-microsoft-submission-for-wmt21","title":"The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task","date":"2021-09-17","arxiv_id":"2109.08724","n_code_links":0,"syntology":null},{"paper":"/paper/levenshtein-training-for-word-level-quality","slug":"levenshtein-training-for-word-level-quality","title":"Levenshtein Training for Word-level Quality Estimation","date":"2021-09-12","arxiv_id":"2109.05611","n_code_links":1,"syntology":null},{"paper":null,"slug":"using-perturbed-length-aware-positional","title":"Using Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translation","date":"2021-07-29","arxiv_id":"2107.13689","n_code_links":0,"syntology":null},{"paper":null,"slug":"how-does-distilled-data-complexity-impact-the","title":"How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?","date":"2021-05-27","arxiv_id":"2105.12900","n_code_links":0,"syntology":null},{"paper":"/paper/editor-an-edit-based-transformer-with","slug":"editor-an-edit-based-transformer-with","title":"EDITOR: an Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints","date":"2020-11-13","arxiv_id":"2011.06868","n_code_links":1,"syntology":null},{"paper":null,"slug":"expand-and-filter-cuni-and-lmu-systems-for","title":"Expand and Filter: CUNI and LMU Systems for the WNGT 2020 Duolingo Shared Task","date":"2020-07-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/lexically-constrained-neural-machine","slug":"lexically-constrained-neural-machine","title":"Lexically Constrained Neural Machine Translation with Levenshtein Transformer","date":"2020-04-27","arxiv_id":"2004.12681","n_code_links":1,"syntology":{"ran":2,"of":4,"n_ran_checked":1,"n_instrument":1,"unverified":2,"pointer_only":4,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified","official":{"repos":["raymondhs/constrained-levt"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/levenshtein-transformer","slug":"levenshtein-transformer","title":"Levenshtein Transformer","date":"2019-05-27","arxiv_id":"1905.11006","n_code_links":3,"syntology":null}],"papers_shown":12,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":12,"with_a_code_link":7,"where_syntology_ran_a_sample":2,"not_listed_spam_title":0,"listed":12,"listed_where_code_ran":2,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":1,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":1,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":2,"papers_where_code_ran_first_page":"/method/levenshtein-transformer/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/levenshtein-transformer/papers/ran/1","json":"/method/levenshtein-transformer/papers/ran/1.json","rows":2}],"tasks":[{"task":"/task/translation","name":"Translation","papers":11},{"task":"/task/machine-translation","name":"Machine Translation","papers":8},{"task":"/task/imitation-learning","name":"Imitation Learning","papers":3},{"task":"/task/knowledge-distillation","name":"Knowledge Distillation","papers":3},{"task":"/task/diversity","name":"Diversity","papers":2},{"task":"/task/nmt","name":"NMT","papers":2},{"task":"/task/automatic-post-editing","name":"Automatic Post-Editing","papers":1},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":1},{"task":"/task/domain-adaptation","name":"Domain Adaptation","papers":1},{"task":"/task/optical-character-recognition","name":"Optical Character Recognition (OCR)","papers":1},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":1},{"task":"/task/retrieval","name":"Retrieval","papers":1},{"task":"/task/scene-text-recognition","name":"Scene Text Recognition","papers":1},{"task":"/task/task-2","name":"Task 2","papers":1},{"task":"/task/text-summarization","name":"Text Summarization","papers":1},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":1},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":1}],"tasks_shown":17,"n_tasks":17,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":3},{"year":"2021","papers":4},{"year":"2022","papers":2},{"year":"2023","papers":1},{"year":"2024","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/levenshtein-transformer"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Levenshtein%20Transformer","read_at":"2026-10-05T00:58:30+00:00","query":"Levenshtein Transformer","kind":"paper","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"arxiv_id":"1905.11006"},"resolution":"paper","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"1905.11006","repositories":{"listed":3,"in_graph":3,"list_truncated":false,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"1905.11006","citations":405},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'Levenshtein Transformer'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"1905.11006","page":"/paper/levenshtein-transformer","syntology_url":"https://syntology.ai/paper/1905.11006","in_graph_layer":false,"samples":0},"line":"repository known, no verified implementation (declared_code; repositories: 3 listed of 3 in the graph, strongest link: official). The lookup matched a paper by its title, arXiv:1905.11006, not a method entry. This answer is about arXiv:1905.11006, the paper the archive credits with this method.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"6b5d69bc810cbc56449b76e8e0bdb8f552c3590f97ec1f05aa71599658115c14","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}