{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/universal-transformer","slug":"universal-transformer","name":"Universal Transformer","full_name":"Universal Transformer","full_name_withheld":false,"description_markdown":"The **Universal Transformer** is a generalization of the [Transformer](https://paperswithcode.com/method/transformer) architecture. Universal Transformers combine the parallelizability and global receptive field of feed-forward sequence models like the Transformer with the recurrent inductive bias of [RNNs](https://paperswithcode.com/methods/category/recurrent-neural-networks). They also utilise a dynamic per-position halting mechanism.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Universal Transformers","paper":"/paper/universal-transformers","first_author":"Mostafa Dehghani","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/universal-transformers"},"source":{"url":"http://arxiv.org/abs/1807.03819v3","title":"Universal Transformers","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Autoregressive Transformers","url":"/methods/category/autoregressive-transformers","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]}],"n_papers_tagged":18,"archive_num_papers":18,"papers_newest_first":[{"paper":null,"slug":"pluto-pathology-universal-transformer","title":"PLUTO: Pathology-Universal Transformer","date":"2024-05-13","arxiv_id":"2405.07905","n_code_links":0,"syntology":null},{"paper":"/paper/recurrent-transformers-with-dynamic-halt","slug":"recurrent-transformers-with-dynamic-halt","title":"Investigating Recurrent Transformers with Dynamic Halt","date":"2024-02-01","arxiv_id":"2402.00976","n_code_links":1,"syntology":null},{"paper":null,"slug":"self-critical-alternate-learning-based","title":"Self-Critical Alternate Learning based Semantic Broadcast Communication","date":"2023-12-03","arxiv_id":"2312.01423","n_code_links":0,"syntology":null},{"paper":"/paper/sparse-universal-transformer","slug":"sparse-universal-transformer","title":"Sparse Universal Transformer","date":"2023-10-11","arxiv_id":"2310.07096","n_code_links":2,"syntology":{"ran":3,"of":5,"n_ran_checked":3,"n_instrument":0,"unverified":2,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","official":null}},{"paper":"/paper/ummaformer-a-universal-multimodal-adaptive-1","slug":"ummaformer-a-universal-multimodal-adaptive-1","title":"UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localization","date":"2023-08-28","arxiv_id":"2308.14395","n_code_links":1,"syntology":null},{"paper":"/paper/beyond-universal-transformer-block-reusing","slug":"beyond-universal-transformer-block-reusing","title":"Beyond Universal Transformer: block reusing with adaptor in Transformer for automatic speech recognition","date":"2023-03-23","arxiv_id":"2303.13072","n_code_links":0,"syntology":null},{"paper":null,"slug":"semantic-communication-with-memory","title":"Semantic Communication with Memory","date":"2023-03-22","arxiv_id":"2303.12335","n_code_links":0,"syntology":null},{"paper":"/paper/towards-autoformalization-of-mathematics-and","slug":"towards-autoformalization-of-mathematics-and","title":"Towards Autoformalization of Mathematics and Code Correctness: Experiments with Elementary Proofs","date":"2023-01-05","arxiv_id":"2301.02195","n_code_links":1,"syntology":{"ran":5,"of":7,"n_ran_checked":5,"n_instrument":0,"unverified":2,"pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","official":{"repos":["gc974517/autoformalization"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"universal-transformer-hawkes-process-with","title":"Universal Transformer Hawkes Process with Adaptive Recursive Iteration","date":"2021-12-29","arxiv_id":"2112.14479","n_code_links":0,"syntology":null},{"paper":"/paper/the-devil-is-in-the-detail-simple-tricks","slug":"the-devil-is-in-the-detail-simple-tricks","title":"The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers","date":"2021-08-26","arxiv_id":"2108.12284","n_code_links":2,"syntology":{"ran":1,"of":2,"n_ran_checked":1,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["robertcsordas/transformer_generalization"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"using-bert-encoding-and-sentence-level","title":"Using BERT Encoding and Sentence-Level Language Model for Sentence Ordering","date":"2021-08-24","arxiv_id":"2108.10986","n_code_links":0,"syntology":null},{"paper":null,"slug":"semantic-communication-with-adaptive","title":"Semantic Communication with Adaptive Universal Transformer","date":"2021-08-20","arxiv_id":"2108.09119","n_code_links":0,"syntology":null},{"paper":null,"slug":"automatically-ranked-russian-paraphrase","title":"Automatically Ranked Russian Paraphrase Corpus for Text Generation","date":"2020-06-17","arxiv_id":"2006.09719","n_code_links":0,"syntology":null},{"paper":"/paper/universal-transforming-geometric-network","slug":"universal-transforming-geometric-network","title":"Universal Transforming Geometric Network","date":"2019-08-02","arxiv_id":"1908.00723","n_code_links":1,"syntology":null},{"paper":null,"slug":"latent-universal-task-specific-bert","title":"Latent Universal Task-Specific BERT","date":"2019-05-16","arxiv_id":"1905.06638","n_code_links":0,"syntology":null},{"paper":null,"slug":"self-attentive-model-for-headline-generation","title":"Self-Attentive Model for Headline Generation","date":"2019-01-23","arxiv_id":"1901.07786","n_code_links":0,"syntology":null},{"paper":"/paper/attending-to-mathematical-language-with","slug":"attending-to-mathematical-language-with","title":"Attending to Mathematical Language with Transformers","date":"2018-12-05","arxiv_id":"1812.02825","n_code_links":3,"syntology":null},{"paper":"/paper/universal-transformers","slug":"universal-transformers","title":"Universal Transformers","date":"2018-07-10","arxiv_id":"1807.03819","n_code_links":8,"syntology":{"ran":17,"of":25,"n_ran_checked":17,"n_instrument":0,"unverified":8,"pointer_only":24,"phrase":"17 ran (of which 8 constructed an object rather than computing a result; 17 with no instrument failure: 1 honoured, 0 violated, 16 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified","official":{"repos":["tensorflow/tensor2tensor"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}}],"papers_shown":18,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":18,"with_a_code_link":8,"where_syntology_ran_a_sample":4,"not_listed_spam_title":0,"listed":18,"listed_where_code_ran":4,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":4,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":4,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":4,"papers_where_code_ran_first_page":"/method/universal-transformer/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/universal-transformer/papers/ran/1","json":"/method/universal-transformer/papers/ran/1.json","rows":4}],"tasks":[{"task":"/task/sentence","name":"Sentence","papers":4},{"task":"/task/language-modeling","name":"Language Modeling","papers":3},{"task":"/task/language-modelling","name":"Language Modelling","papers":3},{"task":"/task/semantic-communication","name":"Semantic Communication","papers":3},{"task":"/task/sentence-similarity","name":"Sentence Similarity","papers":2},{"task":"/task/text-generation","name":"Text Generation","papers":2},{"task":"/task/articles","name":"Articles","papers":1},{"task":"/task/automatic-speech-recognition-2","name":"Automatic Speech Recognition","papers":1},{"task":"/task/automatic-speech-recognition","name":"Automatic Speech Recognition (ASR)","papers":1},{"task":"/task/binary-classification","name":"Binary Classification","papers":1},{"task":"/task/decoder","name":"Decoder","papers":1},{"task":"/task/diagnostic","name":"Diagnostic","papers":1},{"task":"/task/document-summarization","name":"Document Summarization","papers":1},{"task":"/task/headline-generation","name":"Headline Generation","papers":1},{"task":"/task/inductive-bias","name":"Inductive Bias","papers":1},{"task":"/task/instance-segmentation","name":"Instance Segmentation","papers":1},{"task":"/task/lambada","name":"LAMBADA","papers":1},{"task":"/task/learning-to-execute","name":"Learning to Execute","papers":1},{"task":"/task/listops","name":"ListOps","papers":1},{"task":"/task/logical-sequence","name":"Logical Sequence","papers":1}],"tasks_shown":20,"n_tasks":43,"usage_by_year":[{"year":"2018","papers":2},{"year":"2019","papers":3},{"year":"2020","papers":1},{"year":"2021","papers":4},{"year":"2023","papers":6},{"year":"2024","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/universal-transformer"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Universal%20Transformer","read_at":"2026-10-05T01:03:28+00:00","query":"Universal Transformer","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"Universal Transformer","origin_arxiv_id":"1807.03819"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"1807.03819","repositories":{"listed":5,"in_graph":8,"list_truncated":true,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"1807.03819","citations":null},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'Universal Transformer'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[{"name":"Agentic Transformer","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Agentic Transformer')","origin_arxiv_id":"2305.16554"},{"name":"universal technique","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='universal technique')","origin_arxiv_id":"2112.03865"},{"name":"Waypoint Transformer","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='Waypoint Transformer')","origin_arxiv_id":"2306.14069"},{"name":"Block-State Transformer","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Block-State Transformer')","origin_arxiv_id":"2306.09539"},{"name":"Bootstrapped Transformer","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Bootstrapped Transformer')","origin_arxiv_id":"2206.08569"},{"name":"A Universal Error Measure","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='A Universal Error Measure')","origin_arxiv_id":"2205.12850"}],"code_nearby_caption":"Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.","origin_paper":{"arxiv_id":"1807.03819","page":"/paper/universal-transformers","syntology_url":"https://syntology.ai/paper/1807.03819","in_graph_layer":true,"samples":25,"ran":17,"n_ran_checked":17,"n_instrument":0,"unverified":8,"phrase":"17 ran (of which 8 constructed an object rather than computing a result; 17 with no instrument failure: 1 honoured, 0 violated, 16 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"repository known, no verified implementation (declared_code; repositories: 5 listed of 8 in the graph, strongest link: official). This answer is about arXiv:1807.03819, the paper the archive credits with this method.","level_legend":{"state":"quoted","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","card_sha256":"479785a84658dd682a11b10237b3620d651d7f87799f5dc7add970aaa83f6d39","note":"Syntology's served verification levels, quoted verbatim from its MCP server card; a row is split where the card separates clauses with a double hyphen, and the separator is not reproduced","levels":[{"level":"V0","quoted":["imports"]},{"level":"V1","quoted":["executes against a synthesized fixture and returns a value of the recorded type"]},{"level":"V2","quoted":["two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs","neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)"]},{"level":"V3","quoted":["paper-derived property tests pass on both of those same two generations."]}],"no_rung":"No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results."}},"syntology_have_state":"loaded","record_sha256":"26a0b211492d5307627e9f484ad2938703b11178b69a09f72d9f4aa438a63474","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}