{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/sparse-transformer","slug":"sparse-transformer","name":"Sparse Transformer","full_name":"Sparse Transformer","full_name_withheld":false,"description_markdown":"A **Sparse Transformer** is a [Transformer](https://paperswithcode.com/method/transformer) based architecture which utilises sparse factorizations of the attention matrix to reduce time/memory to $O(n \\sqrt{n})$. Other changes to the Transformer architecture include: (a) a restructured [residual block](https://paperswithcode.com/method/residual-block) and weight initialization, (b) A set of sparse attention kernels which efficiently compute subsets of the attention matrix, (c) recomputation of attention weights during the backwards pass to reduce memory usage","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1904.10509v1","title":"Generating Long Sequences with Sparse Transformers","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]}],"n_papers_tagged":47,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"slug":"pyramid-sparse-transformer-enhancing-multi","title":"Pyramid Sparse Transformer: Enhancing Multi-Scale Feature Fusion with Dynamic Token Selection","date":"2025-05-19","arxiv_id":"2505.12772","n_code_links":0,"syntology":null},{"paper":null,"slug":"tractable-transformers-for-flexible","title":"Tractable Transformers for Flexible Conditional Generation","date":"2025-02-11","arxiv_id":"2502.07616","n_code_links":0,"syntology":null},{"paper":"/paper/deepgate4-efficient-and-effective","slug":"deepgate4-efficient-and-effective","title":"DeepGate4: Efficient and Effective Representation Learning for Circuit Design at Scale","date":"2025-02-02","arxiv_id":"2502.01681","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["zyzheng17/DeepGate4-ICLR-25"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"spartan-a-sparse-transformer-learning-local","title":"SPARTAN: A Sparse Transformer Learning Local Causation","date":"2024-11-11","arxiv_id":"2411.06890","n_code_links":0,"syntology":null},{"paper":null,"slug":"extra-global-attention-designation-using","title":"Extra Global Attention Designation Using Keyword Detection in Sparse Transformer Architectures","date":"2024-10-11","arxiv_id":"2410.08971","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-and-scalable-point-cloud-generation","slug":"efficient-and-scalable-point-cloud-generation","title":"Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Models","date":"2024-08-12","arxiv_id":"2408.06145","n_code_links":2,"syntology":null},{"paper":"/paper/stretching-each-dollar-diffusion-training","slug":"stretching-each-dollar-diffusion-training","title":"Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget","date":"2024-07-22","arxiv_id":"2407.15811","n_code_links":1,"syntology":{"ran":4,"of":6,"n_ran_checked":1,"n_instrument":3,"unverified":2,"pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","official":{"repos":["sonyresearch/micro_diffusion"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"hdt-hierarchical-document-transformer","title":"HDT: Hierarchical Document Transformer","date":"2024-07-11","arxiv_id":"2407.08330","n_code_links":0,"syntology":null},{"paper":"/paper/fibottention-inceptive-visual-representation","slug":"fibottention-inceptive-visual-representation","title":"Fibottention: Inceptive Visual Representation Learning with Diverse Attention Across Heads","date":"2024-06-27","arxiv_id":"2406.19391","n_code_links":1,"syntology":null},{"paper":"/paper/a-mixture-of-experts-approach-to-3d-human","slug":"a-mixture-of-experts-approach-to-3d-human","title":"A Mixture of Experts Approach to 3D Human Motion Prediction","date":"2024-05-09","arxiv_id":"2405.06088","n_code_links":1,"syntology":null},{"paper":"/paper/scene-adaptive-sparse-transformer-for-event","slug":"scene-adaptive-sparse-transformer-for-event","title":"Scene Adaptive Sparse Transformer for Event-based Object Detection","date":"2024-04-02","arxiv_id":"2404.01882","n_code_links":1,"syntology":{"ran":19,"of":23,"n_ran_checked":6,"n_instrument":13,"unverified":4,"pointer_only":0,"phrase":"19 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 13 where Syntology's instrument failed) · 4 unverified","official":{"repos":["peterande/sast"],"state":"official (archive's flag): 19 ran","n_ran":19,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":"/paper/smtf-sparse-transformer-with-multiscale","slug":"smtf-sparse-transformer-with-multiscale","title":"SMTF: Sparse transformer with multiscale contextual fusion for medical image segmentation","date":"2024-03-24","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"segmentation-guided-sparse-transformer-for","title":"Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration","date":"2024-03-09","arxiv_id":"2403.05906","n_code_links":0,"syntology":null},{"paper":null,"slug":"do-efficient-transformers-really-save","title":"Do Efficient Transformers Really Save Computation?","date":"2024-02-21","arxiv_id":"2402.13934","n_code_links":0,"syntology":null},{"paper":"/paper/adapt-or-perish-adaptive-sparse-transformer","slug":"adapt-or-perish-adaptive-sparse-transformer","title":"Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration","date":"2024-01-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"towards-more-unified-in-context-visual","title":"Towards More Unified In-context Visual Understanding","date":"2023-12-05","arxiv_id":"2312.02520","n_code_links":0,"syntology":null},{"paper":null,"slug":"spion-layer-wise-sparse-training-of","title":"SPION: Layer-Wise Sparse Training of Transformer via Convolutional Flood Filling","date":"2023-09-22","arxiv_id":"2309.12578","n_code_links":0,"syntology":null},{"paper":"/paper/sparseswin-swin-transformer-with-sparse","slug":"sparseswin-swin-transformer-with-sparse","title":"SparseSwin: Swin Transformer with Sparse Transformer Block","date":"2023-09-11","arxiv_id":"2309.05224","n_code_links":1,"syntology":null},{"paper":"/paper/from-sparse-to-soft-mixtures-of-experts","slug":"from-sparse-to-soft-mixtures-of-experts","title":"From Sparse to Soft Mixtures of Experts","date":"2023-08-02","arxiv_id":"2308.00951","n_code_links":5,"syntology":{"ran":29,"of":33,"n_ran_checked":18,"n_instrument":11,"unverified":4,"pointer_only":8,"phrase":"29 ran (of which 5 constructed an object rather than computing a result; 18 with no instrument failure: 3 honoured, 4 violated, 11 with no contract checked; 11 where Syntology's instrument failed) · 4 unverified","official":{"repos":["google-research/vmoe"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["listed","official","unlocated"]}}},{"paper":"/paper/longcoder-a-long-range-pre-trained-language","slug":"longcoder-a-long-range-pre-trained-language","title":"LongCoder: A Long-Range Pre-trained Language Model for Code Completion","date":"2023-06-26","arxiv_id":"2306.14893","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["microsoft/CodeBERT"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/learning-a-sparse-transformer-network-for","slug":"learning-a-sparse-transformer-network-for","title":"Learning A Sparse Transformer Network for Effective Image Deraining","date":"2023-03-21","arxiv_id":"2303.11950","n_code_links":1,"syntology":{"ran":8,"of":11,"n_ran_checked":8,"n_instrument":0,"unverified":3,"pointer_only":11,"phrase":"8 ran (of which 8 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified; every one of the 8 samples that ran constructed an object rather than computing a result","official":{"repos":["cschenxiang/drsformer"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":8,"n_ran_no_instrument_failure":8,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"sampled-transformer-for-point-sets","title":"Sampled Transformer for Point Sets","date":"2023-02-28","arxiv_id":"2302.14346","n_code_links":0,"syntology":null},{"paper":"/paper/generating-a-structured-summary-of-numerous","slug":"generating-a-structured-summary-of-numerous","title":"Generating a Structured Summary of Numerous Academic Papers: Dataset and Method","date":"2023-02-09","arxiv_id":"2302.04580","n_code_links":1,"syntology":null},{"paper":"/paper/diffuser-efficient-transformers-with-multi","slug":"diffuser-efficient-transformers-with-multi","title":"Diffuser: Efficient Transformers with Multi-hop Attention Diffusion for Long Sequences","date":"2022-10-21","arxiv_id":"2210.11794","n_code_links":1,"syntology":null},{"paper":"/paper/efficient-quantized-sparse-matrix-operations","slug":"efficient-quantized-sparse-matrix-operations","title":"Efficient Quantized Sparse Matrix Operations on Tensor Cores","date":"2022-09-14","arxiv_id":"2209.06979","n_code_links":1,"syntology":null},{"paper":"/paper/dynast-dynamic-sparse-transformer-for","slug":"dynast-dynamic-sparse-transformer-for","title":"DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation","date":"2022-07-13","arxiv_id":"2207.06124","n_code_links":1,"syntology":{"ran":5,"of":8,"n_ran_checked":3,"n_instrument":2,"unverified":3,"pointer_only":0,"phrase":"5 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","official":{"repos":["huage001/dynast"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":2,"n_ran_no_instrument_failure":3,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":"/paper/sparsetir-composable-abstractions-for-sparse","slug":"sparsetir-composable-abstractions-for-sparse","title":"SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning","date":"2022-07-11","arxiv_id":"2207.04606","n_code_links":2,"syntology":{"ran":6,"of":6,"n_ran_checked":5,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 2 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["uwsampl/sparsetir","uwsampl/sparsetir-artifact"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/deepspeed-inference-enabling-efficient","slug":"deepspeed-inference-enabling-efficient","title":"DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale","date":"2022-06-30","arxiv_id":"2207.00032","n_code_links":2,"syntology":null},{"paper":"/paper/video-sparse-transformer-with-attention","slug":"video-sparse-transformer-with-attention","title":"Video Sparse Transformer With Attention-Guided Memory for Video Object Detection","date":"2022-06-17","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/what-dense-graph-do-you-need-for-self","slug":"what-dense-graph-do-you-need-for-self","title":"What Dense Graph Do You Need for Self-Attention?","date":"2022-05-27","arxiv_id":"2205.14014","n_code_links":1,"syntology":null},{"paper":null,"slug":"ernie-sparse-learning-hierarchical-efficient-1","title":"ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention","date":"2022-03-23","arxiv_id":"2203.12276","n_code_links":0,"syntology":null},{"paper":"/paper/coarse-to-fine-sparse-transformer-for","slug":"coarse-to-fine-sparse-transformer-for","title":"Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction","date":"2022-03-09","arxiv_id":"2203.04845","n_code_links":1,"syntology":{"ran":13,"of":16,"n_ran_checked":9,"n_instrument":4,"unverified":3,"pointer_only":0,"phrase":"13 ran (of which 8 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified","official":{"repos":["caiyuanhao1998/MST"],"state":"official (archive's flag): 13 ran","n_ran":13,"n_constructed":8,"n_ran_no_instrument_failure":9,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":"/paper/flow-guided-sparse-transformer-for-video","slug":"flow-guided-sparse-transformer-for-video","title":"Flow-Guided Sparse Transformer for Video Deblurring","date":"2022-01-06","arxiv_id":"2201.01893","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["linjing7/VR-Baseline"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/embracing-single-stride-3d-object-detector","slug":"embracing-single-stride-3d-object-detector","title":"Embracing Single Stride 3D Object Detector with Sparse Transformer","date":"2021-12-13","arxiv_id":"2112.06375","n_code_links":2,"syntology":null},{"paper":null,"slug":"ernie-sparse-learning-hierarchical-efficient","title":"ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention","date":"2021-11-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/yformer-u-net-inspired-transformer-1","slug":"yformer-u-net-inspired-transformer-1","title":"Yformer: U-Net Inspired Transformer Architecture for Far Horizon Time Series Forecasting","date":"2021-10-13","arxiv_id":"2110.08255","n_code_links":1,"syntology":null},{"paper":null,"slug":"ernie-sparse-robust-efficient-transformer","title":"ERNIE-SPARSE: Robust Efficient Transformer Through Hierarchically Unifying Isolated Information","date":"2021-09-29","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"cluster-former-clustering-based-sparse-1","title":"Cluster-Former: Clustering-based Sparse Transformer for Question Answering","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/visualsparta-sparse-transformer-fragment","slug":"visualsparta-sparse-transformer-fragment","title":"VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words","date":"2021-01-01","arxiv_id":"2101.00265","n_code_links":1,"syntology":null},{"paper":"/paper/adversarial-sparse-transformer-for-time","slug":"adversarial-sparse-transformer-for-time","title":"Adversarial Sparse Transformer for Time Series Forecasting","date":"2020-12-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/scene-graph-modification-based-on-natural","slug":"scene-graph-modification-based-on-natural","title":"Scene Graph Modification Based on Natural Language Commands","date":"2020-10-06","arxiv_id":"2010.02591","n_code_links":1,"syntology":null},{"paper":"/paper/cluster-former-clustering-based-sparse","slug":"cluster-former-clustering-based-sparse","title":"Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding","date":"2020-09-13","arxiv_id":"2009.06097","n_code_links":0,"syntology":null},{"paper":"/paper/sparsifying-transformer-models-with","slug":"sparsifying-transformer-models-with","title":"Sparsifying Transformer Models with Trainable Representation Pooling","date":"2020-09-10","arxiv_id":"2009.05169","n_code_links":1,"syntology":null},{"paper":"/paper/gmat-global-memory-augmentation-for","slug":"gmat-global-memory-augmentation-for","title":"GMAT: Global Memory Augmentation for Transformers","date":"2020-06-05","arxiv_id":"2006.03274","n_code_links":1,"syntology":null},{"paper":"/paper/explicit-sparse-transformer-concentrated","slug":"explicit-sparse-transformer-concentrated","title":"Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection","date":"2019-12-25","arxiv_id":"1912.11637","n_code_links":2,"syntology":{"ran":3,"of":3,"n_ran_checked":0,"n_instrument":3,"unverified":0,"pointer_only":3,"phrase":"3 ran (of which 1 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","official":{"repos":["lancopku/Explicit-Sparse-Transformer"],"state":"official: no sample here; runs from other or unrecorded repositories","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["unlocated"]}}},{"paper":"/paper/adaptively-sparse-transformers","slug":"adaptively-sparse-transformers","title":"Adaptively Sparse Transformers","date":"2019-08-30","arxiv_id":"1909.00015","n_code_links":3,"syntology":{"ran":3,"of":3,"n_ran_checked":0,"n_instrument":3,"unverified":0,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","official":{"repos":["deep-spin/entmax"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"paper":"/paper/190410509","slug":"190410509","title":"Generating Long Sequences with Sparse Transformers","date":"2019-04-23","arxiv_id":"1904.10509","n_code_links":7,"syntology":{"ran":5,"of":6,"n_ran_checked":4,"n_instrument":1,"unverified":1,"pointer_only":0,"phrase":"5 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","official":{"repos":["openai/sparse_attention"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}}],"papers_shown":47,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":47,"with_a_code_link":32,"where_syntology_ran_a_sample":13,"not_listed_spam_title":0,"listed":47,"listed_where_code_ran":13,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":8,"every_run_a_failure_of_syntologys_instrument":5,"listed_with_a_run_with_no_instrument_failure":8,"listed_every_run_a_failure_of_syntologys_instrument":5,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":13,"papers_where_code_ran_first_page":"/method/sparse-transformer/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/sparse-transformer/papers/ran/1","json":"/method/sparse-transformer/papers/ran/1.json","rows":13}],"tasks":[{"task":"/task/language-modelling","name":"Language Modelling","papers":6},{"task":"/task/decoder","name":"Decoder","papers":5},{"task":"/task/language-modeling","name":"Language Modeling","papers":4},{"task":"/task/mixture-of-experts","name":"Mixture-of-Experts","papers":4},{"task":"/task/text-classification","name":"Text Classification","papers":4},{"task":"/task/text-classification-1","name":"text-classification","papers":4},{"task":"/task/diversity","name":"Diversity","papers":3},{"task":null,"name":"GPU","papers":3},{"task":"/task/image-restoration","name":"Image Restoration","papers":3},{"task":"/task/machine-translation","name":"Machine Translation","papers":3},{"task":"/task/object","name":"Object","papers":3},{"task":"/task/object-detection","name":"Object Detection","papers":3},{"task":"/task/question-answering","name":"Question Answering","papers":3},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":3},{"task":"/task/translation","name":"Translation","papers":3},{"task":"/task/object-detection-1","name":"object-detection","papers":3},{"task":null,"name":"CPU","papers":2},{"task":"/task/clustering","name":"Clustering","papers":2},{"task":"/task/document-summarization","name":"Document Summarization","papers":2},{"task":"/task/image-captioning","name":"Image Captioning","papers":2}],"tasks_shown":20,"n_tasks":80,"usage_by_year":[{"year":"2019","papers":3},{"year":"2020","papers":5},{"year":"2021","papers":6},{"year":"2022","papers":10},{"year":"2023","papers":8},{"year":"2024","papers":12},{"year":"2025","papers":3}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/sparse-transformer"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Sparse%20Transformer","read_at":"2026-10-05T01:02:19+00:00","query":"Sparse Transformer","kind":"paper","have":"no_code","tier":"paper_only","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"arxiv_id":"openreview_Hye87grYDH"},"resolution":"paper","resolved_name_differs":false,"absence_asserted":"for_this_paper_only","absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":null,"in_graph":0,"list_truncated":null,"strongest_link":null},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"openreview_Hye87grYDH","citations":null},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'Sparse Transformer'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"openreview_Hye87grYDH","page":null,"syntology_url":null,"in_graph_layer":false,"samples":0},"line":"the paper matched by title, the paper Syntology's graph keys as openreview_Hye87grYDH, held, no code (no_code; paper_only). The lookup matched a paper by its title, the paper Syntology's graph keys as openreview_Hye87grYDH, not a method entry. The archive credits no paper with an arXiv id for this method, so whether this answer (about the paper Syntology's graph keys as openreview_Hye87grYDH) is about this method is not checked; it does not say whether this method has code.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"762670033254bc75c2f986ac3510ab06f619d3b7aa3e53c9c607b86bbb18732e","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}