{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/set-transformer","slug":"set-transformer","name":"Set Transformer","full_name":"Set Transformer","full_name_withheld":false,"description_markdown":"Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such problems do not depend on the order of elements of the set, models used to address them should be permutation invariant. We present an attention-based neural network module, the Set Transformer, specifically designed to model interactions among elements in the input set. The model consists of an encoder and a decoder, both of which rely on attention mechanisms. In an effort to reduce computational complexity, we introduce an attention scheme inspired by inducing point methods from sparse Gaussian process literature. It reduces the computation time of self-attention from quadratic to linear in the number of elements in the set. We show that our model is theoretically attractive and we evaluate it on a range of tasks, demonstrating the state-of-the-art performance compared to recent methods for set-structured data.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks","paper":"/paper/set-transformer-a-framework-for-attention","first_author":"Juho Lee","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/set-transformer-a-framework-for-attention"},"source":{"url":"https://arxiv.org/abs/1810.00825v3","title":"Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Attention Mechanisms","url":"/methods/category/attention-mechanisms","pwc_aliases":["attention-mechanisms-1"]}],"n_papers_tagged":20,"archive_num_papers":20,"papers_newest_first":[{"paper":null,"slug":"to-bin-or-not-to-bin-alternative","title":"To Bin or not to Bin: Alternative Representations of Mass Spectra","date":"2025-02-15","arxiv_id":"2502.10851","n_code_links":0,"syntology":null},{"paper":null,"slug":"advances-in-set-function-learning-a-survey-of","title":"Advances in Set Function Learning: A Survey of Techniques and Applications","date":"2025-01-24","arxiv_id":"2501.14991","n_code_links":0,"syntology":null},{"paper":null,"slug":"adaptive-parameters-identification-for","title":"Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networks","date":"2025-01-20","arxiv_id":"2501.11350","n_code_links":0,"syntology":null},{"paper":"/paper/multiset-transformer-advancing-representation","slug":"multiset-transformer-advancing-representation","title":"Multiset Transformer: Advancing Representation Learning in Persistence Diagrams","date":"2024-11-22","arxiv_id":"2411.14662","n_code_links":1,"syntology":null},{"paper":"/paper/graph-as-point-set","slug":"graph-as-point-set","title":"Graph as Point Set","date":"2024-05-05","arxiv_id":"2405.02795","n_code_links":1,"syntology":{"ran":20,"of":27,"n_ran_checked":16,"n_instrument":4,"unverified":7,"pointer_only":27,"phrase":"20 ran (of which 15 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 0 violated, 15 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified","official":{"repos":["GraphPKU/GraphAsSet"],"state":"official (archive's flag): 20 ran","n_ran":20,"n_constructed":15,"n_ran_no_instrument_failure":16,"n_unverified":7,"ran_from_kinds":["official"]}}},{"paper":"/paper/associative-transformer-is-a-sparse","slug":"associative-transformer-is-a-sparse","title":"Associative Transformer","date":"2023-09-22","arxiv_id":"2309.12862","n_code_links":1,"syntology":null},{"paper":null,"slug":"pointersect-neural-rendering-with-cloud-ray","title":"Pointersect: Neural Rendering with Cloud-Ray Intersection","date":"2023-04-24","arxiv_id":"2304.12390","n_code_links":0,"syntology":null},{"paper":"/paper/event-voxel-set-transformer-for","slug":"event-voxel-set-transformer-for","title":"Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams","date":"2023-03-07","arxiv_id":"2303.03856","n_code_links":1,"syntology":null},{"paper":"/paper/set-norm-and-equivariant-skip-connections-1","slug":"set-norm-and-equivariant-skip-connections-1","title":"Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets","date":"2022-06-23","arxiv_id":"2206.11925","n_code_links":1,"syntology":{"ran":1,"of":2,"n_ran_checked":1,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["rajesh-lab/deep_permutation_invariant"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/permutation-invariant-relational-network-for","slug":"permutation-invariant-relational-network-for","title":"Permutation-Invariant Relational Network for Multi-person 3D Pose Estimation","date":"2022-04-11","arxiv_id":"2204.04913","n_code_links":0,"syntology":null},{"paper":"/paper/voxel-set-transformer-a-set-to-set-approach","slug":"voxel-set-transformer-a-set-to-set-approach","title":"Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds","date":"2022-03-19","arxiv_id":"2203.10314","n_code_links":1,"syntology":{"ran":4,"of":6,"n_ran_checked":3,"n_instrument":1,"unverified":2,"pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified","official":{"repos":["skyhehe123/voxset"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/automated-identification-of-cell-populations","slug":"automated-identification-of-cell-populations","title":"Automated Identification of Cell Populations in Flow Cytometry Data with Transformers","date":"2021-08-23","arxiv_id":"2108.10072","n_code_links":1,"syntology":null},{"paper":"/paper/you-are-allset-a-multiset-function-framework","slug":"you-are-allset-a-multiset-function-framework","title":"You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks","date":"2021-06-24","arxiv_id":"2106.13264","n_code_links":1,"syntology":{"ran":0,"of":1,"n_ran_checked":0,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"0 ran · 1 unverified","official":{"repos":["jianhao2016/AllSet"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"paper":"/paper/setvae-learning-hierarchical-composition-for","slug":"setvae-learning-hierarchical-composition-for","title":"SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data","date":"2021-03-29","arxiv_id":"2103.15619","n_code_links":2,"syntology":{"ran":17,"of":27,"n_ran_checked":9,"n_instrument":8,"unverified":10,"pointer_only":0,"phrase":"17 ran (of which 7 constructed an object rather than computing a result; 9 with no instrument failure: 2 honoured, 0 violated, 7 with no contract checked; 8 where Syntology's instrument failed) · 10 unverified","official":{"repos":["jw9730/setvae"],"state":"official (archive's flag): 17 ran","n_ran":17,"n_constructed":7,"n_ran_no_instrument_failure":9,"n_unverified":10,"ran_from_kinds":["official"]}}},{"paper":"/paper/latent-variable-nested-set-transformers","slug":"latent-variable-nested-set-transformers","title":"Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction","date":"2021-02-19","arxiv_id":"2104.00563","n_code_links":2,"syntology":{"ran":11,"of":17,"n_ran_checked":9,"n_instrument":2,"unverified":6,"pointer_only":2,"phrase":"11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 6 unverified","official":{"repos":["roggirg/AutoBots"],"state":"official (archive's flag): 11 ran","n_ran":11,"n_constructed":0,"n_ran_no_instrument_failure":9,"n_unverified":6,"ran_from_kinds":["official"]}}},{"paper":"/paper/gaining-insight-into-sars-cov-2-infection-and","slug":"gaining-insight-into-sars-cov-2-infection-and","title":"Gaining Insight into SARS-CoV-2 Infection and COVID-19 Severity Using Self-supervised Edge Features and Graph Neural Networks","date":"2020-06-23","arxiv_id":"2006.12971","n_code_links":1,"syntology":null},{"paper":"/paper/self-supervised-edge-features-for-improved","slug":"self-supervised-edge-features-for-improved","title":"Self-supervised edge features for improved Graph Neural Network training","date":"2020-06-23","arxiv_id":"2007.04777","n_code_links":1,"syntology":null},{"paper":"/paper/few-shot-learning-as-domain-adaptation","slug":"few-shot-learning-as-domain-adaptation","title":"Few-Shot Learning as Domain Adaptation: Algorithm and Analysis","date":"2020-02-06","arxiv_id":"2002.02050","n_code_links":0,"syntology":null},{"paper":"/paper/learning-set-equivariant-functions-with-swarm","slug":"learning-set-equivariant-functions-with-swarm","title":"Learning Set-equivariant Functions with SWARM Mappings","date":"2019-06-22","arxiv_id":"1906.09400","n_code_links":1,"syntology":null},{"paper":"/paper/set-transformer-a-framework-for-attention","slug":"set-transformer-a-framework-for-attention","title":"Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks","date":"2018-10-01","arxiv_id":"1810.00825","n_code_links":9,"syntology":{"ran":8,"of":9,"n_ran_checked":6,"n_instrument":2,"unverified":1,"pointer_only":4,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 3 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified","official":{"repos":["juho-lee/set_transformer"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["listed","official","unlocated"]}}}],"papers_shown":20,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":20,"with_a_code_link":14,"where_syntology_ran_a_sample":6,"not_listed_spam_title":0,"listed":20,"listed_where_code_ran":6,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":6,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":6,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":6,"papers_where_code_ran_first_page":"/method/set-transformer/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/set-transformer/papers/ran/1","json":"/method/set-transformer/papers/ran/1.json","rows":6}],"tasks":[{"task":"/task/node-classification","name":"Node Classification","papers":3},{"task":"/task/decoder","name":"Decoder","papers":2},{"task":"/task/few-shot-image-classification","name":"Few-Shot Image Classification","papers":2},{"task":"/task/classification","name":"General Classification","papers":2},{"task":"/task/graph-attention","name":"Graph Attention","papers":2},{"task":"/task/graph-neural-network","name":"Graph Neural Network","papers":2},{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/representation-learning","name":"Representation Learning","papers":2},{"task":"/task/3d-multi-person-pose-estimation","name":"3D Multi-Person Pose Estimation","papers":1},{"task":"/task/3d-multi-person-pose-estimation-absolute","name":"3D Multi-Person Pose Estimation (absolute)","papers":1},{"task":"/task/3d-multi-person-pose-estimation-root-relative","name":"3D Multi-Person Pose Estimation (root-relative)","papers":1},{"task":"/task/3d-object-detection","name":"3D Object Detection","papers":1},{"task":"/task/3d-pose-estimation","name":"3D Pose Estimation","papers":1},{"task":"/task/3d-shape-recognition","name":"3D Shape Recognition","papers":1},{"task":"/task/action-recognition-in-videos","name":"Action Recognition","papers":1},{"task":"/task/all","name":"All","papers":1},{"task":"/task/artificial-global-workspace","name":"Artificial Global Workspace","papers":1},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":1},{"task":"/task/benchmarking","name":"Benchmarking","papers":1},{"task":"/task/diversity","name":"Diversity","papers":1}],"tasks_shown":20,"n_tasks":48,"usage_by_year":[{"year":"2018","papers":1},{"year":"2019","papers":1},{"year":"2020","papers":3},{"year":"2021","papers":4},{"year":"2022","papers":3},{"year":"2023","papers":3},{"year":"2024","papers":2},{"year":"2025","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/set-transformer"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Set%20Transformer","read_at":"2026-10-05T01:01:51+00:00","query":"Set Transformer","kind":"paper","have":"no_code","tier":"paper_only","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"arxiv_id":"openreview_Hkgnii09Ym"},"resolution":"paper","resolved_name_differs":false,"absence_asserted":"for_this_paper_only","absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":"1810.00825","repositories":{"listed":null,"in_graph":0,"list_truncated":null,"strongest_link":null},"retrieve_with":null,"name_competition":{"verdict":"outranked","may_be_wrong_sense":true,"bearers_found":2,"served":{"arxiv_id":"openreview_Hkgnii09Ym","citations":null},"leading_other_bearer":{"arxiv_id":"1810.00825","title":"Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks","citations":266,"named_by":["title_prefix"],"bearer_name":"Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks","ask_with":"have('1810.00825')"},"n_other_bearers_listed":1,"note":"'Set Transformer' is ALSO the name of the paper 'Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks' (arXiv:1810.00825, 266 citations), which is cited FAR more than the paper this answer is about (citations not recorded). This answer may be about the wrong Set Transformer. If you meant the better-known one, ask again with that entry of `also_named`."},"name_competition_text":"The same name also belongs to the paper “Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks” (arXiv:1810.00825, 266 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name.","near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"openreview_Hkgnii09Ym","page":null,"syntology_url":null,"in_graph_layer":false,"samples":0},"line":"the paper matched by title, the paper Syntology's graph keys as openreview_Hkgnii09Ym, held, no code (no_code; paper_only). The lookup matched a paper by its title, the paper Syntology's graph keys as openreview_Hkgnii09Ym, not a method entry. The archive credits no paper with an arXiv id for this method, so whether this answer (about the paper Syntology's graph keys as openreview_Hkgnii09Ym) is about this method is not checked; it does not say whether this method has code.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"0e767cd25286734fea7f393b693a1408359e70d8aebe48487d2cf51ce1ede869","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}