{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/lama","slug":"lama","name":"LAMA","full_name":"Low-Rank Factorization-based Multi-Head Attention","full_name_withheld":false,"description_markdown":"**Low-Rank Factorization-based Multi-head Attention Mechanism**, or **LAMA**, is a type of attention module that uses low-rank factorization to reduce computational complexity. It uses low-rank bilinear pooling to construct a structured sentence representation that attends to multiple aspects of a sentence.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/1912.00835v2","title":"Low Rank Factorization for Compact Multi-Head Self-Attention","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/JohnGiorgi/compact-multi-head-self-attention-pytorch/blob/56358eaddde176d539916896d6836c00d1dc5f0c/modules/lama.py#L10","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Attention Modules","url":"/methods/category/attention-modules","pwc_aliases":[]}],"n_papers_tagged":42,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"slug":"humandreamer-generating-controllable-human","title":"HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation","date":"2025-03-31","arxiv_id":"2503.24026","n_code_links":0,"syntology":null},{"paper":null,"slug":"senseexpo-efficient-autonomous-exploration","title":"SenseExpo: Efficient Autonomous Exploration with Prediction Information from Lightweight Neural Networks","date":"2025-03-20","arxiv_id":"2503.16000","n_code_links":0,"syntology":null},{"paper":null,"slug":"one-mind-many-tongues-a-deep-dive-into","title":"One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models","date":"2024-11-26","arxiv_id":"2411.17401","n_code_links":0,"syntology":null},{"paper":null,"slug":"lama-stable-dual-domain-deep-reconstruction","title":"LAMA: Stable Dual-Domain Deep Reconstruction For Sparse-View CT","date":"2024-10-28","arxiv_id":"2410.21111","n_code_links":0,"syntology":null},{"paper":"/paper/discovering-leitmotifs-in-multidimensional","slug":"discovering-leitmotifs-in-multidimensional","title":"Discovering Leitmotifs in Multidimensional Time Series","date":"2024-10-16","arxiv_id":"2410.12293","n_code_links":1,"syntology":null},{"paper":null,"slug":"consolidating-lama-with-best-first-width","title":"Consolidating LAMA with Best-First Width Search","date":"2024-04-26","arxiv_id":"2404.17648","n_code_links":0,"syntology":null},{"paper":"/paper/return-to-tradition-learning-reliable","slug":"return-to-tradition-learning-reliable","title":"Return to Tradition: Learning Reliable Heuristics with Classical Machine Learning","date":"2024-03-25","arxiv_id":"2403.16508","n_code_links":1,"syntology":null},{"paper":"/paper/take-care-of-your-prompt-bias-investigating","slug":"take-care-of-your-prompt-bias-investigating","title":"Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction","date":"2024-03-15","arxiv_id":"2403.09963","n_code_links":1,"syntology":null},{"paper":"/paper/the-cost-of-compression-investigating-the","slug":"the-cost-of-compression-investigating-the","title":"The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models","date":"2023-12-01","arxiv_id":"2312.00960","n_code_links":1,"syntology":{"ran":1,"of":2,"n_ran_checked":0,"n_instrument":1,"unverified":1,"pointer_only":2,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","official":{"repos":["namburisrinath/llmcompression"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"the-minimal-computational-substrate-of-fluid","title":"The minimal computational substrate of fluid intelligence","date":"2023-08-14","arxiv_id":"2308.07039","n_code_links":0,"syntology":null},{"paper":"/paper/dlama-a-framework-for-curating-culturally","slug":"dlama-a-framework-for-curating-culturally","title":"DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Models","date":"2023-06-08","arxiv_id":"2306.05076","n_code_links":1,"syntology":null},{"paper":"/paper/learned-alternating-minimization-algorithm","slug":"learned-alternating-minimization-algorithm","title":"Learned Alternating Minimization Algorithm for Dual-domain Sparse-View CT Reconstruction","date":"2023-06-05","arxiv_id":"2306.02644","n_code_links":1,"syntology":null},{"paper":null,"slug":"locomotion-action-manipulation-synthesizing","title":"Locomotion-Action-Manipulation: Synthesizing Human-Scene Interactions in Complex 3D Environments","date":"2023-01-09","arxiv_id":"2301.02667","n_code_links":0,"syntology":null},{"paper":null,"slug":"rethinking-fast-fourier-convolution-in-image","title":"Rethinking Fast Fourier Convolution in Image Inpainting","date":"2023-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"in-context-learning-distillation-transferring","title":"In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models","date":"2022-12-20","arxiv_id":"2212.10670","n_code_links":0,"syntology":null},{"paper":null,"slug":"context-variance-evaluation-of-pretrained","title":"Context Variance Evaluation of Pretrained Language Models for Prompt-based Biomedical Knowledge Probing","date":"2022-11-18","arxiv_id":"2211.10265","n_code_links":0,"syntology":null},{"paper":null,"slug":"spe-symmetrical-prompt-enhancement-for-fact","title":"SPE: Symmetrical Prompt Enhancement for Fact Probing","date":"2022-11-14","arxiv_id":"2211.07078","n_code_links":0,"syntology":null},{"paper":"/paper/kamel-knowledge-analysis-with-multitoken","slug":"kamel-knowledge-analysis-with-multitoken","title":"KAMEL : Knowledge Analysis with Multitoken Entities in Language Models","date":"2022-11-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/informask-unsupervised-informative-masking","slug":"informask-unsupervised-informative-masking","title":"InforMask: Unsupervised Informative Masking for Language Model Pretraining","date":"2022-10-21","arxiv_id":"2210.11771","n_code_links":1,"syntology":{"ran":0,"of":1,"n_ran_checked":0,"n_instrument":0,"unverified":1,"pointer_only":1,"phrase":"0 ran · 1 unverified","official":{"repos":["nafissadeq/informask"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"paper":"/paper/the-effectiveness-of-masked-language-modeling","slug":"the-effectiveness-of-masked-language-modeling","title":"The Effectiveness of Masked Language Modeling and Adapters for Factual Knowledge Injection","date":"2022-10-03","arxiv_id":"2210.00907","n_code_links":1,"syntology":null},{"paper":null,"slug":"inpainting-at-modern-camera-resolution-by","title":"Inpainting at Modern Camera Resolution by Guided PatchMatch with Auto-Curation","date":"2022-08-06","arxiv_id":"2208.03552","n_code_links":0,"syntology":null},{"paper":null,"slug":"glama-joint-spatial-and-frequency-loss-for","title":"GLaMa: Joint Spatial and Frequency Loss for General Image Inpainting","date":"2022-05-15","arxiv_id":"2205.07162","n_code_links":0,"syntology":null},{"paper":null,"slug":"comparison-of-comodgans-lama-and-glide-for","title":"Comparison of CoModGANs, LaMa and GLIDE for Art Inpainting- Completing M.C Escher's Print Gallery","date":"2022-05-03","arxiv_id":"2205.01741","n_code_links":0,"syntology":null},{"paper":null,"slug":"meta-learning-via-language-model-in-context-1","title":"Meta-learning via Language Model In-context Tuning","date":"2021-11-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"spe-symmetrical-prompt-enhancement-for","title":"SPE: Symmetrical Prompt Enhancement for Factual Knowledge Retrieval","date":"2021-10-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/meta-learning-via-language-model-in-context","slug":"meta-learning-via-language-model-in-context","title":"Meta-learning via Language Model In-context Tuning","date":"2021-10-15","arxiv_id":"2110.07814","n_code_links":1,"syntology":null},{"paper":"/paper/rewire-then-probe-a-contrastive-recipe-for","slug":"rewire-then-probe-a-contrastive-recipe-for","title":"Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models","date":"2021-10-15","arxiv_id":"2110.08173","n_code_links":1,"syntology":null},{"paper":"/paper/p-adapters-robustly-extracting-factual-1","slug":"p-adapters-robustly-extracting-factual-1","title":"P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Prompts","date":"2021-10-14","arxiv_id":"2110.07280","n_code_links":1,"syntology":null},{"paper":"/paper/resolution-robust-large-mask-inpainting-with","slug":"resolution-robust-large-mask-inpainting-with","title":"Resolution-robust Large Mask Inpainting with Fourier Convolutions","date":"2021-09-15","arxiv_id":"2109.07161","n_code_links":8,"syntology":{"ran":3,"of":4,"n_ran_checked":3,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["saic-mdal/lama"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"no-need-to-know-everything-efficiently","title":"No Need to Know Everything! Efficiently Augmenting Language Models With External Knowledge","date":"2021-09-03","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/how-to-query-language-models","slug":"how-to-query-language-models","title":"How to Query Language Models?","date":"2021-08-04","arxiv_id":"2108.01928","n_code_links":1,"syntology":null},{"paper":null,"slug":"how-to-query-language-models-1","title":"How to Query Language Models?","date":"2021-05-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/understanding-by-understanding-not-modeling","slug":"understanding-by-understanding-not-modeling","title":"Understanding by Understanding Not: Modeling Negation in Language Models","date":"2021-05-07","arxiv_id":"2105.03519","n_code_links":1,"syntology":null},{"paper":"/paper/factual-probing-is-mask-learning-vs-learning","slug":"factual-probing-is-mask-learning-vs-learning","title":"Factual Probing Is [MASK]: Learning vs. Learning to Recall","date":"2021-04-12","arxiv_id":"2104.05240","n_code_links":2,"syntology":{"ran":0,"of":1,"n_ran_checked":0,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"0 ran · 1 unverified","official":{"repos":["princeton-nlp/OptiPrompt"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"paper":null,"slug":"detecting-log-anomalies-with-multi-head","title":"Detecting Log Anomalies with Multi-Head Attention (LAMA)","date":"2021-01-07","arxiv_id":"2101.02392","n_code_links":0,"syntology":null},{"paper":"/paper/image-synthesis-from-layout-with-locality","slug":"image-synthesis-from-layout-with-locality","title":"Image Synthesis From Layout With Locality-Aware Mask Adaption","date":"2021-01-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"pretrain-knowledge-aware-language-models","title":"Pretrain Knowledge-Aware Language Models","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"kgplm-knowledge-guided-language-model-pre","title":"KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning","date":"2020-12-07","arxiv_id":"2012.03551","n_code_links":0,"syntology":null},{"paper":"/paper/autoprompt-eliciting-knowledge-from-language","slug":"autoprompt-eliciting-knowledge-from-language","title":"AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts","date":"2020-10-29","arxiv_id":"2010.15980","n_code_links":5,"syntology":{"ran":3,"of":3,"n_ran_checked":1,"n_instrument":2,"unverified":0,"pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["ucinlp/autoprompt"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"paper":"/paper/large-scale-knowledge-graph-based-synthetic","slug":"large-scale-knowledge-graph-based-synthetic","title":"Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training","date":"2020-10-23","arxiv_id":"2010.12688","n_code_links":1,"syntology":null},{"paper":"/paper/finding-macro-actions-with-disentangled","slug":"finding-macro-actions-with-disentangled","title":"Efficient Black-Box Planning Using Macro-Actions with Focused Effects","date":"2020-04-28","arxiv_id":"2004.13242","n_code_links":2,"syntology":{"ran":8,"of":13,"n_ran_checked":8,"n_instrument":0,"unverified":5,"pointer_only":0,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified","official":{"repos":["camall3n/focused-macros","camall3n/skills-for-planning"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":8,"n_unverified":5,"ran_from_kinds":["official"]}}},{"paper":"/paper/low-rank-factorization-for-compact-multi-head","slug":"low-rank-factorization-for-compact-multi-head","title":"Low Rank Factorization for Compact Multi-Head Self-Attention","date":"2019-11-26","arxiv_id":"1912.00835","n_code_links":1,"syntology":null}],"papers_shown":42,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":42,"with_a_code_link":21,"where_syntology_ran_a_sample":4,"not_listed_spam_title":0,"listed":42,"listed_where_code_ran":4,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":3,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":3,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":4,"papers_where_code_ran_first_page":"/method/lama/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/lama/papers/ran/1","json":"/method/lama/papers/ran/1.json","rows":4}],"tasks":[{"task":"/task/language-modelling","name":"Language Modelling","papers":11},{"task":"/task/language-modeling","name":"Language Modeling","papers":10},{"task":"/task/image-inpainting","name":"Image Inpainting","papers":4},{"task":"/task/in-context-learning","name":"In-Context Learning","papers":3},{"task":"/task/inductive-bias","name":"Inductive Bias","papers":3},{"task":"/task/knowledge-probing","name":"Knowledge Probing","papers":3},{"task":"/task/question-answering","name":"Question Answering","papers":3},{"task":"/task/text-classification","name":"Text Classification","papers":3},{"task":"/task/model","name":"model","papers":3},{"task":"/task/text-classification-1","name":"text-classification","papers":3},{"task":"/task/masked-language-modeling","name":"Masked Language Modeling","papers":2},{"task":"/task/meta-learning","name":"Meta-Learning","papers":2},{"task":"/task/object","name":"Object","papers":2},{"task":"/task/retrieval","name":"Retrieval","papers":2},{"task":"/task/ssim","name":"SSIM","papers":2},{"task":"/task/sentiment-analysis","name":"Sentiment Analysis","papers":2},{"task":"/task/4k","name":"4k","papers":1},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":1},{"task":"/task/articles","name":"Articles","papers":1},{"task":"/task/benchmarking","name":"Benchmarking","papers":1}],"tasks_shown":20,"n_tasks":56,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":4},{"year":"2021","papers":14},{"year":"2022","papers":9},{"year":"2023","papers":6},{"year":"2024","papers":6},{"year":"2025","papers":2}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/lama"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=LAMA","read_at":"2026-10-05T00:58:22+00:00","query":"LAMA","kind":"paper","have":"no_code","tier":"paper_only","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"arxiv_id":"2410.21111"},"resolution":"paper","resolved_name_differs":false,"absence_asserted":"for_this_paper_only","absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":null,"in_graph":0,"list_truncated":null,"strongest_link":null},"retrieve_with":null,"name_competition":{"verdict":"outranked","may_be_wrong_sense":false,"bearers_found":3,"served":{"arxiv_id":"2410.21111","citations":null},"leading_other_bearer":{"arxiv_id":"2601.12771","title":"Who Does This Name Remind You of ? Nationality Prediction via Large Language Model Associative Memory","citations":0,"named_by":["acronym_expansion"],"bearer_name":"LLM Associative Memory Agents (LAMA)","ask_with":"have('2601.12771')"},"n_other_bearers_listed":2,"note":"'LAMA' is ALSO the name of 'LLM Associative Memory Agents (LAMA)', a method in 'Who Does This Name Remind You of ? Nationality Prediction via Large Language Model Associative Memory' (arXiv:2601.12771, 0 citations), which is cited more than the paper this answer is about (citations not recorded). The margin is narrow, so this is not a claim that the other one is what you meant -- check `served.arxiv_id` against the paper you had in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"2410.21111","page":null,"syntology_url":null,"in_graph_layer":false,"samples":0},"line":"the paper matched by title, arXiv:2410.21111, held, no code (no_code; paper_only). The lookup matched a paper by its title, arXiv:2410.21111, not a method entry. The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2410.21111) is about this method is not checked; it does not say whether this method has code.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"0a3cd456ac0393d75421dd21a5213108c9771cea7e053eba18d0e377acf1f0f4","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}