{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/lion","slug":"lion","name":"Lion","full_name":"Evolved Sign Momentum","full_name_withheld":false,"description_markdown":"The Lion optimizer is discovered by symbolic program search. It is more memory-efficient than most adaptive optimizers as it only needs to momentum. The update of Lion is produced by the sign function.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":"https://github.com/google/automl/blob/ecbc7a3b5ab1944d5f39a80a21fd07b7606583f0/lion/lion_pytorch.py","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Optimization","url":"/methods/category/optimization","pwc_aliases":[]}],"n_papers_tagged":32,"archive_num_papers":32,"papers_newest_first":[{"paper":null,"slug":"towards-efficient-and-effective-alignment-of","title":"Towards Efficient and Effective Alignment of Large Language Models","date":"2025-06-11","arxiv_id":"2506.09329","n_code_links":0,"syntology":null},{"paper":null,"slug":"lions-and-muons-optimization-via-stochastic","title":"Lions and Muons: Optimization via Stochastic Frank-Wolfe","date":"2025-06-04","arxiv_id":"2506.04192","n_code_links":0,"syntology":null},{"paper":"/paper/linear-attention-for-efficient-bidirectional","slug":"linear-attention-for-efficient-bidirectional","title":"Linear Attention for Efficient Bidirectional Sequence Modeling","date":"2025-02-22","arxiv_id":"2502.16249","n_code_links":1,"syntology":{"ran":14,"of":14,"n_ran_checked":4,"n_instrument":10,"unverified":0,"pointer_only":14,"phrase":"14 ran (of which 1 constructed an object rather than computing a result; 4 with no instrument failure: 3 honoured, 0 violated, 1 with no contract checked; 10 where Syntology's instrument failed) · 0 unverified","official":{"repos":["lions-epfl/lion"],"state":"official (archive's flag): 14 ran","n_ran":14,"n_constructed":1,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"adagc-improving-training-stability-for-large","title":"AdaGC: Improving Training Stability for Large Language Model Pretraining","date":"2025-02-16","arxiv_id":"2502.11034","n_code_links":0,"syntology":null},{"paper":"/paper/connections-between-schedule-free-optimizers","slug":"connections-between-schedule-free-optimizers","title":"Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants","date":"2025-02-04","arxiv_id":"2502.02431","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["depenm/simplified-ademamix"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"how-memory-in-optimization-algorithms","title":"How Memory in Optimization Algorithms Implicitly Modifies the Loss","date":"2025-02-04","arxiv_id":"2502.02132","n_code_links":0,"syntology":null},{"paper":"/paper/grams-gradient-descent-with-adaptive-momentum","slug":"grams-gradient-descent-with-adaptive-momentum","title":"Grams: Gradient Descent with Adaptive Momentum Scaling","date":"2024-12-22","arxiv_id":"2412.17107","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result","official":{"repos":["Gunale0926/Grams"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"lion-cub-minimizing-communication-overhead-in","title":"Lion Cub: Minimizing Communication Overhead in Distributed Lion","date":"2024-11-25","arxiv_id":"2411.16462","n_code_links":0,"syntology":null},{"paper":"/paper/mars-unleashing-the-power-of-variance","slug":"mars-unleashing-the-power-of-variance","title":"MARS: Unleashing the Power of Variance Reduction for Training Large Models","date":"2024-11-15","arxiv_id":"2411.10438","n_code_links":2,"syntology":{"ran":3,"of":5,"n_ran_checked":2,"n_instrument":1,"unverified":2,"pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified","official":{"repos":["AGI-Arena/MARS"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official","unlocated"]}}},{"paper":null,"slug":"convergence-rate-analysis-of-lion","title":"Convergence Rate Analysis of LION","date":"2024-11-12","arxiv_id":"2411.07724","n_code_links":0,"syntology":null},{"paper":null,"slug":"talkmosaic-interactive-photomosaic-with-multi","title":"TalkMosaic: Interactive PhotoMosaic with Multi-modal LLM Q&A Interactions","date":"2024-09-20","arxiv_id":"2409.13941","n_code_links":0,"syntology":null},{"paper":null,"slug":"learning-spatially-aware-language-and-audio","title":"Learning Spatially-Aware Language and Audio Embeddings","date":"2024-09-17","arxiv_id":"2409.11369","n_code_links":0,"syntology":null},{"paper":"/paper/lion-linear-group-rnn-for-3d-object-detection","slug":"lion-linear-group-rnn-for-3d-object-detection","title":"LION: Linear Group RNN for 3D Object Detection in Point Clouds","date":"2024-07-25","arxiv_id":"2407.18232","n_code_links":1,"syntology":{"ran":6,"of":7,"n_ran_checked":5,"n_instrument":1,"unverified":1,"pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","official":{"repos":["happinesslz/LION"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"deconstructing-what-makes-a-good-optimizer","title":"Deconstructing What Makes a Good Optimizer for Language Models","date":"2024-07-10","arxiv_id":"2407.07972","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamic-multi-objective-lion-swarm","title":"Dynamic Multi-Objective Lion Swarm Optimization with Multi-strategy Fusion: An application in 6R robot trajectory planning","date":"2024-05-31","arxiv_id":"2406.00114","n_code_links":0,"syntology":null},{"paper":null,"slug":"surge-phenomenon-in-optimal-learning-rate-and","title":"Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling","date":"2024-05-23","arxiv_id":"2405.14578","n_code_links":0,"syntology":null},{"paper":"/paper/audio-visual-speech-recognition-based-on","slug":"audio-visual-speech-recognition-based-on","title":"Audio-Visual Speech Recognition based on Regulated Transformer and Spatio-Temporal Fusion Strategy for Driver Assistive Systems","date":"2024-05-09","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"communication-efficient-distributed-training","title":"Communication Efficient Distributed Training with Distributed Lion","date":"2024-03-30","arxiv_id":"2404.00438","n_code_links":0,"syntology":null},{"paper":"/paper/fedlion-faster-adaptive-federated","slug":"fedlion-faster-adaptive-federated","title":"FedLion: Faster Adaptive Federated Optimization with Fewer Communication","date":"2024-02-15","arxiv_id":"2402.09941","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["tzw1998/fedlion"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"mada-meta-adaptive-optimizers-through-hyper","title":"MADA: Meta-Adaptive Optimizers through hyper-gradient Descent","date":"2024-01-17","arxiv_id":"2401.08893","n_code_links":0,"syntology":null},{"paper":"/paper/collie-collaborative-training-of-large","slug":"collie-collaborative-training-of-large","title":"CoLLiE: Collaborative Training of Large Language Models in an Efficient Way","date":"2023-12-01","arxiv_id":"2312.00407","n_code_links":1,"syntology":{"ran":4,"of":7,"n_ran_checked":2,"n_instrument":2,"unverified":3,"pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","official":{"repos":["openlmlab/collie"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"rethinking-sign-training-provable-nonconvex","title":"Rethinking SIGN Training: Provable Nonconvex Acceleration without First- and Second-Order Gradient Lipschitz","date":"2023-10-23","arxiv_id":"2310.14616","n_code_links":0,"syntology":null},{"paper":null,"slug":"qft-quantized-full-parameter-tuning-of-llms","title":"QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources","date":"2023-10-11","arxiv_id":"2310.07147","n_code_links":0,"syntology":null},{"paper":null,"slug":"lion-secretly-solves-constrained-optimization","title":"Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts","date":"2023-10-09","arxiv_id":"2310.05898","n_code_links":0,"syntology":null},{"paper":"/paper/handwritten-stenography-recognition-and-the","slug":"handwritten-stenography-recognition-and-the","title":"Handwritten Stenography Recognition and the LION Dataset","date":"2023-08-15","arxiv_id":"2308.07799","n_code_links":1,"syntology":null},{"paper":null,"slug":"convergence-guarantees-for-stochastic","title":"Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization","date":"2023-07-19","arxiv_id":"2307.10053","n_code_links":0,"syntology":null},{"paper":null,"slug":"automatic-trade-off-adaptation-in-offline-rl","title":"Automatic Trade-off Adaptation in Offline RL","date":"2023-06-16","arxiv_id":"2306.09744","n_code_links":0,"syntology":null},{"paper":"/paper/rotational-optimizers-simple-robust-dnn","slug":"rotational-optimizers-simple-robust-dnn","title":"Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks","date":"2023-05-26","arxiv_id":"2305.17212","n_code_links":2,"syntology":{"ran":5,"of":6,"n_ran_checked":5,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["epfml/req","epfml/rotational-optimizers"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"lion-implicit-vision-prompt-tuning","title":"LION: Implicit Vision Prompt Tuning","date":"2023-03-17","arxiv_id":"2303.09992","n_code_links":0,"syntology":null},{"paper":null,"slug":"evolving-deep-neural-network-by-customized","title":"Evolving Deep Neural Network by Customized Moth Flame Optimization Algorithm for Underwater Targets Recognition","date":"2023-02-16","arxiv_id":"2303.00922","n_code_links":0,"syntology":null},{"paper":"/paper/symbolic-discovery-of-optimization-algorithms-1","slug":"symbolic-discovery-of-optimization-algorithms-1","title":"Symbolic Discovery of Optimization Algorithms","date":"2023-02-13","arxiv_id":"2302.06675","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":null}},{"paper":null,"slug":"is-boeing-737-max-still-safe-analysis-and","title":"Analysis and Prevention of MCAS-Induced Crashes","date":"2023-01-20","arxiv_id":"2301.08779","n_code_links":0,"syntology":null}],"papers_shown":32,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":32,"with_a_code_link":11,"where_syntology_ran_a_sample":9,"not_listed_spam_title":0,"listed":32,"listed_where_code_ran":9,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":8,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":8,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":9,"papers_where_code_ran_first_page":"/method/lion/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/lion/papers/ran/1","json":"/method/lion/papers/ran/1.json","rows":9}],"tasks":[{"task":"/task/language-modeling","name":"Language Modeling","papers":4},{"task":"/task/language-modelling","name":"Language Modelling","papers":4},{"task":"/task/quantization","name":"Quantization","papers":3},{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":2},{"task":null,"name":"GPU","papers":2},{"task":null,"name":"Position","papers":2},{"task":"/task/stochastic-optimization","name":"Stochastic Optimization","papers":2},{"task":"/task/parameter-efficient-fine-tuning","name":"parameter-efficient fine-tuning","papers":2},{"task":"/task/3d-object-detection","name":"3D Object Detection","papers":1},{"task":"/task/attribute","name":"Attribute","papers":1},{"task":"/task/audio-visual-speech-recognition","name":"Audio-Visual Speech Recognition","papers":1},{"task":"/task/diversity","name":"Diversity","papers":1},{"task":"/task/federated-learning","name":"Federated Learning","papers":1},{"task":"/task/handwritten-text-recognition","name":"Handwritten Text Recognition","papers":1},{"task":"/task/image-classification","name":"Image Classification","papers":1},{"task":"/task/l2-regularization","name":"L2 Regularization","papers":1},{"task":"/task/lambada","name":"LAMBADA","papers":1},{"task":"/task/large-language-model","name":"Large Language Model","papers":1},{"task":"/task/lip-reading","name":"Lip Reading","papers":1},{"task":"/task/lipreading","name":"Lipreading","papers":1}],"tasks_shown":20,"n_tasks":38,"usage_by_year":[{"year":"2023","papers":12},{"year":"2024","papers":14},{"year":"2025","papers":6}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/lion"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Lion","read_at":"2026-10-05T00:58:36+00:00","query":"Lion","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"Lion","origin_arxiv_id":"2305.12870"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":null,"repositories":{"listed":1,"in_graph":1,"list_truncated":false,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"outranked","may_be_wrong_sense":true,"bearers_found":8,"served":{"arxiv_id":"2305.12870","citations":49},"leading_other_bearer":{"arxiv_id":"2210.06978","title":"LION: Latent Point Diffusion Models for 3D Shape Generation","citations":733,"named_by":["method_name","title_prefix"],"bearer_name":"LION","ask_with":"have('2210.06978')"},"n_other_bearers_listed":5,"note":"'Lion' is ALSO the name of the paper 'LION: Latent Point Diffusion Models for 3D Shape Generation' (arXiv:2210.06978, 733 citations), which is cited FAR more than the paper this answer is about (49). This answer may be about the wrong Lion. If you meant the better-known one, ask again with that entry of `also_named`."},"name_competition_text":"The same name also belongs to “LION”, a method in the paper “LION: Latent Point Diffusion Models for 3D Shape Generation” (arXiv:2210.06978, 733 citations), which the lookup finds cited far more than the paper this answer is about (49 citations), so this answer may be about a different sense of the name.","near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"2305.12870","page":"/paper/lion-adversarial-distillation-of-closed","syntology_url":"https://syntology.ai/paper/2305.12870","in_graph_layer":true,"samples":11,"ran":6,"n_ran_checked":4,"n_instrument":2,"unverified":5,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: official). The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2305.12870) is about this method is not checked.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"c34785c123009eeb25b3cff274794235d5b7483fbd9b4be2d0507cc39c7662fd","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}