Methods › General › Normalization › Layer Normalization
Layer Normalization
Introduced by Jimmy Lei Ba et al. in Layer Normalization
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs to the neurons within a hidden layer so the normalization does not introduce any new dependencies between training cases. It works well for RNNs and improves both the training time and the generalization performance of several existing RNN models. More recently, it has been used with Transformer models.
We compute the layer normalization statistics over all the hidden units in the same layer as follows:
μˡ = 1/H∑ᴴᵢ₌₁aᵢˡ
σˡ = √(1/H∑ᴴᵢ₌₁(aᵢˡ-μˡ)²)
where H denotes the number of hidden units in a layer. Under layer normalization, all the hidden units in a layer share the same normalization terms μ and σ, but different training cases have different normalization terms. Unlike batch normalization, layer normalization does not impose any constraint on the size of the mini-batch and it can be used in the pure online regime with batch size 1.
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 34 in the graph, strongest link: mentioned on GitHub). This answer is about arXiv:1607.06450, the paper the archive credits with this method.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- Frequency Adaptive Normalization (FAN) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Frequency Adaptive Normalization (FAN)') - Adversarial Batch Normalization (AdvBN) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Adversarial Batch Normalization (AdvBN)') - Style-Adaptive Layer Normalization (SALN) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Style-Adaptive Layer Normalization (SALN)') - DuBIN (Dual-Batch-and-Instance Normalization) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='DuBIN (Dual-Batch-and-Instance Normalization)')
Origin paper of this answer, arXiv:1607.06450 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 7 ran (of which 5 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 10 unverified (of 17 harvested samples) · 8 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 24,980 · with a code link: 11,273 · where Syntology ran a sample: 3,471 (2,817 with a run with no instrument failure, 654 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (3,471 of 24,980 tagged: 2,817 with a run with no instrument failure, 654 where every run was a failure of Syntology's instrument)
Page 1 of 250: papers 1 to 100 of 24,980, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
DASViT: Differentiable Architecture Search for Vision Transformer 17 Jul 2025 · 0 repositories · arXiv:2507.13079
-
Making Language Model a Hierarchical Classifier and Generator 17 Jul 2025 · 1 repository · arXiv:2507.12930
-
Best Practices for Large-Scale, Pixel-Wise Crop Mapping and Transfer Learning Workflows 16 Jul 2025 · 1 repository · arXiv:2507.12590
-
DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition 16 Jul 2025 · 1 repository · arXiv:2507.12426
-
Biological Processing Units: Leveraging an Insect Connectome to Pioneer Biofidelic Neural Architectures 15 Jul 2025 · 0 repositories · arXiv:2507.10951
-
Generative Click-through Rate Prediction with Applications to Search Advertising 15 Jul 2025 · 0 repositories · arXiv:2507.11246
-
Hashed Watermark as a Filter: Defeating Forging and Overwriting Attacks in Weight-based Neural Network Watermarking 15 Jul 2025 · 1 repository · arXiv:2507.11137
-
KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding 15 Jul 2025 · 1 repository · arXiv:2507.11273Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Langevin Flows for Modeling Neural Latent Dynamics 15 Jul 2025 · 1 repository · arXiv:2507.11531
-
Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI 13 Jul 2025 · 0 repositories · arXiv:2507.09702
-
Learning from Synthetic Labs: Language Models as Auction Participants 12 Jul 2025 · 0 repositories · arXiv:2507.09083
-
Comparative Analysis of Vision Transformers and Traditional Deep Learning Approaches for Automated Pneumonia Detection in Chest X-Rays 11 Jul 2025 · 0 repositories · arXiv:2507.10589
-
A Wireless Foundation Model for Multi-Task Prediction 8 Jul 2025 · 0 repositories · arXiv:2507.05938
-
Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving 8 Jul 2025 · 1 repository · arXiv:2507.06229Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Chat-Ghosting: A Comparative Study of Methods for Auto-Completion in Dialog Systems 8 Jul 2025 · 0 repositories · arXiv:2507.05940
-
Geo-Registration of Terrestrial LiDAR Point Clouds with Satellite Images without GNSS 8 Jul 2025 · 0 repositories · arXiv:2507.05999
-
Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate 8 Jul 2025 · 1 repository · arXiv:2507.07129
-
SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression 8 Jul 2025 · 0 repositories · arXiv:2507.05633
-
Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identification 8 Jul 2025 · 1 repository · arXiv:2507.06093
-
AI Generated Text Detection Using Instruction Fine-tuned Large Language and Transformer-Based Models 7 Jul 2025 · 0 repositories · arXiv:2507.05157
-
Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations 7 Jul 2025 · 1 repository · arXiv:2507.04886
-
Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning 7 Jul 2025 · 0 repositories · arXiv:2507.05526
-
SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model 7 Jul 2025 · 1 repository · arXiv:2507.05148
-
Behaviour Space Analysis of LLM-driven Meta-heuristic Discovery 4 Jul 2025 · 0 repositories · arXiv:2507.03605
-
CyberRAG: An agentic RAG cyber attack classification and reporting tool 3 Jul 2025 · 0 repositories · arXiv:2507.02424
-
DeepGesture: A conversational gesture synthesis system based on emotions and semantics 3 Jul 2025 · 1 repository · arXiv:2507.03147
-
Fast and Simplex: 2-Simplicial Attention in Triton 3 Jul 2025 · 0 repositories · arXiv:2507.02754
-
Knowledge Protocol Engineering: A New Paradigm for AI in Domain-Specific Knowledge Work 3 Jul 2025 · 0 repositories · arXiv:2507.02760
-
AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation 2 Jul 2025 · 0 repositories · arXiv:2507.01961
-
Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer 2 Jul 2025 · 1 repository · arXiv:2507.02199Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 2 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 11 pointer-only (licence)
-
A Unified Transformer-Based Framework with Pretraining For Whole Body Grasping Motion Generation 1 Jul 2025 · 1 repository · arXiv:2507.00676
-
Large Language Models Don't Make Sense of Word Problems. A Scoping Review from a Mathematics Education Perspective 30 Jun 2025 · 0 repositories · arXiv:2506.24006
-
Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking 30 Jun 2025 · 1 repository · arXiv:2506.23783
-
Robustness of Misinformation Classification Systems to Adversarial Examples Through BeamAttack 30 Jun 2025 · 1 repository · arXiv:2506.23661
-
CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation 29 Jun 2025 · 1 repository · arXiv:2506.23347
-
Agent-to-Agent Theory of Mind: Testing Interlocutor Awareness among Large Language Models 28 Jun 2025 · 0 repositories · arXiv:2506.22957
-
Attention to Burstiness: Low-Rank Bilinear Prompt Tuning 28 Jun 2025 · 1 repository · arXiv:2506.22908
-
Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Features 26 Jun 2025 · 1 repository · arXiv:2506.21046
-
Cat and Mouse -- Can Fake Text Generation Outpace Detector Systems? 26 Jun 2025 · 0 repositories · arXiv:2506.21274
-
Chain-of-Thought Enhanced Shallow Transformers for Wireless Symbol Detection 26 Jun 2025 · 0 repositories · arXiv:2506.21093
-
DBConformer: Dual-Branch Convolutional Transformer for EEG Decoding 26 Jun 2025 · 1 repository · arXiv:2506.21140
-
Distributed Cross-Channel Hierarchical Aggregation for Foundation Models 26 Jun 2025 · 0 repositories · arXiv:2506.21411
-
Early Stopping Tabular In-Context Learning 26 Jun 2025 · 0 repositories · arXiv:2506.21387
-
EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora 26 Jun 2025 · 1 repository · arXiv:2506.20963
-
Forecasting Geopolitical Events with a Sparse Temporal Fusion Transformer and Gaussian Process Hybrid: A Case Study in Middle Eastern and U.S. Conflict Dynamics 26 Jun 2025 · 0 repositories · arXiv:2506.20935
-
Large Language Models Acing Chartered Accountancy 26 Jun 2025 · 0 repositories · arXiv:2506.21031
-
Leveraging LLM-Assisted Query Understanding for Live Retrieval-Augmented Generation 26 Jun 2025 · 0 repositories · arXiv:2506.21384
-
M2SFormer: Multi-Spectral and Multi-Scale Attention with Edge-Aware Difficulty Guidance for Image Forgery Localization 26 Jun 2025 · 0 repositories · arXiv:2506.20922
-
Progtuning: Progressive Fine-tuning Framework for Transformer-based Language Models 26 Jun 2025 · 0 repositories · arXiv:2506.21119
-
PsyLite Technical Report 26 Jun 2025 · 1 repository · arXiv:2506.21536
-
Real-Time ESFP: Estimating, Smoothing, Filtering, and Pose-Mapping 26 Jun 2025 · 1 repository · arXiv:2506.21234
-
Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality 26 Jun 2025 · 1 repository · arXiv:2506.20978Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation 26 Jun 2025 · 1 repository · arXiv:2506.21154Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Video Virtual Try-on with Conditional Diffusion Transformer Inpainter 26 Jun 2025 · 0 repositories · arXiv:2506.21270
-
A foundation model with multi-variate parallel attention to generate neuronal activity 25 Jun 2025 · 1 repository · arXiv:2506.20354
-
AdvMIM: Adversarial Masked Image Modeling for Semi-Supervised Medical Image Segmentation 25 Jun 2025 · 0 repositories · arXiv:2506.20563
-
AI Assistants to Enhance and Exploit the PETSc Knowledge Base 25 Jun 2025 · 0 repositories · arXiv:2506.20608
-
CCRS: A Zero-Shot LLM-as-a-Judge Framework for Comprehensive RAG Evaluation 25 Jun 2025 · 0 repositories · arXiv:2506.20128
-
Differential Transformer-driven 6G Physical Layer for Collaborative Perception Enhancement 25 Jun 2025 · 0 repositories · arXiv:2506.20597
-
Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation 25 Jun 2025 · 0 repositories · arXiv:2506.20869
-
Feature Hallucination for Self-supervised Action Recognition 25 Jun 2025 · 0 repositories · arXiv:2506.20342
-
ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset 25 Jun 2025 · 0 repositories · arXiv:2506.20093
-
Knowledge-Aware Diverse Reranking for Cross-Source Question Answering 25 Jun 2025 · 0 repositories · arXiv:2506.20476
-
Large Language Model-Driven Code Compliance Checking in Building Information Modeling 25 Jun 2025 · 0 repositories · arXiv:2506.20551
-
Med-Art: Diffusion Transformer for 2D Medical Text-to-Image Generation 25 Jun 2025 · 0 repositories · arXiv:2506.20449
-
Memento: Note-Taking for Your Future Self 25 Jun 2025 · 0 repositories · arXiv:2506.20642
-
MS-IQA: A Multi-Scale Feature Fusion Network for PET/CT Image Quality Assessment 25 Jun 2025 · 1 repository · arXiv:2506.20200
-
Probing AI Safety with Source Code 25 Jun 2025 · 1 repository · arXiv:2506.20471
-
X-SiT: Inherently Interpretable Surface Vision Transformers for Dementia Diagnosis 25 Jun 2025 · 0 repositories · arXiv:2506.20267
-
Accurate and Energy Efficient: Local Retrieval-Augmented Generation Models Outperform Commercial Large Language Models in Medical Tasks 24 Jun 2025 · 0 repositories · arXiv:2506.20009
-
Controlled Retrieval-augmented Context Evaluation for Long-form RAG 24 Jun 2025 · 0 repositories · arXiv:2506.20051
-
Inference Scaled GraphRAG: Improving Multi Hop Question Answering on Knowledge Graphs 24 Jun 2025 · 0 repositories · arXiv:2506.19967
-
KunLunBaizeRAG: Reinforcement Learning Driven Inference Performance Leap for Large Language Models 24 Jun 2025 · 0 repositories · arXiv:2506.19466
-
Lost in Translation? Converting RegExes for Log Parsing into Dynatrace Pattern Language 24 Jun 2025 · 0 repositories · arXiv:2506.19539
-
Toward the Explainability of Protein Language Models for Sequence Design 24 Jun 2025 · 0 repositories · arXiv:2506.19532
-
Unlocking Insights Addressing Alcohol Inference Mismatch through Database-Narrative Alignment 24 Jun 2025 · 0 repositories · arXiv:2506.19342
-
Vision Transformer-Based Time-Series Image Reconstruction for Cloud-Filling Applications 24 Jun 2025 · 0 repositories · arXiv:2506.19591
-
WebGuard++:Interpretable Malicious URL Detection via Bidirectional Fusion of HTML Subgraphs and Multi-Scale Convolutional BERT 24 Jun 2025 · 0 repositories · arXiv:2506.19356
-
A Study of Dynamic Stock Relationship Modeling and S&P500 Price Forecasting Based on Differential Graph Transformer 23 Jun 2025 · 0 repositories · arXiv:2506.18717
-
An Audio-centric Multi-task Learning Framework for Streaming Ads Targeting on Spotify 23 Jun 2025 · 0 repositories · arXiv:2506.18735
-
Deep CNN Face Matchers Inherently Support Revocable Biometric Templates 23 Jun 2025 · 0 repositories · arXiv:2506.18731
-
Fully Few-shot Class-incremental Audio Classification Using Multi-level Embedding Extractor and Ridge Regression Classifier 23 Jun 2025 · 1 repository · arXiv:2506.18406
-
MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners 23 Jun 2025 · 0 repositories · arXiv:2506.18729
-
Rethinking Decoder Design: Improving Biomarker Segmentation Using Depth-to-Space Restoration and Residual Linear Attention 23 Jun 2025 · 1 repository · arXiv:2506.18335Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Security Assessment of DeepSeek and GPT Series Models against Jailbreak Attacks 23 Jun 2025 · 0 repositories · arXiv:2506.18543
-
Semantic similarity estimation for domain specific data using BERT and other techniques 23 Jun 2025 · 0 repositories · arXiv:2506.18602
-
SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications 23 Jun 2025 · 0 repositories · arXiv:2506.18951
-
T-CPDL: A Temporal Causal Probabilistic Description Logic for Developing Logic-RAG Agent 23 Jun 2025 · 0 repositories · arXiv:2506.18559
-
Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learning 23 Jun 2025 · 1 repository · arXiv:2506.18537
-
Leveraging Large Language Model for Intelligent Log Processing and Autonomous Debugging in Cloud AI Platforms 22 Jun 2025 · 0 repositories · arXiv:2506.17900
-
In-Context Learning Strategies Emerge Rationally 21 Jun 2025 · 0 repositories · arXiv:2506.17859
-
MTSIC: Multi-stage Transformer-based GAN for Spectral Infrared Image Colorization 21 Jun 2025 · 0 repositories · arXiv:2506.17540
-
SSAVSV: Towards Unified Model for Self-Supervised Audio-Visual Speaker Verification 21 Jun 2025 · 0 repositories · arXiv:2506.17694
-
IsoNet: Causal Analysis of Multimodal Transformers for Neuromuscular Gesture Classification 20 Jun 2025 · 0 repositories · arXiv:2506.16744
-
Leveraging LLMs to Assess Tutor Moves in Real-Life Dialogues: A Feasibility Study 20 Jun 2025 · 0 repositories · arXiv:2506.17410
-
Multimodal Political Bias Identification and Neutralization 20 Jun 2025 · 0 repositories · arXiv:2506.17372
-
Optimal Depth of Neural Networks 20 Jun 2025 · 0 repositories · arXiv:2506.16862
-
Trans²-CBCT: A Dual-Transformer Framework for Sparse-View CBCT Reconstruction 20 Jun 2025 · 0 repositories · arXiv:2506.17425
-
UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generation 20 Jun 2025 · 1 repository · arXiv:2506.17202Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 8 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples) · 8 pointer-only (licence)
-
VMRA-MaR: An Asymmetry-Aware Temporal Framework for Longitudinal Breast Cancer Risk Prediction 20 Jun 2025 · 1 repository · arXiv:2506.17412
Tasks archive 2025-07-28
20 shown of 2,597 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 3,010 |
| Language Modeling | 2,355 |
| Retrieval | 1,818 |
| Question Answering | 1,490 |
| Decoder | 1,393 |
| RAG | 1,355 |
| Sentence | 1,334 |
| Retrieval-augmented Generation | 1,176 |
| Translation | 1,035 |
| Machine Translation | 925 |
| Semantic Segmentation | 862 |
| Large Language Model | 841 |
| Text Generation | 763 |
| Image Classification | 720 |
| Transfer Learning | 715 |
| Object Detection | 668 |
| Representation Learning | 622 |
| Sentiment Analysis | 608 |
| Classification | 597 |
| object-detection | 597 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections