Methods › General › Stochastic Optimization › Stochastic Weight Averaging
Stochastic Weight Averaging
Introduced by Pavel Izmailov et al. in Averaging Weights Leads to Wider Optima and Better Generalization
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Stochastic Weight Averaging is an optimization procedure that averages multiple points along the trajectory of SGD, with a cyclical or constant learning rate. On the one hand it averages weights, but it also has the property that, with a cyclical or constant learning rate, SGD proposals are approximately sampling from the loss surface of the network, leading to stochastic weights and helping to discover broader optima.
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: Boosting Data Augmentation with Stochastic Weight Averaging; Clinically-Grounded Architecture using Riemannian Alignment and Stochastic Weight Averaging; Early Weight Averaging; weight averaging technique; Adaptive Weighted Averaging; precision weighted averaging; Trainable Weight Averaging (TWA); stochastic frame-averaging (SFA).
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 45 · with a code link: 21 · where Syntology ran a sample: 10 (8 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (10 of 45 tagged: 8 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 45 of 45, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
An Effective End-to-End Solution for Multimodal Action Recognition 11 Jun 2025 · 0 repositories · arXiv:2506.09345
-
Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections 23 Apr 2025 · 0 repositories · arXiv:2504.16612
-
Understanding Flatness in Generative Models: Its Role and Benefits 14 Mar 2025 · 0 repositories · arXiv:2503.11078
-
SeWA: Selective Weight Average via Probabilistic Masking 14 Feb 2025 · 0 repositories · arXiv:2502.10119
-
Divide and Conquer: Grounding a Bleeding Areas in Gastrointestinal Image with Two-Stage Model 21 Dec 2024 · 1 repository · arXiv:2412.16723
-
A Unified Analysis for Finite Weight Averaging 20 Nov 2024 · 0 repositories · arXiv:2411.13169
-
Adaptive Stochastic Weight Averaging 27 Jun 2024 · 1 repository · arXiv:2406.19092
-
Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations 28 May 2024 · 4 repositories · arXiv:2405.18392Syntology official (archive's flag): 6 ran · 10 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples) · 4 pointer-only (licence)
-
Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models 6 May 2024 · 1 repository · arXiv:2405.03425Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples) · 1 pointer-only (licence)
-
Post-Hoc Reversal: Are We Selecting Models Prematurely? 11 Apr 2024 · 1 repository · arXiv:2404.07815Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample) · 1 pointer-only (licence)
-
Instance Segmentation under Occlusions via Location-aware Copy-Paste Data Augmentation 27 Oct 2023 · 1 repository · arXiv:2310.17949
-
SimBIG: Field-level Simulation-Based Inference of Galaxy Clustering 23 Oct 2023 · 0 repositories · arXiv:2310.15256
-
Weight Averaging Improves Knowledge Distillation under Domain Shift 20 Sep 2023 · 1 repository · arXiv:2309.11446
-
A Novel Training Framework for Physics-informed Neural Networks: Towards Real-time Applications in Ultrafast Ultrasound Blood Flow Imaging 9 Sep 2023 · 0 repositories · arXiv:2309.04755
-
Fine-grained building roof instance segmentation based on domain adapted pretraining and composite dual-backbone 10 Aug 2023 · 0 repositories · arXiv:2308.05358
-
The Split Matters: Flat Minima Methods for Improving the Performance of GNNs 15 Jun 2023 · 1 repository · arXiv:2306.09121
-
Decoupled Training for Long-Tailed Classification With Stochastic Representations 19 Apr 2023 · 0 repositories · arXiv:2304.09426Syntology 10 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified (of 14 harvested samples) · 2 pointer-only (licence)
-
Training trajectories, mini-batch losses and the curious role of the learning rate 5 Jan 2023 · 0 repositories · arXiv:2301.02312
-
Frequency Regularization for Improving Adversarial Robustness 24 Dec 2022 · 0 repositories · arXiv:2212.12732
-
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging 12 Dec 2022 · 0 repositories · arXiv:2212.05956
-
Deep Combinatorial Aggregation 12 Oct 2022 · 1 repository · arXiv:2210.06436Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
ARQ-based Average Consensus over Directed Network Topologies with Unreliable Communication Links 29 Sep 2022 · 0 repositories · arXiv:2209.14699
-
Robust Simulation-Based Inference in Cosmology with Bayesian Neural Networks 18 Jul 2022 · 1 repository · arXiv:2207.08435Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Trainable Weight Averaging: A General Approach for Subspace Training 26 May 2022 · 1 repository · arXiv:2205.13104Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 5 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 8 harvested samples)
-
UNBUS: Uncertainty-aware Deep Botnet Detection System in Presence of Perturbed Samples 18 Apr 2022 · 0 repositories · arXiv:2204.09502
-
Improving Generalization in Federated Learning by Seeking Flat Minima 22 Mar 2022 · 1 repository · arXiv:2203.11834Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
PFGE: Parsimonious Fast Geometric Ensembling of DNNs 14 Feb 2022 · 1 repository · arXiv:2202.06658
-
When Do Flat Minima Optimizers Work? 1 Feb 2022 · 1 repository · arXiv:2202.00661
-
SpectraNet: Learned Recognition of Artificial Satellites From High Contrast Spectroscopic Imagery 10 Jan 2022 · 0 repositories · arXiv:2201.03614
-
Stochastic Weight Averaging Revisited 3 Jan 2022 · 1 repository · arXiv:2201.00519
-
5th Place Solution for VSPW 2021 Challenge 13 Dec 2021 · 0 repositories · arXiv:2112.06379
-
How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task 18 Nov 2021 · 1 repository · arXiv:2111.09612
-
Reduced-Lead ECG Classifier Model Trained with DivideMix and Model Ensemble 24 Sep 2021 · 0 repositories · arXiv:2109.12063
-
Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression 16 Sep 2021 · 0 repositories · arXiv:2109.08248
-
Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines 10 Jul 2021 · 0 repositories · arXiv:2107.10650
-
Learning Neural Network Subspaces 20 Feb 2021 · 1 repository · arXiv:2102.10472
-
SWAD: Domain Generalization by Seeking Flat Minima 17 Feb 2021 · 4 repositories · arXiv:2102.08604Syntology official (archive's flag): 2 ran · 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 2 samples that ran constructed an object rather than computing a result (of 3 harvested samples)
-
Robust Overfitting may be mitigated by properly learned smoothening 1 Jan 2021 · 0 repositories
-
The 2ST-UNet for Pneumothorax Segmentation in Chest X-Rays using ResNet34 as a Backbone for U-Net 6 Sep 2020 · 0 repositories · arXiv:2009.02805
-
DHARI Report to EPIC-Kitchens 2020 Object Detection Challenge 28 Jun 2020 · 0 repositories · arXiv:2006.15553
-
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well 7 Jan 2020 · 0 repositories · arXiv:2001.02312
-
Object Segmentation using Pixel-wise Adversarial Loss 23 Sep 2019 · 0 repositories · arXiv:1909.10341
-
On Model Stability as a Function of Random Seed 23 Sep 2019 · 1 repository · arXiv:1909.10447
-
A Simple Baseline for Bayesian Uncertainty in Deep Learning 7 Feb 2019 · 8 repositories · arXiv:1902.02476Syntology official (archive's flag): 8 ran · 11 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 2 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 6 unverified (of 17 harvested samples) · 9 pointer-only (licence)
-
Averaging Weights Leads to Wider Optima and Better Generalization 14 Mar 2018 · 17 repositories · arXiv:1803.05407Syntology official (archive's flag): 2 ran · 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 1 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 4 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 69 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections