Methods › General › Optimization › SAGA
SAGA
Introduced by Aaron Defazio et al. in SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
SAGA is a method in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem.
In Syntology about another paper, arXiv:2504.21034: repository known, no verified implementation (declared_code; repositories: 2 listed of 2 in the graph, strongest link: listed only). About another paper: this answer is about arXiv:2504.21034, while the archive credits arXiv:1407.0202.
From the lookup's note on this name: The same name also belongs to the paper “SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives” (arXiv:1407.0202, 2,023 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name. For arXiv:1407.0202: its page on this site · ask the lookup about it.
Origin paper of this answer, arXiv:2504.21034 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 81 · with a code link: 15 · where Syntology ran a sample: 5 (4 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (5 of 81 tagged: 4 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 81 of 81, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Rethinking Verification for LLM Code Generation: From Generation to Testing 9 Jul 2025 · 1 repository · arXiv:2507.06920
-
ATMM-SAGA: Alternating Training for Multi-Module with Score-Aware Gated Attention SASV system 23 May 2025 · 0 repositories · arXiv:2505.18273
-
A digital perspective on the role of a stemma in material-philological transmission studies 11 May 2025 · 0 repositories · arXiv:2505.06938
-
SAGA: A Security Architecture for Governing AI Agentic Systems 27 Apr 2025 · 0 repositories · arXiv:2504.21034Syntology 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples)
-
SAGA: Semantic-Aware Gray color Augmentation for Visible-to-Thermal Domain Adaptation across Multi-View Drone and Ground-Based Vision Systems 22 Apr 2025 · 2 repositories · arXiv:2504.15728
-
Variance-Reduced Fast Operator Splitting Methods for Stochastic Generalized Equations 17 Apr 2025 · 0 repositories · arXiv:2504.13046
-
SAGA: Surface-Aligned Gaussian Avatar 1 Dec 2024 · 0 repositories · arXiv:2412.00845
-
On the SAGA algorithm with decreasing step 2 Oct 2024 · 0 repositories · arXiv:2410.03760
-
SAGA: Synthesis Augmentation with Genetic Algorithms for In-Memory Sequence Optimization 14 Jun 2024 · 0 repositories · arXiv:2406.09677
-
Variance-Reduced Fast Krasnoselkii-Mann Methods for Finite-Sum Root-Finding Problems 4 Jun 2024 · 0 repositories · arXiv:2406.02413
-
SPABA: A Single-Loop and Probabilistic Stochastic Bilevel Algorithm Achieving Optimal Sample Complexity 29 May 2024 · 0 repositories · arXiv:2405.18777
-
Segment Any 3D Gaussians 1 Dec 2023 · 0 repositories · arXiv:2312.00860
-
PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates 5 Sep 2023 · 1 repository · arXiv:2309.02014Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
Variance reduction techniques for stochastic proximal point algorithms 18 Aug 2023 · 1 repository · arXiv:2308.09310
-
Growing and Serving Large Open-domain Knowledge Graphs 16 May 2023 · 0 repositories · arXiv:2305.09464
-
Attacking the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples 7 Sep 2022 · 0 repositories · arXiv:2209.03358
-
Tackling Data Heterogeneity: A New Unified Framework for Decentralized SGD with Sample-induced Topology 8 Jul 2022 · 0 repositories · arXiv:2207.03730
-
An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms 28 Apr 2022 · 0 repositories · arXiv:2205.02273
-
Saga: A Platform for Continuous Construction and Serving of Knowledge At Scale 15 Apr 2022 · 0 repositories · arXiv:2204.07309
-
A framework for bilevel optimization that enables stochastic and global variance reduction algorithms 31 Jan 2022 · 1 repository · arXiv:2201.13409Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
SAGA: Stochastic Whole-Body Grasping with Contact 19 Dec 2021 · 1 repository · arXiv:2112.10103Syntology 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 2 pointer-only (licence)
-
Siamese Attribute-missing Graph Auto-encoder 9 Dec 2021 · 0 repositories · arXiv:2112.04842
-
Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selection 17 Nov 2021 · 1 repository · arXiv:2111.09074
-
ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computations 29 Sep 2021 · 0 repositories
-
ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computation 2 Mar 2021 · 0 repositories · arXiv:2103.01447
-
A New Framework for Variance-Reduced Hamiltonian Monte Carlo 9 Feb 2021 · 0 repositories · arXiv:2102.04613
-
Segmentation and genome annotation algorithms 3 Jan 2021 · 0 repositories · arXiv:2101.00688
-
Tight Lower Complexity Bounds for Strongly Convex Finite-Sum Optimization 17 Oct 2020 · 0 repositories · arXiv:2010.08766
-
Asynchronous Distributed Optimization with Stochastic Delays 22 Sep 2020 · 0 repositories · arXiv:2009.10717
-
Variance Reduced EXTRA and DIGing and Their Optimal Acceleration for Strongly Convex Decentralized Optimization 9 Sep 2020 · 0 repositories · arXiv:2009.04373
-
Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning 14 Aug 2020 · 0 repositories · arXiv:2008.06233
-
Langevin Monte Carlo: random coordinate descent and variance reduction 26 Jul 2020 · 0 repositories · arXiv:2007.14209
-
A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization 12 Jun 2020 · 0 repositories · arXiv:2006.07013
-
Sampling and Update Frequencies in Proximal Variance-Reduced Stochastic Gradient Methods 13 Feb 2020 · 0 repositories · arXiv:2002.05545
-
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks 29 Dec 2019 · 0 repositories · arXiv:1912.12716
-
On the Global Convergence of (Fast) Incremental Expectation Maximization Methods 28 Oct 2019 · 0 repositories · arXiv:1910.12521
-
Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking -- Part II: GT-SVRG 8 Oct 2019 · 0 repositories · arXiv:1910.04057
-
A Stochastic Proximal Point Algorithm for Saddle-Point Problems 13 Sep 2019 · 0 repositories · arXiv:1909.06946
-
ASYNC: A Cloud Engine with Asynchrony and History for Distributed Machine Learning 19 Jul 2019 · 0 repositories · arXiv:1907.08526
-
Estimate Sequences for Variance-Reduced Stochastic Composite Optimization 7 May 2019 · 0 repositories · arXiv:1905.02374
-
Cocoercivity, Smoothness and Bias in Variance-Reduced Stochastic Gradient Methods 21 Mar 2019 · 0 repositories · arXiv:1903.09009
-
Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization 16 Feb 2019 · 0 repositories · arXiv:1902.06158
-
Optimal mini-batch and step sizes for SAGA 31 Jan 2019 · 2 repositories · arXiv:1902.00071
-
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it 27 Jan 2019 · 0 repositories · arXiv:1901.09437
-
Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noise 25 Jan 2019 · 0 repositories · arXiv:1901.08788
-
SAGA with Arbitrary Sampling 24 Jan 2019 · 0 repositories · arXiv:1901.08669
-
Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity 1 Dec 2018 · 0 repositories
-
Stochastic Primal-Dual Method for Empirical Risk Minimization with 𝒪(1) Per-Iteration Complexity 3 Nov 2018 · 0 repositories · arXiv:1811.01182
-
Fast Variance Reduction Method with Stochastic Batch Size 7 Aug 2018 · 0 repositories · arXiv:1808.02169
-
Local Convergence Properties of SAGA/Prox-SVRG and Acceleration 1 Jul 2018 · 1 repository
-
Direct Acceleration of SAGA using Sampled Negative Momentum 28 Jun 2018 · 0 repositories · arXiv:1806.11048
-
Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data 8 Jun 2018 · 0 repositories · arXiv:1806.02927
-
Nonlinear Acceleration of CNNs 1 Jun 2018 · 1 repository · arXiv:1806.00370Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
SUCAG: Stochastic Unbiased Curvature-aided Gradient Method for Distributed Optimization 22 Mar 2018 · 0 repositories · arXiv:1803.08198
-
Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization 26 Feb 2018 · 0 repositories · arXiv:1802.09933
-
On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo 15 Feb 2018 · 0 repositories · arXiv:1802.05431
-
Faster Learning by Reduction of Data Access Time 18 Jan 2018 · 0 repositories · arXiv:1801.05931
-
Improved asynchronous parallel optimization analysis for stochastic incremental methods 11 Jan 2018 · 0 repositories · arXiv:1801.03749
-
Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice 15 Dec 2017 · 1 repository · arXiv:1712.05654
-
Nonlinear Acceleration of Stochastic Algorithms 1 Dec 2017 · 0 repositories
-
Duality-free Methods for Stochastic Composition Optimization 26 Oct 2017 · 0 repositories · arXiv:1710.09554
-
Stochastic Optimization with Bandit Sampling 8 Aug 2017 · 0 repositories · arXiv:1708.02544
-
Variance-Reduced Stochastic Learning by Networked Agents under Random Reshuffling 4 Aug 2017 · 0 repositories · arXiv:1708.01384
-
Variance-Reduced Stochastic Learning under Random Reshuffling 4 Aug 2017 · 0 repositories · arXiv:1708.01383
-
Breaking the Nonsmooth Barrier: A Scalable Parallel Method for Composite Optimization 20 Jul 2017 · 1 repository · arXiv:1707.06468
-
A Unified Analysis of Stochastic Optimization Methods Using Jump System Theory and Quadratic Constraints 25 Jun 2017 · 0 repositories · arXiv:1706.08141
-
Distributed SAGA: Maintaining linear convergence rate with limited communication 29 May 2017 · 0 repositories · arXiv:1705.10405
-
Catalyst Acceleration for Gradient-Based Non-Convex Optimization 31 Mar 2017 · 0 repositories · arXiv:1703.10993
-
Guaranteed Sufficient Decrease for Variance Reduced Stochastic Gradient Descent 20 Mar 2017 · 0 repositories · arXiv:1703.06807
-
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient 1 Mar 2017 · 0 repositories · arXiv:1703.00102
-
SAGA and Restricted Strong Convexity 19 Feb 2017 · 0 repositories · arXiv:1702.05683
-
Dimension-Free Iteration Complexity of Finite Sum Optimization Problems 30 Jun 2016 · 0 repositories · arXiv:1606.09333
-
ASAGA: Asynchronous Parallel SAGA 15 Jun 2016 · 1 repository · arXiv:1606.04809
-
Fast Incremental Method for Nonconvex Optimization 19 Mar 2016 · 0 repositories · arXiv:1603.06159
-
A Simple Practical Accelerated Method for Finite Sums 8 Feb 2016 · 1 repository · arXiv:1602.02442Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
-
Speed learning on the fly 8 Nov 2015 · 0 repositories · arXiv:1511.02540
-
New Optimisation Methods for Machine Learning 9 Oct 2015 · 0 repositories · arXiv:1510.02533
-
On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants 23 Jun 2015 · 0 repositories · arXiv:1506.06840
-
Variance Reduced Stochastic Gradient Descent with Neighbors 11 Jun 2015 · 0 repositories · arXiv:1506.03662
-
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields 16 Apr 2015 · 0 repositories · arXiv:1504.04406
-
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives 1 Jul 2014 · 5 repositories · arXiv:1407.0202
Tasks archive 2025-07-28
20 shown of 39 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections