{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/sgd-with-momentum","slug":"sgd-with-momentum","name":"SGD with Momentum","full_name":"SGD with Momentum","full_name_withheld":false,"description_markdown":"### Why SGD with  Momentum?\r\nIn deep learning, we have used stochastic gradient descent as one of the optimizers because at the end we will find the minimum weight and bias at which the model loss is lowest. In the SGD we have some issues in which the SGD does not work perfectly because in deep learning we got a non-convex cost function graph and if use the simple SGD then it leads to low performance. There are 3 main reasons why it does not work:\r\n\r\n<img src=\"https://www.cs.umd.edu/~tomg/img/landscapes/shortHighRes.png\" alt=\"Non-convex graph\" style=\"width:400px;height :300px;\" />\r\n\r\n1) We end up in local minima and not able to reach global minima\r\nAt the start, we randomly start at some point and we are going to end up at the local minimum and not able to reach the global minimum.\r\n\r\n2) Saddle Point will be the stop for reaching global minima\r\nA saddle point is a point where in one direction the surface goes in the upward direction and in another direction it goes downwards. So that the slope is changing very gradually so the speed of changing is going to slow and as result, the training also going to slow.\r\n\r\n3) High curvature can be a reason\r\nThe larger radius leads to low curvature and vice-versa. It will be difficult to traverse in the large curvature which was generally high in non-convex optimization.\r\nBy using the SGD with Momentum optimizer we can overcome the problems like high curvature, consistent gradient, and noisy gradient.\r\n\r\n### What is SGD with Momentum?\r\nSGD with Momentum is an optimization technique designed to improve the performance of neural networks. To understand the intuition behind it, consider the analogy of a ball rolling down a slope. As it rolls, the ball gains speed over time due to momentum.\r\n\r\nSuppose we have a starting point A and want to reach B. If we are uncertain about the correct direction, we can ask four other points that have already reached B. If all four indicate the same direction, we can move more confidently and at a faster pace. This is the key idea behind SGD with Momentum.\r\n\r\n<img src=\"https://cdn-images-1.medium.com/max/1000/1*zNbZqU_uDIV13c9ZCJOEXA.jpeg\" alt=\"Non-convex graph\" style=\"width:400px;height :250px;\" />\r\n### How does SGD with Momentum work?\r\nTo understand its working, we must first explore the concept of the Exponentially Weighted Moving Average (EWMA), a technique used to identify trends in time-series data. The formula for EWMA is:\r\n\r\n<img src=\"https://cdn-images-1.medium.com/max/1000/1*O9Wcq-mbRgNOdRNTivSefw.png\" alt=\"Non-convex graph\" style=\"width:400px;height :100px;\" />\r\n\r\n In the formula, β represents the weightage that is going to assign to the past values of the gradient. The values of β is from 0 < β < 1. If the value of the beta is 0.5 then it means that the 1/1–0.5 = 2 so it represents that the calculated average was from the previous 2 readings. \r\n\r\nThe value of Vt depends on β. The higher the value of β the more we try to get an average of more past data and vice-versa. For example, let's take the value of β 0.98 and 0.5 for two different scenarios so if we do 1/1-β then we get 50 and 10 respectively so it was clear that to calculate the average we take past 50 and 10 outcomes respectively for both cases.\r\nNow in SGD with Momentum, we use the same concept of EWMA. Here we introduce the term velocity v which is used to denote the change in the gradient to get to the global minima. The change in the weights is denoted by the formula:\r\n\r\n<img src=\"https://cdn-images-1.medium.com/max/1000/0*i_r3u7LACa6dQyXd\" alt=\"Non-convex graph\" style=\"width:400px;height :100px;\" />\r\n\r\nthe β part of the V formula denotes and is useful to compute the confidence or we can say the past velocity for calculating Vt we have to calculate Vt-1 and for calculating Vt-1 we have to calculate Vt-2 and likewise. So we are using the history of velocity to calculate the momentum and this is the part that provides acceleration to the formula.\r\n\r\n<img src=\"https://cdn-images-1.medium.com/max/1000/1*L5lNKxAHLPYNc6-Zs4Vscw.png\" alt=\"Non-convex graph\" style=\"width:300px;height :100px;\" />\r\n\r\nHere we have to consider two cases:\r\n1. β=0 then, as per the formula weight updating is going to just work as a Stochastic gradient descent. Here we called β a decaying factor because it is defining the speed of past velocity.\r\n\r\n2. β=1 then, there will be no decay. It involves the dynamic equilibrium which is not desired so we generally use the value of β like 0.9,0.99or 0.5 only.\r\n\r\n### Advantages of SGD with Momentum\r\n1. **Faster Convergence**: Momentum accelerates training, making it faster than standard SGD.\r\n2. **Escaping Local Minima**: Momentum helps the optimizer overcome small local minima and reach the global minimum.\r\n\r\n<img src=\"https://cdn-images-1.medium.com/max/1000/1*Nb39bHHUWGXqgisr2WcLGQ.gif\" alt=\"Non-convex graph\" style=\"width:400px;height :300px;\" />\r\n\r\nIn the following visualization, purple represents SGD with Momentum, while light blue represents standard SGD. As shown, SGD with Momentum successfully reaches the global minimum, whereas standard SGD gets stuck in local minima.\r\n\r\nHowever, momentum itself can sometimes be problematic. When the optimizer reaches the global minimum, excessive momentum may cause oscillations before stabilizing, leading to longer training times. Despite this, SGD with Momentum remains significantly faster than standard SGD and is widely used in optimization.","description_state":"present","introduced_year":1999,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Stochastic Optimization","url":"/methods/category/stochastic-optimization","pwc_aliases":[]}],"n_papers_tagged":144,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/adams-momentum-itself-can-be-a-normalizer-for","slug":"adams-momentum-itself-can-be-a-normalizer-for","title":"AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training","date":"2025-05-22","arxiv_id":"2505.16363","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["pku-huzhang/AdamS"],"state":"official: no sample here; runs from other or unrecorded repositories","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["found_in_text"]}}},{"paper":null,"slug":"understanding-gradient-orthogonalization-for","title":"Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization","date":"2025-03-16","arxiv_id":"2503.12645","n_code_links":0,"syntology":null},{"paper":null,"slug":"algorithmic-stability-of-stochastic-gradient","title":"Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise","date":"2025-02-02","arxiv_id":"2502.00885","n_code_links":0,"syntology":null},{"paper":null,"slug":"sofim-stochastic-optimization-using","title":"SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrix","date":"2024-03-05","arxiv_id":"2403.02833","n_code_links":0,"syntology":null},{"paper":null,"slug":"should-i-try-multiple-optimizers-when-fine","title":"Should I try multiple optimizers when fine-tuning pre-trained Transformers for NLP tasks? Should I tune their hyperparameters?","date":"2024-02-10","arxiv_id":"2402.06948","n_code_links":0,"syntology":null},{"paper":null,"slug":"role-of-momentum-in-smoothing-objective","title":"Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent","date":"2024-02-04","arxiv_id":"2402.02325","n_code_links":0,"syntology":null},{"paper":"/paper/adaplus-integrating-nesterov-momentum-and","slug":"adaplus-integrating-nesterov-momentum-and","title":"AdaPlus: Integrating Nesterov Momentum and Precise Stepsize Adjustment on AdamW Basis","date":"2023-09-05","arxiv_id":"2309.01966","n_code_links":1,"syntology":null},{"paper":null,"slug":"when-and-why-momentum-accelerates-sgd-an","title":"When and Why Momentum Accelerates SGD:An Empirical Study","date":"2023-06-15","arxiv_id":"2306.09000","n_code_links":0,"syntology":null},{"paper":"/paper/fast-diffusion-model","slug":"fast-diffusion-model","title":"Fast Diffusion Model","date":"2023-06-12","arxiv_id":"2306.06991","n_code_links":1,"syntology":{"ran":14,"of":17,"n_ran_checked":5,"n_instrument":9,"unverified":3,"pointer_only":9,"phrase":"14 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 4 honoured, 0 violated, 1 with no contract checked; 9 where Syntology's instrument failed) · 3 unverified","official":{"repos":["sail-sg/fdm"],"state":"official (archive's flag): 14 ran","n_ran":14,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"layer-wise-adaptive-step-sizes-for-stochastic","title":"Layer-wise Adaptive Step-Sizes for Stochastic First-Order Methods for Deep Learning","date":"2023-05-23","arxiv_id":"2305.13664","n_code_links":0,"syntology":null},{"paper":"/paper/momo-momentum-models-for-adaptive-learning","slug":"momo-momentum-models-for-adaptive-learning","title":"MoMo: Momentum Models for Adaptive Learning Rates","date":"2023-05-12","arxiv_id":"2305.07583","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["fabian-sp/MoMo"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/a-view-of-mini-batch-sgd-via-generating","slug":"a-view-of-mini-batch-sgd-via-generating","title":"A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta","date":"2022-06-22","arxiv_id":"2206.11124","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":0,"n_instrument":2,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["godofnothing/powerlawoptimization"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/computing-the-variance-of-shuffling","slug":"computing-the-variance-of-shuffling","title":"Computing the Variance of Shuffling Stochastic Gradient Algorithms via Power Spectral Density Analysis","date":"2022-06-01","arxiv_id":"2206.00632","n_code_links":1,"syntology":null},{"paper":"/paper/an-adaptive-gradient-method-with-energy-and","slug":"an-adaptive-gradient-method-with-energy-and","title":"An Adaptive Gradient Method with Energy and Momentum","date":"2022-03-23","arxiv_id":"2203.12191","n_code_links":1,"syntology":null},{"paper":null,"slug":"momentum-doesn-t-change-the-implicit-bias","title":"Does Momentum Change the Implicit Regularization on Separable Data?","date":"2021-10-08","arxiv_id":"2110.03891","n_code_links":0,"syntology":null},{"paper":"/paper/using-a-one-dimensional-parabolic-model-of","slug":"using-a-one-dimensional-parabolic-model-of","title":"Using a one dimensional parabolic model of the full-batch loss to estimate learning rates during training","date":"2021-08-31","arxiv_id":"2108.13880","n_code_links":1,"syntology":null},{"paper":null,"slug":"on-the-hyperparameters-in-stochastic-gradient","title":"On the Hyperparameters in Stochastic Gradient Descent with Momentum","date":"2021-08-09","arxiv_id":"2108.03947","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-new-adaptive-gradient-method-with-gradient","title":"A New Adaptive Gradient Method with Gradient Decomposition","date":"2021-07-18","arxiv_id":"2107.08377","n_code_links":0,"syntology":null},{"paper":"/paper/lrtuner-a-learning-rate-tuner-for-deep-neural","slug":"lrtuner-a-learning-rate-tuner-for-deep-neural","title":"LRTuner: A Learning Rate Tuner for Deep Neural Networks","date":"2021-05-30","arxiv_id":"2105.14526","n_code_links":2,"syntology":null},{"paper":"/paper/fast-federated-learning-by-balancing","slug":"fast-federated-learning-by-balancing","title":"Fast Federated Learning by Balancing Communication Trade-Offs","date":"2021-05-23","arxiv_id":"2105.11028","n_code_links":2,"syntology":null},{"paper":"/paper/positive-negative-momentum-manipulating","slug":"positive-negative-momentum-manipulating","title":"Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization","date":"2021-03-31","arxiv_id":"2103.17182","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result","official":{"repos":["zeke-xie/Positive-Negative-Momentum"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/bottleneck-transformers-for-visual","slug":"bottleneck-transformers-for-visual","title":"Bottleneck Transformers for Visual Recognition","date":"2021-01-27","arxiv_id":"2101.11605","n_code_links":13,"syntology":{"ran":26,"of":49,"n_ran_checked":19,"n_instrument":7,"unverified":23,"pointer_only":8,"phrase":"26 ran (of which 9 constructed an object rather than computing a result; 19 with no instrument failure: 1 honoured, 0 violated, 18 with no contract checked; 7 where Syntology's instrument failed) · 23 unverified","official":null}},{"paper":"/paper/ikshana-a-theory-of-human-scene-understanding","slug":"ikshana-a-theory-of-human-scene-understanding","title":"The Ikshana Hypothesis of Human Scene Understanding","date":"2021-01-21","arxiv_id":"2101.10837","n_code_links":2,"syntology":null},{"paper":"/paper/repvgg-making-vgg-style-convnets-great-again","slug":"repvgg-making-vgg-style-convnets-great-again","title":"RepVGG: Making VGG-style ConvNets Great Again","date":"2021-01-11","arxiv_id":"2101.03697","n_code_links":25,"syntology":{"ran":13,"of":16,"n_ran_checked":8,"n_instrument":5,"unverified":3,"pointer_only":6,"phrase":"13 ran (of which 7 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified","official":{"repos":["DingXiaoH/RepVGG"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":2,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["listed","official","unlocated"]}}},{"paper":"/paper/magneto-an-efficient-deep-learning-method-for-1","slug":"magneto-an-efficient-deep-learning-method-for-1","title":"MAGNeto: An Efficient Deep Learning Method for the Extractive Tags Summarization Problem","date":"2020-11-09","arxiv_id":"2011.04349","n_code_links":1,"syntology":null},{"paper":null,"slug":"dual-averaging-is-surprisingly-effective-for-1","title":"Dual Averaging is Surprisingly Effective for Deep Learning Optimization","date":"2020-10-20","arxiv_id":"2010.10502","n_code_links":0,"syntology":null},{"paper":"/paper/aegd-adaptive-gradient-decent-with-energy","slug":"aegd-adaptive-gradient-decent-with-energy","title":"AEGD: Adaptive Gradient Descent with Energy","date":"2020-10-10","arxiv_id":"2010.05109","n_code_links":1,"syntology":null},{"paper":"/paper/regularizing-neural-networks-via-adversarial","slug":"regularizing-neural-networks-via-adversarial","title":"Regularizing Neural Networks via Adversarial Model Perturbation","date":"2020-10-10","arxiv_id":"2010.04925","n_code_links":1,"syntology":{"ran":8,"of":12,"n_ran_checked":8,"n_instrument":0,"unverified":4,"pointer_only":0,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified","official":{"repos":["hiyouga/AMP-Regularizer"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":8,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":"/paper/understanding-the-role-of-momentum-in-non","slug":"understanding-the-role-of-momentum-in-non","title":"Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization","date":"2020-10-01","arxiv_id":"2010.00406","n_code_links":1,"syntology":null},{"paper":"/paper/optimization-of-graph-neural-networks-with","slug":"optimization-of-graph-neural-networks-with","title":"Optimization of Graph Neural Networks with Natural Gradient Descent","date":"2020-08-21","arxiv_id":"2008.09624","n_code_links":1,"syntology":null},{"paper":"/paper/spinalnet-deep-neural-network-with-gradual-1","slug":"spinalnet-deep-neural-network-with-gradual-1","title":"SpinalNet: Deep Neural Network with Gradual Input","date":"2020-07-07","arxiv_id":"2007.03347","n_code_links":3,"syntology":null},{"paper":null,"slug":"adaptive-braking-for-mitigating-gradient","title":"Adaptive Braking for Mitigating Gradient Delay","date":"2020-07-02","arxiv_id":"2007.01397","n_code_links":0,"syntology":null},{"paper":null,"slug":"squarm-sgd-communication-efficient-momentum","title":"SQuARM-SGD: Communication-Efficient Momentum SGD for Decentralized Optimization","date":"2020-05-13","arxiv_id":"2005.07041","n_code_links":0,"syntology":null},{"paper":"/paper/supervised-contrastive-learning","slug":"supervised-contrastive-learning","title":"Supervised Contrastive Learning","date":"2020-04-23","arxiv_id":"2004.11362","n_code_links":26,"syntology":{"ran":17,"of":23,"n_ran_checked":12,"n_instrument":5,"unverified":6,"pointer_only":5,"phrase":"17 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 2 honoured, 0 violated, 10 with no contract checked; 5 where Syntology's instrument failed) · 6 unverified","official":{"repos":["HobbitLong/SupContrast","google-research/google-research"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/yolov4-optimal-speed-and-accuracy-of-object","slug":"yolov4-optimal-speed-and-accuracy-of-object","title":"YOLOv4: Optimal Speed and Accuracy of Object Detection","date":"2020-04-23","arxiv_id":"2004.10934","n_code_links":223,"syntology":{"ran":142,"of":184,"n_ran_checked":128,"n_instrument":14,"unverified":42,"pointer_only":21,"phrase":"142 ran (of which 0 constructed an object rather than computing a result; 128 with no instrument failure: 8 honoured, 0 violated, 120 with no contract checked; 14 where Syntology's instrument failed) · 42 unverified","official":{"repos":["AlexeyAB/darknet"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/resnest-split-attention-networks","slug":"resnest-split-attention-networks","title":"ResNeSt: Split-Attention Networks","date":"2020-04-19","arxiv_id":"2004.08955","n_code_links":36,"syntology":{"ran":28,"of":48,"n_ran_checked":21,"n_instrument":7,"unverified":20,"pointer_only":23,"phrase":"28 ran (of which 0 constructed an object rather than computing a result; 21 with no instrument failure: 0 honoured, 0 violated, 21 with no contract checked; 7 where Syntology's instrument failed) · 20 unverified","official":{"repos":["zhanghang1989/ResNeSt"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":1,"ran_from_kinds":["listed","official"]}}},{"paper":"/paper/designing-network-design-spaces","slug":"designing-network-design-spaces","title":"Designing Network Design Spaces","date":"2020-03-30","arxiv_id":"2003.13678","n_code_links":26,"syntology":{"ran":48,"of":53,"n_ran_checked":41,"n_instrument":7,"unverified":5,"pointer_only":7,"phrase":"48 ran (of which 0 constructed an object rather than computing a result; 41 with no instrument failure: 4 honoured, 0 violated, 37 with no contract checked; 7 where Syntology's instrument failed) · 5 unverified","official":{"repos":["facebookresearch/pycls"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/hit-detector-hierarchical-trinity","slug":"hit-detector-hierarchical-trinity","title":"Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection","date":"2020-03-26","arxiv_id":"2003.11818","n_code_links":1,"syntology":null},{"paper":"/paper/revisiting-the-sibling-head-in-object","slug":"revisiting-the-sibling-head-in-object","title":"Revisiting the Sibling Head in Object Detector","date":"2020-03-17","arxiv_id":"2003.07540","n_code_links":2,"syntology":null},{"paper":"/paper/improved-baselines-with-momentum-contrastive","slug":"improved-baselines-with-momentum-contrastive","title":"Improved Baselines with Momentum Contrastive Learning","date":"2020-03-09","arxiv_id":"2003.04297","n_code_links":36,"syntology":{"ran":28,"of":43,"n_ran_checked":21,"n_instrument":7,"unverified":15,"pointer_only":15,"phrase":"28 ran (of which 0 constructed an object rather than computing a result; 21 with no instrument failure: 0 honoured, 1 violated, 20 with no contract checked; 7 where Syntology's instrument failed) · 15 unverified","official":{"repos":["facebookresearch/moco"],"state":"official: no sample here; runs from other or unrecorded repositories","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed","unlocated"]}}},{"paper":null,"slug":"momentum-improves-normalized-sgd","title":"Momentum Improves Normalized SGD","date":"2020-02-09","arxiv_id":"2002.03305","n_code_links":0,"syntology":null},{"paper":"/paper/compounding-the-performance-improvements-of","slug":"compounding-the-performance-improvements-of","title":"Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network","date":"2020-01-17","arxiv_id":"2001.06268","n_code_links":1,"syntology":{"ran":3,"of":11,"n_ran_checked":2,"n_instrument":1,"unverified":8,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified","official":{"repos":["clovaai/assembled-cnn"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":8,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"training-deep-networks-with-stochastic","title":"Training Deep Networks with Stochastic Gradient Normalized by Layerwise Adaptive Second Moments","date":"2020-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/large-scale-learning-of-general-visual","slug":"large-scale-learning-of-general-visual","title":"Big Transfer (BiT): General Visual Representation Learning","date":"2019-12-24","arxiv_id":"1912.11370","n_code_links":9,"syntology":{"ran":9,"of":10,"n_ran_checked":7,"n_instrument":2,"unverified":1,"pointer_only":0,"phrase":"9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 1 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified","official":{"repos":["google-research/big_transfer"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/pointrend-image-segmentation-as-rendering","slug":"pointrend-image-segmentation-as-rendering","title":"PointRend: Image Segmentation as Rendering","date":"2019-12-17","arxiv_id":"1912.08193","n_code_links":14,"syntology":{"ran":16,"of":18,"n_ran_checked":11,"n_instrument":5,"unverified":2,"pointer_only":0,"phrase":"16 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified","official":{"repos":["facebookresearch/detectron2"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/spinenet-learning-scale-permuted-backbone-for","slug":"spinenet-learning-scale-permuted-backbone-for","title":"SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization","date":"2019-12-10","arxiv_id":"1912.05027","n_code_links":13,"syntology":null},{"paper":"/paper/bridging-the-gap-between-anchor-based-and","slug":"bridging-the-gap-between-anchor-based-and","title":"Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection","date":"2019-12-05","arxiv_id":"1912.02424","n_code_links":13,"syntology":{"ran":4,"of":4,"n_ran_checked":4,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["sfzhang15/ATSS"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/whats-hidden-in-a-randomly-weighted-neural","slug":"whats-hidden-in-a-randomly-weighted-neural","title":"What's Hidden in a Randomly Weighted Neural Network?","date":"2019-11-29","arxiv_id":"1911.13299","n_code_links":4,"syntology":{"ran":9,"of":9,"n_ran_checked":5,"n_instrument":4,"unverified":0,"pointer_only":5,"phrase":"9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","official":{"repos":["allenai/hidden-networks"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"paper":"/paper/cspnet-a-new-backbone-that-can-enhance","slug":"cspnet-a-new-backbone-that-can-enhance","title":"CSPNet: A New Backbone that can Enhance Learning Capability of CNN","date":"2019-11-27","arxiv_id":"1911.11929","n_code_links":123,"syntology":null},{"paper":"/paper/ghostnet-more-features-from-cheap-operations","slug":"ghostnet-more-features-from-cheap-operations","title":"GhostNet: More Features from Cheap Operations","date":"2019-11-27","arxiv_id":"1911.11907","n_code_links":33,"syntology":{"ran":19,"of":23,"n_ran_checked":16,"n_instrument":3,"unverified":4,"pointer_only":5,"phrase":"19 ran (of which 0 constructed an object rather than computing a result; 16 with no instrument failure: 3 honoured, 0 violated, 13 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified","official":{"repos":["huawei-noah/ghostnet"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/learning-spatial-fusion-for-single-shot","slug":"learning-spatial-fusion-for-single-shot","title":"Learning Spatial Fusion for Single-Shot Object Detection","date":"2019-11-21","arxiv_id":"1911.09516","n_code_links":1,"syntology":null},{"paper":"/paper/efficientdet-scalable-and-efficient-object","slug":"efficientdet-scalable-and-efficient-object","title":"EfficientDet: Scalable and Efficient Object Detection","date":"2019-11-20","arxiv_id":"1911.09070","n_code_links":64,"syntology":{"ran":55,"of":70,"n_ran_checked":48,"n_instrument":7,"unverified":15,"pointer_only":7,"phrase":"55 ran (of which 1 constructed an object rather than computing a result; 48 with no instrument failure: 4 honoured, 0 violated, 44 with no contract checked; 7 where Syntology's instrument failed) · 15 unverified","official":{"repos":["google/automl"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"paper":"/paper/centermask-real-time-anchor-free-instance-1","slug":"centermask-real-time-anchor-free-instance-1","title":"CenterMask : Real-Time Anchor-Free Instance Segmentation","date":"2019-11-15","arxiv_id":"1911.06667","n_code_links":8,"syntology":{"ran":2,"of":2,"n_ran_checked":2,"n_instrument":0,"unverified":0,"pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 1 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["youngwanLEE/CenterMask"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"paper":"/paper/momentum-contrast-for-unsupervised-visual","slug":"momentum-contrast-for-unsupervised-visual","title":"Momentum Contrast for Unsupervised Visual Representation Learning","date":"2019-11-13","arxiv_id":"1911.05722","n_code_links":44,"syntology":{"ran":28,"of":42,"n_ran_checked":23,"n_instrument":5,"unverified":14,"pointer_only":19,"phrase":"28 ran (of which 16 constructed an object rather than computing a result; 23 with no instrument failure: 0 honoured, 0 violated, 23 with no contract checked; 5 where Syntology's instrument failed) · 14 unverified","official":{"repos":["facebookresearch/moco","ppwwyyxx/moco.tensorflow"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"an-exponential-learning-rate-schedule-for-1","title":"An Exponential Learning Rate Schedule for Deep Learning","date":"2019-10-16","arxiv_id":"1910.07454","n_code_links":0,"syntology":null},{"paper":"/paper/decaying-momentum-helps-neural-network","slug":"decaying-momentum-helps-neural-network","title":"Demon: Improved Neural Network Training with Momentum Decay","date":"2019-10-11","arxiv_id":"1910.04952","n_code_links":2,"syntology":null},{"paper":"/paper/eca-net-efficient-channel-attention-for-deep","slug":"eca-net-efficient-channel-attention-for-deep","title":"ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks","date":"2019-10-08","arxiv_id":"1910.03151","n_code_links":13,"syntology":{"ran":3,"of":8,"n_ran_checked":2,"n_instrument":1,"unverified":5,"pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified","official":{"repos":["BangguWu/ECANet"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":5,"ran_from_kinds":["official"]}}},{"paper":"/paper/deformable-kernels-adapting-effective","slug":"deformable-kernels-adapting-effective","title":"Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation","date":"2019-10-07","arxiv_id":"1910.02940","n_code_links":2,"syntology":null},{"paper":"/paper/randaugment-practical-data-augmentation-with","slug":"randaugment-practical-data-augmentation-with","title":"RandAugment: Practical automated data augmentation with a reduced search space","date":"2019-09-30","arxiv_id":"1909.13719","n_code_links":19,"syntology":{"ran":58,"of":65,"n_ran_checked":7,"n_instrument":51,"unverified":7,"pointer_only":17,"phrase":"58 ran (of which 1 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 1 violated, 6 with no contract checked; 51 where Syntology's instrument failed) · 7 unverified","official":null}},{"paper":"/paper/diffgrad-an-optimization-method-for","slug":"diffgrad-an-optimization-method-for","title":"diffGrad: An Optimization Method for Convolutional Neural Networks","date":"2019-09-12","arxiv_id":"1909.11015","n_code_links":1,"syntology":null},{"paper":"/paper/lip-local-importance-based-pooling","slug":"lip-local-importance-based-pooling","title":"LIP: Local Importance-based Pooling","date":"2019-08-12","arxiv_id":"1908.04156","n_code_links":1,"syntology":null},{"paper":"/paper/attentive-normalization","slug":"attentive-normalization","title":"Attentive Normalization","date":"2019-08-04","arxiv_id":"1908.01259","n_code_links":2,"syntology":null},{"paper":"/paper/moga-searching-beyond-mobilenetv3","slug":"moga-searching-beyond-mobilenetv3","title":"MoGA: Searching Beyond MobileNetV3","date":"2019-08-04","arxiv_id":"1908.01314","n_code_links":2,"syntology":null},{"paper":"/paper/compact-global-descriptor-for-neural-networks","slug":"compact-global-descriptor-for-neural-networks","title":"Compact Global Descriptor for Neural Networks","date":"2019-07-23","arxiv_id":"1907.09665","n_code_links":1,"syntology":null},{"paper":"/paper/densely-connected-search-space-for-more","slug":"densely-connected-search-space-for-more","title":"Densely Connected Search Space for More Flexible Neural Architecture Search","date":"2019-06-23","arxiv_id":"1906.09607","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":0,"n_instrument":2,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["JaminFong/DenseNAS"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/contrastive-multiview-coding","slug":"contrastive-multiview-coding","title":"Contrastive Multiview Coding","date":"2019-06-13","arxiv_id":"1906.05849","n_code_links":8,"syntology":{"ran":4,"of":4,"n_ran_checked":1,"n_instrument":3,"unverified":0,"pointer_only":2,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","official":{"repos":["HobbitLong/CMC"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"paper":"/paper/stochastic-gradient-methods-with-layer-wise","slug":"stochastic-gradient-methods-with-layer-wise","title":"Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks","date":"2019-05-27","arxiv_id":"1905.11286","n_code_links":3,"syntology":{"ran":13,"of":16,"n_ran_checked":12,"n_instrument":1,"unverified":3,"pointer_only":1,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified","official":null}},{"paper":"/paper/spatial-group-wise-enhance-improving-semantic","slug":"spatial-group-wise-enhance-improving-semantic","title":"Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks","date":"2019-05-23","arxiv_id":"1905.09646","n_code_links":3,"syntology":{"ran":2,"of":4,"n_ran_checked":0,"n_instrument":2,"unverified":2,"pointer_only":4,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","official":{"repos":["implus/PytorchInsight"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/cutmix-regularization-strategy-to-train","slug":"cutmix-regularization-strategy-to-train","title":"CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features","date":"2019-05-13","arxiv_id":"1905.04899","n_code_links":30,"syntology":{"ran":17,"of":24,"n_ran_checked":10,"n_instrument":7,"unverified":7,"pointer_only":5,"phrase":"17 ran (of which 6 constructed an object rather than computing a result; 10 with no instrument failure: 2 honoured, 0 violated, 8 with no contract checked; 7 where Syntology's instrument failed) · 7 unverified","official":{"repos":["clovaai/CutMix-PyTorch"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"on-the-linear-speedup-analysis-of","title":"On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization","date":"2019-05-09","arxiv_id":"1905.03817","n_code_links":0,"syntology":null},{"paper":"/paper/billion-scale-semi-supervised-learning-for","slug":"billion-scale-semi-supervised-learning-for","title":"Billion-scale semi-supervised learning for image classification","date":"2019-05-02","arxiv_id":"1905.00546","n_code_links":4,"syntology":null},{"paper":"/paper/gcnet-non-local-networks-meet-squeeze","slug":"gcnet-non-local-networks-meet-squeeze","title":"GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond","date":"2019-04-25","arxiv_id":"1904.11492","n_code_links":9,"syntology":null},{"paper":"/paper/190409925","slug":"190409925","title":"Attention Augmented Convolutional Networks","date":"2019-04-22","arxiv_id":"1904.09925","n_code_links":14,"syntology":{"ran":3,"of":6,"n_ran_checked":1,"n_instrument":2,"unverified":3,"pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","official":null}},{"paper":"/paper/190409460","slug":"190409460","title":"Data-Driven Neuron Allocation for Scale Aggregation Networks","date":"2019-04-20","arxiv_id":"1904.09460","n_code_links":1,"syntology":null},{"paper":"/paper/foveabox-beyond-anchor-based-object-detector","slug":"foveabox-beyond-anchor-based-object-detector","title":"FoveaBox: Beyond Anchor-based Object Detector","date":"2019-04-08","arxiv_id":"1904.03797","n_code_links":7,"syntology":null},{"paper":"/paper/adaptive-nms-refining-pedestrian-detection-in","slug":"adaptive-nms-refining-pedestrian-detection-in","title":"Adaptive NMS: Refining Pedestrian Detection in a Crowd","date":"2019-04-07","arxiv_id":"1904.03629","n_code_links":0,"syntology":null},{"paper":"/paper/exploring-randomly-wired-neural-networks-for","slug":"exploring-randomly-wired-neural-networks-for","title":"Exploring Randomly Wired Neural Networks for Image Recognition","date":"2019-04-02","arxiv_id":"1904.01569","n_code_links":9,"syntology":{"ran":10,"of":31,"n_ran_checked":10,"n_instrument":0,"unverified":21,"pointer_only":1,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 21 unverified","official":null}},{"paper":"/paper/fcos-fully-convolutional-one-stage-object","slug":"fcos-fully-convolutional-one-stage-object","title":"FCOS: Fully Convolutional One-Stage Object Detection","date":"2019-04-02","arxiv_id":"1904.01355","n_code_links":87,"syntology":{"ran":37,"of":40,"n_ran_checked":30,"n_instrument":7,"unverified":3,"pointer_only":19,"phrase":"37 ran (of which 0 constructed an object rather than computing a result; 30 with no instrument failure: 2 honoured, 2 violated, 26 with no contract checked; 7 where Syntology's instrument failed) · 3 unverified","official":{"repos":["tianzhi0549/FCOS"],"state":"official: no sample here; runs from other or unrecorded repositories","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed","unlocated"]}}},{"paper":"/paper/res2net-a-new-multi-scale-backbone","slug":"res2net-a-new-multi-scale-backbone","title":"Res2Net: A New Multi-scale Backbone Architecture","date":"2019-04-02","arxiv_id":"1904.01169","n_code_links":34,"syntology":{"ran":5,"of":9,"n_ran_checked":2,"n_instrument":3,"unverified":4,"pointer_only":9,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified","official":null}},{"paper":"/paper/feature-intertwiner-for-object-detection-1","slug":"feature-intertwiner-for-object-detection-1","title":"Feature Intertwiner for Object Detection","date":"2019-03-28","arxiv_id":"1903.11851","n_code_links":2,"syntology":null},{"paper":"/paper/thundernet-towards-real-time-generic-object","slug":"thundernet-towards-real-time-generic-object","title":"ThunderNet: Towards Real-time Generic Object Detection","date":"2019-03-28","arxiv_id":"1903.11752","n_code_links":3,"syntology":null},{"paper":"/paper/detnas-neural-architecture-search-on-object","slug":"detnas-neural-architecture-search-on-object","title":"DetNAS: Backbone Search for Object Detection","date":"2019-03-26","arxiv_id":"1903.10979","n_code_links":2,"syntology":{"ran":7,"of":8,"n_ran_checked":5,"n_instrument":2,"unverified":1,"pointer_only":3,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified","official":{"repos":["megvii-model/DetNAS"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/srm-a-style-based-recalibration-module-for","slug":"srm-a-style-based-recalibration-module-for","title":"SRM : A Style-based Recalibration Module for Convolutional Neural Networks","date":"2019-03-26","arxiv_id":"1903.10829","n_code_links":1,"syntology":{"ran":3,"of":3,"n_ran_checked":0,"n_instrument":3,"unverified":0,"pointer_only":2,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","official":null}},{"paper":"/paper/feature-selective-anchor-free-module-for","slug":"feature-selective-anchor-free-module-for","title":"Feature Selective Anchor-Free Module for Single-Shot Object Detection","date":"2019-03-02","arxiv_id":"1903.00621","n_code_links":4,"syntology":{"ran":2,"of":8,"n_ran_checked":2,"n_instrument":0,"unverified":6,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 6 unverified","official":null}},{"paper":"/paper/retinamask-learning-to-predict-masks-improves","slug":"retinamask-learning-to-predict-masks-improves","title":"RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free","date":"2019-01-10","arxiv_id":"1901.03353","n_code_links":53,"syntology":null},{"paper":"/paper/elastic-improving-cnns-with-instance-specific","slug":"elastic-improving-cnns-with-instance-specific","title":"ELASTIC: Improving CNNs with Dynamic Scaling Policies","date":"2018-12-13","arxiv_id":"1812.05262","n_code_links":1,"syntology":{"ran":3,"of":10,"n_ran_checked":2,"n_instrument":1,"unverified":7,"pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified","official":{"repos":["allenai/elastic"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":7,"ran_from_kinds":["official"]}}},{"paper":"/paper/fbnet-hardware-aware-efficient-convnet-design","slug":"fbnet-hardware-aware-efficient-convnet-design","title":"FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search","date":"2018-12-09","arxiv_id":"1812.03443","n_code_links":5,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["facebookresearch/mobile-vision"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/pelee-a-real-time-object-detection-system-on-1","slug":"pelee-a-real-time-object-detection-system-on-1","title":"Pelee: A Real-Time Object Detection System on Mobile Devices","date":"2018-12-01","arxiv_id":null,"n_code_links":2,"syntology":null},{"paper":"/paper/grid-r-cnn","slug":"grid-r-cnn","title":"Grid R-CNN","date":"2018-11-29","arxiv_id":"1811.12030","n_code_links":2,"syntology":{"ran":3,"of":3,"n_ran_checked":3,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":null}},{"paper":"/paper/deformable-convnets-v2-more-deformable-better","slug":"deformable-convnets-v2-more-deformable-better","title":"Deformable ConvNets v2: More Deformable, Better Results","date":"2018-11-27","arxiv_id":"1811.11168","n_code_links":26,"syntology":{"ran":10,"of":13,"n_ran_checked":9,"n_instrument":1,"unverified":3,"pointer_only":4,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified","official":null}},{"paper":"/paper/m2det-a-single-shot-object-detector-based-on","slug":"m2det-a-single-shot-object-detector-based-on","title":"M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network","date":"2018-11-12","arxiv_id":"1811.04533","n_code_links":11,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":null}},{"paper":"/paper/biologically-plausible-learning-algorithms","slug":"biologically-plausible-learning-algorithms","title":"Biologically-plausible learning algorithms can scale to large datasets","date":"2018-11-08","arxiv_id":"1811.03567","n_code_links":2,"syntology":null},{"paper":"/paper/kalman-gradient-descent-adaptive-variance","slug":"kalman-gradient-descent-adaptive-variance","title":"Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization","date":"2018-10-29","arxiv_id":"1810.12273","n_code_links":1,"syntology":null},{"paper":"/paper/psanet-point-wise-spatial-attention-network","slug":"psanet-point-wise-spatial-attention-network","title":"PSANet: Point-wise Spatial Attention Network for Scene Parsing","date":"2018-09-01","arxiv_id":null,"n_code_links":4,"syntology":null},{"paper":null,"slug":"on-the-convergence-of-weighted-adagrad-with","title":"A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration","date":"2018-08-10","arxiv_id":"1808.03408","n_code_links":0,"syntology":null},{"paper":"/paper/acquisition-of-localization-confidence-for","slug":"acquisition-of-localization-confidence-for","title":"Acquisition of Localization Confidence for Accurate Object Detection","date":"2018-07-30","arxiv_id":"1807.11590","n_code_links":4,"syntology":{"ran":17,"of":18,"n_ran_checked":13,"n_instrument":4,"unverified":1,"pointer_only":2,"phrase":"17 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified","official":{"repos":["vacancy/PreciseRoIPooling"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/cbam-convolutional-block-attention-module","slug":"cbam-convolutional-block-attention-module","title":"CBAM: Convolutional Block Attention Module","date":"2018-07-17","arxiv_id":"1807.06521","n_code_links":31,"syntology":{"ran":13,"of":22,"n_ran_checked":10,"n_instrument":3,"unverified":9,"pointer_only":3,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 0 violated, 9 with no contract checked; 3 where Syntology's instrument failed) · 9 unverified","official":null}},{"paper":"/paper/representation-learning-with-contrastive","slug":"representation-learning-with-contrastive","title":"Representation Learning with Contrastive Predictive Coding","date":"2018-07-10","arxiv_id":"1807.03748","n_code_links":28,"syntology":{"ran":35,"of":45,"n_ran_checked":30,"n_instrument":5,"unverified":10,"pointer_only":22,"phrase":"35 ran (of which 21 constructed an object rather than computing a result; 30 with no instrument failure: 1 honoured, 0 violated, 29 with no contract checked; 5 where Syntology's instrument failed) · 10 unverified","official":null}},{"paper":null,"slug":"interpolatron-interpolation-or-extrapolation","title":"Interpolatron: Interpolation or Extrapolation Schemes to Accelerate Optimization for Deep Neural Networks","date":"2018-05-17","arxiv_id":"1805.06753","n_code_links":0,"syntology":null},{"paper":"/paper/exploring-the-limits-of-weakly-supervised","slug":"exploring-the-limits-of-weakly-supervised","title":"Exploring the Limits of Weakly Supervised Pretraining","date":"2018-05-02","arxiv_id":"1805.00932","n_code_links":4,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":144,"with_a_code_link":121,"where_syntology_ran_a_sample":72,"not_listed_spam_title":0,"listed":144,"listed_where_code_ran":72,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":61,"every_run_a_failure_of_syntologys_instrument":11,"listed_with_a_run_with_no_instrument_failure":61,"listed_every_run_a_failure_of_syntologys_instrument":11,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":72,"papers_where_code_ran_first_page":"/method/sgd-with-momentum/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/sgd-with-momentum/papers/2","json":"/method/sgd-with-momentum/papers/2.json","rows":44},{"order":"ran","page":1,"of":1,"url":"/method/sgd-with-momentum/papers/ran/1","json":"/method/sgd-with-momentum/papers/ran/1.json","rows":72}],"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":63},{"task":"/task/object-detection","name":"Object Detection","papers":51},{"task":"/task/object-detection-1","name":"object-detection","papers":28},{"task":"/task/image-classification","name":"image-classification","papers":26},{"task":"/task/classification","name":"General Classification","papers":23},{"task":"/task/object","name":"Object","papers":23},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":23},{"task":"/task/instance-segmentation","name":"Instance Segmentation","papers":14},{"task":"/task/real-time-object-detection","name":"Real-Time Object Detection","papers":10},{"task":"/task/architecture-search","name":"Neural Architecture Search","papers":9},{"task":"/task/segmentation","name":"Segmentation","papers":9},{"task":"/task/classification-1","name":"Classification","papers":7},{"task":"/task/representation-learning","name":"Representation Learning","papers":7},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":6},{"task":"/task/stochastic-optimization","name":"Stochastic Optimization","papers":6},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":5},{"task":"/task/deep-learning","name":"Deep Learning","papers":5},{"task":"/task/domain-generalization","name":"Domain Generalization","papers":5},{"task":null,"name":"GPU","papers":5},{"task":"/task/object-recognition","name":"Object Recognition","papers":5}],"tasks_shown":20,"n_tasks":137,"usage_by_year":[{"year":"2012","papers":1},{"year":"2013","papers":2},{"year":"2014","papers":3},{"year":"2015","papers":10},{"year":"2016","papers":10},{"year":"2017","papers":12},{"year":"2018","papers":21},{"year":"2019","papers":42},{"year":"2020","papers":19},{"year":"2021","papers":10},{"year":"2022","papers":3},{"year":"2023","papers":5},{"year":"2024","papers":3},{"year":"2025","papers":3}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/sgd-with-momentum"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=SGD%20with%20Momentum","read_at":"2026-10-05T01:01:53+00:00","query":"SGD with Momentum","kind":"method","have":"unknown_name","tier":null,"verified_implementation":null,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":null,"resolution":null,"resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":null,"archive_paper_arxiv_id":null,"repositories":null,"retrieve_with":null,"name_competition":{"verdict":"none","may_be_wrong_sense":false,"bearers_found":0,"served":null,"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing in this corpus is named 'SGD with Momentum' -- no method node, no paper announcing it in its title, no expanded form declaring it as an acronym."},"name_competition_text":null,"near_matches":["Cayley SGD with momentum","Frequency Stochastic Gradient Descent with Momentum (FSGDM)","∞-SGD","SGD-KV","INO-SGD","FBi-SGD","badr-sgd","DP-SGD-RC"],"code_nearby":[],"code_nearby_caption":null,"origin_paper":null,"line":"name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"b4071696c842d71d2b90ad6b2a4b0257ae515bc02c90446af51be1d8bf3bb863","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}