{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/gradient-clipping","slug":"gradient-clipping","name":"Gradient Clipping","full_name":"Gradient Clipping","full_name_withheld":false,"description_markdown":"One difficulty that arises with optimization of deep neural networks is that large parameter gradients can lead an [SGD](https://paperswithcode.com/method/sgd) optimizer to update the parameters strongly into a region where the loss function is much greater, effectively undoing much of the work that was needed to get to the current solution.\r\n\r\n**Gradient Clipping** clips the size of the gradients to ensure optimization performs more reasonably near sharp areas of the loss surface. It can be performed in a number of ways. One option is to simply clip the parameter gradient element-wise before a parameter update. Another option is to clip the norm ||$\\textbf{g}$|| of the gradient $\\textbf{g}$ before a parameter update:\r\n\r\n$$\\text{ if } ||\\textbf{g}||  > v \\text{ then } \\textbf{g} \\leftarrow \\frac{\\textbf{g}{v}}{||\\textbf{g}||}$$\r\n\r\nwhere $v$ is a norm threshold.\r\n\r\nSource: Deep Learning, Goodfellow et al\r\n\r\nImage Source: [Pascanu et al](https://arxiv.org/pdf/1211.5063.pdf)","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Optimization","url":"/methods/category/optimization","pwc_aliases":[]}],"n_papers_tagged":167,"archive_num_papers":167,"papers_newest_first":[{"paper":"/paper/2506-08347","slug":"2506-08347","title":"Differentially Private Relational Learning with Entity-level Privacy Guarantees","date":"2025-06-10","arxiv_id":"2506.08347","n_code_links":1,"syntology":null},{"paper":null,"slug":"geoclip-geometry-aware-clipping-for","title":"GeoClip: Geometry-Aware Clipping for Differentially Private SGD","date":"2025-06-06","arxiv_id":"2506.06549","n_code_links":0,"syntology":null},{"paper":null,"slug":"gcfl-a-gradient-correction-based-federated","title":"GCFL: A Gradient Correction-based Federated Learning Framework for Privacy-preserving CPSS","date":"2025-06-04","arxiv_id":"2506.03618","n_code_links":0,"syntology":null},{"paper":"/paper/sample-and-computationally-efficient-1","slug":"sample-and-computationally-efficient-1","title":"Sample and Computationally Efficient Continuous-Time Reinforcement Learning with General Function Approximation","date":"2025-05-20","arxiv_id":"2505.14821","n_code_links":1,"syntology":null},{"paper":"/paper/a-training-framework-for-optimal-and-stable","slug":"a-training-framework-for-optimal-and-stable","title":"A Training Framework for Optimal and Stable Training of Polynomial Neural Networks","date":"2025-05-16","arxiv_id":"2505.11589","n_code_links":1,"syntology":null},{"paper":null,"slug":"dyn-d-2-p-dynamic-differentially-private","title":"Dyn-D$^2$P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee","date":"2025-05-10","arxiv_id":"2505.06651","n_code_links":0,"syntology":null},{"paper":null,"slug":"can-local-representation-alignment-rnns-solve","title":"Can Local Representation Alignment RNNs Solve Temporal Tasks?","date":"2025-04-18","arxiv_id":"2504.13531","n_code_links":0,"syntology":null},{"paper":null,"slug":"technical-report-full-version-of-analyzing","title":"Technical Report: Full Version of Analyzing and Optimizing Perturbation of DP-SGD Geometrically","date":"2025-04-08","arxiv_id":"2504.05618","n_code_links":0,"syntology":null},{"paper":"/paper/zclip-adaptive-spike-mitigation-for-llm-pre","slug":"zclip-adaptive-spike-mitigation-for-llm-pre","title":"ZClip: Adaptive Spike Mitigation for LLM Pre-Training","date":"2025-04-03","arxiv_id":"2504.02507","n_code_links":1,"syntology":{"ran":0,"of":1,"n_ran_checked":0,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"0 ran · 1 unverified","official":{"repos":["bluorion-com/ZClip"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"paper":"/paper/world-model-agents-with-change-based","slug":"world-model-agents-with-change-based","title":"World Model Agents with Change-Based Intrinsic Motivation","date":"2025-03-26","arxiv_id":"2503.21047","n_code_links":1,"syntology":null},{"paper":null,"slug":"tractable-representations-for-convergent","title":"Tractable Representations for Convergent Approximation of Distributional HJB Equations","date":"2025-03-07","arxiv_id":"2503.05563","n_code_links":0,"syntology":null},{"paper":null,"slug":"adagc-improving-training-stability-for-large","title":"AdaGC: Improving Training Stability for Large Language Model Pretraining","date":"2025-02-16","arxiv_id":"2502.11034","n_code_links":0,"syntology":null},{"paper":null,"slug":"local-differential-privacy-is-not-enough-a","title":"Local Differential Privacy is Not Enough: A Sample Reconstruction Attack against Federated Learning with Local Differential Privacy","date":"2025-02-12","arxiv_id":"2502.08151","n_code_links":0,"syntology":null},{"paper":"/paper/mitigating-unintended-memorization-with-lora","slug":"mitigating-unintended-memorization-with-lora","title":"Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs","date":"2025-02-07","arxiv_id":"2502.05087","n_code_links":1,"syntology":null},{"paper":null,"slug":"bmg-q-localized-bipartite-match-graph","title":"BMG-Q: Localized Bipartite Match Graph Attention Q-Learning for Ride-Pooling Order Dispatch","date":"2025-01-23","arxiv_id":"2501.13448","n_code_links":0,"syntology":null},{"paper":null,"slug":"integrating-llms-with-its-recent-advances","title":"Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions","date":"2025-01-08","arxiv_id":"2501.04437","n_code_links":0,"syntology":null},{"paper":null,"slug":"on-the-convergence-of-dp-sgd-with-adaptive","title":"On the Convergence of DP-SGD with Adaptive Clipping","date":"2024-12-27","arxiv_id":"2412.19916","n_code_links":0,"syntology":null},{"paper":"/paper/optimized-gradient-clipping-for-noisy-label","slug":"optimized-gradient-clipping-for-noisy-label","title":"Optimized Gradient Clipping for Noisy Label Learning","date":"2024-12-12","arxiv_id":"2412.08941","n_code_links":1,"syntology":null},{"paper":null,"slug":"a-combined-encoder-and-transformer-approach","title":"A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generation","date":"2024-11-19","arxiv_id":"2411.12157","n_code_links":0,"syntology":null},{"paper":null,"slug":"gradient-normalization-with-out-clipping","title":"Gradient Normalization Provably Benefits Nonconvex SGD under Heavy-Tailed Noise","date":"2024-10-21","arxiv_id":"2410.16561","n_code_links":0,"syntology":null},{"paper":null,"slug":"from-gradient-clipping-to-normalization-for","title":"From Gradient Clipping to Normalization for Heavy Tailed SGD","date":"2024-10-17","arxiv_id":"2410.13849","n_code_links":0,"syntology":null},{"paper":null,"slug":"independently-normalized-sgd-for-generalized","title":"Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization","date":"2024-10-17","arxiv_id":"2410.14054","n_code_links":0,"syntology":null},{"paper":null,"slug":"private-and-communication-efficient-federated","title":"Private and Communication-Efficient Federated Learning based on Differentially Private Sketches","date":"2024-10-08","arxiv_id":"2410.05733","n_code_links":0,"syntology":null},{"paper":null,"slug":"methods-for-convex-l-0-l-1-smooth","title":"Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity","date":"2024-09-23","arxiv_id":"2409.14989","n_code_links":0,"syntology":null},{"paper":null,"slug":"robust-federated-learning-over-the-air","title":"Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping","date":"2024-09-23","arxiv_id":"2409.15100","n_code_links":0,"syntology":null},{"paper":null,"slug":"convergence-of-distributed-adaptive","title":"Convergence of Distributed Adaptive Optimization with Local Updates","date":"2024-09-20","arxiv_id":"2409.13155","n_code_links":0,"syntology":null},{"paper":"/paper/denomamba-a-fused-state-space-model-for-low","slug":"denomamba-a-fused-state-space-model-for-low","title":"DenoMamba: A fused state-space model for low-dose CT denoising","date":"2024-09-19","arxiv_id":"2409.13094","n_code_links":1,"syntology":null},{"paper":"/paper/causal-temporal-representation-learning-with","slug":"causal-temporal-representation-learning-with","title":"Causal Temporal Representation Learning with Nonstationary Sparse Transition","date":"2024-09-05","arxiv_id":"2409.03142","n_code_links":1,"syntology":{"ran":7,"of":9,"n_ran_checked":3,"n_instrument":4,"unverified":2,"pointer_only":9,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified","official":{"repos":["xiangchensong/ctrlns"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/machine-learning-models-for-daily-rainfall","slug":"machine-learning-models-for-daily-rainfall","title":"Machine learning models for daily rainfall forecasting in Northern Tropical Africa using tropical wave predictors","date":"2024-08-29","arxiv_id":"2408.16349","n_code_links":1,"syntology":null},{"paper":null,"slug":"revisit-micro-batch-clipping-adaptive-data","title":"Revisit Micro-batch Clipping: Adaptive Data Pruning via Gradient Manipulation","date":"2024-08-29","arxiv_id":"2408.16204","n_code_links":0,"syntology":null},{"paper":null,"slug":"doppler-differentially-private-optimizers","title":"DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction","date":"2024-08-24","arxiv_id":"2408.13460","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-adaptive-gradient-regularization-method","title":"Adaptive Gradient Regularization: A Faster and Generalizable Optimization Technique for Deep Neural Networks","date":"2024-07-24","arxiv_id":"2407.16944","n_code_links":0,"syntology":null},{"paper":null,"slug":"privacy-of-the-last-iterate-in-cyclically","title":"Privacy of the last iterate in cyclically-sampled DP-SGD on nonconvex composite losses","date":"2024-07-07","arxiv_id":"2407.05237","n_code_links":0,"syntology":null},{"paper":"/paper/towards-efficient-and-scalable-training-of","slug":"towards-efficient-and-scalable-training-of","title":"Towards Efficient and Scalable Training of Differentially Private Deep Learning","date":"2024-06-25","arxiv_id":"2406.17298","n_code_links":1,"syntology":null},{"paper":"/paper/gradient-clipping-improves-adagrad-when-the","slug":"gradient-clipping-improves-adagrad-when-the","title":"Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed","date":"2024-06-06","arxiv_id":"2406.04443","n_code_links":1,"syntology":null},{"paper":null,"slug":"polyak-meets-parameter-free-clipped-gradient","title":"Parameter-free Clipped Gradient Descent Meets Polyak","date":"2024-05-23","arxiv_id":"2405.15010","n_code_links":0,"syntology":null},{"paper":"/paper/dreamer-xl-towards-high-resolution-text-to-3d","slug":"dreamer-xl-towards-high-resolution-text-to-3d","title":"Dreamer XL: Towards High-Resolution Text-to-3D Generation via Trajectory Score Matching","date":"2024-05-18","arxiv_id":"2405.11252","n_code_links":1,"syntology":null},{"paper":null,"slug":"ctrl-continuous-time-representation-learning","title":"CTRL: Continuous-Time Representation Learning on Temporal Heterogeneous Information Network","date":"2024-05-11","arxiv_id":"2405.08013","n_code_links":0,"syntology":null},{"paper":null,"slug":"hyp-oc-hyperbolic-one-class-classification","title":"Hyp-OC: Hyperbolic One Class Classification for Face Anti-Spoofing","date":"2024-04-22","arxiv_id":"2404.14406","n_code_links":0,"syntology":null},{"paper":null,"slug":"privacy-at-a-price-exploring-its-dual-impact","title":"Privacy at a Price: Exploring its Dual Impact on AI Fairness","date":"2024-04-15","arxiv_id":"2404.09391","n_code_links":0,"syntology":null},{"paper":"/paper/regularized-gradient-clipping-provably-trains","slug":"regularized-gradient-clipping-provably-trains","title":"Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks","date":"2024-04-12","arxiv_id":"2404.08624","n_code_links":1,"syntology":null},{"paper":null,"slug":"poisoning-decentralized-collaborative","title":"Poisoning Decentralized Collaborative Recommender System and Its Countermeasures","date":"2024-04-01","arxiv_id":"2404.01177","n_code_links":0,"syntology":null},{"paper":"/paper/an-embarrassingly-simple-defense-against","slug":"an-embarrassingly-simple-defense-against","title":"Towards Adversarial Robustness And Backdoor Mitigation in SSL","date":"2024-03-23","arxiv_id":"2403.15918","n_code_links":1,"syntology":null},{"paper":null,"slug":"a-stochastic-quasi-newton-method-for-non","title":"A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothness","date":"2024-03-22","arxiv_id":"2403.15244","n_code_links":0,"syntology":null},{"paper":"/paper/two-trades-is-not-baffled-condense-graph-via","slug":"two-trades-is-not-baffled-condense-graph-via","title":"Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching","date":"2024-02-07","arxiv_id":"2402.04924","n_code_links":1,"syntology":{"ran":3,"of":3,"n_ran_checked":2,"n_instrument":1,"unverified":0,"pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["nus-hpc-ai-lab/ctrl"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"breaking-mlperf-training-a-case-study-on","title":"Breaking MLPerf Training: A Case Study on Optimizing BERT","date":"2024-02-04","arxiv_id":"2402.02447","n_code_links":0,"syntology":null},{"paper":null,"slug":"auto-dp-sgd-dual-improvements-of-privacy-and","title":"DP-SGD-Global-Adapt-V2-S: Triad Improvements of Privacy, Accuracy and Fairness via Step Decay Noise Multiplier and Step Decay Upper Clipping Threshold","date":"2023-12-05","arxiv_id":"2312.02400","n_code_links":0,"syntology":null},{"paper":"/paper/differentially-private-sgd-without-clipping","slug":"differentially-private-sgd-without-clipping","title":"Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach","date":"2023-11-24","arxiv_id":"2311.14632","n_code_links":1,"syntology":{"ran":2,"of":5,"n_ran_checked":2,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","official":{"repos":["564612540/DiceSGD"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"byzantine-robustness-and-partial","title":"Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences","date":"2023-11-23","arxiv_id":"2311.14127","n_code_links":0,"syntology":null},{"paper":null,"slug":"on-the-accuracy-and-efficiency-of-group-wise","title":"On the accuracy and efficiency of group-wise clipping in differentially private optimization","date":"2023-10-30","arxiv_id":"2310.19215","n_code_links":0,"syntology":null},{"paper":"/paper/enhancing-high-resolution-3d-generation","slug":"enhancing-high-resolution-3d-generation","title":"Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping","date":"2023-10-19","arxiv_id":"2310.12474","n_code_links":1,"syntology":{"ran":3,"of":9,"n_ran_checked":3,"n_instrument":0,"unverified":6,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 6 unverified","official":{"repos":["fudan-zvg/pgc-3d"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":6,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"unintended-memorization-in-large-asr-models","title":"Unintended Memorization in Large ASR Models, and How to Mitigate It","date":"2023-10-18","arxiv_id":"2310.11739","n_code_links":0,"syntology":null},{"paper":null,"slug":"user-inference-attacks-on-large-language","title":"User Inference Attacks on Large Language Models","date":"2023-10-13","arxiv_id":"2310.09266","n_code_links":0,"syntology":null},{"paper":null,"slug":"high-probability-convergence-for-composite","title":"High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise","date":"2023-10-03","arxiv_id":"2310.01860","n_code_links":0,"syntology":null},{"paper":"/paper/cleanba-a-reproducible-and-efficient","slug":"cleanba-a-reproducible-and-efficient","title":"Cleanba: A Reproducible and Efficient Distributed Reinforcement Learning Platform","date":"2023-09-29","arxiv_id":"2310.00036","n_code_links":1,"syntology":{"ran":4,"of":8,"n_ran_checked":4,"n_instrument":0,"unverified":4,"pointer_only":8,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified","official":{"repos":["vwxyzjn/cleanba"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":"/paper/automatic-clipping-differentially-private","slug":"automatic-clipping-differentially-private","title":"Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger","date":"2023-09-21","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/differentially-private-image-classification-1","slug":"differentially-private-image-classification-1","title":"Differentially Private Image Classification by Learning Priors from Random Processes","date":"2023-09-21","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/federated-learning-with-client-subsampling","slug":"federated-learning-with-client-subsampling","title":"Federated Learning with Client Subsampling, Data Heterogeneity, and Unbounded Smoothness: A New Algorithm and Lower Bounds","date":"2023-09-21","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"high-probability-analysis-for-non-convex","title":"High Probability Analysis for Non-Convex Stochastic Optimization with Clipping","date":"2023-07-25","arxiv_id":"2307.13680","n_code_links":0,"syntology":null},{"paper":"/paper/enhancing-continuous-time-series-modelling","slug":"enhancing-continuous-time-series-modelling","title":"Enhancing Continuous Time Series Modelling with a Latent ODE-LSTM Approach","date":"2023-07-11","arxiv_id":"2307.05126","n_code_links":1,"syntology":null},{"paper":null,"slug":"differentially-private-video-activity","title":"Differentially Private Video Activity Recognition","date":"2023-06-27","arxiv_id":"2306.15742","n_code_links":0,"syntology":null},{"paper":null,"slug":"provably-robust-temporal-difference-learning","title":"Provably Robust Temporal Difference Learning for Heavy-Tailed Rewards","date":"2023-06-20","arxiv_id":"2306.11455","n_code_links":0,"syntology":null},{"paper":null,"slug":"adaptive-strategies-in-non-convex","title":"Adaptive Strategies in Non-convex Optimization","date":"2023-06-17","arxiv_id":"2306.10278","n_code_links":0,"syntology":null},{"paper":"/paper/differentially-private-sharpness-aware","slug":"differentially-private-sharpness-aware","title":"Differentially Private Sharpness-Aware Training","date":"2023-06-09","arxiv_id":"2306.05651","n_code_links":1,"syntology":{"ran":9,"of":11,"n_ran_checked":9,"n_instrument":0,"unverified":2,"pointer_only":1,"phrase":"9 ran (of which 1 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","official":{"repos":["jinseongp/dpsat"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["found_in_text","official"]}}},{"paper":null,"slug":"ctrl-connect-tabular-and-language-model-for","title":"CTRL: Connect Collaborative and Language Model for CTR Prediction","date":"2023-06-05","arxiv_id":"2306.02841","n_code_links":0,"syntology":null},{"paper":null,"slug":"convex-and-non-convex-optimization-under","title":"Convex and Non-convex Optimization Under Generalized Smoothness","date":"2023-06-02","arxiv_id":"2306.01264","n_code_links":0,"syntology":null},{"paper":null,"slug":"clip21-error-feedback-for-gradient-clipping","title":"Clip21: Error Feedback for Gradient Clipping","date":"2023-05-30","arxiv_id":"2305.18929","n_code_links":0,"syntology":null},{"paper":"/paper/exploring-the-promise-and-limits-of-real-time","slug":"exploring-the-promise-and-limits-of-real-time","title":"Exploring the Promise and Limits of Real-Time Recurrent Learning","date":"2023-05-30","arxiv_id":"2305.19044","n_code_links":1,"syntology":null},{"paper":null,"slug":"domo-ac-doubly-multi-step-off-policy-actor","title":"DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm","date":"2023-05-29","arxiv_id":"2305.18501","n_code_links":0,"syntology":null},{"paper":null,"slug":"dpformer-learning-differentially-private","title":"DPFormer: Learning Differentially Private Transformer on Long-Tailed Data","date":"2023-05-28","arxiv_id":"2305.17633","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-framework-for-provably-stable-and","title":"A Framework for Provably Stable and Consistent Training of Deep Feedforward Networks","date":"2023-05-20","arxiv_id":"2305.12125","n_code_links":0,"syntology":null},{"paper":"/paper/sharing-lifelong-reinforcement-learning","slug":"sharing-lifelong-reinforcement-learning","title":"Sharing Lifelong Reinforcement Learning Knowledge via Modulating Masks","date":"2023-05-18","arxiv_id":"2305.10997","n_code_links":2,"syntology":null},{"paper":null,"slug":"convergence-and-privacy-of-decentralized","title":"Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compression","date":"2023-05-17","arxiv_id":"2305.09896","n_code_links":0,"syntology":null},{"paper":null,"slug":"privacy-loss-of-noisy-stochastic-gradient","title":"Privacy Loss of Noisy Stochastic Gradient Descent Might Converge Even for Non-Convex Losses","date":"2023-05-17","arxiv_id":"2305.09903","n_code_links":0,"syntology":null},{"paper":"/paper/an-intelligent-sdwn-routing-algorithm-based","slug":"an-intelligent-sdwn-routing-algorithm-based","title":"An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning","date":"2023-05-12","arxiv_id":"2305.10441","n_code_links":1,"syntology":null},{"paper":"/paper/dpmlbench-holistic-evaluation-of","slug":"dpmlbench-holistic-evaluation-of","title":"DPMLBench: Holistic Evaluation of Differentially Private Machine Learning","date":"2023-05-10","arxiv_id":"2305.05900","n_code_links":1,"syntology":null},{"paper":null,"slug":"adam-family-methods-for-nonsmooth","title":"Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees","date":"2023-05-06","arxiv_id":"2305.03938","n_code_links":0,"syntology":null},{"paper":"/paper/noise-is-not-the-main-factor-behind-the-gap","slug":"noise-is-not-the-main-factor-behind-the-gap","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","date":"2023-04-27","arxiv_id":"2304.13960","n_code_links":1,"syntology":null},{"paper":"/paper/swectrl-mini-a-data-transparent-transformer","slug":"swectrl-mini-a-data-transparent-transformer","title":"SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in Swedish","date":"2023-04-27","arxiv_id":"2304.13994","n_code_links":1,"syntology":null},{"paper":"/paper/stable-and-low-precision-training-for-large","slug":"stable-and-low-precision-training-for-large","title":"Stable and low-precision training for large-scale vision-language models","date":"2023-04-25","arxiv_id":"2304.13013","n_code_links":1,"syntology":{"ran":0,"of":3,"n_ran_checked":0,"n_instrument":0,"unverified":3,"pointer_only":0,"phrase":"0 ran · 3 unverified","official":{"repos":["mlfoundations/open_clip"],"state":"official: harvested for another paper","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":[]}}},{"paper":"/paper/once-detected-never-lost-surpassing-human","slug":"once-detected-never-lost-surpassing-human","title":"Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection","date":"2023-04-24","arxiv_id":"2304.12315","n_code_links":2,"syntology":{"ran":3,"of":5,"n_ran_checked":0,"n_instrument":3,"unverified":2,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","official":{"repos":["tusen-ai/sst"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"dpaf-image-synthesis-via-differentially","title":"DPAF: Image Synthesis via Differentially Private Aggregation in Forward Phase","date":"2023-04-20","arxiv_id":"2304.12185","n_code_links":0,"syntology":null},{"paper":null,"slug":"manipulating-federated-recommender-systems","title":"Manipulating Federated Recommender Systems: Poisoning with Synthetic Users and Its Countermeasures","date":"2023-04-06","arxiv_id":"2304.03054","n_code_links":0,"syntology":null},{"paper":"/paper/weaktr-exploring-plain-vision-transformer-for","slug":"weaktr-exploring-plain-vision-transformer-for","title":"WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentation","date":"2023-04-03","arxiv_id":"2304.01184","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["hustvl/weaktr"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/elastic-weight-removal-for-faithful-and","slug":"elastic-weight-removal-for-faithful-and","title":"Elastic Weight Removal for Faithful and Abstractive Dialogue Generation","date":"2023-03-30","arxiv_id":"2303.17574","n_code_links":1,"syntology":null},{"paper":null,"slug":"mixing-backward-with-forward-chaining-for","title":"Mixing Backward- with Forward-Chaining for Metacognitive Skill Acquisition and Transfer","date":"2023-03-18","arxiv_id":"2303.12223","n_code_links":0,"syntology":null},{"paper":"/paper/episode-episodic-gradient-clipping-with","slug":"episode-episodic-gradient-clipping-with","title":"EPISODE: Episodic Gradient Clipping with Periodic Resampled Corrections for Federated Learning with Heterogeneous Data","date":"2023-02-14","arxiv_id":"2302.07155","n_code_links":1,"syntology":null},{"paper":null,"slug":"u-clip-on-average-unbiased-stochastic","title":"U-Clip: On-Average Unbiased Stochastic Gradient Clipping","date":"2023-02-06","arxiv_id":"2302.02971","n_code_links":0,"syntology":null},{"paper":"/paper/lifelong-reinforcement-learning-with","slug":"lifelong-reinforcement-learning-with","title":"Lifelong Reinforcement Learning with Modulating Masks","date":"2022-12-21","arxiv_id":"2212.11110","n_code_links":4,"syntology":null},{"paper":null,"slug":"adap-dp-fl-differentially-private-federated","title":"Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise","date":"2022-11-29","arxiv_id":"2211.15893","n_code_links":0,"syntology":null},{"paper":null,"slug":"alternative-formulations-for-gilthead","title":"Alternative formulations for gilthead seabream diets: towards a more sustainable production","date":"2022-11-03","arxiv_id":"2211.02430","n_code_links":0,"syntology":null},{"paper":"/paper/controllable-factuality-in-document-grounded","slug":"controllable-factuality-in-document-grounded","title":"Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model","date":"2022-10-31","arxiv_id":"2210.17418","n_code_links":1,"syntology":null},{"paper":"/paper/unsupervised-learning-of-structured","slug":"unsupervised-learning-of-structured","title":"Unsupervised Learning of Structured Representations via Closed-Loop Transcription","date":"2022-10-30","arxiv_id":"2210.16782","n_code_links":1,"syntology":null},{"paper":"/paper/gaussian-bernoulli-rbms-without-tears","slug":"gaussian-bernoulli-rbms-without-tears","title":"Gaussian-Bernoulli RBMs Without Tears","date":"2022-10-19","arxiv_id":"2210.10318","n_code_links":1,"syntology":{"ran":2,"of":3,"n_ran_checked":2,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["lrjconan/grbm"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/dpis-an-enhanced-mechanism-for-differentially","slug":"dpis-an-enhanced-mechanism-for-differentially","title":"DPIS: An Enhanced Mechanism for Differentially Private SGD with Importance Sampling","date":"2022-10-18","arxiv_id":"2210.09634","n_code_links":1,"syntology":null},{"paper":null,"slug":"a-closer-look-at-the-calibration-of","title":"A Closer Look at the Calibration of Differentially Private Learners","date":"2022-10-15","arxiv_id":"2210.08248","n_code_links":0,"syntology":null},{"paper":"/paper/demystifying-self-supervised-trojan-attacks","slug":"demystifying-self-supervised-trojan-attacks","title":"An Embarrassingly Simple Backdoor Attack on Self-supervised Learning","date":"2022-10-13","arxiv_id":"2210.07346","n_code_links":4,"syntology":{"ran":2,"of":2,"n_ran_checked":0,"n_instrument":2,"unverified":0,"pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["meet-cjli/ctrl","CCCjiang/CTRL"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"differentially-private-deep-learning-with-2","title":"Differentially Private Deep Learning with ModelMix","date":"2022-10-07","arxiv_id":"2210.03843","n_code_links":0,"syntology":null},{"paper":"/paper/continual-learning-fast-and-slow","slug":"continual-learning-fast-and-slow","title":"Continual Learning, Fast and Slow","date":"2022-09-06","arxiv_id":"2209.02370","n_code_links":1,"syntology":null},{"paper":"/paper/ctrl-clustering-training-losses-for-label","slug":"ctrl-clustering-training-losses-for-label","title":"CTRL: Clustering Training Losses for Label Error Detection","date":"2022-08-17","arxiv_id":"2208.08464","n_code_links":1,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":167,"with_a_code_link":78,"where_syntology_ran_a_sample":28,"not_listed_spam_title":0,"listed":167,"listed_where_code_ran":28,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":22,"every_run_a_failure_of_syntologys_instrument":6,"listed_with_a_run_with_no_instrument_failure":22,"listed_every_run_a_failure_of_syntologys_instrument":6,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":28,"papers_where_code_ran_first_page":"/method/gradient-clipping/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/gradient-clipping/papers/2","json":"/method/gradient-clipping/papers/2.json","rows":67},{"order":"ran","page":1,"of":1,"url":"/method/gradient-clipping/papers/ran/1","json":"/method/gradient-clipping/papers/ran/1.json","rows":28}],"tasks":[{"task":"/task/federated-learning","name":"Federated Learning","papers":12},{"task":"/task/language-modelling","name":"Language Modelling","papers":10},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":10},{"task":"/task/image-classification","name":"Image Classification","papers":9},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":9},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":9},{"task":"/task/deep-learning","name":"Deep Learning","papers":8},{"task":"/task/language-modeling","name":"Language Modeling","papers":7},{"task":null,"name":"GPU","papers":6},{"task":"/task/privacy-preserving","name":"Privacy Preserving","papers":6},{"task":"/task/representation-learning","name":"Representation Learning","papers":6},{"task":"/task/image-classification","name":"image-classification","papers":6},{"task":"/task/stochastic-optimization","name":"Stochastic Optimization","papers":5},{"task":"/task/text-generation","name":"Text Generation","papers":5},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":3},{"task":"/task/denoising","name":"Denoising","papers":3},{"task":"/task/fairness","name":"Fairness","papers":3},{"task":"/task/classification","name":"General Classification","papers":3},{"task":"/task/lemma","name":"LEMMA","papers":3},{"task":"/task/large-language-model","name":"Large Language Model","papers":3}],"tasks_shown":20,"n_tasks":150,"usage_by_year":[{"year":"2015","papers":1},{"year":"2016","papers":3},{"year":"2017","papers":3},{"year":"2018","papers":2},{"year":"2019","papers":10},{"year":"2020","papers":13},{"year":"2021","papers":21},{"year":"2022","papers":26},{"year":"2023","papers":42},{"year":"2024","papers":30},{"year":"2025","papers":16}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/gradient-clipping"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Gradient%20Clipping","read_at":"2026-10-05T00:57:12+00:00","query":"Gradient Clipping","kind":"method","have":"unknown_name","tier":null,"verified_implementation":null,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":null,"resolution":null,"resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":null,"archive_paper_arxiv_id":null,"repositories":null,"retrieve_with":null,"name_competition":{"verdict":"none","may_be_wrong_sense":false,"bearers_found":0,"served":null,"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing in this corpus is named 'Gradient Clipping' -- no method node, no paper announcing it in its title, no expanded form declaring it as an acronym."},"name_competition_text":null,"near_matches":["gradient norm clipping strategy","Pixel-wise Gradient Clipping (PGC)","Component-Wise Gradient Norm Clipping","Adaptive Group-wise Gradient Clipping (AGGC)","stochastic methods for composite and distributed optimization based on the clipping of stochastic gradient differences","gradient EM","FAT-Clipping","Gradient Cuff"],"code_nearby":[],"code_nearby_caption":null,"origin_paper":null,"line":"name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"32e942743cf963c50183da604c20d6f50ee6e7a0f92c2e3b9734f16e71271b4f","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}