{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/distributed-statistical-machine-learning-in","title":"Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent","arxiv_id":"1705.05491","date":"2017-05-16","proceeding":null,"authors":["Yudong Chen","Lili Su","Jiaming Xu"],"abstract":"We consider the problem of distributed statistical machine learning in\nadversarial settings, where some unknown and time-varying subset of working\nmachines may be compromised and behave arbitrarily to prevent an accurate model\nfrom being learned. This setting captures the potential adversarial attacks\nfaced by Federated Learning -- a modern machine learning paradigm that is\nproposed by Google researchers and has been intensively studied for ensuring\nuser privacy. Formally, we focus on a distributed system consisting of a\nparameter server and $m$ working machines. Each working machine keeps $N/m$\ndata samples, where $N$ is the total number of samples. The goal is to\ncollectively learn the underlying true model parameter of dimension $d$.\n  In classical batch gradient descent methods, the gradients reported to the\nserver by the working machines are aggregated via simple averaging, which is\nvulnerable to a single Byzantine failure. In this paper, we propose a Byzantine\ngradient descent method based on the geometric median of means of the\ngradients. We show that our method can tolerate $q \\le (m-1)/2$ Byzantine\nfailures, and the parameter estimate converges in $O(\\log N)$ rounds with an\nestimation error of $\\sqrt{d(2q+1)/N}$, hence approaching the optimal error\nrate $\\sqrt{d/N}$ in the centralized and failure-free setting. The total\ncomputational complexity of our algorithm is of $O((Nd/m) \\log N)$ at each\nworking machine and $O(md + kd \\log^3 N)$ at the central server, and the total\ncommunication cost is of $O(m d \\log N)$. We further provide an application of\nour general results to the linear regression problem.\n  A key challenge arises in the above problem is that Byzantine failures create\narbitrary and unspecified dependency among the iterations and the aggregated\ngradients. We prove that the aggregated gradient converges uniformly to the\ntrue gradient function.","url_abs":"http://arxiv.org/abs/1705.05491v2","url_pdf":"http://arxiv.org/pdf/1705.05491v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distributed-statistical-machine-learning-in","repo_url":"https://github.com/bladesteam/blades","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"distributed-statistical-machine-learning-in","repo_url":"https://github.com/lishenghui/blades","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"linear-regression","method_name":"Linear Regression"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1705.05491","atlas_url":"https://app.syntology.ai/?focus=1705.05491","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1705.05491"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1705.05491\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1705.05491"},"call":"get_concepts_for_paper(arxiv_id=\"1705.05491\")"}],"by_repo":[{"repo":"bladesteam/blades","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["1903.03936"],"harvested_for_calls":[{"arxiv_id":"1903.03936","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"1903.03936"},"call":"get_harvested_code_for_paper(arxiv_id=\"1903.03936\")"}]},{"repo":"lishenghui/blades","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"a8d577b639566e99d902346f8f4bb61c1c3fd5aec1ea97cfe08bca08f124eb37","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}