Papers › A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Data

A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Data

28 Oct 2024arXiv:2410.20659archive 2025-07-28

Saptarshi Chakraborty, Peter L. Bartlett

Federated Learning (FL) has emerged as a groundbreaking paradigm in collaborative machine learning, emphasizing decentralized model training to address data privacy concerns. While significant progress has been made in optimizing federated learning, the exploration of generalization error, particularly in heterogeneous settings, has been limited, focusing mainly on parametric cases. This paper investigates the generalization properties of deep federated regression within a two-stage sampling model. Our findings highlight that the intrinsic dimension, defined by the entropic dimension, is crucial for determining convergence rates when appropriate network sizes are used. Specifically, if the true relationship between response and explanatory variables is charecterized by a β-Hölder function and there are n independent and identically distributed (i.i.d.) samples from m participating clients, the error rate for participating clients scales at most as Õ((mn)^(-2β/(2β+ d̅₂ᵦ(λ)))), and for non-participating clients, it scales as Õ(Δ·m^(-2β/(2β+ d̅₂ᵦ(λ))) + (mn)^(-2β/(2β+ d̅₂ᵦ(λ)))). Here, d̅₂ᵦ(λ) represents the 2β-entropic dimension of λ, the marginal distribution of the explanatory variables, and Δ characterizes the dependence between the sampling stages. Our results explicitly account for the "closeness" of clients, demonstrating that the convergence rates of deep federated learners depend on intrinsic rather than nominal high-dimensionality.

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

saptarshic27/fl officialmentioned in paperpytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Federated Learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections