{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/private-empirical-risk-minimization-beyond","title":"Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry","arxiv_id":"1411.5417","date":"2014-11-20","proceeding":null,"authors":["Kunal Talwar","Abhradeep Thakurta","Li Zhang"],"abstract":"Empirical Risk Minimization (ERM) is a standard technique in machine\nlearning, where a model is selected by minimizing a loss function over\nconstraint set. When the training dataset consists of private information, it\nis natural to use a differentially private ERM algorithm, and this problem has\nbeen the subject of a long line of work started with Chaudhuri and Monteleoni\n2008. A private ERM algorithm outputs an approximate minimizer of the loss\nfunction and its error can be measured as the difference from the optimal value\nof the loss function. When the constraint set is arbitrary, the required error\nbounds are fairly well understood \\cite{BassilyST14}. In this work, we show\nthat the geometric properties of the constraint set can be used to derive\nsignificantly better results. Specifically, we show that a differentially\nprivate version of Mirror Descent leads to error bounds of the form\n$\\tilde{O}(G_{\\mathcal{C}}/n)$ for a lipschitz loss function, improving on the\n$\\tilde{O}(\\sqrt{p}/n)$ bounds in Bassily, Smith and Thakurta 2014. Here $p$ is\nthe dimensionality of the problem, $n$ is the number of data points in the\ntraining set, and $G_{\\mathcal{C}}$ denotes the Gaussian width of the\nconstraint set that we optimize over. We show similar improvements for strongly\nconvex functions, and for smooth functions. In addition, we show that when the\nloss function is Lipschitz with respect to the $\\ell_1$ norm and $\\mathcal{C}$\nis $\\ell_1$-bounded, a differentially private version of the Frank-Wolfe\nalgorithm gives error bounds of the form $\\tilde{O}(n^{-2/3})$. This captures\nthe important and common case of sparse linear regression (LASSO), when the\ndata $x_i$ satisfies $|x_i|_{\\infty} \\leq 1$ and we optimize over the $\\ell_1$\nball. We show new lower bounds for this setting, that together with known\nbounds, imply that all our upper bounds are tight.","url_abs":"http://arxiv.org/abs/1411.5417v3","url_pdf":"http://arxiv.org/pdf/1411.5417v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"private-empirical-risk-minimization-beyond","repo_url":"https://github.com/sunblaze-ucb/dpml-benchmark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"linear-regression","method_name":"Linear Regression"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1411.5417","atlas_url":"https://app.syntology.ai/?focus=1411.5417","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1411.5417"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1411.5417\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1411.5417"},"call":"get_concepts_for_paper(arxiv_id=\"1411.5417\")"}],"by_repo":[{"repo":"sunblaze-ucb/dpml-benchmark","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["1607.00133"],"harvested_for_calls":[{"arxiv_id":"1607.00133","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"1607.00133"},"call":"get_harvested_code_for_paper(arxiv_id=\"1607.00133\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"5cc60464cc40957d5667232d77bacc1eedac6dea3a647b999bb0bd229d1ad7bd","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}