{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/the-zipml-framework-for-training-models-with","title":"The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning","arxiv_id":"1611.05402","date":"2016-11-16","proceeding":null,"authors":["Hantian Zhang","Jerry Li","Kaan Kara","Dan Alistarh","Ji Liu","Ce Zhang"],"abstract":"Recently there has been significant interest in training machine-learning\nmodels at low precision: by reducing precision, one can reduce computation and\ncommunication by one order of magnitude. We examine training at reduced\nprecision, both from a theoretical and practical perspective, and ask: is it\npossible to train models at end-to-end low precision with provable guarantees?\nCan this lead to consistent order-of-magnitude speedups? We present a framework\ncalled ZipML to answer these questions. For linear models, the answer is yes.\nWe develop a simple framework based on one simple but novel strategy called\ndouble sampling. Our framework is able to execute training at low precision\nwith no bias, guaranteeing convergence, whereas naive quantization would\nintroduce significant bias. We validate our framework across a range of\napplications, and show that it enables an FPGA prototype that is up to 6.5x\nfaster than an implementation using full 32-bit precision. We further develop a\nvariance-optimal stochastic quantization strategy and show that it can make a\nsignificant difference in a variety of settings. When applied to linear models\ntogether with double sampling, we save up to another 1.7x in data movement\ncompared with uniform quantization. When training deep networks with quantized\nmodels, we achieve higher accuracy than the state-of-the-art XNOR-Net. Finally,\nwe extend our framework through approximation to non-linear models, such as\nSVM. We show that, although using low-precision data induces bias, we can\nappropriately bound and control the bias. We find in practice 8-bit precision\nis often sufficient to converge to the correct solution. Interestingly,\nhowever, in practice we notice that our framework does not always outperform\nthe naive rounding approach. We discuss this negative result in detail.","url_abs":"http://arxiv.org/abs/1611.05402v3","url_pdf":"http://arxiv.org/pdf/1611.05402v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-zipml-framework-for-training-models-with","repo_url":"https://github.com/IST-DASLab/smart-quantizer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1611.05402","atlas_url":"https://app.syntology.ai/?focus=1611.05402","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1611.05402"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1611.05402\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1611.05402"},"call":"get_concepts_for_paper(arxiv_id=\"1611.05402\")"}],"by_repo":[{"repo":"IST-DASLab/smart-quantizer","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"2bd05367501894300b99eff84b7c05fc902781800ac276fcfd6c55532f310e50","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}