{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/deep-learning-as-a-mixed-convex-combinatorial","title":"Deep Learning as a Mixed Convex-Combinatorial Optimization Problem","arxiv_id":"1710.11573","date":"2017-10-31","proceeding":"ICLR 2018 1","authors":["Abram L. Friesen","Pedro Domingos"],"abstract":"As neural networks grow deeper and wider, learning networks with\nhard-threshold activations is becoming increasingly important, both for network\nquantization, which can drastically reduce time and energy requirements, and\nfor creating large integrated systems of deep networks, which may have\nnon-differentiable components and must avoid vanishing and exploding gradients\nfor effective learning. However, since gradient descent is not applicable to\nhard-threshold functions, it is not clear how to learn networks of them in a\nprincipled way. We address this problem by observing that setting targets for\nhard-threshold hidden units in order to minimize loss is a discrete\noptimization problem, and can be solved as such. The discrete optimization goal\nis to find a set of targets such that each unit, including the output, has a\nlinearly separable problem to solve. Given these targets, the network\ndecomposes into individual perceptrons, which can then be learned with standard\nconvex approaches. Based on this, we develop a recursive mini-batch algorithm\nfor learning deep hard-threshold networks that includes the popular but poorly\njustified straight-through estimator as a special case. Empirically, we show\nthat our algorithm improves classification accuracy in a number of settings,\nincluding for AlexNet and ResNet-18 on ImageNet, when compared to the\nstraight-through estimator.","url_abs":"http://arxiv.org/abs/1710.11573v3","url_pdf":"http://arxiv.org/pdf/1710.11573v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-learning-as-a-mixed-convex-combinatorial","repo_url":"https://github.com/afriesen/ftprop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"combinatorial-optimization","task_name":"Combinatorial Optimization"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"afriesen/ftprop","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"0db05e1b18dc3dd8042ccbbca66e62c577dbf5475b60d417ee61ed873099bc72","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}