{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/memory-efficient-implementation-of-densenets","title":"Memory-Efficient Implementation of DenseNets","arxiv_id":"1707.06990","date":"2017-07-21","proceeding":null,"authors":["Geoff Pleiss","Danlu Chen","Gao Huang","Tongcheng Li","Laurens van der Maaten","Kilian Q. Weinberger"],"abstract":"The DenseNet architecture is highly computationally efficient as a result of\nfeature reuse. However, a naive DenseNet implementation can require a\nsignificant amount of GPU memory: If not properly managed, pre-activation batch\nnormalization and contiguous convolution operations can produce feature maps\nthat grow quadratically with network depth. In this technical report, we\nintroduce strategies to reduce the memory consumption of DenseNets during\ntraining. By strategically using shared memory allocations, we reduce the\nmemory cost for storing feature maps from quadratic to linear. Without the GPU\nmemory bottleneck, it is now possible to train extremely deep DenseNets.\nNetworks with 14M parameters can be trained on a single GPU, up from 4M. A\n264-layer DenseNet (73M parameters), which previously would have been\ninfeasible to train, can now be trained on a single workstation with 8 NVIDIA\nTesla M40 GPUs. On the ImageNet ILSVRC classification dataset, this large\nDenseNet obtains a state-of-the-art single-crop top-1 error of 20.26%.","url_abs":"http://arxiv.org/abs/1707.06990v1","url_pdf":"http://arxiv.org/pdf/1707.06990v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"memory-efficient-implementation-of-densenets","repo_url":"https://github.com/gpleiss/efficient_densenet_pytorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"memory-efficient-implementation-of-densenets","repo_url":"https://github.com/facebookresearch/ResNeXt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"memory-efficient-implementation-of-densenets","repo_url":"https://github.com/joeyearsley/efficient_densenet_tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"memory-efficient-implementation-of-densenets","repo_url":"https://github.com/npielawski/pytorch_tiramisu","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"memory-efficient-implementation-of-densenets","repo_url":"https://github.com/zhourunlong/densenet-ag_news-nlp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"memory-efficient-implementation-of-densenets","repo_url":"https://github.com/code-implementation1/Code2/tree/main/3D_DenseNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":null,"task_name":"GPU"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-block","method_name":"Dense Block"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1707.06990","atlas_url":"https://app.syntology.ai/?focus=1707.06990","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1707.06990"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1707.06990\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1707.06990"},"call":"get_concepts_for_paper(arxiv_id=\"1707.06990\")"}],"by_repo":[{"repo":"gpleiss/efficient_densenet_pytorch","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"facebookresearch/ResNeXt","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"joeyearsley/efficient_densenet_tensorflow","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"npielawski/pytorch_tiramisu","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["1609.05158"],"harvested_for_calls":[{"arxiv_id":"1609.05158","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"1609.05158"},"call":"get_harvested_code_for_paper(arxiv_id=\"1609.05158\")"}]},{"repo":"zhourunlong/densenet-ag_news-nlp","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"code-implementation1/Code2","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"c80d5e12be79ab3450010c6d4fe52ad0b9204297ac6ac537136debce0a4b2e6b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}