{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/tictac-accelerating-distributed-deep-learning","title":"TicTac: Accelerating Distributed Deep Learning with Communication Scheduling","arxiv_id":"1803.03288","date":"2018-03-08","proceeding":null,"authors":["Sayed Hadi Hashemi","Sangeetha Abdu Jyothi","Roy H. Campbell"],"abstract":"State-of-the-art deep learning systems rely on iterative distributed training\nto tackle the increasing complexity of models and input data. The iteration\ntime in these communication-heavy systems depends on the computation time,\ncommunication time and the extent of overlap of computation and communication.\n  In this work, we identify a shortcoming in systems with graph representation\nfor computation, such as TensorFlow and PyTorch, that result in high variance\nin iteration time --- random order of received parameters across workers. We\ndevelop a system, TicTac, to improve the iteration time by fixing this issue in\ndistributed deep learning with Parameter Servers while guaranteeing\nnear-optimal overlap of communication and computation. TicTac identifies and\nenforces an order of network transfers which improves the iteration time using\nprioritization. Our system is implemented over TensorFlow and requires no\nchanges to the model or developer inputs. TicTac improves the throughput by up\nto $37.7\\%$ in inference and $19.2\\%$ in training, while also reducing\nstraggler effect by up to $2.3\\times$. Our code is publicly available.","url_abs":"http://arxiv.org/abs/1803.03288v2","url_pdf":"http://arxiv.org/pdf/1803.03288v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tictac-accelerating-distributed-deep-learning","repo_url":"https://github.com/xldrx/tictac","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"scheduling","task_name":"Scheduling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1803.03288","atlas_url":"https://app.syntology.ai/?focus=1803.03288","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1803.03288"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1803.03288\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1803.03288"},"call":"get_concepts_for_paper(arxiv_id=\"1803.03288\")"}],"by_repo":[{"repo":"xldrx/tictac","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"9bda9a2499c5cd25a548e18dd37f190ef8f72d4838eb1ed59d44fe7e23fb4337","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}