{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/cumf_sgd-fast-and-scalable-matrix","title":"CuMF_SGD: Fast and Scalable Matrix Factorization","arxiv_id":"1610.05838","date":"2016-10-19","proceeding":null,"authors":["Xiaolong Xie","Wei Tan","Liana L. Fong","Yun Liang"],"abstract":"Matrix factorization (MF) has been widely used in e.g., recommender systems,\ntopic modeling and word embedding. Stochastic gradient descent (SGD) is popular\nin solving MF problems because it can deal with large data sets and is easy to\ndo incremental learning. We observed that SGD for MF is memory bound.\nMeanwhile, single-node CPU systems with caching performs well only for small\ndata sets; distributed systems have higher aggregated memory bandwidth but\nsuffer from relatively slow network connection. This observation inspires us to\naccelerate MF by utilizing GPUs's high memory bandwidth and fast intra-node\nconnection. We present cuMF_SGD, a CUDA-based SGD solution for large-scale MF\nproblems. On a single CPU, we design two workload schedule schemes, i.e.,\nbatch-Hogwild! and wavefront-update that fully exploit the massive amount of\ncores. Especially, batch-Hogwild! as a vectorized version of Hogwild! overcomes\nthe issue of memory discontinuity. We also develop highly-optimized kernels for\nSGD update, leveraging cache, warp-shuffle instructions and half-precision\nfloats. We also design a partition scheme to utilize multiple GPUs while\naddressing the well-known convergence issue when parallelizing SGD. On three\ndata sets with only one Maxwell or Pascal GPU, cuMF_SGD runs 3.1X-28.2X as fast\ncompared with state-of-art CPU solutions on 1-64 CPU nodes. Evaluations also\nshow that cuMF_SGD scales well on multiple GPUs in large data sets.","url_abs":"http://arxiv.org/abs/1610.05838v3","url_pdf":"http://arxiv.org/pdf/1610.05838v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cumf_sgd-fast-and-scalable-matrix","repo_url":"https://github.com/cumf/cumf_sgd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"cumf_sgd-fast-and-scalable-matrix","repo_url":"https://github.com/MehdiChelh/CuMF_SGD","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"small-data","task_name":"Small Data Image Classification"}],"methods":[{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"cumf/cumf_sgd","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"MehdiChelh/CuMF_SGD","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"0ef55a80fe41555f2cd2c341d9da83dc3dcc33e4c81a4a150ea92cd591b13f3f","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}