{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/backprop-q-generalized-backpropagation-for","title":"Backprop-Q: Generalized Backpropagation for Stochastic Computation Graphs","arxiv_id":"1807.09511","date":"2018-07-25","proceeding":null,"authors":["Xiaoran Xu","Songpeng Zu","Yuan Zhang","Hanning Zhou","Wei Feng"],"abstract":"In real-world scenarios, it is appealing to learn a model carrying out\nstochastic operations internally, known as stochastic computation graphs\n(SCGs), rather than learning a deterministic mapping. However, standard\nbackpropagation is not applicable to SCGs. We attempt to address this issue\nfrom the angle of cost propagation, with local surrogate costs, called\nQ-functions, constructed and learned for each stochastic node in an SCG. Then,\nthe SCG can be trained based on these surrogate costs using standard\nbackpropagation. We propose the entire framework as a solution to generalize\nbackpropagation for SCGs, which resembles an actor-critic architecture but\nbased on a graph. For broad applicability, we study a variety of SCG structures\nfrom one cost to multiple costs. We utilize recent advances in reinforcement\nlearning (RL) and variational Bayes (VB), such as off-policy critic learning\nand unbiased-and-low-variance gradient estimation, and review them in the\ncontext of SCGs. The generalized backpropagation extends transported learning\nsignals beyond gradients between stochastic nodes while preserving the benefit\nof backpropagating gradients through deterministic nodes. Experimental\nsuggestions and concerns are listed to help design and test any specific model\nusing this framework.","url_abs":"http://arxiv.org/abs/1807.09511v2","url_pdf":"http://arxiv.org/pdf/1807.09511v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"backprop-q-generalized-backpropagation-for","repo_url":"https://github.com/netpaladinx/experiments-for-Backprop-Q","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1807.09511","atlas_url":"https://app.syntology.ai/?focus=1807.09511","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1807.09511"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1807.09511\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1807.09511"},"call":"get_concepts_for_paper(arxiv_id=\"1807.09511\")"}],"by_repo":[{"repo":"netpaladinx/experiments-for-Backprop-Q","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"dae18ba7ae4a6697d2d748aaff2e526709608cd2f184fe37a9748a7a57205864","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}