{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/multimodal-residual-learning-for-visual-qa","title":"Multimodal Residual Learning for Visual QA","arxiv_id":"1606.01455","date":"2016-06-05","proceeding":"NeurIPS 2016 12","authors":["Jin-Hwa Kim","Sang-Woo Lee","Dong-Hyun Kwak","Min-Oh Heo","Jeonghee Kim","Jung-Woo Ha","Byoung-Tak Zhang"],"abstract":"Deep neural networks continue to advance the state-of-the-art of image\nrecognition tasks with various methods. However, applications of these methods\nto multimodality remain limited. We present Multimodal Residual Networks (MRN)\nfor the multimodal residual learning of visual question-answering, which\nextends the idea of the deep residual learning. Unlike the deep residual\nlearning, MRN effectively learns the joint representation from vision and\nlanguage information. The main idea is to use element-wise multiplication for\nthe joint residual mappings exploiting the residual learning of the attentional\nmodels in recent studies. Various alternative models introduced by\nmultimodality are explored based on our study. We achieve the state-of-the-art\nresults on the Visual QA dataset for both Open-Ended and Multiple-Choice tasks.\nMoreover, we introduce a novel method to visualize the attention effect of the\njoint representations for each learning block using back-propagation algorithm,\neven though the visual features are collapsed without spatial information.","url_abs":"http://arxiv.org/abs/1606.01455v2","url_pdf":"http://arxiv.org/pdf/1606.01455v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multimodal-residual-learning-for-visual-qa","repo_url":"https://github.com/jnhwkim/nips-mrn-vqa","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"multiple-choice","task_name":"Multiple-choice"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-coco-visual-1","task":"Visual Question Answering (VQA)","dataset":"COCO Visual Question Answering (VQA) real images 1.0 multiple choice","model":"MRN","rank_in_archive_order":6,"of":10,"metrics":{"Percentage correct":"66.3"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-coco-visual-4","task":"Visual Question Answering (VQA)","dataset":"COCO Visual Question Answering (VQA) real images 1.0 open ended","model":"MRN + global features","rank_in_archive_order":7,"of":14,"metrics":{"Percentage correct":"61.8"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1606.01455","atlas_url":"https://app.syntology.ai/?focus=1606.01455","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1606.01455"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1606.01455\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1606.01455"},"call":"get_concepts_for_paper(arxiv_id=\"1606.01455\")"}],"by_repo":[{"repo":"jnhwkim/nips-mrn-vqa","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"a31d99a2f019b352ee66ec0679f35445779d3722d207fe7da323593e6974021c","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}