{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/r-vqa-learning-visual-relation-facts-with","title":"R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering","arxiv_id":"1805.09701","date":"2018-05-24","proceeding":null,"authors":["Pan Lu","Lei Ji","Wei zhang","Nan Duan","Ming Zhou","Jianyong Wang"],"abstract":"Recently, Visual Question Answering (VQA) has emerged as one of the most\nsignificant tasks in multimodal learning as it requires understanding both\nvisual and textual modalities. Existing methods mainly rely on extracting image\nand question features to learn their joint feature embedding via multimodal\nfusion or attention mechanism. Some recent studies utilize external\nVQA-independent models to detect candidate entities or attributes in images,\nwhich serve as semantic knowledge complementary to the VQA task. However, these\ncandidate entities or attributes might be unrelated to the VQA task and have\nlimited semantic capacities. To better utilize semantic knowledge in images, we\npropose a novel framework to learn visual relation facts for VQA. Specifically,\nwe build up a Relation-VQA (R-VQA) dataset based on the Visual Genome dataset\nvia a semantic similarity module, in which each data consists of an image, a\ncorresponding question, a correct answer and a supporting relation fact. A\nwell-defined relation detector is then adopted to predict visual\nquestion-related relation facts. We further propose a multi-step attention\nmodel composed of visual attention and semantic attention sequentially to\nextract related visual knowledge and semantic knowledge. We conduct\ncomprehensive experiments on the two benchmark datasets, demonstrating that our\nmodel achieves state-of-the-art performance and verifying the benefit of\nconsidering visual relation facts.","url_abs":"http://arxiv.org/abs/1805.09701v2","url_pdf":"http://arxiv.org/pdf/1805.09701v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"r-vqa-learning-visual-relation-facts-with","repo_url":"https://github.com/lupantech/rvqa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":null,"task_name":"Relation"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-coco-visual-1","task":"Visual Question Answering (VQA)","dataset":"COCO Visual Question Answering (VQA) real images 1.0 multiple choice","model":"RelAtt","rank_in_archive_order":3,"of":10,"metrics":{"Percentage correct":"69.60"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-coco-visual-4","task":"Visual Question Answering (VQA)","dataset":"COCO Visual Question Answering (VQA) real images 1.0 open ended","model":"RelAtt","rank_in_archive_order":4,"of":14,"metrics":{"Percentage correct":"65.69"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1805.09701","atlas_url":"https://app.syntology.ai/?focus=1805.09701","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1805.09701"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1805.09701\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1805.09701"},"call":"get_concepts_for_paper(arxiv_id=\"1805.09701\")"}],"by_repo":[{"repo":"lupantech/rvqa","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"aa3823cae8d3d3f7bba2454acd1ad747894a3874bf9d60b6db91402e19af06bc","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}