{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/towards-vqa-models-that-can-read","title":"Towards VQA Models That Can Read","arxiv_id":"1904.08920","date":"2019-04-18","proceeding":"CVPR 2019 6","authors":["Amanpreet Singh","Vivek Natarajan","Meet Shah","Yu Jiang","Xinlei Chen","Dhruv Batra","Devi Parikh","Marcus Rohrbach"],"abstract":"Studies have shown that a dominant class of questions asked by visually impaired users on images of their surroundings involves reading text in the image. But today's VQA models can not read! Our paper takes a first step towards addressing this problem. First, we introduce a new \"TextVQA\" dataset to facilitate progress on this important problem. Existing datasets either have a small proportion of questions about text (e.g., the VQA dataset) or are too small (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408 images that require reasoning about text to answer. Second, we introduce a novel model architecture that reads text in the image, reasons about it in the context of the image and the question, and predicts an answer which might be a deduction based on the text and the image or composed of the strings found in the image. Consequently, we call our approach Look, Read, Reason & Answer (LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on our TextVQA dataset. We find that the gap between human performance and machine performance is significantly larger on TextVQA than on VQA 2.0, suggesting that TextVQA is well-suited to benchmark progress along directions complementary to VQA 2.0.","url_abs":"https://arxiv.org/abs/1904.08920v2","url_pdf":"https://arxiv.org/pdf/1904.08920v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-vqa-models-that-can-read","repo_url":"https://github.com/facebookresearch/pythia","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-vqa-models-that-can-read","repo_url":"https://github.com/ZephyrZhuQi/ssbaseline","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-vqa-models-that-can-read","repo_url":"https://github.com/allenai/pythia","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-vqa-models-that-can-read","repo_url":"https://github.com/facebookresearch/mmf","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-vqa-models-that-can-read","repo_url":"https://github.com/jackroos/pythia","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-vqa-models-that-can-read","repo_url":"https://github.com/ronghanghu/pythia","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-vqa-models-that-can-read","repo_url":"https://github.com/zwxalgorithm/pythia","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":null,"task_name":"TextVQA"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[{"slug":"textvqa","name":"TextVQA","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-vqa-v2-test-dev","task":"Visual Question Answering (VQA)","dataset":"VQA v2 test-dev","model":"Pythia v0.3 + LoRRA","rank_in_archive_order":34,"of":56,"metrics":{"Accuracy":"69.21"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-vizwiz-2018-1","task":"Visual Question Answering (VQA)","dataset":"VizWiz 2018","model":"Pythia v0.3","rank_in_archive_order":3,"of":10,"metrics":{"overall":"54.72"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1904.08920","atlas_url":"https://app.syntology.ai/?focus=1904.08920","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1904.08920"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1904.08920\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1904.08920"},"call":"get_concepts_for_paper(arxiv_id=\"1904.08920\")"}],"by_repo":[{"repo":"facebookresearch/pythia","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"ZephyrZhuQi/ssbaseline","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"allenai/pythia","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"facebookresearch/mmf","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"jackroos/pythia","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"ronghanghu/pythia","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"zwxalgorithm/pythia","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"75b1001ae7cccff42d4f45f0b0a6c957b4feac71dd8c7b4bafe70eddc062fc49","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}