Papers › GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering

GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering

25 Feb 2019CVPR 2019 6arXiv:1902.09506archive 2025-07-28

Drew A. Hudson, Christopher D. Manning

We introduce GQA, a new dataset for real-world visual reasoning and compositional question answering, seeking to address key shortcomings of previous VQA datasets. We have developed a strong and robust question engine that leverages scene graph structures to create 22M diverse reasoning questions, all come with functional programs that represent their semantics. We use the programs to gain tight control over the answer distribution and present a new tunable smoothing technique to mitigate question biases. Accompanying the dataset is a suite of new metrics that evaluate essential qualities such as consistency, grounding and plausibility. An extensive analysis is performed for baselines as well as state-of-the-art models, providing fine-grained results for different question types and topologies. Whereas a blind LSTM obtains mere 42.1%, and strong VQA models achieve 54.1%, human performance tops at 89.3%, offering ample opportunity for new research to explore. We strongly hope GQA will provide an enabling resource for the next generation of models with enhanced robustness, improved consistency, and deeper semantic understanding for images and language.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

stanfordnlp/mac-network officialtfApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Learning by Abstraction: The Neural State Machine (arXiv:1907.03950; that paper's own run record: 13 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 2 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified); nothing checks that they implement this paper's method. report
adapter-hub/xgqa mentioned on GitHubSyntology: not harvested report
kakao/DAFT mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Neural Ordinary Differential Equations (arXiv:1806.07366; that paper's own run record: 98 ran (of which 44 constructed an object rather than computing a result; 64 with no instrument failure: 2 honoured, 5 violated, 57 with no contract checked; 34 where Syntology's instrument failed) · 26 unverified); nothing checks that they implement this paper's method. report
luomancs/alternative_answer_set mentioned on GitHubSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to VQA: Visual Question Answering (arXiv:1505.00468; that paper's own run record: 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
weixin-liang/metashift mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and... (arXiv:2202.06523; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified) and SwordBench: Evaluating Orthogonality of Steering Image Representations (arXiv:2605.16372; that paper's own run record: 14 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 13 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Question AnsweringVisual Question Answering (VQA)Visual Reasoning

Datasets

Introduced by this paper, per the archive.

GQA

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Visual Question Answering (VQA) GQA test-std MAC Accuracy 54.06 #6 of 7 Archive leaderboard report
Visual Question Answering (VQA) GQA test-std CNN+LSTM Accuracy 46.55 #7 of 7 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

LSTMSigmoid ActivationTanh Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections