Browse State-of-the-Art › Grounded language learning
Grounded language learning
23 papers with code · 0 benchmarks · 1 dataset archive 2025-07-28
Acquire the meaning of language in situated environments.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
1 dataset whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 56 · with a code link: 23 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 56 tagged: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 56 of 56, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
16 Jul 2021 6 repositories listed Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 3 pointer-only (licence)Most existing methods employ a transformer-based multimodal encoder to jointly model visual tokens (region-based image features) and word tokens.
-
18 Oct 2018 6 repositories listed Syntology official: harvested, nothing ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 12 unverified (of 14 harvested samples) · 1 pointer-only (licence)Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific reasons, but given the poor data efficiency of the current learning methods…
-
21 Mar 2024 2 repositories listedToday's most accurate language models are trained on orders of magnitude more language data than human language learners receive - but with no supervision from other sensory modalities that play a crucial role in human…
-
3 Sep 2020 2 repositories listedRecent work has shown that large text-based neural language models, trained with conventional supervised learning objectives, acquire a surprising propensity for few- and one-shot learning.
-
2 May 2020 2 repositories listedTo study this human-like language acquisition ability, we present VisCOLL, a visually grounded language learning task, which simulates the continual acquisition of compositional phrases from streaming visual scenes.
-
2 Apr 2019 2 repositories listedWe improve the informativeness of models for conditional text generation using techniques from computational pragmatics.
-
6 Jul 2022 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples)We provide a study of how induced model sparsity can help achieve compositional generalization and better sample efficiency in grounded language learning problems.
-
5 Jul 2022 1 repository listedLexical semantics and cognitive science point to affordances (i. e. the actions that objects support) as critical for understanding and representing nouns and verbs.
-
22 Feb 2022 1 repository listedAfter training on an augmented dataset with almost forty times more adverbs than the original problem, a non-modular baseline is not able to systematically generalize to a novel combination of a known verb and adverb.
-
21 Feb 2022 1 repository listedIn this paper we create visually grounded word embeddings by combining English text and images and compare them to popular text-based methods, to see if visual information allows our model to better capture cognitive…
-
20 Oct 2021 1 repository listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)We hope SILG enables the community to quickly identify new methodologies for language grounding that generalize to a diverse set of environments and their associated challenges.
-
16 Jun 2021 1 repository listedThis study addresses the question whether visually grounded speech recognition (VGS) models learn to capture sentence semantics without access to any prior linguistic knowledge.
-
13 Feb 2021 1 repository listedWe present a novel interactive learning protocol that enables training request-fulfilling agents by verbally describing their activities.
-
15 Apr 2020 1 repository listedTo facilitate the research on language-guided agents with domain adaption, we propose a novel zero-shot compositional policy learning task, where the environments are characterized as a composition of different…
-
4 Nov 2019 1 repository listedAlthough their encodeing method is not compositional like natural languages from a perspective of human beings, the emergent languages can be generalised to unseen inputs and, more importantly, are easier for models to…
-
9 Sep 2019 1 repository listedHumans learn language by interaction with their environment and listening to other humans.
-
Learning semantic sentence representations from visually grounded language without lexical knowledge27 Mar 2019 1 repository listedThe system achieves state-of-the-art results on several of these benchmarks, which shows that a system trained solely on multimodal data, without assuming any word representations, is able to capture sentence level…
-
26 Nov 2018 1 repository listedWe introduce a new inference task - Visual Entailment (VE) - which differs from traditional Textual Entailment (TE) tasks whereby a premise is defined by an image, rather than a natural language sentence as in TE tasks.
-
1 Oct 2018 1 repository listedPrevious work on grounded language learning did not fully capture the semantics underlying the correspondences between structured world state representations and texts, especially those between numerical values and…
-
20 Sep 2018 1 repository listedRecent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language.
-
21 May 2018 1 repository listedOur goal is to develop a model that can learn to follow new instructions given prior instruction-perception-action examples.
-
Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game26 Apr 2018 1 repository listedBuilding intelligent agents that can communicate with and learn from humans in natural language is of great value.
-
20 Jun 2017 1 repository listedTrained via a combination of reinforcement and unsupervised learning, and beginning with minimal prior knowledge, the agent learns to relate linguistic symbols to emergent perceptual representations of its physical…
-
Visually Grounded Language Learning: a review of language games, datasets, tasks, and models5 Dec 2023 0 repositories listedIn recent years, several machine learning models have been proposed.
-
Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment4 Dec 2023 0 repositories listedLanguage models have been supervised with both language-only objective and visual grounding in existing studies of visual-grounded language learning.
-
Improved Compositional Generalization by Generating Demonstrations for Meta-Learning22 May 2023 0 repositories listedMeta-learning and few-shot prompting are viable methods to induce certain types of compositional behaviour.
-
20 Jan 2023 0 repositories listedCurrent work on image-based story generation suffers from the fact that the existing image sequence collections do not have coherent plots behind them.
-
Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches15 Nov 2022 0 repositories listedPeople rely heavily on context to enrich meaning beyond what is literally said, enabling concise but effective communication.
-
Not Cheating on the Turing Test: Towards Grounded Language Learning in Artificial Intelligence29 Jun 2022 0 repositories listedRecent hype surrounding the increasing sophistication of language processing models has renewed optimism regarding machines achieving a human-like command of natural language.
-
SILG: The Multi-domain Symbolic Interactive Language Grounding Benchmark1 Dec 2021 0 repositories listedExisting work in language grounding typically study single environments.
-
Grounding Language Representation with Visual Object Information via Cross Modal Pretraining29 Sep 2021 0 repositories listed
-
Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning20 Jul 2021 0 repositories listed
-
Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction18 Apr 2021 0 repositories listed
-
28 Oct 2020 0 repositories listed
-
Practical Cross-modal Manifold Alignment for Grounded Language1 Sep 2020 0 repositories listed
-
Presentation and Analysis of a Multimodal Dataset for Grounded Language Learning29 Jul 2020 0 repositories listed
-
CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language Learning3 Jun 2020 0 repositories listed
-
Ecological Semantics: Programming Environments for Situated Language Understanding10 Mar 2020 0 repositories listed
-
The Emergence of Compositional Languages for Numeric Concepts Through Iterated Learning in Neural Agents11 Oct 2019 0 repositories listed
-
Large-scale representation learning from visually grounded untranscribed speech19 Sep 2019 0 repositories listed
-
?`Es un plátano? Exploring the Application of a Physically Grounded Language Acquisition System to Spanish1 Jun 2019 0 repositories listed
-
Grounded Word Sense Translation1 Jun 2019 0 repositories listed
-
A Gold Standard to Measure Relative Linguistic Complexity with a Grounded Language Learning Model1 Aug 2018 0 repositories listed
-
Affordances in Grounded Language Learning1 Jul 2018 0 repositories listed
-
Understanding Grounded Language Learning Agents1 Jan 2018 0 repositories listed
-
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent21 Nov 2017 0 repositories listed
-
Understanding Early Word Learning in Situated Artificial Agents26 Oct 2017 0 repositories listed
-
Guiding Interaction Behaviors for Multi-modal Grounded Language Learning1 Aug 2017 0 repositories listed
-
From phonemes to images: levels of representation in a recurrent neural model of visually-grounded language learning11 Oct 2016 0 repositories listed
-
Discriminative Unsupervised Alignment of Natural Language Instructions with Corresponding Video Segments1 May 2015 0 repositories listed
-
UnixMan Corpus: A Resource for Language Learning in the Unix Domain1 May 2014 0 repositories listed
-
Adapting Discriminative Reranking to Grounded Language Learning1 Aug 2013 0 repositories listed
-
Grounded Language Learning from Video Described with Sentences1 Aug 2013 0 repositories listed
-
Parsing entire discourses as very long strings: Capturing topic continuity in grounded language learning1 Jan 2013 0 repositories listed
-
Exploiting Social Information in Grounded Language Learning via Grammatical Reduction1 Jul 2012 0 repositories listed
-
Unsupervised PCFG Induction for Grounded Language Learning with Highly Ambiguous Supervision1 Jul 2012 0 repositories listed
Syntology lines on 4 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections