Browse State-of-the-Art › Cross-Modal Information Retrieval
Cross-Modal Information Retrieval
8 papers with code · 0 benchmarks · 1 dataset archive 2025-07-28
Cross-Modal Information Retrieval (CMIR) is the task of finding relevant items across different modalities. For example, given an image, find a text or vice versa. The main challenge in CMIR is known as the heterogeneity gap: since items from different modalities have different data types, the similarity between them cannot be measured directly. Therefore, the majority of CMIR methods published to date attempt to bridge this gap by learning a latent representation space, where the similarity between items from different modalities can be measured.
Source: Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
1 dataset whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
1 subtask in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 16 · with a code link: 8 · where Syntology ran a sample: 1 (1 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (1 of 16 tagged: 1 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 16 of 16, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from.
-
23 Jun 2016 2 repositories listedWe choose to implement the actual search process as a similarity search in a visual feature space, by learning to translate a textual query into a visual representation.
-
10 Oct 2022 1 repository listedVisual Semantic Embedding (VSE) aims to extract the semantics of images and their descriptions, and embed them into the same latent space for cross-modal information retrieval.
-
1 Jan 2021 1 repository listedTo the best of our knowledge, VisualSparta is the first transformer-based text-to-image retrieval model that can achieve real-time searching for large-scale datasets, with significant accuracy improvement compared to…
-
9 Nov 2020 1 repository listedVisual Semantic Embedding (VSE) is a dominant approach for vision-language retrieval, which aims at learning a deep embedding space such that visual data are embedded close to their semantic text labels or descriptions.
-
12 Aug 2020 1 repository listed Syntology official (archive's flag): 14 ran · 14 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 1 violated, 12 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 16 harvested samples) · 2 pointer-only (licence)In this work, we tackle the task of cross-modal retrieval through image-sentence matching based on word-region alignments, using supervision only at the global image-sentence level.
-
23 Jul 2020 1 repository listedMost existing algorithms for cross-modal Information Retrieval are based on a supervised train-test setup, where a model learns to align the mode of the query (e. g., text) to the mode of the documents (e. g., images)…
-
22 Nov 2019 1 repository listedModeling the relationship between chemical structure and molecular activity is a key goal in drug development.
-
9 Apr 2019 1 repository listedIn particular, we are interested in two application scenarios: i) cross-modal retrieval between panchromatic (PAN) and multi-spectral imagery, and ii) multi-label image retrieval between very high resolution (VHR)…
-
Multimodal Representation Alignment for Cross-modal Information Retrieval10 Jun 2025 0 repositories listedDifferent machine learning models can represent the same underlying concept in different ways.
-
jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images11 Dec 2024 0 repositories listedContrastive Language-Image Pretraining (CLIP) has been widely used for crossmodal information retrieval and multimodal understanding tasks.
-
Is Cross-modal Information Retrieval Possible without Training?20 Apr 2023 0 repositories listedEncoded representations from a pretrained deep learning model (e.g., BERT text embeddings, penultimate CNN layer activations of an image) convey a rich set of features beneficial for information retrieval.
-
Multi-modal Machine Learning in Engineering Design: A Review and Future Directions14 Feb 2023 0 repositories listedIn the rapidly advancing field of multi-modal machine learning (MMML), the convergence of multiple data modalities has the potential to reshape various applications.
-
VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval13 Feb 2023 0 repositories listedThe relations expressed in user queries are vital for cross-modal information retrieval.
-
2 Mar 2022 0 repositories listedThe volume of available data has grown dramatically in recent years in many applications.
-
Scene Graph Reasoning with Prior Visual Relationship for Visual Question Answering23 Dec 2018 0 repositories listedOne of the key issues of Visual Question Answering (VQA) is to reason with semantic clues in the visual content under the guidance of the question, how to model relational semantics still remains as a great challenge.
-
Modeling Text with Graph Convolutional Network for Cross-Modal Information Retrieval3 Feb 2018 0 repositories listedCross-modal information retrieval aims to find heterogeneous data of various modalities from a given query of one modality.
Syntology lines on 1 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections