Browse State-of-the-Art › Multimodal Recommendation
Multimodal Recommendation
33 papers with code · 5 benchmarks · 6 datasets archive 2025-07-28
The multimodal recommendation task involves developing systems that leverage and integrate multiple types of data—such as text, images, audio, and user interactions—to predict and suggest items that align with a user's preferences. Unlike traditional recommendation approaches that rely on a single data modality, multimodal recommendation harnesses the diverse information from various sources to create richer and more nuanced representations of both users and items. This integration enables the system to understand and capture complex relationships and attributes across different data types, thereby enhancing the accuracy and relevance of the recommendations. The primary goal is to provide personalized suggestions by effectively merging and processing heterogeneous data to better match users with items they are likely to engage with or find valuable.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| Amazon Baby (18 rows) | FREEDOM (CLIP) | Ducho meets Elliot: Large-scale Benchmarks for Multimodal Recommendation | code | — | Compare |
| Amazon Beauty (18 rows) | LATTICE (ALIGN) | Ducho meets Elliot: Large-scale Benchmarks for Multimodal Recommendation | code | — | Compare |
| Amazon Digital Music (18 rows) | LATTICE (AltCLIP) | — | — | — | Compare |
| Amazon Office Products (18 rows) | LATTICE (ResNet50+ Sentence Bert) | Ducho meets Elliot: Large-scale Benchmarks for Multimodal Recommendation | code | — | Compare |
| Amazon Toys & Games (18 rows) | FREEDOM (MMFashion + Sentence Bert) | Ducho meets Elliot: Large-scale Benchmarks for Multimodal Recommendation | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
6 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 59 · with a code link: 33 · where Syntology ran a sample: 5 (3 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (5 of 59 tagged: 3 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 59 of 59, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
9 Feb 2023 2 repositories listedRecommendation systems have become popular and effective tools to help users discover their interesting items by modeling the user preference and item property based on implicit interactions (e. g., purchasing and…
-
13 Nov 2022 2 repositories listedBased on this finding, we propose a simple yet effective model, dubbed as FREEDOM, that FREEzes the item-item graph and DenOises the user-item interaction graph simultaneously for Multimodal recommendation.
-
24 Apr 2025 1 repository listedMultimodal click-through rate (CTR) prediction is a key technique in industrial recommender systems.
-
COHESION: Composite Graph Convolutional Network with Dual-Stage Fusion for Multimodal Recommendation6 Apr 2025 1 repository listedTwo key processes in multimodal recommendations are modality fusion and representation learning.
-
12 Feb 2025 1 repository listedHowever, when graph-structured side information (e. g., multimodal similarity graphs or social networks) is integrated into the U-I bipartite graph, existing graph collaborative filtering methods fall short of achieving…
-
21 Jan 2025 1 repository listedHowever, two significant gaps persist: 1) the difficulty in accurately generating missing data due to the limited ability to capture modality distributions; and 2) the critical but overlooked visibility bias, where…
-
13 Jan 2025 1 repository listedIn this paper, we develop a novel meta-learning-based multimodal fusion framework called Meta Multimodal Fusion (MetaMMF), which dynamically assigns parameters to the multimodal fusion function for each micro-video…
-
19 Dec 2024 1 repository listedMoreover, the variation of noise that is unique within each modality results in noise alleviation and fusion being more challenging.
-
18 Dec 2024 1 repository listedOur results indicate that the optimal K for certain modalities on specific datasets can be as low as 1 or 2, which may restrict the GNNs' capacity to capture global information.
-
16 Dec 2024 1 repository listedGiven the large volume of side information from different modalities, multimodal recommender systems have become increasingly vital, as they exploit richer semantic information beyond user-item interactions.
-
16 Dec 2024 1 repository listedIn order to combine the two distinct types of information, some additional challenges are encountered: 1) Modality erasure: Vanilla graph convolution, which proves rather useful in collaborative filtering, however…
-
26 Sep 2024 1 repository listedWe show that SiBraR's recommendations are accurate in missing modality scenarios, and that the model is able to map different modalities to the same region of the shared embedding space, hence reducing the modality gap.
-
25 Sep 2024 1 repository listedOur fMRLRec captures item features at different granularities, learning informative representations for efficient recommendation across multiple dimensions.
-
24 Sep 2024 1 repository listedIn specific domains like fashion, music, and movie recommendation, the multi-faceted features characterizing products and services may influence each customer on online selling platforms differently, paving the way to…
-
21 Aug 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)Generally, items with missing modalities are dropped in multimodal recommendation.
-
19 Aug 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)Recent advances in Large Language Models (LLMs) have demonstrated significant potential in the field of Recommendation Systems (RSs).
-
26 Jul 2024 1 repository listedTo address these issues, we propose a Counterfactual Knowledge Distillation method that could solve the imbalance problem and make the best use of all modalities.
-
17 Jul 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 6 pointer-only (licence)By using the user modality enhancement strategy to maximize mutual information between these two features, we improve the generalization ability of user modality representations.
-
9 Apr 2024 1 repository listedIn this paper, we propose an industrial multimodal recommendation framework named EM3: End-to-end training of Multimodal Model and ranking Model, which sufficiently utilizes multimodal information and allows…
-
19 Mar 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Each alignment is characterized by a specific objective function and is integrated into our multimodal recommendation framework.
-
7 Mar 2024 1 repository listedIn this work, we introduce Ducho 2.
-
29 Feb 2024 1 repository listedIt utilizes multimodal information to alleviate the data sparsity problem in recommendation systems, thus improving recommendation accuracy.
-
25 Feb 2024 1 repository listedWhile the incorporation of multimodal information could enhance the interpretability of these systems, current multimodal models represent users and items utilizing entangled numerical vectors, rendering them arduous to…
-
17 Feb 2024 1 repository listedThis strategy can implicitly enhance the model's robustness during the optimization process, mitigating instability risks arising from multimodal information inputs.
-
27 Dec 2023 1 repository listedThe multimodal recommendation has gradually become the infrastructure of online media platforms, enabling them to provide personalized service to users through a joint modeling of user historical behaviors (e. g.…
-
26 Oct 2023 1 repository listedRecently, multimodal recommendations (MMR) have gained increasing attention for alleviating the data sparsity problem of traditional recommender systems by incorporating modality-based representations.
-
6 Aug 2023 1 repository listedThe teacher model first extracts rich modality features from the generic modality feature by considering both the semantic information of items and the complementary information of multiple modalities.
-
18 Jul 2023 1 repository listedConsidering its challenges in effectiveness and efficiency, we propose a novel Transformer-based recommendation model, termed as Light Graph Transformer model (LightGT).
-
29 Jun 2023 1 repository listedMotivated by the outlined aspects, we propose \framework, a unified framework for the extraction of multimodal features in recommendation.
-
2 Feb 2023 1 repository listedThis paper presents an open-source toolbox, MMRec for multimodal recommendation.
-
28 Jan 2023 1 repository listed
-
19 Apr 2021 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
19 Oct 2019 1 repository listed
-
MDVT: Enhancing Multimodal Recommendation with Model-Agnostic Multimodal-Driven Virtual Triplets22 May 2025 0 repositories listed
-
A Survey on Large Language Models in Multimodal Recommender Systems14 May 2025 0 repositories listed
-
Modality Reliability Guided Multimodal Recommendation23 Apr 2025 0 repositories listed
-
HistLLM: A Unified Framework for LLM-Based Multimodal Recommendation with User History Encoding and Compression14 Apr 2025 0 repositories listed
-
Multi-Modal Hypergraph Enhanced LLM Learning for Recommendation13 Apr 2025 0 repositories listed
-
Training-Free Graph Filtering via Multimodal Feature Refinement for Extremely Fast Multimodal Recommendation6 Mar 2025 0 repositories listed
-
Bridging Domain Gaps between Pretrained Multimodal Models and Recommendations21 Feb 2025 0 repositories listed
-
Don't Lose Yourself: Boosting Multimodal Recommendation via Reducing Node-neighbor Discrepancy in Graph Convolutional Network25 Dec 2024 0 repositories listed
-
Multimodal Graph Neural Network for Recommendation with Dynamic De-redundancy and Modality-Guided Feature De-noisy3 Nov 2024 0 repositories listed
-
Learning ID-free Item Representation with Token Crossing for Multimodal Recommendation25 Oct 2024 0 repositories listed
-
Dynamic Fusion Strategies for Federated Multimodal Recommendations11 Oct 2024 0 repositories listed
-
Multimodal Point-of-Interest Recommendation4 Oct 2024 0 repositories listed
-
ATFLRec: A Multimodal Recommender System with Audio-Text Fusion and Low-Rank Adaptation via Instruction-Tuned Large Language Model13 Sep 2024 0 repositories listed
-
X-Reflect: Cross-Reflection Prompting for Multimodal Recommendation27 Aug 2024 0 repositories listed
-
SynerGraph: An Integrated Graph Convolution Network for Multimodal Recommendation29 May 2024 0 repositories listed
-
Navigating the Future of Federated Recommendation Systems with Foundation Models12 May 2024 0 repositories listed
-
Multimodal Pretraining and Generation for Recommendation: A Tutorial11 May 2024 0 repositories listed
-
MMGRec: Multimodal Generative Recommendation with Transformer Model25 Apr 2024 0 repositories listed
-
DREAM: A Dual Representation Learning Model for Multimodal Recommendation17 Apr 2024 0 repositories listed
-
Dealing with Missing Modalities in Multimodal Recommendation: a Feature Propagation-based Approach28 Mar 2024 0 repositories listed
-
Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models13 Feb 2024 0 repositories listed
-
Attribute-driven Disentangled Representation Learning for Multimodal Recommendation22 Dec 2023 0 repositories listed
-
ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation10 Nov 2023 0 repositories listed
-
Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and Benchmark26 May 2023 0 repositories listed
-
Knowledge Soft Integration for Multimodal Recommendation12 May 2023 0 repositories listed
-
Attention-guided Multi-step Fusion: A Hierarchical Fusion Network for Multimodal Recommendation24 Apr 2023 0 repositories listed
Syntology lines on 5 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections