Browse State-of-the-Art › MME
MME
47 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
MME is a comprehensive evaluation benchmark for multimodal large language models. It measures both perception and cognition abilities on a total of 14 subtasks, including existence, count, position, color, poster, celebrity, scene, landmark, artwork, OCR, commonsense reasoning, numerical calculation, text translation, and code reasoning.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 95 · with a code link: 47 · where Syntology ran a sample: 27 (21 with a run with no instrument failure, 6 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (27 of 95 tagged: 21 with a run with no instrument failure, 6 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 95 of 95, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
13 Apr 2019 5 repositories listed Syntology 19 ran (of which 0 constructed an object rather than computing a result; 18 with no instrument failure: 4 honoured, 0 violated, 14 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified (of 24 harvested samples) · 7 pointer-only (licence)Contemporary domain adaptation methods are very effective at aligning feature distributions of source and target domains without any target supervision.
-
22 Nov 2024 4 repositories listedAs a prominent direction of Artificial General Intelligence (AGI), Multimodal Large Language Models (MLLMs) have garnered increased attention from both industry and academia.
-
23 Jun 2023 4 repositories listedMultimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image.
-
26 Sep 2023 3 repositories listedWe propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition.
-
20 May 2025 2 repositories listed Syntology 12 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified (of 14 harvested samples) · 5 pointer-only (licence)Large multimodal models (LMMs) have recently emerged as a powerful tool for long video understanding (LVU), prompting the development of standardized LVU benchmarks to evaluate their performance.
-
24 Apr 2025 2 repositories listed Syntology official: harvested, nothing ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 12 harvested samples) · 1 pointer-only (licence)Remarkably, our experiments demonstrate that DTD achieves an 82.
-
2 Apr 2025 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)Motivated by the success of Reinforcement Learning with Verifiable Reward (RLVR) in unlocking LLM reasoning abilities, this work aims to improve MLLMs in video spatial reasoning through the RLVR paradigm.
-
24 Jun 2024 2 repositories listed Syntology official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)By simply extrapolating the context length of the language backbone, we enable LMMs to comprehend orders of magnitude more visual tokens without any video training.
-
27 Mar 2024 2 repositories listed Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 6 unverified (of 8 harvested samples) · 2 pointer-only (licence)Our method is inspired by our observation that what we call disturbance instructions significantly exacerbate hallucinations in multimodal fusion modules.
-
19 Dec 2023 2 repositories listedThey endow Large Language Models (LLMs) with powerful capabilities in visual understanding, enabling them to tackle diverse multi-modal tasks.
-
14 Sep 2023 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 7 pointer-only (licence)In this paper, we address the limitation above by 1) introducing vision-language Model with Multi-Modal In-Context Learning(MMICL), a new approach to allow the VLM to deal with multi-modal inputs efficiently; 2)…
-
12 Oct 2022 2 repositories listedThe latest attempts seek to learn a representation model by predicting the appearance contents in the masked regions.
-
8 Jul 2025 1 repository listedState-of-the-art large multi-modal models (LMMs) face challenges when processing high-resolution images, as these inputs are converted into enormous visual tokens, many of which are irrelevant to the downstream task.
-
30 Jun 2025 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 7 where Syntology's instrument failed) · 4 unverified (of 11 harvested samples)Benefiting from the advances in large language models and cross-modal alignment, existing multimodal large language models have achieved prominent performance in image and short video understanding.
-
12 Jun 2025 1 repository listedLong video understanding (LVU) presents a significant challenge for current multi-modal large language models (MLLMs) due to the task's inherent complexity and context window constraint.
-
30 May 2025 1 repository listedIn the second stage, language descriptions are fed into a powerful reasoning LLM to solve complex video-language understanding tasks.
-
24 Apr 2025 1 repository listedHowever, the model size and performance of these long context models are still limited due to the computational cost in both training and inference.
-
21 Apr 2025 1 repository listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)We introduce Eagle 2.
-
27 Mar 2025 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)In this paper, we introduce BOLT, a method to BOost Large VLMs without additional Training through a comprehensive study of frame selection strategies.
-
24 Mar 2025 1 repository listedDespite the significant success of Large Vision-Language models(LVLMs), these models still suffer hallucinations when describing images, generating answers that include non-existent objects.
-
11 Mar 2025 1 repository listed Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Specifically, (i) QuoTA strategically allocates frame-level importance scores based on query relevance, enabling one-time visual token assignment before cross-modal interactions in decoder layers, (ii) we decouple the…
-
2 Mar 2025 1 repository listed Syntology official: harvested, nothing ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples) · 4 pointer-only (licence)Multimodal instruction tuning has proven to be an effective strategy for achieving zero-shot generalization by fine-tuning pre-trained Large Multimodal Models (LMMs) with instruction-following data.
-
18 Feb 2025 1 repository listed Syntology official: no sample here; runs from other or unrecorded repositories · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples)In this work, we introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences, and the model is required to understand the semantics from the interleaved…
-
24 Dec 2024 1 repository listedWe found current VLLMs to exhibit a contradiction of over-sensitivity to language instructions and under-sensitivity to visual positional information.
-
12 Dec 2024 1 repository listed Syntology official (archive's flag): 14 ran · 14 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 19 harvested samples) · 4 pointer-only (licence)As Multi-modal Large Language Models (MLLMs) evolve, expanding beyond single-domain capabilities is essential to meet the demands for more versatile and efficient AI.
-
20 Nov 2024 1 repository listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Existing large video-language models (LVLMs) struggle to comprehend long videos correctly due to limited context.
-
11 Oct 2024 1 repository listedLarge Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across multimodal tasks such as visual perception and reasoning, leading to good performance on various multimodal evaluation benchmarks.
-
9 Oct 2024 1 repository listedHowever, there is still considerable debate on constructing MLLM architectures, particularly regarding the selection of appropriate connectors for perception tasks of varying granularities.
-
7 Oct 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples)These biases arise from the visual encoder and the Large Language Model (LLM) backbone, affecting the attention mechanism responsible for aligning multimodal inputs.
-
5 Oct 2024 1 repository listedLarge Vision-Language Models (LVLMs) have achieved remarkable progress on visual perception and linguistic interpretation.
-
23 Aug 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 1 pointer-only (licence)
-
7 Aug 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)
-
Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment21 Jul 2024 1 repository listed
-
29 May 2024 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 11 harvested samples)
-
6 Apr 2024 1 repository listed
-
21 Mar 2024 1 repository listed
-
20 Mar 2024 1 repository listed
-
11 Dec 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
7 Dec 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples)
-
28 Nov 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 1 pointer-only (licence)
-
21 Nov 2023 1 repository listed
-
2 Nov 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
19 Aug 2023 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 1 pointer-only (licence)
-
8 Aug 2023 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 1 violated, 1 with no contract checked; 8 where Syntology's instrument failed) · 2 unverified (of 12 harvested samples) · 1 pointer-only (licence)
-
1 Jan 2023 1 repository listed
-
15 Aug 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs27 Jun 2025 0 repositories listed
-
Language-Vision Planner and Executor for Text-to-Visual Reasoning9 Jun 2025 0 repositories listed
-
DynTok: Dynamic Compression of Visual Tokens for Efficient and Effective Video Understanding4 Jun 2025 0 repositories listed
-
Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering1 Jun 2025 0 repositories listed
-
EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models28 May 2025 0 repositories listed
-
MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs27 May 2025 0 repositories listed
-
Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Hallucination in Large Vision-Language Models26 May 2025 0 repositories listed
-
Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models18 May 2025 0 repositories listed
-
VISTA: Enhancing Vision-Text Alignment in MLLMs via Cross-Modal Mutual Information Maximization16 May 2025 0 repositories listed
-
Visual Instruction Tuning with Chain of Region-of-Interest11 May 2025 0 repositories listed
-
An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubes21 Apr 2025 0 repositories listed
-
MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models4 Apr 2025 0 repositories listed
-
Improving LLM Video Understanding with 16 Frames Per Second18 Mar 2025 0 repositories listed
-
Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understanding17 Mar 2025 0 repositories listed
-
Ultra-High-Frequency Harmony: mmWave Radar and Event Camera Orchestrate Accurate Drone Landing20 Feb 2025 0 repositories listed
-
AIDE: Agentically Improve Visual Language Model with Domain Experts13 Feb 2025 0 repositories listed
-
MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency13 Feb 2025 0 repositories listed
-
Hummingbird: High Fidelity Image Generation via Multimodal Context Alignment7 Feb 2025 0 repositories listed
-
Mitigating Hallucinations in Large Vision-Language Models with Internal Fact-based Contrastive Decoding3 Feb 2025 0 repositories listed
-
MME-Industry: A Cross-Industry Multimodal Evaluation Benchmark28 Jan 2025 0 repositories listed
-
Temporal Preference Optimization for Long-Form Video Understanding23 Jan 2025 0 repositories listed
-
GFormer: Accelerating Large Language Models with Optimized Transformers on Gaudi Processors19 Dec 2024 0 repositories listed
-
Apollo: An Exploration of Video Understanding in Large Multimodal Models13 Dec 2024 0 repositories listed
-
EACO: Enhancing Alignment in Multimodal LLMs via Critical Observation6 Dec 2024 0 repositories listed
-
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads28 Nov 2024 0 repositories listed
-
SAVEn-Vid: Synergistic Audio-Visual Integration for Enhanced Understanding in Long Video Context25 Nov 2024 0 repositories listed
-
Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy23 Nov 2024 0 repositories listed
-
The economic value of empowering older patients transitioning from hospital to home: Evidence from the 'Your Care Needs You' intervention7 Nov 2024 0 repositories listed
-
MME-Finance: A Multimodal Finance Benchmark for Expert-level Understanding and Reasoning5 Nov 2024 0 repositories listed
-
ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification11 Oct 2024 0 repositories listed
-
Temporal Reasoning Transfer from Text to Video8 Oct 2024 0 repositories listed
-
DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct 2024 0 repositories listed
-
MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?23 Aug 2024 0 repositories listed
-
DrVideo: Document Retrieval Based Long Video Understanding18 Jun 2024 0 repositories listed
-
Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models28 May 2024 0 repositories listed
-
RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in Large Vision Language Models28 May 2024 0 repositories listed
-
17 Dec 2023 0 repositories listed
-
The Use of Symmetry for Models with Variable-size Variables15 Nov 2023 0 repositories listed
-
Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Model31 Oct 2023 0 repositories listed
-
Benchmarking and In-depth Performance Study of Large Language Models on Habana Gaudi Processors29 Sep 2023 0 repositories listed
-
Domain Adaptation via Minimax Entropy for Real/Bogus Classification of Astronomical Alerts15 Aug 2023 0 repositories listed
-
Multi-Modal Evaluation Approach for Medical Image Segmentation8 Feb 2023 0 repositories listed
-
MME-CRS: Multi-Metric Evaluation Based on Correlation Re-Scaling for Evaluating Open-Domain Dialogue19 Jun 2022 0 repositories listed
-
Machine Learning Methods for Inferring the Number of UAV Emitters via Massive MIMO Receive Array2 Mar 2022 0 repositories listed
-
Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation9 Apr 2020 0 repositories listed
-
Learning Multilingual Meta-Embeddings for Code-Switching Named Entity Recognition1 Aug 2019 0 repositories listed
-
Deep Learning for Hybrid 5G Services in Mobile Edge Computing Systems: Learn from a Digital Twin30 Jun 2019 0 repositories listed
-
Scalable K-Medoids via True Error Bound and Familywise Bandits27 May 2019 0 repositories listed
Syntology lines on 27 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections