{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/llava-uhd-an-lmm-perceiving-any-aspect-ratio","title":"LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images","arxiv_id":"2403.11703","date":"2024-03-18","proceeding":null,"authors":["Ruyi Xu","Yuan YAO","Zonghao Guo","Junbo Cui","Zanlin Ni","Chunjiang Ge","Tat-Seng Chua","Zhiyuan Liu","Maosong Sun","Gao Huang"],"abstract":"Visual encoding constitutes the basis of large multimodal models (LMMs) in understanding the visual world. Conventional LMMs process images in fixed sizes and limited resolutions, while recent explorations in this direction are limited in adaptivity, efficiency, and even correctness. In this work, we first take GPT-4V and LLaVA-1.5 as representative examples and expose systematic flaws rooted in their visual encoding strategy. To address the challenges, we present LLaVA-UHD, a large multimodal model that can efficiently perceive images in any aspect ratio and high resolution. LLaVA-UHD includes three key components: (1) An image modularization strategy that divides native-resolution images into smaller variable-sized slices for efficient and extensible encoding, (2) a compression module that further condenses image tokens from visual encoders, and (3) a spatial schema to organize slice tokens for LLMs. Comprehensive experiments show that LLaVA-UHD outperforms established LMMs trained with 2-3 orders of magnitude more data on 9 benchmarks. Notably, our model built on LLaVA-1.5 336x336 supports 6 times larger (i.e., 672x1088) resolution images using only 94% inference computation, and achieves 6.4 accuracy improvement on TextVQA. Moreover, the model can be efficiently trained in academic settings, within 23 hours on 8 A100 GPUs (vs. 26 hours of LLaVA-1.5). We make the data and code publicly available at https://github.com/thunlp/LLaVA-UHD.","url_abs":"https://arxiv.org/abs/2403.11703v1","url_pdf":"https://arxiv.org/pdf/2403.11703v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"llava-uhd-an-lmm-perceiving-any-aspect-ratio","repo_url":"https://github.com/thunlp/llava-uhd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"long-context-understanding","task_name":"Long-Context Understanding"},{"task_slug":null,"task_name":"TextVQA"}],"methods":[{"method_slug":"1d-cnn","method_name":"1D CNN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/long-context-understanding-on-mmneedle","task":"Long-Context Understanding","dataset":"MMNeedle","model":"LLaVA-Llama-3","rank_in_archive_order":5,"of":12,"metrics":{"1 Image, 2*2 Stitching, Exact Accuracy":"43.8","1 Image, 4*4 Stitching, Exact Accuracy":"17.5","1 Image, 8*8 Stitching, Exact Accuracy":"3.3","10 Images, 1*1 Stitching, Exact Accuracy":"0","10 Images, 2*2 Stitching, Exact Accuracy":"0","10 Images, 4*4 Stitching, Exact Accuracy":"0","10 Images, 8*8 Stitching, Exact Accuracy":"0"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.11703","atlas_url":"https://app.syntology.ai/?focus=2403.11703","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2403.11703"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2403.11703\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2403.11703"},"call":"get_concepts_for_paper(arxiv_id=\"2403.11703\")"}],"by_repo":[{"repo":"thunlp/llava-uhd","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2412.13871"],"harvested_for_calls":[{"arxiv_id":"2412.13871","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2412.13871"},"call":"get_harvested_code_for_paper(arxiv_id=\"2412.13871\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"848d1e92d3897fdf4478a45d09693b15ca0846d4af9a84e6fda3451a65dc8a3e","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}