{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/llava-onevision-easy-visual-task-transfer","title":"LLaVA-OneVision: Easy Visual Task Transfer","arxiv_id":"2408.03326","date":"2024-08-06","proceeding":null,"authors":["Bo Li","Yuanhan Zhang","Dong Guo","Renrui Zhang","Feng Li","Hao Zhang","Kaichen Zhang","Peiyuan Zhang","Yanwei Li","Ziwei Liu","Chunyuan Li"],"abstract":"We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series. Our experimental results demonstrate that LLaVA-OneVision is the first single model that can simultaneously push the performance boundaries of open LMMs in three important computer vision scenarios: single-image, multi-image, and video scenarios. Importantly, the design of LLaVA-OneVision allows strong transfer learning across different modalities/scenarios, yielding new emerging capabilities. In particular, strong video understanding and cross-scenario capabilities are demonstrated through task transfer from images to videos.","url_abs":"https://arxiv.org/abs/2408.03326v3","url_pdf":"https://arxiv.org/pdf/2408.03326v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"llava-onevision-easy-visual-task-transfer","repo_url":"https://github.com/evolvinglmms-lab/lmms-eval","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"llava-onevision-easy-visual-task-transfer","repo_url":"https://github.com/MindSpore-scientific-2/code-14/tree/main/llava_next","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"3d-question-answering-3d-qa","task_name":"3D Question Answering (3D-QA)"},{"task_slug":"multiple-choice","task_name":"Multiple-choice"},{"task_slug":"temporal-relation-extraction","task_name":"Temporal Relation Extraction"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"video-question-answering","task_name":"Video Question Answering"},{"task_slug":"video-understanding","task_name":"Video Understanding"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"},{"task_slug":"zeroshot-video-question-answer","task_name":"Zero-Shot Video Question Answer"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/on-implicitqa","task":"","dataset":"ImplicitQA","model":"LLaVA-OneVision - 7B","rank_in_archive_order":4,"of":7,"metrics":{"Average Accuracy":"43.4","Macro Average Accuracy":"46.4"},"uses_additional_data":false},{"leaderboard":"/sota/3d-question-answering-3d-qa-on-sqa3d","task":"3D Question Answering (3D-QA)","dataset":"SQA3D","model":"LLaVA-NeXT-Video","rank_in_archive_order":13,"of":13,"metrics":{"Exact Match":"34.2"},"uses_additional_data":false},{"leaderboard":"/sota/3d-question-answering-3d-qa-on-scanqa-test-w","task":"3D Question Answering (3D-QA)","dataset":"ScanQA Test w/ objects","model":"LLaVA-NeXT-Video","rank_in_archive_order":16,"of":18,"metrics":{"BLEU-4":"9.8","CIDEr":"46.2","Exact Match":"18.7","METEOR":"9.1","ROUGE":"27.8"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-relation-extraction-on-vinoground","task":"Temporal Relation Extraction","dataset":"Vinoground","model":"LLaVA-OneVision-Qwen2-72B","rank_in_archive_order":4,"of":24,"metrics":{"Group Score":"21.8","Text Score":"48.4","Video Score":"35.2"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-relation-extraction-on-vinoground","task":"Temporal Relation Extraction","dataset":"Vinoground","model":"LLaVA-OneVision-Qwen2-7B","rank_in_archive_order":5,"of":24,"metrics":{"Group Score":"14.6","Text Score":"41.6","Video Score":"29.4"},"uses_additional_data":false},{"leaderboard":"/sota/video-question-answering-on-next-qa","task":"Video Question Answering","dataset":"NExT-QA","model":"LLaVA-OV(72B)","rank_in_archive_order":13,"of":47,"metrics":{"Accuracy":"80.2"},"uses_additional_data":false},{"leaderboard":"/sota/video-question-answering-on-next-qa","task":"Video Question Answering","dataset":"NExT-QA","model":"LLaVA-OV(7B)","rank_in_archive_order":15,"of":47,"metrics":{"Accuracy":"79.4"},"uses_additional_data":false},{"leaderboard":"/sota/video-question-answering-on-ovbench","task":"Video Question Answering","dataset":"OVBench","model":"LLaVA-OneVision (7B)","rank_in_archive_order":5,"of":16,"metrics":{"AVG":"49.5"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-mm-vet","task":"Visual Question Answering","dataset":"MM-Vet","model":"LLaVA-OneVision-72B","rank_in_archive_order":26,"of":231,"metrics":{"GPT-4 score":"63.7"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-mm-vet","task":"Visual Question Answering","dataset":"MM-Vet","model":"LLaVA-OneVision-7B","rank_in_archive_order":43,"of":231,"metrics":{"GPT-4 score":"57.5"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-mm-vet","task":"Visual Question Answering","dataset":"MM-Vet","model":"LLaVA-OneVision-0.5B","rank_in_archive_order":212,"of":231,"metrics":{"GPT-4 score":"29.1"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-v-bench","task":"Visual Question Answering","dataset":"V*bench","model":"LLaVA-OneVision7B","rank_in_archive_order":5,"of":5,"metrics":{"Accuracy":"74.46"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-vqa-on-vlm2-bench","task":"Visual Question Answering (VQA)","dataset":"VLM2-Bench","model":"LLaVA-OneVision-7B","rank_in_archive_order":8,"of":9,"metrics":{"Average Score on VLM2-bench (9 subtasks)":"39.35","GC-mat":"16.60","GC-trk":"13.70","OC-cnt":"56.17","OC-cpr":"47.22","OC-grp":"27.50","PC-VID":"47.25","PC-cnt":"46.67","PC-cpr":"62.00","PC-grp":"37.00"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-video-question-answer-on-vnbench","task":"Zero-Shot Video Question Answer","dataset":"VNBench","model":"LLaVA-OneVision-72B","rank_in_archive_order":3,"of":9,"metrics":{"Accuracy":"58.7"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-video-question-answer-on-vnbench","task":"Zero-Shot Video Question Answer","dataset":"VNBench","model":"LLaVA-OneVision-7B","rank_in_archive_order":4,"of":9,"metrics":{"Accuracy":"51.8"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2408.03326","atlas_url":"https://app.syntology.ai/?focus=2408.03326","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2408.03326"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2408.03326\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2408.03326"},"call":"get_concepts_for_paper(arxiv_id=\"2408.03326\")"}],"by_repo":[{"repo":"evolvinglmms-lab/lmms-eval","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2602.01649"],"harvested_for_calls":[{"arxiv_id":"2602.01649","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2602.01649"},"call":"get_harvested_code_for_paper(arxiv_id=\"2602.01649\")"}]},{"repo":"MindSpore-scientific-2/code-14","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"90ed259663454ce78b68097c405e6c114b02d0fe328d60616539189915abddc1","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}