{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/flash-vstream-memory-based-real-time","title":"Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams","arxiv_id":"2406.08085","date":"2024-06-12","proceeding":null,"authors":["Haoji Zhang","Yiqin Wang","Yansong Tang","Yong liu","Jiashi Feng","Jifeng Dai","Xiaojie Jin"],"abstract":"Benefiting from the advancements in large language models and cross-modal alignment, existing multi-modal video understanding methods have achieved prominent performance in offline scenario. However, online video streams, as one of the most common media forms in the real world, have seldom received attention. Compared to offline videos, the 'dynamic' nature of online video streams poses challenges for the direct application of existing models and introduces new problems, such as the storage of extremely long-term information, interaction between continuous visual content and 'asynchronous' user questions. Therefore, in this paper we present Flash-VStream, a video-language model that simulates the memory mechanism of human. Our model is able to process extremely long video streams in real-time and respond to user queries simultaneously. Compared to existing models, Flash-VStream achieves significant reductions in inference latency and VRAM consumption, which is intimately related to performing understanding of online streaming video. In addition, given that existing video understanding benchmarks predominantly concentrate on offline scenario, we propose VStream-QA, a novel question answering benchmark specifically designed for online video streaming understanding. Comparisons with popular existing methods on the proposed benchmark demonstrate the superiority of our method for such challenging setting. To verify the generalizability of our approach, we further evaluate it on existing video understanding benchmarks and achieves state-of-the-art performance in offline scenarios as well. All code, models, and datasets are available at the https://invinciblewyq.github.io/vstream-page/","url_abs":"https://arxiv.org/abs/2406.08085v2","url_pdf":"https://arxiv.org/pdf/2406.08085v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"flash-vstream-memory-based-real-time","repo_url":"https://github.com/IVGSZ/Flash-VStream","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"video-question-answering","task_name":"Video Question Answering"},{"task_slug":"video-understanding","task_name":"Video Understanding"},{"task_slug":"zeroshot-video-question-answer","task_name":"Zero-Shot Video Question Answer"},{"task_slug":"cross-modal-alignment","task_name":"cross-modal alignment"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-next-qa-open-ended","task":"Question Answering","dataset":"NExT-QA (Open-ended VideoQA)","model":"Flash-VStream","rank_in_archive_order":1,"of":6,"metrics":{"Accuracy":"61.6","Confidence Score":"3.4"},"uses_additional_data":false},{"leaderboard":"/sota/video-question-answering-on-ovbench","task":"Video Question Answering","dataset":"OVBench","model":"Flash-Vstream (7B)","rank_in_archive_order":12,"of":16,"metrics":{"AVG":"31.2"},"uses_additional_data":false},{"leaderboard":"/sota/zeroshot-video-question-answer-on-activitynet","task":"Zero-Shot Video Question Answer","dataset":"ActivityNet-QA","model":"Flash-VStream","rank_in_archive_order":9,"of":28,"metrics":{"Accuracy":"51.9","Confidence Score":"3.4"},"uses_additional_data":false},{"leaderboard":"/sota/zeroshot-video-question-answer-on-msrvtt-qa","task":"Zero-Shot Video Question Answer","dataset":"MSRVTT-QA","model":"Flash-VStream","rank_in_archive_order":1,"of":30,"metrics":{"Accuracy":"72.4","Confidence Score":"3.4"},"uses_additional_data":false},{"leaderboard":"/sota/zeroshot-video-question-answer-on-msvd-qa","task":"Zero-Shot Video Question Answer","dataset":"MSVD-QA","model":"Flash-VStream","rank_in_archive_order":2,"of":28,"metrics":{"Accuracy":"80.3","Confidence Score":"3.9"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.08085","atlas_url":"https://app.syntology.ai/?focus=2406.08085","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2406.08085"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2406.08085\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2406.08085"},"call":"get_concepts_for_paper(arxiv_id=\"2406.08085\")"}],"by_repo":[{"repo":"IVGSZ/Flash-VStream","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2506.23825"],"harvested_for_calls":[{"arxiv_id":"2506.23825","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2506.23825"},"call":"get_harvested_code_for_paper(arxiv_id=\"2506.23825\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"e7b36019e0dcf7df9136487382ef1776d52ac9878dd627373f3293d4ba27e6fa","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}