{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/egovlpv2-egocentric-video-language-pre","title":"EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone","arxiv_id":"2307.05463","date":"2023-07-11","proceeding":"ICCV 2023 1","authors":["Shraman Pramanick","Yale Song","Sayan Nag","Kevin Qinghong Lin","Hardik Shah","Mike Zheng Shou","Rama Chellappa","Pengchuan Zhang"],"abstract":"Video-language pre-training (VLP) has become increasingly important due to its ability to generalize to various vision and language tasks. However, existing egocentric VLP frameworks utilize separate video and language encoders and learn task-specific cross-modal information only during fine-tuning, limiting the development of a unified system. In this work, we introduce the second generation of egocentric video-language pre-training (EgoVLPv2), a significant improvement from the previous generation, by incorporating cross-modal fusion directly into the video and language backbones. EgoVLPv2 learns strong video-text representation during pre-training and reuses the cross-modal attention modules to support different downstream tasks in a flexible and efficient manner, reducing fine-tuning costs. Moreover, our proposed fusion in the backbone strategy is more lightweight and compute-efficient than stacking additional fusion-specific layers. Extensive experiments on a wide range of VL tasks demonstrate the effectiveness of EgoVLPv2 by achieving consistent state-of-the-art performance over strong baselines across all downstream. Our project page can be found at https://shramanpramanick.github.io/EgoVLPv2/.","url_abs":"https://arxiv.org/abs/2307.05463v2","url_pdf":"https://arxiv.org/pdf/2307.05463v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"egovlpv2-egocentric-video-language-pre","repo_url":"https://github.com/facebookresearch/EgoVLPv2","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"moment-queries","task_name":"Moment Queries"},{"task_slug":"multi-instance-retrieval","task_name":"Multi-Instance Retrieval"},{"task_slug":"natural-language-queries","task_name":"Natural Language Queries"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"video-summarization","task_name":"Video Summarization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-recognition-on-charades-ego","task":"Action Recognition","dataset":"Charades-Ego","model":"EgoVLPv2","rank_in_archive_order":2,"of":6,"metrics":{"mAP":"34.1"},"uses_additional_data":false},{"leaderboard":"/sota/natural-language-queries-on-ego4d","task":"Natural Language Queries","dataset":"Ego4D","model":"EgoVLPv2","rank_in_archive_order":10,"of":10,"metrics":{"R@1 IoU=0.3":"12.95","R@1 IoU=0.5":"7.91","R@5 IoU=0.3":"23.80","R@5 IoU=0.5":"16.11"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-egotaskqa","task":"Question Answering","dataset":"EgoTaskQA","model":"EgoVLPv2","rank_in_archive_order":1,"of":4,"metrics":{"Direct":"46.26"},"uses_additional_data":false},{"leaderboard":"/sota/video-summarization-on-query-focused-video","task":"Video Summarization","dataset":"Query-Focused Video Summarization Dataset","model":"EgoVLPv2","rank_in_archive_order":1,"of":2,"metrics":{"F1 (avg)":"52.08"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2307.05463","atlas_url":"https://app.syntology.ai/?focus=2307.05463","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2307.05463"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2307.05463\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2307.05463"},"call":"get_concepts_for_paper(arxiv_id=\"2307.05463\")"}],"by_repo":[{"repo":"facebookresearch/EgoVLPv2","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2311.18259"],"harvested_for_calls":[{"arxiv_id":"2311.18259","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2311.18259"},"call":"get_harvested_code_for_paper(arxiv_id=\"2311.18259\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"be9893d436e92aaf8794d5be100c5c2f952eda31dd62eff992458f14a94ff4ba","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}