{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/many-to-many-splatting-for-efficient-video","title":"Many-to-many Splatting for Efficient Video Frame Interpolation","arxiv_id":"2204.03513","date":"2022-04-07","proceeding":"CVPR 2022 1","authors":["Ping Hu","Simon Niklaus","Stan Sclaroff","Kate Saenko"],"abstract":"Motion-based video frame interpolation commonly relies on optical flow to warp pixels from the inputs to the desired interpolation instant. Yet due to the inherent challenges of motion estimation (e.g. occlusions and discontinuities), most state-of-the-art interpolation approaches require subsequent refinement of the warped result to generate satisfying outputs, which drastically decreases the efficiency for multi-frame interpolation. In this work, we propose a fully differentiable Many-to-Many (M2M) splatting framework to interpolate frames efficiently. Specifically, given a frame pair, we estimate multiple bidirectional flows to directly forward warp the pixels to the desired time step, and then fuse any overlapping pixels. In doing so, each source pixel renders multiple target pixels and each target pixel can be synthesized from a larger area of visual context. This establishes a many-to-many splatting scheme with robustness to artifacts like holes. Moreover, for each input frame pair, M2M only performs motion estimation once and has a minuscule computational overhead when interpolating an arbitrary number of in-between frames, hence achieving fast multi-frame interpolation. We conducted extensive experiments to analyze M2M, and found that it significantly improves efficiency while maintaining high effectiveness.","url_abs":"https://arxiv.org/abs/2204.03513v1","url_pdf":"https://arxiv.org/pdf/2204.03513v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"many-to-many-splatting-for-efficient-video","repo_url":"https://github.com/feinanshan/m2m_vfi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"motion-estimation","task_name":"Motion Estimation"},{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"video-frame-interpolation","task_name":"Video Frame Interpolation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-frame-interpolation-on-atd-12k","task":"Video Frame Interpolation","dataset":"ATD-12K","model":"M2M-PWC","rank_in_archive_order":1,"of":1,"metrics":{"PSNR":"29.03","SSIM":"0.959"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-ucf101-1","task":"Video Frame Interpolation","dataset":"UCF101","model":"M2M-PWC","rank_in_archive_order":14,"of":19,"metrics":{"PSNR":"35.17","SSIM":"0.97"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-vimeo90k","task":"Video Frame Interpolation","dataset":"Vimeo90K","model":"M2M-PWC","rank_in_archive_order":12,"of":23,"metrics":{"PSNR":"35.4","SSIM":"0.978","Speed (ms/f)":"32 (Titan X)"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-x4k1000fps","task":"Video Frame Interpolation","dataset":"X4K1000FPS","model":"M2M-PWC","rank_in_archive_order":7,"of":20,"metrics":{"PSNR":"30.81","SSIM":"0.912","Speed (ms/f)":"200 (Titan X)"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-x4k1000fps-2k","task":"Video Frame Interpolation","dataset":"X4K1000FPS-2K","model":"M2M-PWC","rank_in_archive_order":3,"of":4,"metrics":{"PSNR":"32.07","SSIM":"0.923"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-xiph-2k","task":"Video Frame Interpolation","dataset":"Xiph-2K","model":"M2M-PWC","rank_in_archive_order":4,"of":4,"metrics":{"PSNR":"36.45","SSIM":"0.967"},"uses_additional_data":false},{"leaderboard":"/sota/video-frame-interpolation-on-xiph-4k-crop","task":"Video Frame Interpolation","dataset":"Xiph-4K (Crop)","model":"M2M-PWC","rank_in_archive_order":1,"of":1,"metrics":{"PSNR":"33.93","SSIM":"0.945"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2204.03513","atlas_url":"https://app.syntology.ai/?focus=2204.03513","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2204.03513"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2204.03513\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2204.03513"},"call":"get_concepts_for_paper(arxiv_id=\"2204.03513\")"}],"by_repo":[{"repo":"feinanshan/m2m_vfi","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"e57f6a0be050bfebab254ad7ff40ef7607eaa8e62958b4d316e3b8b19ee9a546","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}