{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/mar-masked-autoencoders-for-efficient-action","title":"MAR: Masked Autoencoders for Efficient Action Recognition","arxiv_id":"2207.11660","date":"2022-07-24","proceeding":null,"authors":["Zhiwu Qing","Shiwei Zhang","Ziyuan Huang","Xiang Wang","Yuehuan Wang","Yiliang Lv","Changxin Gao","Nong Sang"],"abstract":"Standard approaches for video recognition usually operate on the full input videos, which is inefficient due to the widely present spatio-temporal redundancy in videos. Recent progress in masked video modelling, i.e., VideoMAE, has shown the ability of vanilla Vision Transformers (ViT) to complement spatio-temporal contexts given only limited visual contents. Inspired by this, we propose propose Masked Action Recognition (MAR), which reduces the redundant computation by discarding a proportion of patches and operating only on a part of the videos. MAR contains the following two indispensable components: cell running masking and bridging classifier. Specifically, to enable the ViT to perceive the details beyond the visible patches easily, cell running masking is presented to preserve the spatio-temporal correlations in videos, which ensures the patches at the same spatial location can be observed in turn for easy reconstructions. Additionally, we notice that, although the partially observed features can reconstruct semantically explicit invisible patches, they fail to achieve accurate classification. To address this, a bridging classifier is proposed to bridge the semantic gap between the ViT encoded features for reconstruction and the features specialized for classification. Our proposed MAR reduces the computational cost of ViT by 53% and extensive experiments show that MAR consistently outperforms existing ViT models with a notable margin. Especially, we found a ViT-Large trained by MAR outperforms the ViT-Huge trained by a standard training scheme by convincing margins on both Kinetics-400 and Something-Something v2 datasets, while our computation overhead of ViT-Large is only 14.5% of ViT-Huge.","url_abs":"https://arxiv.org/abs/2207.11660v1","url_pdf":"https://arxiv.org/pdf/2207.11660v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mar-masked-autoencoders-for-efficient-action","repo_url":"https://github.com/alibaba-mmai-research/masked-action-recognition","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"video-recognition","task_name":"Video Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"MAR (50% mask, ViT-L, 16x4)","rank_in_archive_order":55,"of":207,"metrics":{"Acc@1":"85.3","Acc@5":"96.3"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"MAR (75% mask, ViT-L, 16x4)","rank_in_archive_order":64,"of":207,"metrics":{"Acc@1":"83.9","Acc@5":"96.0"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"MAR (50% mask, ViT-B, 16x4)","rank_in_archive_order":90,"of":207,"metrics":{"Acc@1":"81.0","Acc@5":"94.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"MAR (75% mask, ViT-B, 16x4)","rank_in_archive_order":113,"of":207,"metrics":{"Acc@1":"79.4","Acc@5":"93.7"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"MAR (50% mask, ViT-L, 16x4)","rank_in_archive_order":12,"of":123,"metrics":{"GFLOPs":"276x6","Parameters":"311","Top-1 Accuracy":"74.7","Top-5 Accuracy":"94.9"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"MAR (75% mask, ViT-L, 16x4)","rank_in_archive_order":16,"of":123,"metrics":{"GFLOPs":"131x6","Parameters":"311","Top-1 Accuracy":"73.8","Top-5 Accuracy":"94.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"MAR (50% mask, ViT-B, 16x4)","rank_in_archive_order":33,"of":123,"metrics":{"GFLOPs":"86x6","Parameters":"94","Top-1 Accuracy":"71.0","Top-5 Accuracy":"92.8"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"MAR (75% mask, ViT-B, 16x4)","rank_in_archive_order":45,"of":123,"metrics":{"GFLOPs":"41x6","Parameters":"94","Top-1 Accuracy":"69.5","Top-5 Accuracy":"91.9"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2207.11660","atlas_url":"https://app.syntology.ai/?focus=2207.11660","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2207.11660"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2207.11660\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2207.11660"},"call":"get_concepts_for_paper(arxiv_id=\"2207.11660\")"}],"by_repo":[{"repo":"alibaba-mmai-research/masked-action-recognition","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"8ba876c3b777a6cfe564874c17227d11f98bea473a282726e02bb07182a2dd0e","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}