{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/dropmae-masked-autoencoders-with-spatial","title":"DropMAE: Masked Autoencoders with Spatial-Attention Dropout for Tracking Tasks","arxiv_id":"2304.00571","date":"2023-04-02","proceeding":"CVPR 2023 1","authors":["Qiangqiang Wu","Tianyu Yang","Ziquan Liu","Baoyuan Wu","Ying Shan","Antoni B. Chan"],"abstract":"In this paper, we study masked autoencoder (MAE) pretraining on videos for matching-based downstream tasks, including visual object tracking (VOT) and video object segmentation (VOS). A simple extension of MAE is to randomly mask out frame patches in videos and reconstruct the frame pixels. However, we find that this simple baseline heavily relies on spatial cues while ignoring temporal relations for frame reconstruction, thus leading to sub-optimal temporal matching representations for VOT and VOS. To alleviate this problem, we propose DropMAE, which adaptively performs spatial-attention dropout in the frame reconstruction to facilitate temporal correspondence learning in videos. We show that our DropMAE is a strong and efficient temporal matching learner, which achieves better finetuning results on matching-based tasks than the ImageNetbased MAE with 2X faster pre-training speed. Moreover, we also find that motion diversity in pre-training videos is more important than scene diversity for improving the performance on VOT and VOS. Our pre-trained DropMAE model can be directly loaded in existing ViT-based trackers for fine-tuning without further modifications. Notably, DropMAE sets new state-of-the-art performance on 8 out of 9 highly competitive video tracking and segmentation datasets. Our code and pre-trained models are available at https://github.com/jimmy-dq/DropMAE.git.","url_abs":"https://arxiv.org/abs/2304.00571v2","url_pdf":"https://arxiv.org/pdf/2304.00571v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dropmae-masked-autoencoders-with-spatial","repo_url":"https://github.com/jimmy-dq/dropmae","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"},{"task_slug":"visual-object-tracking","task_name":"Visual Object Tracking"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"mae","method_name":"MAE"},{"method_slug":"vos","method_name":"VOS"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-object-tracking-on-got-10k","task":"Visual Object Tracking","dataset":"GOT-10k","model":"DropMAE","rank_in_archive_order":18,"of":42,"metrics":{"Average Overlap":"75.9","Success Rate 0.5":"86.8","Success Rate 0.75":"72"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-itb","task":"Visual Object Tracking","dataset":"ITB","model":"DropTrack","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.65"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-lasot","task":"Visual Object Tracking","dataset":"LaSOT","model":"DropTrack","rank_in_archive_order":23,"of":46,"metrics":{"AUC":"71.8","Normalized Precision":"81.8","Precision":"78.1"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-lasot-ext","task":"Visual Object Tracking","dataset":"LaSOT-ext","model":"DropTrack","rank_in_archive_order":13,"of":18,"metrics":{"AUC":"52.7","Precision":"60.2"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-tnl2k","task":"Visual Object Tracking","dataset":"TNL2K","model":"DropTrack","rank_in_archive_order":15,"of":16,"metrics":{"AUC":"56.9","precision":"57.9"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-trackingnet","task":"Visual Object Tracking","dataset":"TrackingNet","model":"DropTrack","rank_in_archive_order":38,"of":40,"metrics":{"AUC":"0.841","Normalized Precision":"88.9"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"jimmy-dq/dropmae","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"6c42a8388bba3a848fe016489a7bc7b04d78d5da0a4307effab1b74ff5e83b87","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}