{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/adafocus-towards-end-to-end-weakly-supervised","title":"Towards Weakly Supervised End-to-end Learning for Long-video Action Recognition","arxiv_id":"2311.17118","date":"2023-11-28","proceeding":null,"authors":["Jiaming Zhou","Hanjun Li","Kun-Yu Lin","Junwei Liang"],"abstract":"Developing end-to-end action recognition models on long videos is fundamental and crucial for long-video action understanding. Due to the unaffordable cost of end-to-end training on the whole long videos, existing works generally train models on short clips trimmed from long videos. However, this ``trimming-then-training'' practice requires action interval annotations for clip-level supervision, i.e., knowing which actions are trimmed into the clips. Unfortunately, collecting such annotations is very expensive and prevents model training at scale. To this end, this work aims to build a weakly supervised end-to-end framework for training recognition models on long videos, with only video-level action category labels. Without knowing the precise temporal locations of actions in long videos, our proposed weakly supervised framework, namely AdaptFocus, estimates where and how likely the actions will occur to adaptively focus on informative action clips for end-to-end training. The effectiveness of the proposed AdaptFocus framework is demonstrated on three long-video datasets. Furthermore, for downstream long-video tasks, our AdaptFocus framework provides a weakly supervised feature extraction pipeline for extracting more robust long-video features, such that the state-of-the-art methods on downstream tasks are significantly advanced. We will release the code and models.","url_abs":"https://arxiv.org/abs/2311.17118v2","url_pdf":"https://arxiv.org/pdf/2311.17118v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"action-segmentation","task_name":"Action Segmentation"},{"task_slug":"action-understanding","task_name":"Action Understanding"},{"task_slug":"long-video-activity-recognition","task_name":"Long-video Activity Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"temporal-sentence-grounding","task_name":"Temporal Sentence Grounding"},{"task_slug":"weakly-supervised-action-segmentation-action","task_name":"Weakly Supervised Action Segmentation (Action Set))"},{"task_slug":"weakly-supervised-learning","task_name":"Weakly-supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-charades","task":"Action Classification","dataset":"Charades","model":"AdaFocus (weak supervision, MViT-B-24, 32x3)","rank_in_archive_order":13,"of":49,"metrics":{"MAP":"47.8"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-charades","task":"Action Classification","dataset":"Charades","model":"AdaFocus (weak supervision, MViT-B-K400-pretrain, 16x4)","rank_in_archive_order":28,"of":49,"metrics":{"MAP":"41.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-charades","task":"Action Classification","dataset":"Charades","model":"AdaFocus (weak supervision, X3D-L, 32x3)","rank_in_archive_order":29,"of":49,"metrics":{"MAP":"41.2"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-charades","task":"Action Classification","dataset":"Charades","model":"AdaFocus (weak supervision, Slowfast-R50, 16x8)","rank_in_archive_order":35,"of":49,"metrics":{"MAP":"39.3"},"uses_additional_data":false},{"leaderboard":"/sota/action-segmentation-on-breakfast-1","task":"Action Segmentation","dataset":"Breakfast","model":"AdaFocus (newly extracted I3D-features, LT-Context model)","rank_in_archive_order":1,"of":37,"metrics":{"Acc":"78.0","Average F1":"76.2","Edit":"78.3","F1@10%":"82.1","F1@25%":"79.0","F1@50%":"67.5"},"uses_additional_data":false},{"leaderboard":"/sota/long-video-activity-recognition-on-breakfast","task":"Long-video Activity Recognition","dataset":"Breakfast","model":"AdaFocus (MViT-Breakfast-Pretrain-feature, GHRM)","rank_in_archive_order":1,"of":8,"metrics":{"mAP":"79.5"},"uses_additional_data":false},{"leaderboard":"/sota/long-video-activity-recognition-on-breakfast","task":"Long-video Activity Recognition","dataset":"Breakfast","model":"AdaFocus (MViT-Breakfast-Pretrain-feature, Timeception)","rank_in_archive_order":2,"of":8,"metrics":{"mAP":"79.2"},"uses_additional_data":false},{"leaderboard":"/sota/long-video-activity-recognition-on-breakfast","task":"Long-video Activity Recognition","dataset":"Breakfast","model":"AdaFocus (I3D-Breakfast-Pretrain-feature, Timeception)","rank_in_archive_order":3,"of":8,"metrics":{"mAP":"70.4"},"uses_additional_data":false},{"leaderboard":"/sota/long-video-activity-recognition-on-breakfast","task":"Long-video Activity Recognition","dataset":"Breakfast","model":"AdaFocus (I3D-Breakfast-Pretrain-feature, GHRM)","rank_in_archive_order":4,"of":8,"metrics":{"mAP":"69.6"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-sentence-grounding-on-charades-sta","task":"Temporal Sentence Grounding","dataset":"Charades-STA","model":"AdaFocus (Full, MViT-Charades-Pretrain-feature, MMN model)","rank_in_archive_order":2,"of":13,"metrics":{"R1@0.5":"62.4","R1@0.7":"38.6","R5@0.5":"89.4","R5@0.7":"66.4"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-sentence-grounding-on-charades-sta","task":"Temporal Sentence Grounding","dataset":"Charades-STA","model":"AdaFocus (Full, I3D-Charades-Pretrain-feature, MMN model)","rank_in_archive_order":3,"of":13,"metrics":{"R1@0.5":"56.7","R1@0.7":"35.6","R5@0.5":"87.9","R5@0.7":"65.0"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-sentence-grounding-on-charades-sta","task":"Temporal Sentence Grounding","dataset":"Charades-STA","model":"AdaFocus (Weak, MViT-Charades-Pretrain-feature, CPL model)","rank_in_archive_order":6,"of":13,"metrics":{"R1@0.5":"51.7","R1@0.7":"23.2","R5@0.5":"85.2","R5@0.7":"52.6"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-sentence-grounding-on-charades-sta","task":"Temporal Sentence Grounding","dataset":"Charades-STA","model":"AdaFocus (Weak, I3D-Charades-Pretrain-feature, CPL model)","rank_in_archive_order":7,"of":13,"metrics":{"R1@0.5":"49.1","R1@0.7":"22.4","R5@0.5":"84.2","R5@0.7":"51.8"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-sentence-grounding-on-charades-sta","task":"Temporal Sentence Grounding","dataset":"Charades-STA","model":"AdaFocus (Semi-weak, MViT-Charades-Pretrain-feature, D3G model)","rank_in_archive_order":8,"of":13,"metrics":{"R1@0.5":"50.1","R1@0.7":"21.8","R5@0.5":"86.1","R5@0.7":"54.6"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-sentence-grounding-on-charades-sta","task":"Temporal Sentence Grounding","dataset":"Charades-STA","model":"AdaFocus (Semi-weak, I3D-Charades-Pretrain-feature, D3G model)","rank_in_archive_order":10,"of":13,"metrics":{"R1@0.5":"46.9","R1@0.7":"21.1","R5@0.5":"79.3","R5@0.7":"49.2"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-action-segmentation-action","task":"Weakly Supervised Action Segmentation (Action Set))","dataset":"Breakfast","model":"AdaFocus (newly extracted I3D-features, POC model)","rank_in_archive_order":1,"of":2,"metrics":{"Acc":"49.6"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2311.17118","atlas_url":"https://app.syntology.ai/?focus=2311.17118","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2311.17118"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2311.17118\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2311.17118"},"call":"get_concepts_for_paper(arxiv_id=\"2311.17118\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"aaafcbb12619fc50ea212d6b330c48b097d3748a7968b701d6e5dec12c8dd54a","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}