{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/rethinking-spatiotemporal-feature-learning","title":"Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification","arxiv_id":"1712.04851","date":"2017-12-13","proceeding":"ECCV 2018 9","authors":["Saining Xie","Chen Sun","Jonathan Huang","Zhuowen Tu","Kevin Murphy"],"abstract":"Despite the steady progress in video analysis led by the adoption of\nconvolutional neural networks (CNNs), the relative improvement has been less\ndrastic as that in 2D static image classification. Three main challenges exist\nincluding spatial (image) feature representation, temporal information\nrepresentation, and model/computation complexity. It was recently shown by\nCarreira and Zisserman that 3D CNNs, inflated from 2D networks and pretrained\non ImageNet, could be a promising way for spatial and temporal representation\nlearning. However, as for model/computation complexity, 3D CNNs are much more\nexpensive than 2D CNNs and prone to overfit. We seek a balance between speed\nand accuracy by building an effective and efficient video classification system\nthrough systematic exploration of critical network design choices. In\nparticular, we show that it is possible to replace many of the 3D convolutions\nby low-cost 2D convolutions. Rather surprisingly, best result (in both speed\nand accuracy) is achieved when replacing the 3D convolutions at the bottom of\nthe network, suggesting that temporal representation learning on high-level\nsemantic features is more useful. Our conclusion generalizes to datasets with\nvery different properties. When combined with several other cost-effective\ndesigns including separable spatial/temporal convolution and feature gating,\nour system results in an effective video classification system that that\nproduces very competitive results on several action classification benchmarks\n(Kinetics, Something-something, UCF101 and HMDB), as well as two action\ndetection (localization) benchmarks (JHMDB and UCF101-24).","url_abs":"http://arxiv.org/abs/1712.04851v2","url_pdf":"http://arxiv.org/pdf/1712.04851v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rethinking-spatiotemporal-feature-learning","repo_url":"https://github.com/3dperceptionlab/visual-wetlandbirds","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"rethinking-spatiotemporal-feature-learning","repo_url":"https://github.com/kylemin/S3D","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-detection","task_name":"Action Detection"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"video-classification","task_name":"Video Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"S3D-G (RGB+Flow, ImageNet pretrained)","rank_in_archive_order":144,"of":207,"metrics":{"Acc@1":"77.2","Acc@5":"93"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"S3D-G (RGB, ImageNet pretrained)","rank_in_archive_order":160,"of":207,"metrics":{"Acc@1":"74.7","Acc@5":"93.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"S3D-G (Flow, ImageNet pretrained)","rank_in_archive_order":184,"of":207,"metrics":{"Acc@1":"68","Acc@5":"87.6"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-600","task":"Action Classification","dataset":"Kinetics-600","model":"S3D-G (RGB+Flow)","rank_in_archive_order":55,"of":65,"metrics":{"Top-1 Accuracy":"78.6"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-600","task":"Action Classification","dataset":"Kinetics-600","model":"S3D-G (RGB)","rank_in_archive_order":58,"of":65,"metrics":{"Top-1 Accuracy":"76.6"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-600","task":"Action Classification","dataset":"Kinetics-600","model":"S3D-G (Flow)","rank_in_archive_order":63,"of":65,"metrics":{"Top-1 Accuracy":"69.7"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-hmdb-51","task":"Action Recognition","dataset":"HMDB-51","model":"S3D-G (ImageNet, Kinetics-400 pretrained)","rank_in_archive_order":37,"of":77,"metrics":{"Average accuracy of 3 splits":"75.9"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-something-1","task":"Action Recognition","dataset":"Something-Something V1","model":"S3D-G (ImageNet pretrained)","rank_in_archive_order":62,"of":74,"metrics":{"Top 1 Accuracy":"48.2","Top 5 Accuracy":"78.7"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-something-1","task":"Action Recognition","dataset":"Something-Something V1","model":"S3D","rank_in_archive_order":64,"of":74,"metrics":{"Top 1 Accuracy":"47.3","Top 5 Accuracy":"78.1"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-ucf101","task":"Action Recognition","dataset":"UCF101","model":"S3D-G (ImageNet, Kinetics-400 pretrained)","rank_in_archive_order":31,"of":91,"metrics":{"3-fold Accuracy":"96.8"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/1712.04851","atlas_url":"https://app.syntology.ai/?focus=1712.04851","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1712.04851"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1712.04851\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1712.04851"},"call":"get_concepts_for_paper(arxiv_id=\"1712.04851\")"}],"by_repo":[{"repo":"3dperceptionlab/visual-wetlandbirds","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"kylemin/S3D","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"c8d438327961fd2c04887c05b719172500e5f51077b81141e32185f07793a07b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}