{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/seqtrack-sequence-to-sequence-learning-for","title":"Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking","arxiv_id":"2304.14394","date":"2023-04-27","proceeding":"CVPR 2023 1","authors":["Xin Chen","Ben Kang","Jiawen Zhu","Dong Wang","Houwen Peng","Huchuan Lu"],"abstract":"In this paper, we introduce a new sequence-to-sequence learning framework for RGB-based and multi-modal object tracking. First, we present SeqTrack for RGB-based tracking. It casts visual tracking as a sequence generation task, forecasting object bounding boxes in an autoregressive manner. This differs from previous trackers, which depend on the design of intricate head networks, such as classification and regression heads. SeqTrack employs a basic encoder-decoder transformer architecture. The encoder utilizes a bidirectional transformer for feature extraction, while the decoder generates bounding box sequences autoregressively using a causal transformer. The loss function is a plain cross-entropy. Second, we introduce SeqTrackv2, a unified sequence-to-sequence framework for multi-modal tracking tasks. Expanding upon SeqTrack, SeqTrackv2 integrates a unified interface for auxiliary modalities and a set of task-prompt tokens to specify the task. This enables it to manage multi-modal tracking tasks using a unified model and parameter set. This sequence learning paradigm not only simplifies the tracking framework, but also showcases superior performance across 14 challenging benchmarks spanning five single- and multi-modal tracking tasks. The code and models are available at https://github.com/chenxin-dlut/SeqTrackv2.","url_abs":"https://arxiv.org/abs/2304.14394v3","url_pdf":"https://arxiv.org/pdf/2304.14394v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"seqtrack-sequence-to-sequence-learning-for","repo_url":"https://github.com/chenxin-dlut/seqtrackv2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"rgb-t-tracking","task_name":"Rgb-T Tracking"},{"task_slug":"visual-object-tracking","task_name":"Visual Object Tracking"},{"task_slug":"visual-tracking","task_name":"Visual Tracking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/rgb-t-tracking-on-lasher","task":"Rgb-T Tracking","dataset":"LasHeR","model":"SeqTrackv2-L384","rank_in_archive_order":3,"of":39,"metrics":{"Precision":"76.7","Success":"61.0"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-t-tracking-on-lasher","task":"Rgb-T Tracking","dataset":"LasHeR","model":"SeqTrackv2-L256","rank_in_archive_order":9,"of":39,"metrics":{"Precision":"74.1","Success":"58.8"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-t-tracking-on-lasher","task":"Rgb-T Tracking","dataset":"LasHeR","model":"SeqTrackv2-B384","rank_in_archive_order":18,"of":39,"metrics":{"Precision":"71.5","Success":"56.2"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-t-tracking-on-lasher","task":"Rgb-T Tracking","dataset":"LasHeR","model":"SeqTrackv2-B256","rank_in_archive_order":21,"of":39,"metrics":{"Precision":"70.4","Success":"55.8"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-t-tracking-on-rgbt234","task":"Rgb-T Tracking","dataset":"RGBT234","model":"SeqTrackv2-L256","rank_in_archive_order":3,"of":42,"metrics":{"Precision":"92.3","Success":"68.5"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-t-tracking-on-rgbt234","task":"Rgb-T Tracking","dataset":"RGBT234","model":"SeqTrackv2-L384","rank_in_archive_order":5,"of":42,"metrics":{"Precision":"91.3","Success":"68.0"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-t-tracking-on-rgbt234","task":"Rgb-T Tracking","dataset":"RGBT234","model":"SeqTrackv2-B384","rank_in_archive_order":10,"of":42,"metrics":{"Precision":"90.0","Success":"66.3"},"uses_additional_data":false},{"leaderboard":"/sota/rgb-t-tracking-on-rgbt234","task":"Rgb-T Tracking","dataset":"RGBT234","model":"SeqTrackv2-B256","rank_in_archive_order":20,"of":42,"metrics":{"Precision":"88.0","Success":"64.7"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-got-10k","task":"Visual Object Tracking","dataset":"GOT-10k","model":"SeqTrack-L384","rank_in_archive_order":22,"of":42,"metrics":{"Average Overlap":"74.8","Success Rate 0.5":"81.9","Success Rate 0.75":"72.2"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-lasot","task":"Visual Object Tracking","dataset":"LaSOT","model":"SeqTrack-L384","rank_in_archive_order":18,"of":46,"metrics":{"AUC":"72.5","Normalized Precision":"81.5","Precision":"79.3"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-lasot-ext","task":"Visual Object Tracking","dataset":"LaSOT-ext","model":"SeqTrack-L384","rank_in_archive_order":15,"of":18,"metrics":{"AUC":"50.7","Normalized Precision":"61.6","Precision":"57.5"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-needforspeed","task":"Visual Object Tracking","dataset":"NeedForSpeed","model":"SeqTrack-L384","rank_in_archive_order":9,"of":10,"metrics":{"AUC":"0.662"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-otb-2015","task":"Visual Object Tracking","dataset":"OTB-2015","model":"SeqTrack-L384","rank_in_archive_order":12,"of":18,"metrics":{"AUC":"0.683"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-tnl2k","task":"Visual Object Tracking","dataset":"TNL2K","model":"SeqTrack-L384","rank_in_archive_order":13,"of":16,"metrics":{"AUC":"57.8"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-trackingnet","task":"Visual Object Tracking","dataset":"TrackingNet","model":"SeqTrack-L384","rank_in_archive_order":12,"of":40,"metrics":{"Accuracy":"85.5","Normalized Precision":"89.8","Precision":"85.8"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-uav123","task":"Visual Object Tracking","dataset":"UAV123","model":"SeqTrack-L384","rank_in_archive_order":12,"of":16,"metrics":{"AUC":"0.685"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2304.14394","atlas_url":"https://app.syntology.ai/?focus=2304.14394","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2304.14394"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2304.14394\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2304.14394"},"call":"get_concepts_for_paper(arxiv_id=\"2304.14394\")"}],"by_repo":[{"repo":"chenxin-dlut/seqtrackv2","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"e304a1accecd586b71ae855333d163755e02c724aebca5e93f8993486db6a743","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}