{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/two-stream-network-for-sign-language","title":"Two-Stream Network for Sign Language Recognition and Translation","arxiv_id":"2211.01367","date":"2022-11-02","proceeding":null,"authors":["Yutong Chen","Ronglai Zuo","Fangyun Wei","Yu Wu","Shujie Liu","Brian Mak"],"abstract":"Sign languages are visual languages using manual articulations and non-manual elements to convey information. For sign language recognition and translation, the majority of existing approaches directly encode RGB videos into hidden representations. RGB videos, however, are raw signals with substantial visual redundancy, leading the encoder to overlook the key information for sign language understanding. To mitigate this problem and better incorporate domain knowledge, such as handshape and body movement, we introduce a dual visual encoder containing two separate streams to model both the raw videos and the keypoint sequences generated by an off-the-shelf keypoint estimator. To make the two streams interact with each other, we explore a variety of techniques, including bidirectional lateral connection, sign pyramid network with auxiliary supervision, and frame-level self-distillation. The resulting model is called TwoStream-SLR, which is competent for sign language recognition (SLR). TwoStream-SLR is extended to a sign language translation (SLT) model, TwoStream-SLT, by simply attaching an extra translation network. Experimentally, our TwoStream-SLR and TwoStream-SLT achieve state-of-the-art performance on SLR and SLT tasks across a series of datasets including Phoenix-2014, Phoenix-2014T, and CSL-Daily. Code and models are available at: https://github.com/FangyunWei/SLRT.","url_abs":"https://arxiv.org/abs/2211.01367v2","url_pdf":"https://arxiv.org/pdf/2211.01367v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"two-stream-network-for-sign-language","repo_url":"https://github.com/FangyunWei/SLRT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"sign-language-recognition","task_name":"Sign Language Recognition"},{"task_slug":"sign-language-translation","task_name":"Sign Language Translation"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"two","task_name":"Vocal Bursts Valence Prediction"}],"methods":[{"method_slug":"slr","method_name":"SLR"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sign-language-recognition-on-csl-daily","task":"Sign Language Recognition","dataset":"CSL-Daily","model":"TwoStream-SLR","rank_in_archive_order":2,"of":14,"metrics":{"Word Error Rate (WER)":"25.3"},"uses_additional_data":false},{"leaderboard":"/sota/sign-language-recognition-on-rwth-phoenix","task":"Sign Language Recognition","dataset":"RWTH-PHOENIX-Weather 2014","model":"TwoStream-SLR","rank_in_archive_order":2,"of":22,"metrics":{"Word Error Rate (WER)":"18.4"},"uses_additional_data":false},{"leaderboard":"/sota/sign-language-recognition-on-rwth-phoenix-1","task":"Sign Language Recognition","dataset":"RWTH-PHOENIX-Weather 2014 T","model":"TwoStream-SLR","rank_in_archive_order":2,"of":15,"metrics":{"Word Error Rate (WER)":"19.3"},"uses_additional_data":false},{"leaderboard":"/sota/sign-language-translation-on-csl-daily","task":"Sign Language Translation","dataset":"CSL-Daily","model":"TwoStream-SLT","rank_in_archive_order":1,"of":8,"metrics":{"BLEU-4":"25.79"},"uses_additional_data":false},{"leaderboard":"/sota/sign-language-translation-on-rwth-phoenix","task":"Sign Language Translation","dataset":"RWTH-PHOENIX-Weather 2014 T","model":"TwoStream-SLT","rank_in_archive_order":2,"of":11,"metrics":{"BLEU-4":"28.95"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2211.01367","atlas_url":"https://app.syntology.ai/?focus=2211.01367","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2211.01367"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2211.01367\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2211.01367"},"call":"get_concepts_for_paper(arxiv_id=\"2211.01367\")"}],"by_repo":[{"repo":"FangyunWei/SLRT","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2401.05336"],"harvested_for_calls":[{"arxiv_id":"2401.05336","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2401.05336"},"call":"get_harvested_code_for_paper(arxiv_id=\"2401.05336\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"d1b2cd9a1cda001907e691ebd8d2698225082b1ba247642a92495e0450e9773b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}