{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/deep-word-embeddings-for-visual-speech","title":"Deep word embeddings for visual speech recognition","arxiv_id":"1710.11201","date":"2017-10-30","proceeding":null,"authors":["Themos Stafylakis","Georgios Tzimiropoulos"],"abstract":"In this paper we present a deep learning architecture for extracting word\nembeddings for visual speech recognition. The embeddings summarize the\ninformation of the mouth region that is relevant to the problem of word\nrecognition, while suppressing other types of variability such as speaker, pose\nand illumination. The system is comprised of a spatiotemporal convolutional\nlayer, a Residual Network and bidirectional LSTMs and is trained on the\nLipreading in-the-wild database. We first show that the proposed architecture\ngoes beyond state-of-the-art on closed-set word identification, by attaining\n11.92% error rate on a vocabulary of 500 words. We then examine the capacity of\nthe embeddings in modelling words unseen during training. We deploy\nProbabilistic Linear Discriminant Analysis (PLDA) to model the embeddings and\nperform low-shot learning experiments on words unseen during training. The\nexperiments demonstrate that word-level visual speech recognition is feasible\neven in cases where the target words are not included in the training set.","url_abs":"http://arxiv.org/abs/1710.11201v1","url_pdf":"http://arxiv.org/pdf/1710.11201v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-word-embeddings-for-visual-speech","repo_url":"https://github.com/tstafylakis/Lipreading-ResNet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"lipreading","task_name":"Lipreading"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"visual-speech-recognition","task_name":"Visual Speech Recognition"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"tstafylakis/Lipreading-ResNet","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"06f0ebda4284837d33103c76c44c28e9ad5a6c9f61418da0b097c233977276f7","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}