{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/zero-shot-keyword-spotting-for-visual-speech","title":"Zero-shot keyword spotting for visual speech recognition in-the-wild","arxiv_id":"1807.08469","date":"2018-07-23","proceeding":"ECCV 2018 9","authors":["Themos Stafylakis","Georgios Tzimiropoulos"],"abstract":"Visual keyword spotting (KWS) is the problem of estimating whether a text\nquery occurs in a given recording using only video information. This paper\nfocuses on visual KWS for words unseen during training, a real-world, practical\nsetting which so far has received no attention by the community. To this end,\nwe devise an end-to-end architecture comprising (a) a state-of-the-art visual\nfeature extractor based on spatiotemporal Residual Networks, (b) a\ngrapheme-to-phoneme model based on sequence-to-sequence neural networks, and\n(c) a stack of recurrent neural networks which learn how to correlate visual\nfeatures with the keyword representation. Different to prior works on KWS,\nwhich try to learn word representations merely from sequences of graphemes\n(i.e. letters), we propose the use of a grapheme-to-phoneme encoder-decoder\nmodel which learns how to map words to their pronunciation. We demonstrate that\nour system obtains very promising visual-only KWS results on the challenging\nLRS2 database, for keywords unseen during training. We also show that our\nsystem outperforms a baseline which addresses KWS via automatic speech\nrecognition (ASR), while it drastically improves over other recently proposed\nASR-free KWS methods.","url_abs":"http://arxiv.org/abs/1807.08469v2","url_pdf":"http://arxiv.org/pdf/1807.08469v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zero-shot-keyword-spotting-for-visual-speech","repo_url":"https://github.com/lilianemomeni/KWS-Net","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"automatic-speech-recognition-2","task_name":"Automatic Speech Recognition"},{"task_slug":"automatic-speech-recognition","task_name":"Automatic Speech Recognition (ASR)"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"keyword-spotting","task_name":"Keyword Spotting"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"visual-keyword-spotting","task_name":"Visual Keyword Spotting"},{"task_slug":"visual-speech-recognition","task_name":"Visual Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1807.08469","atlas_url":"https://app.syntology.ai/?focus=1807.08469","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1807.08469"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1807.08469\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1807.08469"},"call":"get_concepts_for_paper(arxiv_id=\"1807.08469\")"}],"by_repo":[{"repo":"lilianemomeni/KWS-Net","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"936446417b50727443dea0c56919b5c11cd270786c7352c0f9070289f9381531","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}