{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/end-to-end-concept-word-detection-for-video","title":"End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answering","arxiv_id":"1610.02947","date":"2016-10-10","proceeding":"CVPR 2017 7","authors":["Youngjae Yu","Hyungjin Ko","Jongwook Choi","Gunhee Kim"],"abstract":"We propose a high-level concept word detector that can be integrated with any\nvideo-to-language models. It takes a video as input and generates a list of\nconcept words as useful semantic priors for language generation models. The\nproposed word detector has two important properties. First, it does not require\nany external knowledge sources for training. Second, the proposed word detector\nis trainable in an end-to-end manner jointly with any video-to-language models.\nTo maximize the values of detected words, we also develop a semantic attention\nmechanism that selectively focuses on the detected concept words and fuse them\nwith the word encoding and decoding in the language model. In order to\ndemonstrate that the proposed approach indeed improves the performance of\nmultiple video-to-language tasks, we participate in four tasks of LSMDC 2016.\nOur approach achieves the best accuracies in three of them, including\nfill-in-the-blank, multiple-choice test, and movie retrieval. We also attain\ncomparable performance for the other task, movie description.","url_abs":"http://arxiv.org/abs/1610.02947v3","url_pdf":"http://arxiv.org/pdf/1610.02947v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"multiple-choice","task_name":"Multiple-choice"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"video-captioning","task_name":"Video Captioning"},{"task_slug":"video-retrieval","task_name":"Video Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-retrieval-on-lsmdc","task":"Video Retrieval","dataset":"LSMDC","model":"CT-SAN","rank_in_archive_order":37,"of":38,"metrics":{"text-to-video Median Rank":"46","text-to-video R@1":"5.1","text-to-video R@10":"25.2","text-to-video R@5":"16.3"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1610.02947","atlas_url":"https://app.syntology.ai/?focus=1610.02947","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1610.02947"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1610.02947\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1610.02947"},"call":"get_concepts_for_paper(arxiv_id=\"1610.02947\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"0efb3b04cd4030009d46568349d6efc9726cf3b4eaf109e6530547ae46a3126e","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}