{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/end-to-end-dense-video-captioning-with-masked","title":"End-to-End Dense Video Captioning with Masked Transformer","arxiv_id":"1804.00819","date":"2018-04-03","proceeding":"CVPR 2018 4","authors":["Luowei Zhou","Yingbo Zhou","Jason J. Corso","Richard Socher","Caiming Xiong"],"abstract":"Dense video captioning aims to generate text descriptions for all events in\nan untrimmed video. This involves both detecting and describing events.\nTherefore, all previous methods on dense video captioning tackle this problem\nby building two models, i.e. an event proposal and a captioning model, for\nthese two sub-problems. The models are either trained separately or in\nalternation. This prevents direct influence of the language description to the\nevent proposal, which is important for generating accurate descriptions. To\naddress this problem, we propose an end-to-end transformer model for dense\nvideo captioning. The encoder encodes the video into appropriate\nrepresentations. The proposal decoder decodes from the encoding with different\nanchors to form video event proposals. The captioning decoder employs a masking\nnetwork to restrict its attention to the proposal event over the encoding\nfeature. This masking network converts the event proposal to a differentiable\nmask, which ensures the consistency between the proposal and captioning during\ntraining. In addition, our model employs a self-attention mechanism, which\nenables the use of efficient non-recurrent structure during encoding and leads\nto performance improvements. We demonstrate the effectiveness of this\nend-to-end model on ActivityNet Captions and YouCookII datasets, where we\nachieved 10.12 and 6.58 METEOR score, respectively.","url_abs":"http://arxiv.org/abs/1804.00819v1","url_pdf":"http://arxiv.org/pdf/1804.00819v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-dense-video-captioning-with-masked","repo_url":"https://github.com/salesforce/densecap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"dense-video-captioning","task_name":"Dense Video Captioning"},{"task_slug":"video-captioning","task_name":"Video Captioning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-captioning-on-youcook2","task":"Video Captioning","dataset":"YouCook2","model":"Zhou","rank_in_archive_order":12,"of":14,"metrics":{"BLEU-3":"7.53","BLEU-4":"4.38","CIDEr":"0.38","METEOR":"11.55","ROUGE-L":"27.44"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.00819","atlas_url":"https://app.syntology.ai/?focus=1804.00819","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1804.00819"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1804.00819\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1804.00819"},"call":"get_concepts_for_paper(arxiv_id=\"1804.00819\")"}],"by_repo":[{"repo":"salesforce/densecap","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2312.01575"],"harvested_for_calls":[{"arxiv_id":"2312.01575","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2312.01575"},"call":"get_harvested_code_for_paper(arxiv_id=\"2312.01575\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"80e840fac57cd788cafd989b62a3ef1c6f390cc06e7aaf93947a753ba1464bd5","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}