{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/imagine-this-scripts-to-compositions-to","title":"Imagine This! Scripts to Compositions to Videos","arxiv_id":"1804.03608","date":"2018-04-10","proceeding":"ECCV 2018 9","authors":["Tanmay Gupta","Dustin Schwenk","Ali Farhadi","Derek Hoiem","Aniruddha Kembhavi"],"abstract":"Imagining a scene described in natural language with realistic layout and\nappearance of entities is the ultimate test of spatial, visual, and semantic\nworld knowledge. Towards this goal, we present the Composition, Retrieval, and\nFusion Network (CRAFT), a model capable of learning this knowledge from\nvideo-caption data and applying it while generating videos from novel captions.\nCRAFT explicitly predicts a temporal-layout of mentioned entities (characters\nand objects), retrieves spatio-temporal entity segments from a video database\nand fuses them to generate scene videos. Our contributions include sequential\ntraining of components of CRAFT while jointly modeling layout and appearances,\nand losses that encourage learning compositional representations for retrieval.\nWe evaluate CRAFT on semantic fidelity to caption, composition consistency, and\nvisual quality. CRAFT outperforms direct pixel generation approaches and\ngeneralizes well to unseen captions and to unseen video databases with no text\nannotations. We demonstrate CRAFT on FLINTSTONES, a new richly annotated\nvideo-caption dataset with over 25000 videos. For a glimpse of videos generated\nby CRAFT, see https://youtu.be/688Vv86n0z8.","url_abs":"http://arxiv.org/abs/1804.03608v1","url_pdf":"http://arxiv.org/pdf/1804.03608v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"imagine-this-scripts-to-compositions-to","repo_url":"https://github.com/Leiree/Paginas-importantes","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"imagine-this-scripts-to-compositions-to","repo_url":"https://github.com/ghost1412/CRAFT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"imagine-this-scripts-to-compositions-to","repo_url":"https://github.com/ishandutta2007/Craft","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"imagine-this-scripts-to-compositions-to","repo_url":"https://github.com/ubc-vision/make-a-story","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"imagine-this-scripts-to-compositions-to","repo_url":"https://github.com/xiaoqian-shen/StoryGPT-V","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.03608","atlas_url":"https://app.syntology.ai/?focus=1804.03608","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1804.03608"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1804.03608\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1804.03608"},"call":"get_concepts_for_paper(arxiv_id=\"1804.03608\")"}],"by_repo":[{"repo":"Leiree/Paginas-importantes","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"ghost1412/CRAFT","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"ishandutta2007/Craft","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"ubc-vision/make-a-story","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2211.13319"],"harvested_for_calls":[{"arxiv_id":"2211.13319","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2211.13319"},"call":"get_harvested_code_for_paper(arxiv_id=\"2211.13319\")"}]},{"repo":"xiaoqian-shen/StoryGPT-V","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2312.02252"],"harvested_for_calls":[{"arxiv_id":"2312.02252","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2312.02252"},"call":"get_harvested_code_for_paper(arxiv_id=\"2312.02252\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"12130182bda6473d290c5ea42c709dcc191d6850f96461aa7e3b435c47a0ca2b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}