{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/multimodal-convolutional-neural-networks-for","title":"Multimodal Convolutional Neural Networks for Matching Image and Sentence","arxiv_id":"1504.06063","date":"2015-04-23","proceeding":"ICCV 2015 12","authors":["Lin Ma","Zhengdong Lu","Lifeng Shang","Hang Li"],"abstract":"In this paper, we propose multimodal convolutional neural networks (m-CNNs)\nfor matching image and sentence. Our m-CNN provides an end-to-end framework\nwith convolutional architectures to exploit image representation, word\ncomposition, and the matching relations between the two modalities. More\nspecifically, it consists of one image CNN encoding the image content, and one\nmatching CNN learning the joint representation of image and sentence. The\nmatching CNN composes words to different semantic fragments and learns the\ninter-modal relations between image and the composed fragments at different\nlevels, thus fully exploit the matching relations between image and sentence.\nExperimental results on benchmark databases of bidirectional image and sentence\nretrieval demonstrate that the proposed m-CNNs can effectively capture the\ninformation necessary for image and sentence matching. Specifically, our\nproposed m-CNNs for bidirectional image and sentence retrieval on Flickr30K and\nMicrosoft COCO databases achieve the state-of-the-art performances.","url_abs":"http://arxiv.org/abs/1504.06063v5","url_pdf":"http://arxiv.org/pdf/1504.06063v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multimodal-convolutional-neural-networks-for","repo_url":"https://github.com/2281123066/visual-semantic-embedding","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"multimodal-convolutional-neural-networks-for","repo_url":"https://github.com/ryankiros/visual-semantic-embedding","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"multimodal-convolutional-neural-networks-for","repo_url":"https://github.com/2023-MindSpore-1/ms-code-18/tree/main/MCNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-retrieval","task_name":"Sentence Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-retrieval-on-flickr30k-1k-test","task":"Image Retrieval","dataset":"Flickr30K 1K test","model":"mCNN","rank_in_archive_order":16,"of":18,"metrics":{"R@1":"26.2","R@10":"69.6","R@5":"56.3"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1504.06063","atlas_url":"https://app.syntology.ai/?focus=1504.06063","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1504.06063"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1504.06063\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1504.06063"},"call":"get_concepts_for_paper(arxiv_id=\"1504.06063\")"}],"by_repo":[{"repo":"2281123066/visual-semantic-embedding","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"ryankiros/visual-semantic-embedding","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"2023-MindSpore-1/ms-code-18","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"56af3487fc69d838ec2f32245c52bc3d368cedfbe6d284707b856af88b5254ba","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}