{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/dual-path-convolutional-image-text-embedding","title":"Dual-Path Convolutional Image-Text Embeddings with Instance Loss","arxiv_id":"1711.05535","date":"2017-11-15","proceeding":null,"authors":["Zhedong Zheng","Liang Zheng","Michael Garrett","Yi Yang","Mingliang Xu","Yi-Dong Shen"],"abstract":"Matching images and sentences demands a fine understanding of both modalities. In this paper, we propose a new system to discriminatively embed the image and text to a shared visual-textual space. In this field, most existing works apply the ranking loss to pull the positive image / text pairs close and push the negative pairs apart from each other. However, directly deploying the ranking loss is hard for network learning, since it starts from the two heterogeneous features to build inter-modal relationship. To address this problem, we propose the instance loss which explicitly considers the intra-modal data distribution. It is based on an unsupervised assumption that each image / text group can be viewed as a class. So the network can learn the fine granularity from every image/text group. The experiment shows that the instance loss offers better weight initialization for the ranking loss, so that more discriminative embeddings can be learned. Besides, existing works usually apply the off-the-shelf features, i.e., word2vec and fixed visual feature. So in a minor contribution, this paper constructs an end-to-end dual-path convolutional network to learn the image and text representations. End-to-end learning allows the system to directly learn from the data and fully utilize the supervision. On two generic retrieval datasets (Flickr30k and MSCOCO), experiments demonstrate that our method yields competitive accuracy compared to state-of-the-art methods. Moreover, in language based person retrieval, we improve the state of the art by a large margin. The code has been made publicly available.","url_abs":"https://arxiv.org/abs/1711.05535v4","url_pdf":"https://arxiv.org/pdf/1711.05535v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dual-path-convolutional-image-text-embedding","repo_url":"https://github.com/layumi/Image-Text-Embedding","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"dual-path-convolutional-image-text-embedding","repo_url":"https://github.com/pshroff04/Dual_Path_CNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"content-based-image-retrieval","task_name":"Content-Based Image Retrieval"},{"task_slug":"cross-modal-retrieval","task_name":"Cross-Modal Retrieval"},{"task_slug":"nlp-based-person-retrival-1","task_name":"NLP based Person Retrival"},{"task_slug":"person-retrieval","task_name":"Person Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"nlp-based-person-retrival","task_name":"Text based Person Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/cross-modal-retrieval-on-cuhk-pedes","task":"Cross-Modal Retrieval","dataset":"CUHK-PEDES","model":"Dual Path","rank_in_archive_order":1,"of":1,"metrics":{"Text-to-image Medr":"2"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-flickr30k","task":"Cross-Modal Retrieval","dataset":"Flickr30k","model":"Dual-Path\n  (ResNet)","rank_in_archive_order":22,"of":27,"metrics":{"Image-to-text R@1":"55.6","Image-to-text R@5":"81.9"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-flickr30k","task":"Cross-Modal Retrieval","dataset":"Flickr30k","model":"Dual-Path (ResNet)","rank_in_archive_order":26,"of":27,"metrics":{"Image-to-text R@10":"89.5","Text-to-image R@1":"39.1","Text-to-image R@10":"80.9","Text-to-image R@5":"69.2"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-mscoco-1k","task":"Cross-Modal Retrieval","dataset":"MSCOCO-1k","model":"Dual-path CNN","rank_in_archive_order":2,"of":2,"metrics":{"Image-to-text R@1":"41.2","Text-to-image R@1":"25.3"},"uses_additional_data":false},{"leaderboard":"/sota/nlp-based-person-retrival-on-cuhk-pedes","task":"Text based Person Retrieval","dataset":"CUHK-PEDES","model":"Dual Path","rank_in_archive_order":19,"of":21,"metrics":{"R@1":"44.4","R@10":"75.07","R@5":"66.26"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1711.05535","atlas_url":"https://app.syntology.ai/?focus=1711.05535","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1711.05535"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1711.05535\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1711.05535"},"call":"get_concepts_for_paper(arxiv_id=\"1711.05535\")"}],"by_repo":[{"repo":"layumi/Image-Text-Embedding","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"pshroff04/Dual_Path_CNN","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"26f98df802b91a136c28bb1b3cf30a90217ac5c4a02293d9a581ad0c85de1295","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}