{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/huse-hierarchical-universal-semantic","title":"HUSE: Hierarchical Universal Semantic Embeddings","arxiv_id":"1911.05978","date":"2019-11-14","proceeding":null,"authors":["Pradyumna Narayana","Aniket Pednekar","Abishek Krishnamoorthy","Kazoo Sone","Sugato Basu"],"abstract":"There is a recent surge of interest in cross-modal representation learning corresponding to images and text. The main challenge lies in mapping images and text to a shared latent space where the embeddings corresponding to a similar semantic concept lie closer to each other than the embeddings corresponding to different semantic concepts, irrespective of the modality. Ranking losses are commonly used to create such shared latent space -- however, they do not impose any constraints on inter-class relationships resulting in neighboring clusters to be completely unrelated. The works in the domain of visual semantic embeddings address this problem by first constructing a semantic embedding space based on some external knowledge and projecting image embeddings onto this fixed semantic embedding space. These works are confined only to image domain and constraining the embeddings to a fixed space adds additional burden on learning. This paper proposes a novel method, HUSE, to learn cross-modal representation with semantic information. HUSE learns a shared latent space where the distance between any two universal embeddings is similar to the distance between their corresponding class embeddings in the semantic embedding space. HUSE also uses a classification objective with a shared classification layer to make sure that the image and text embeddings are in the same shared latent space. Experiments on UPMC Food-101 show our method outperforms previous state-of-the-art on retrieval, hierarchical precision and classification results.","url_abs":"https://arxiv.org/abs/1911.05978v1","url_pdf":"https://arxiv.org/pdf/1911.05978v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/1sh1vam/E-commerce-products-classification-using-images-and-text","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/AnkushMalaker/HUSE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/amanjain1397/HUSE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/arpytanshu/HUSE-PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/gupta-keshav/HUSE-TF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/guramritpalsaggu/HUSE-Tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/harshitsinghai77/hierarchical-universal-semantic-embeddings","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"huse-hierarchical-universal-semantic","repo_url":"https://github.com/vijaynchakole/HUSE-Hierarchical-Universal-Semantic-Embeddings","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1911.05978","atlas_url":"https://app.syntology.ai/?focus=1911.05978","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1911.05978"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1911.05978\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1911.05978"},"call":"get_concepts_for_paper(arxiv_id=\"1911.05978\")"}],"by_repo":[{"repo":"1sh1vam/E-commerce-products-classification-using-images-and-text","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"AnkushMalaker/HUSE","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"amanjain1397/HUSE","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"arpytanshu/HUSE-PyTorch","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"gupta-keshav/HUSE-TF","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"guramritpalsaggu/HUSE-Tensorflow","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"harshitsinghai77/hierarchical-universal-semantic-embeddings","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"vijaynchakole/HUSE-Hierarchical-Universal-Semantic-Embeddings","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"7bc797a530f2d273977ba216c39c3cfce3b93691ab68f1054d409fc2de8960ce","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}