{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/avgzslnet-audio-visual-generalized-zero-shot","title":"AVGZSLNet: Audio-Visual Generalized Zero-Shot Learning by Reconstructing Label Features from Multi-Modal Embeddings","arxiv_id":"2005.13402","date":"2020-05-27","proceeding":null,"authors":["Pratik Mazumder","Pravendra Singh","Kranti Kumar Parida","Vinay P. Namboodiri"],"abstract":"In this paper, we propose a novel approach for generalized zero-shot learning in a multi-modal setting, where we have novel classes of audio/video during testing that are not seen during training. We use the semantic relatedness of text embeddings as a means for zero-shot learning by aligning audio and video embeddings with the corresponding class label text feature space. Our approach uses a cross-modal decoder and a composite triplet loss. The cross-modal decoder enforces a constraint that the class label text features can be reconstructed from the audio and video embeddings of data points. This helps the audio and video embeddings to move closer to the class label text embedding. The composite triplet loss makes use of the audio, video, and text embeddings. It helps bring the embeddings from the same class closer and push away the embeddings from different classes in a multi-modal setting. This helps the network to perform better on the multi-modal zero-shot learning task. Importantly, our multi-modal zero-shot learning approach works even if a modality is missing at test time. We test our approach on the generalized zero-shot classification and retrieval tasks and show that our approach outperforms other models in the presence of a single modality as well as in the presence of multiple modalities. We validate our approach by comparing it with previous approaches and using various ablations.","url_abs":"https://arxiv.org/abs/2005.13402v3","url_pdf":"https://arxiv.org/pdf/2005.13402v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"gzsl-video-classification","task_name":"GZSL Video Classification"},{"task_slug":"generalized-zero-shot-learning","task_name":"Generalized Zero-Shot Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":null,"task_name":"Triplet"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":null,"task_name":"zero-shot-classification"}],"methods":[{"method_slug":"triplet-loss","method_name":"Triplet Loss"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/gzsl-video-classification-on-activitynet-gzsl-1","task":"GZSL Video Classification","dataset":"ActivityNet-GZSL(main)","model":"AVGZSLNet","rank_in_archive_order":6,"of":7,"metrics":{"HM":"6.44","ZSL":"5.40"},"uses_additional_data":false},{"leaderboard":"/sota/gzsl-video-classification-on-ucf-gzsl-main","task":"GZSL Video Classification","dataset":"UCF-GZSL(main)","model":"AVGZSLNet","rank_in_archive_order":6,"of":7,"metrics":{"HM":"18.05","ZSL":"13.65"},"uses_additional_data":false},{"leaderboard":"/sota/gzsl-video-classification-on-vggsound-gzsl-1","task":"GZSL Video Classification","dataset":"VGGSound-GZSL(main)","model":"AVGZSLNet","rank_in_archive_order":6,"of":7,"metrics":{"HM":"5.83","ZSL":"5.28"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2005.13402","atlas_url":"https://app.syntology.ai/?focus=2005.13402","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2005.13402"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2005.13402\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2005.13402"},"call":"get_concepts_for_paper(arxiv_id=\"2005.13402\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"ce9bd5023f7992a9dd6380a41e0b89d99a67a41811cf294373ba3a2531e4cf6f","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}