{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/feta-towards-specializing-foundation-models","title":"FETA: Towards Specializing Foundation Models for Expert Task Applications","arxiv_id":"2209.03648","date":"2022-09-08","proceeding":null,"authors":["Amit Alfassy","Assaf Arbelle","Oshri Halimi","Sivan Harary","Roei Herzig","Eli Schwartz","Rameswar Panda","Michele Dolfi","Christoph Auer","Kate Saenko","PeterW. J. Staar","Rogerio Feris","Leonid Karlinsky"],"abstract":"Foundation Models (FMs) have demonstrated unprecedented capabilities including zero-shot learning, high fidelity data synthesis, and out of domain generalization. However, as we show in this paper, FMs still have poor out-of-the-box performance on expert tasks (e.g. retrieval of car manuals technical illustrations from language queries), data for which is either unseen or belonging to a long-tail part of the data distribution of the huge datasets used for FM pre-training. This underlines the necessity to explicitly evaluate and finetune FMs on such expert tasks, arguably ones that appear the most in practical real-world applications. In this paper, we propose a first of its kind FETA benchmark built around the task of teaching FMs to understand technical documentation, via learning to match their graphical illustrations to corresponding language descriptions. Our FETA benchmark focuses on text-to-image and image-to-text retrieval in public car manuals and sales catalogue brochures. FETA is equipped with a procedure for completely automatic annotation extraction (code would be released upon acceptance), allowing easy extension of FETA to more documentation types and application domains in the future. Our automatic annotation leads to an automated performance metric shown to be consistent with metrics computed on human-curated annotations (also released). We provide multiple baselines and analysis of popular FMs on FETA leading to several interesting findings that we believe would be very valuable to the FM community, paving the way towards real-world application of FMs for practical expert tasks currently 'overlooked' by standard benchmarks focusing on common objects.","url_abs":"https://arxiv.org/abs/2209.03648v2","url_pdf":"https://arxiv.org/pdf/2209.03648v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"feta-towards-specializing-foundation-models","repo_url":"https://github.com/alfassy/FETA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"image-to-text","task_name":"Image to text"},{"task_slug":"image-text-retrieval","task_name":"Image-text Retrieval"},{"task_slug":"image-to-text-retrieval","task_name":"Image-to-Text Retrieval"},{"task_slug":"one-shot-learning","task_name":"One-Shot Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-retrieval","task_name":"Text Retrieval"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":"zero-shot-image-retrieval","task_name":"Zero-shot Image Retrieval"}],"methods":[],"datasets_introduced":[{"slug":"feta-car-manuals","name":"FETA Car-Manuals","full_name":"FETA Car-Manuals dataset, image-text retrieval for foundation models' expert data performance."},{"slug":"feta-ikea","name":"FETA IKEA","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-retrieval-on-feta-car-manuals","task":"Image Retrieval","dataset":"FETA Car-Manuals","model":"FETA's CLIP-MIL (Many-Shot Image-to-text)","rank_in_archive_order":1,"of":1,"metrics":{"R@1":"29","R@10":"72.6","R@5":"59.9"},"uses_additional_data":false},{"leaderboard":"/sota/image-to-text-retrieval-on-feta-car-manuals","task":"Image-to-Text Retrieval","dataset":"FETA Car-Manuals","model":"FETA's CLIP-MIL (Many-Shot Image-to-text)","rank_in_archive_order":1,"of":1,"metrics":{"R@1":"35.5","R@10":"67","R@5":"58.3"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2209.03648","atlas_url":"https://app.syntology.ai/?focus=2209.03648","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2209.03648"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2209.03648\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2209.03648"},"call":"get_concepts_for_paper(arxiv_id=\"2209.03648\")"}],"by_repo":[{"repo":"alfassy/FETA","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"3a05736c341cc2053a5a9090165d4b0a851c09b713658c1b5e97e19287235e54","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}