{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/task/multimodal-deep-learning","name":"Multimodal Deep Learning","slug":"multimodal-deep-learning","description_markdown":"**Multimodal deep learning** is a type of deep learning that combines information from multiple modalities, such as text, image, audio, and video, to make more accurate and comprehensive predictions. It involves training deep neural networks on data that includes multiple types of information and using the network to make predictions based on this combined data.\r\n\r\nOne of the key challenges in multimodal deep learning is how to effectively combine information from multiple modalities. This can be done using a variety of techniques, such as fusing the features extracted from each modality, or using attention mechanisms to weight the contribution of each modality based on its importance for the task at hand.\r\n\r\nMultimodal deep learning has many applications, including image captioning, speech recognition, natural language processing, and autonomous vehicles. By combining information from multiple modalities, multimodal deep learning can improve the accuracy and robustness of models, enabling them to perform better in real-world scenarios where multiple types of information are present.","categories":[{"name":"Methodology","url":"/area/methodology"},{"name":"Natural Language Processing","url":"/area/natural-language-processing"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":213,"papers_with_code":97,"benchmarks":1,"benchmark_tables_in_archive":1,"benchmark_tables_shown":1,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":22,"subtasks":1,"parent_tasks":0},"benchmarks":[{"leaderboard":"/sota/multimodal-deep-learning-on-cub-200-2011","slug":"multimodal-deep-learning-on-cub-200-2011","dataset":"CUB-200-2011","dataset_url":"/dataset/cub-200-2011","rows_in_archive":1,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"Two Branch Network (Text - Bert + Image - Nts-Net)","paper_title":"Are These Birds Similar: Learning Branched Networks for Fine-grained Representations","paper_url":"/paper/are-these-birds-similar-learning-branched","paper_date":"2020-01-16","arxiv_id":null,"code_links":[{"title":"nicolalandro/ntsnet-cub200","url":"https://github.com/nicolalandro/ntsnet-cub200"},{"title":"Mind23-2/MindCode-101","url":"https://github.com/Mind23-2/MindCode-101/tree/main/ntsnet"},{"title":"artelabsuper/ivcnz-2019-bird","url":"https://gitlab.com/artelabsuper/ivcnz-2019-bird"}],"syntology":null}}],"datasets":[{"url":"/dataset/cub-200-2011","name":"CUB-200-2011","full_name":"Caltech-UCSD Birds-200-2011","num_papers_in_archive":2235},{"url":"/dataset/scienceqa","name":"ScienceQA","full_name":"Science Question Answering","num_papers_in_archive":339},{"url":"/dataset/quilt-1m","name":"QUILT-1M","full_name":"","num_papers_in_archive":17},{"url":"/dataset/olives-dataset","name":"OLIVES Dataset","full_name":"Ophthalmic Labels for Investigating Visual Eye Semantics","num_papers_in_archive":10},{"url":"/dataset/marine-video-kit","name":"MVK","full_name":"Marine Video Kit","num_papers_in_archive":9},{"url":"/dataset/bioscan-5m","name":"BIOSCAN-5M","full_name":"","num_papers_in_archive":4},{"url":"/dataset/mute","name":"MUTE","full_name":"Multimodal Bengali Hateful Memes Dataset","num_papers_in_archive":3},{"url":"/dataset/luma","name":"LUMA","full_name":"Learning from Uncertain and Multimodal Data","num_papers_in_archive":2},{"url":"/dataset/au-dataset-for-visuo-haptic-object","name":"AU Dataset for Visuo-Haptic Object Recognition for Robots","full_name":"","num_papers_in_archive":1},{"url":"/dataset/boombox","name":"Boombox","full_name":"","num_papers_in_archive":1},{"url":"/dataset/correlated-corrupted-dataset","name":"Correlated Corrupted Dataset","full_name":"CCD","num_papers_in_archive":1},{"url":"/dataset/gaze-cifar-10","name":"Gaze-CIFAR-10","full_name":"","num_papers_in_archive":1},{"url":"/dataset/gebid","name":"GeBiD","full_name":"Geometric shapes Bimodal Dataset","num_papers_in_archive":1},{"url":"/dataset/mminstruct-gpt4v","name":"MMInstruct-GPT4V","full_name":"MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity","num_papers_in_archive":1},{"url":"/dataset/multimodal-pisa","name":"Multimodal PISA","full_name":"Multimodal Piano Skills Assessment","num_papers_in_archive":1},{"url":"/dataset/rebus","name":"REBUS","full_name":"A Robust Evaluation Benchmark of Understanding Symbols","num_papers_in_archive":1},{"url":"/dataset/ucd","name":"Uncorrelated Corrupted Dataset","full_name":"UCD","num_papers_in_archive":1},{"url":"/dataset/webli","name":"WebLI","full_name":"Web Language Image","num_papers_in_archive":1},{"url":"/dataset/mimic-meme-dataset","name":"MIMIC Meme Dataset","full_name":"Misogyny Identification in Multimodal Internet Content in Hindi-English Code-Mix Language","num_papers_in_archive":0},{"url":"/dataset/mudestreda","name":"Mudestreda","full_name":"Mudestreda Multimodal Device State Recognition Dataset","num_papers_in_archive":0},{"url":"/dataset/sf-tl54","name":"SF-TL54: A Thermal Facial Landmark Dataset with Visual Pairs","full_name":"SF-TL54: A Thermal Facial Landmark Dataset with Visual Pairs","num_papers_in_archive":0},{"url":"/dataset/vidimu-multimodal-video-and-imu-kinematic","name":"VIDIMU: Multimodal video and IMU kinematic dataset on daily life activities using affordable devices","full_name":"https://zenodo.org/record/8210563","num_papers_in_archive":0}],"subtasks":[{"url":"/task/multimodal-text-and-image-classification","name":"Multimodal Text and Image Classification"}],"parent_tasks":[],"papers":{"order":"repositories listed in the archive (most first), then date (newest first), then slug; the archive holds no stars","population":"every paper the archive tags with this task (distinct papers; spam titles not listed). 'of' counts the papers listed; until 2026-09-25 it counted only the papers with a code link, and that number is of_with_code","page":1,"pages_in_order":3,"rows_per_page":100,"shown":100,"of":213,"of_with_code":97,"tagged_in_all":213,"counts":{"archive_papers_tagged":213,"with_a_code_link":97,"where_syntology_ran_a_sample":21,"not_listed_spam_title":0,"listed":213,"listed_where_code_ran":21,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":15,"every_run_a_failure_of_syntologys_instrument":6,"listed_with_a_run_with_no_instrument_failure":15,"listed_every_run_a_failure_of_syntologys_instrument":6,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"items":[{"url":"/paper/llama-adapter-efficient-fine-tuning-of","slug":"llama-adapter-efficient-fine-tuning-of","title":"LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention","date":"2023-03-28","arxiv_id":"2303.16199","repositories_listed":7,"syntology":null},{"url":"/paper/languagebind-extending-video-language","slug":"languagebind-extending-video-language","title":"LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment","date":"2023-10-03","arxiv_id":"2310.01852","repositories_listed":6,"syntology":{"n":14,"n_ran":10,"n_constructed":0,"n_ran_checked":9,"n_instrument":1,"n_unverified":4,"n_honours":3,"n_violates":0,"n_no_contract":6,"n_pointer_only":3,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 3 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/languagebind-extending-video-language#ran","syntology_url":"https://syntology.ai/paper/2310.01852","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01852"}},"official":{"repos":["pku-yuangroup/languagebind"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"url":"/paper/imagebind-one-embedding-space-to-bind-them","slug":"imagebind-one-embedding-space-to-bind-them","title":"ImageBind: One Embedding Space To Bind Them All","date":"2023-05-09","arxiv_id":"2305.05665","repositories_listed":3,"syntology":{"n":34,"n_ran":24,"n_constructed":14,"n_ran_checked":21,"n_instrument":3,"n_unverified":10,"n_honours":1,"n_violates":1,"n_no_contract":19,"n_pointer_only":32,"phrase":"24 ran (of which 14 constructed an object rather than computing a result; 21 with no instrument failure: 1 honoured, 1 violated, 19 with no contract checked; 3 where Syntology's instrument failed) · 10 unverified","sample_list":"/paper/imagebind-one-embedding-space-to-bind-them#ran","syntology_url":"https://syntology.ai/paper/2305.05665","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.05665"}},"official":{"repos":["facebookresearch/imagebind"],"state":"official (archive's flag): 18 ran","n_ran":18,"n_constructed":10,"n_ran_no_instrument_failure":15,"n_unverified":7,"ran_from_kinds":["listed","official"]}}},{"url":"/paper/aimotive-dataset-a-multimodal-dataset-for","slug":"aimotive-dataset-a-multimodal-dataset-for","title":"aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception","date":"2022-11-17","arxiv_id":"2211.09445","repositories_listed":3,"syntology":{"n":5,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/aimotive-dataset-a-multimodal-dataset-for#ran","syntology_url":"https://syntology.ai/paper/2211.09445","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.09445"}},"official":{"repos":["aimotive/aimotive-dataset-loader","aimotive/aimotive_dataset"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/supervised-video-summarization-via-multiple","slug":"supervised-video-summarization-via-multiple","title":"Supervised Video Summarization via Multiple Feature Sets with Parallel Attention","date":"2021-04-23","arxiv_id":"2104.11530","repositories_listed":3,"syntology":null},{"url":"/paper/are-these-birds-similar-learning-branched","slug":"are-these-birds-similar-learning-branched","title":"Are These Birds Similar: Learning Branched Networks for Fine-grained Representations","date":"2020-01-16","arxiv_id":null,"repositories_listed":3,"syntology":null},{"url":"/paper/multimodal-deep-networks-for-text-and-image","slug":"multimodal-deep-networks-for-text-and-image","title":"Multimodal deep networks for text and image-based document classification","date":"2019-07-15","arxiv_id":"1907.06370","repositories_listed":3,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/multimodal-deep-networks-for-text-and-image#ran","syntology_url":"https://syntology.ai/paper/1907.06370","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1907.06370"}},"official":{"repos":["Quicksign/ocrized-text-dataset"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/shapeworld-a-new-test-methodology-for","slug":"shapeworld-a-new-test-methodology-for","title":"ShapeWorld - A new test methodology for multimodal language understanding","date":"2017-04-14","arxiv_id":"1704.04517","repositories_listed":3,"syntology":null},{"url":"/paper/vitextvqa-a-large-scale-visual-question","slug":"vitextvqa-a-large-scale-visual-question","title":"ViTextVQA: A Large-Scale Visual Question Answering Dataset for Evaluating Vietnamese Text Comprehension in Images","date":"2024-04-16","arxiv_id":"2404.10652","repositories_listed":2,"syntology":null},{"url":"/paper/healnet-hybrid-multi-modal-fusion-for","slug":"healnet-hybrid-multi-modal-fusion-for","title":"HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data","date":"2023-11-15","arxiv_id":"2311.09115","repositories_listed":2,"syntology":{"n":11,"n_ran":8,"n_constructed":0,"n_ran_checked":6,"n_instrument":2,"n_unverified":3,"n_honours":1,"n_violates":3,"n_no_contract":2,"n_pointer_only":3,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 3 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/healnet-hybrid-multi-modal-fusion-for#ran","syntology_url":"https://syntology.ai/paper/2311.09115","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.09115"}},"official":{"repos":["konst-int-i/healnet"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"url":"/paper/asymmetric-contrastive-multimodal-learning","slug":"asymmetric-contrastive-multimodal-learning","title":"Advancing Drug Discovery with Enhanced Chemical Understanding via Asymmetric Contrastive Multimodal Learning","date":"2023-11-11","arxiv_id":"2311.06456","repositories_listed":2,"syntology":null},{"url":"/paper/hymnet-a-multimodal-deep-learning-system-for","slug":"hymnet-a-multimodal-deep-learning-system-for","title":"HyMNet: a Multimodal Deep Learning System for Hypertension Classification using Fundus Photographs and Cardiometabolic Risk Factors","date":"2023-10-02","arxiv_id":"2310.01099","repositories_listed":2,"syntology":null},{"url":"/paper/bi-bimodal-modality-fusion-for-correlation","slug":"bi-bimodal-modality-fusion-for-correlation","title":"Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis","date":"2021-07-28","arxiv_id":"2107.13669","repositories_listed":2,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":1,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":1,"n_no_contract":0,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/bi-bimodal-modality-fusion-for-correlation#ran","syntology_url":"https://syntology.ai/paper/2107.13669","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.13669"}},"official":{"repos":["declare-lab/multimodal-deep-learning"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/a-multimodal-deep-learning-framework-for","slug":"a-multimodal-deep-learning-framework-for","title":"A multimodal deep learning framework for scalable content based visual media retrieval","date":"2021-05-18","arxiv_id":"2105.08665","repositories_listed":2,"syntology":null},{"url":"/paper/multimodal-deep-learning-for-robust-rgb-d","slug":"multimodal-deep-learning-for-robust-rgb-d","title":"Multimodal Deep Learning for Robust RGB-D Object Recognition","date":"2015-07-24","arxiv_id":"1507.06821","repositories_listed":2,"syntology":null},{"url":"/paper/gaze-guided-learning-avoiding-shortcut-bias","slug":"gaze-guided-learning-avoiding-shortcut-bias","title":"Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification","date":"2025-04-08","arxiv_id":"2504.05583","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-deep-learning-for-subtype","slug":"multimodal-deep-learning-for-subtype","title":"Multimodal Deep Learning for Subtype Classification in Breast Cancer Using Histopathological Images and Gene Expression Data","date":"2025-03-04","arxiv_id":"2503.02849","repositories_listed":1,"syntology":null},{"url":"/paper/a-multimodal-pde-foundation-model-for","slug":"a-multimodal-pde-foundation-model-for","title":"A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions","date":"2025-02-09","arxiv_id":"2502.06026","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-marvels-of-deep-learning-in","slug":"multimodal-marvels-of-deep-learning-in","title":"Multimodal Marvels of Deep Learning in Medical Diagnosis: A Comprehensive Review of COVID-19 Detection","date":"2025-01-16","arxiv_id":"2501.09506","repositories_listed":1,"syntology":null},{"url":"/paper/clasp-contrastive-language-speech-pretraining","slug":"clasp-contrastive-language-speech-pretraining","title":"CLASP: Contrastive Language-Speech Pretraining for Multilingual Multimodal Information Retrieval","date":"2024-12-17","arxiv_id":"2412.13071","repositories_listed":1,"syntology":null},{"url":"/paper/frozen-large-scale-pretrained-vision-language","slug":"frozen-large-scale-pretrained-vision-language","title":"Frozen Large-scale Pretrained Vision-Language Models are the Effective Foundational Backbone for Multimodal Breast Cancer Prediction","date":"2024-11-27","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/cardiolab-laboratory-values-estimation-and","slug":"cardiolab-laboratory-values-estimation-and","title":"CardioLab: Laboratory Values Estimation and Monitoring from Electrocardiogram Signals -- A Multimodal Deep Learning Approach","date":"2024-11-22","arxiv_id":"2411.14886","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/cardiolab-laboratory-values-estimation-and#ran","syntology_url":"https://syntology.ai/paper/2411.14886","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.14886"}},"official":{"repos":["ai4healthuol/cardiolab"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/uncovering-the-genetic-basis-of-glioblastoma","slug":"uncovering-the-genetic-basis-of-glioblastoma","title":"Uncovering the Genetic Basis of Glioblastoma Heterogeneity through Multimodal Analysis of Whole Slide Images and RNA Sequencing Data","date":"2024-10-23","arxiv_id":"2410.18710","repositories_listed":1,"syntology":null},{"url":"/paper/phemonet-a-multimodal-network-for","slug":"phemonet-a-multimodal-network-for","title":"PHemoNet: A Multimodal Network for Physiological Signals","date":"2024-09-13","arxiv_id":"2410.00010","repositories_listed":1,"syntology":null},{"url":"/paper/dual-level-cross-modal-contrastive-clustering","slug":"dual-level-cross-modal-contrastive-clustering","title":"Dual-Level Cross-Modal Contrastive Clustering","date":"2024-09-06","arxiv_id":"2409.04561","repositories_listed":1,"syntology":null},{"url":"/paper/mvx-vit-multimodal-collaborative-perception-1","slug":"mvx-vit-multimodal-collaborative-perception-1","title":"MVX-ViT: Multimodal Collaborative Perception for 6G V2X Network Management Decisions Using Vision Transformer.","date":"2024-08-30","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/focus-on-focus-focus-oriented-representation","slug":"focus-on-focus-focus-oriented-representation","title":"Focus on Focus: Focus-oriented Representation Learning and Multi-view Cross-modal Alignment for Glioma Grading","date":"2024-08-16","arxiv_id":"2408.08527","repositories_listed":1,"syntology":null},{"url":"/paper/modeling-of-spatially-embedded-networks-via","slug":"modeling-of-spatially-embedded-networks-via","title":"Modeling of spatially embedded networks via regional spatial graph convolutional networks","date":"2024-06-20","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/luma-a-benchmark-dataset-for-learning-from","slug":"luma-a-benchmark-dataset-for-learning-from","title":"LUMA: A Benchmark Dataset for Learning from Uncertain and Multimodal Data","date":"2024-06-14","arxiv_id":"2406.09864","repositories_listed":1,"syntology":null},{"url":"/paper/automatic-fused-multimodal-deep-learning-for","slug":"automatic-fused-multimodal-deep-learning-for","title":"Automatic Fused Multimodal Deep Learning for Plant Identification","date":"2024-06-03","arxiv_id":"2406.01455","repositories_listed":1,"syntology":null},{"url":"/paper/towards-precision-healthcare-robust-fusion-of","slug":"towards-precision-healthcare-robust-fusion-of","title":"Towards Precision Healthcare: Robust Fusion of Time Series and Image Data","date":"2024-05-24","arxiv_id":"2405.15442","repositories_listed":1,"syntology":null},{"url":"/paper/an-interpretable-adaptive-multiscale","slug":"an-interpretable-adaptive-multiscale","title":"An Interpretable Adaptive Multiscale Attention Deep Neural Network for Tabular Data","date":"2024-05-15","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/honeybee-a-scalable-modular-framework-for","slug":"honeybee-a-scalable-modular-framework-for","title":"HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Models","date":"2024-05-13","arxiv_id":"2405.07460","repositories_listed":1,"syntology":null},{"url":"/paper/feature-importance-to-explain-multimodal","slug":"feature-importance-to-explain-multimodal","title":"Feature importance to explain multimodal prediction models. A clinical use case","date":"2024-04-29","arxiv_id":"2404.18631","repositories_listed":1,"syntology":null},{"url":"/paper/in-the-search-for-optimal-multi-view-learning","slug":"in-the-search-for-optimal-multi-view-learning","title":"In the Search for Optimal Multi-view Learning Models for Crop Classification with Global Remote Sensing Data","date":"2024-03-25","arxiv_id":"2403.16582","repositories_listed":1,"syntology":null},{"url":"/paper/mope-parameter-efficient-and-scalable","slug":"mope-parameter-efficient-and-scalable","title":"MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion","date":"2024-03-14","arxiv_id":"2403.10568","repositories_listed":1,"syntology":null},{"url":"/paper/restoring-ancient-ideograph-a-multimodal","slug":"restoring-ancient-ideograph-a-multimodal","title":"Restoring Ancient Ideograph: A Multimodal Multitask Neural Network Approach","date":"2024-03-11","arxiv_id":"2403.06682","repositories_listed":1,"syntology":null},{"url":"/paper/deepseek-vl-towards-real-world-vision","slug":"deepseek-vl-towards-real-world-vision","title":"DeepSeek-VL: Towards Real-World Vision-Language Understanding","date":"2024-03-08","arxiv_id":"2403.05525","repositories_listed":1,"syntology":{"n":11,"n_ran":11,"n_constructed":0,"n_ran_checked":4,"n_instrument":7,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":1,"phrase":"11 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 7 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/deepseek-vl-towards-real-world-vision#ran","syntology_url":"https://syntology.ai/paper/2403.05525","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.05525"}},"official":{"repos":["deepseek-ai/deepseek-vl"],"state":"official (archive's flag): 11 ran","n_ran":11,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/cultural-aware-ai-model-for-emotion","slug":"cultural-aware-ai-model-for-emotion","title":"Cultural-Aware AI Model for Emotion Recognition","date":"2024-03-06","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/robust-clip-unsupervised-adversarial-fine","slug":"robust-clip-unsupervised-adversarial-fine","title":"Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models","date":"2024-02-19","arxiv_id":"2402.12336","repositories_listed":1,"syntology":{"n":9,"n_ran":7,"n_constructed":0,"n_ran_checked":1,"n_instrument":6,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/robust-clip-unsupervised-adversarial-fine#ran","syntology_url":"https://syntology.ai/paper/2402.12336","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.12336"}},"official":{"repos":["chs20/robustvlm"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/multimodal-enhanced-objectness-learner-for","slug":"multimodal-enhanced-objectness-learner-for","title":"Multimodal-Enhanced Objectness Learner for Corner Case Detection in Autonomous Driving","date":"2024-02-03","arxiv_id":"2402.02026","repositories_listed":1,"syntology":null},{"url":"/paper/high-quality-image-restoration-following","slug":"high-quality-image-restoration-following","title":"InstructIR: High-Quality Image Restoration Following Human Instructions","date":"2024-01-29","arxiv_id":"2401.16468","repositories_listed":1,"syntology":null},{"url":"/paper/uncertainty-aware-hardware-trojan-detection","slug":"uncertainty-aware-hardware-trojan-detection","title":"Uncertainty-Aware Hardware Trojan Detection Using Multimodal Deep Learning","date":"2024-01-15","arxiv_id":"2401.09479","repositories_listed":1,"syntology":null},{"url":"/paper/linguistic-aware-patch-slimming-framework-for","slug":"linguistic-aware-patch-slimming-framework-for","title":"Linguistic-Aware Patch Slimming Framework for Fine-grained Cross-Modal Alignment","date":"2024-01-01","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/enhancing-scene-graph-generation-with","slug":"enhancing-scene-graph-generation-with","title":"Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge","date":"2023-11-21","arxiv_id":"2311.12889","repositories_listed":1,"syntology":null},{"url":"/paper/dynamic-task-and-weight-prioritization","slug":"dynamic-task-and-weight-prioritization","title":"Dynamic Task and Weight Prioritization Curriculum Learning for Multimodal Imagery","date":"2023-10-29","arxiv_id":"2310.19109","repositories_listed":1,"syntology":null},{"url":"/paper/formalizing-multimedia-recommendation-through","slug":"formalizing-multimedia-recommendation-through","title":"Formalizing Multimedia Recommendation through Multimodal Deep Learning","date":"2023-09-11","arxiv_id":"2309.05273","repositories_listed":1,"syntology":null},{"url":"/paper/multi-modality-guidance-network-for-missing","slug":"multi-modality-guidance-network-for-missing","title":"Multimodal Guidance Network for Missing-Modality Inference in Content Moderation","date":"2023-09-07","arxiv_id":"2309.03452","repositories_listed":1,"syntology":{"n":10,"n_ran":4,"n_constructed":0,"n_ran_checked":4,"n_instrument":0,"n_unverified":6,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 6 unverified","sample_list":"/paper/multi-modality-guidance-network-for-missing#ran","syntology_url":"https://syntology.ai/paper/2309.03452","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.03452"}},"official":{"repos":["zhuokaizhao/multimodal-guidance-network"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":6,"ran_from_kinds":["official"]}}},{"url":"/paper/multimodal-foundation-models-for","slug":"multimodal-foundation-models-for","title":"Multimodal Foundation Models For Echocardiogram Interpretation","date":"2023-08-29","arxiv_id":"2308.15670","repositories_listed":1,"syntology":null},{"url":"/paper/on-the-adversarial-robustness-of-multi-modal","slug":"on-the-adversarial-robustness-of-multi-modal","title":"On the Adversarial Robustness of Multi-Modal Foundation Models","date":"2023-08-21","arxiv_id":"2308.10741","repositories_listed":1,"syntology":{"n":4,"n_ran":4,"n_constructed":0,"n_ran_checked":0,"n_instrument":4,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":4,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/on-the-adversarial-robustness-of-multi-modal#ran","syntology_url":"https://syntology.ai/paper/2308.10741","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.10741"}},"official":{"repos":["chs20/robustvlm"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/promptstyler-prompt-driven-style-generation","slug":"promptstyler-prompt-driven-style-generation","title":"PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization","date":"2023-07-27","arxiv_id":"2307.15199","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/promptstyler-prompt-driven-style-generation#ran","syntology_url":"https://syntology.ai/paper/2307.15199","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.15199"}},"official":null}},{"url":"/paper/multizoo-multibench-a-standardized-toolkit","slug":"multizoo-multibench-a-standardized-toolkit","title":"MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning","date":"2023-06-28","arxiv_id":"2306.16413","repositories_listed":1,"syntology":null},{"url":"/paper/cross-modal-attribute-insertions-for","slug":"cross-modal-attribute-insertions-for","title":"Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning","date":"2023-06-19","arxiv_id":"2306.11065","repositories_listed":1,"syntology":null},{"url":"/paper/towards-balanced-active-learning-for","slug":"towards-balanced-active-learning-for","title":"Towards Balanced Active Learning for Multimodal Classification","date":"2023-06-14","arxiv_id":"2306.08306","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-neural-databases","slug":"multimodal-neural-databases","title":"Multimodal Neural Databases","date":"2023-05-02","arxiv_id":"2305.01447","repositories_listed":1,"syntology":null},{"url":"/paper/building-multimodal-ai-chatbots","slug":"building-multimodal-ai-chatbots","title":"Building Multimodal AI Chatbots","date":"2023-04-21","arxiv_id":"2305.03512","repositories_listed":1,"syntology":null},{"url":"/paper/zorro-the-masked-multimodal-transformer","slug":"zorro-the-masked-multimodal-transformer","title":"Zorro: the masked multimodal transformer","date":"2023-01-23","arxiv_id":"2301.09595","repositories_listed":1,"syntology":{"n":3,"n_ran":3,"n_constructed":0,"n_ran_checked":3,"n_instrument":0,"n_unverified":0,"n_honours":2,"n_violates":1,"n_no_contract":0,"n_pointer_only":1,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/zorro-the-masked-multimodal-transformer#ran","syntology_url":"https://syntology.ai/paper/2301.09595","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.09595"}},"official":null}},{"url":"/paper/multimodal-deep-learning","slug":"multimodal-deep-learning","title":"Multimodal Deep Learning","date":"2023-01-12","arxiv_id":"2301.04856","repositories_listed":1,"syntology":null},{"url":"/paper/learning-semantic-relationship-among","slug":"learning-semantic-relationship-among","title":"Learning Semantic Relationship Among Instances for Image-Text Matching","date":"2023-01-01","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/learning-multimodal-data-augmentation-in","slug":"learning-multimodal-data-augmentation-in","title":"Learning Multimodal Data Augmentation in Feature Space","date":"2022-12-29","arxiv_id":"2212.14453","repositories_listed":1,"syntology":{"n":10,"n_ran":9,"n_constructed":1,"n_ran_checked":3,"n_instrument":6,"n_unverified":1,"n_honours":0,"n_violates":2,"n_no_contract":1,"n_pointer_only":0,"phrase":"9 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 2 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/learning-multimodal-data-augmentation-in#ran","syntology_url":"https://syntology.ai/paper/2212.14453","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.14453"}},"official":{"repos":["lzcemma/lemda"],"state":"official (archive's flag): 9 ran","n_ran":9,"n_constructed":1,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/common-practices-and-taxonomy-in-deep-multi","slug":"common-practices-and-taxonomy-in-deep-multi","title":"Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications","date":"2022-12-20","arxiv_id":"2301.01200","repositories_listed":1,"syntology":null},{"url":"/paper/medical-diagnosis-with-large-scale-multimodal","slug":"medical-diagnosis-with-large-scale-multimodal","title":"Medical Diagnosis with Large Scale Multimodal Transformers: Leveraging Diverse Data for More Accurate Diagnosis","date":"2022-12-18","arxiv_id":"2212.09162","repositories_listed":1,"syntology":null},{"url":"/paper/variational-prompt-tuning-improves","slug":"variational-prompt-tuning-improves","title":"Bayesian Prompt Learning for Image-Language Model Generalization","date":"2022-10-05","arxiv_id":"2210.02390","repositories_listed":1,"syntology":{"n":6,"n_ran":4,"n_constructed":0,"n_ran_checked":1,"n_instrument":3,"n_unverified":2,"n_honours":1,"n_violates":0,"n_no_contract":0,"n_pointer_only":3,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/variational-prompt-tuning-improves#ran","syntology_url":"https://syntology.ai/paper/2210.02390","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.02390"}},"official":{"repos":["saic-fi/bayesian-prompt-learning"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/language-aware-soft-prompting-for-vision","slug":"language-aware-soft-prompting-for-vision","title":"LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models","date":"2022-10-03","arxiv_id":"2210.01115","repositories_listed":1,"syntology":{"n":7,"n_ran":4,"n_constructed":0,"n_ran_checked":1,"n_instrument":3,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":2,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/language-aware-soft-prompting-for-vision#ran","syntology_url":"https://syntology.ai/paper/2210.01115","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.01115"}},"official":{"repos":["1adrianb/lasp"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/learn-to-explain-multimodal-reasoning-via","slug":"learn-to-explain-multimodal-reasoning-via","title":"Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering","date":"2022-09-20","arxiv_id":"2209.09513","repositories_listed":1,"syntology":{"n":5,"n_ran":4,"n_constructed":0,"n_ran_checked":0,"n_instrument":4,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":2,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/learn-to-explain-multimodal-reasoning-via#ran","syntology_url":"https://syntology.ai/paper/2209.09513","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.09513"}},"official":{"repos":["lupantech/ScienceQA"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official","unlocated"]}}},{"url":"/paper/lavis-a-library-for-language-vision","slug":"lavis-a-library-for-language-vision","title":"LAVIS: A Library for Language-Vision Intelligence","date":"2022-09-15","arxiv_id":"2209.09019","repositories_listed":1,"syntology":null},{"url":"/paper/tmss-an-end-to-end-transformer-based","slug":"tmss-an-end-to-end-transformer-based","title":"TMSS: An End-to-End Transformer-based Multimodal Network for Segmentation and Survival Prediction","date":"2022-09-12","arxiv_id":"2209.05036","repositories_listed":1,"syntology":null},{"url":"/paper/multi-modal-experience-inspired-ai-creation","slug":"multi-modal-experience-inspired-ai-creation","title":"Multi-Modal Experience Inspired AI Creation","date":"2022-09-02","arxiv_id":"2209.02427","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-attention-based-deep-learning-for","slug":"multimodal-attention-based-deep-learning-for","title":"Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis","date":"2022-06-17","arxiv_id":"2206.08826","repositories_listed":1,"syntology":null},{"url":"/paper/deepgravilens-a-multi-modal-architecture-for","slug":"deepgravilens-a-multi-modal-architecture-for","title":"DeepGraviLens: a Multi-Modal Architecture for Classifying Gravitational Lensing Data","date":"2022-05-02","arxiv_id":"2205.00701","repositories_listed":1,"syntology":null},{"url":"/paper/learning-multi-view-aggregation-in-the-wild","slug":"learning-multi-view-aggregation-in-the-wild","title":"Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation","date":"2022-04-15","arxiv_id":"2204.07548","repositories_listed":1,"syntology":null},{"url":"/paper/contrastive-language-image-pre-training-for","slug":"contrastive-language-image-pre-training-for","title":"Contrastive Language-Image Pre-training for the Italian Language","date":"2021-08-19","arxiv_id":"2108.08688","repositories_listed":1,"syntology":null},{"url":"/paper/pan-cancer-integrative-histology-genomic","slug":"pan-cancer-integrative-histology-genomic","title":"Pan-Cancer Integrative Histology-Genomic Analysis via Interpretable Multimodal Deep Learning","date":"2021-08-04","arxiv_id":"2108.02278","repositories_listed":1,"syntology":null},{"url":"/paper/multi-modal-understanding-and-generation-for","slug":"multi-modal-understanding-and-generation-for","title":"Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training","date":"2021-05-24","arxiv_id":"2105.11333","repositories_listed":1,"syntology":{"n":2,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/multi-modal-understanding-and-generation-for#ran","syntology_url":"https://syntology.ai/paper/2105.11333","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.11333"}},"official":{"repos":["SuperSupermoon/MedViLL"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/distilling-audio-visual-knowledge-by","slug":"distilling-audio-visual-knowledge-by","title":"Distilling Audio-Visual Knowledge by Compositional Contrastive Learning","date":"2021-04-22","arxiv_id":"2104.10955","repositories_listed":1,"syntology":{"n":1,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"0 ran · 1 unverified","sample_list":"/paper/distilling-audio-visual-knowledge-by#ran","syntology_url":"https://syntology.ai/paper/2104.10955","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.10955"}},"official":{"repos":["yanbeic/CCL"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"url":"/paper/robust-sensor-fusion-algorithms-against","slug":"robust-sensor-fusion-algorithms-against","title":"Robust Sensor Fusion Algorithms Against Voice Command Attacks in Autonomous Vehicles","date":"2021-04-20","arxiv_id":"2104.09872","repositories_listed":1,"syntology":null},{"url":"/paper/subverting-the-jewtocracy-online-antisemitism","slug":"subverting-the-jewtocracy-online-antisemitism","title":"\"Subverting the Jewtocracy\": Online Antisemitism Detection Using Multimodal Deep Learning","date":"2021-04-13","arxiv_id":"2104.05947","repositories_listed":1,"syntology":null},{"url":"/paper/minkloc-lidar-and-monocular-image-fusion-for","slug":"minkloc-lidar-and-monocular-image-fusion-for","title":"MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition","date":"2021-04-12","arxiv_id":"2104.05327","repositories_listed":1,"syntology":{"n":5,"n_ran":3,"n_constructed":0,"n_ran_checked":1,"n_instrument":2,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":2,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/minkloc-lidar-and-monocular-image-fusion-for#ran","syntology_url":"https://syntology.ai/paper/2104.05327","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.05327"}},"official":{"repos":["jac99/MinkLocMultimodal"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/deep-learning-for-android-malware-defenses-a","slug":"deep-learning-for-android-malware-defenses-a","title":"Deep Learning for Android Malware Defenses: a Systematic Literature Review","date":"2021-03-09","arxiv_id":"2103.05292","repositories_listed":1,"syntology":null},{"url":"/paper/piano-skills-assessment","slug":"piano-skills-assessment","title":"Piano Skills Assessment","date":"2021-01-13","arxiv_id":"2101.04884","repositories_listed":1,"syntology":null},{"url":"/paper/detecting-hate-speech-in-memes-using","slug":"detecting-hate-speech-in-memes-using","title":"Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challenge","date":"2020-12-23","arxiv_id":"2012.12975","repositories_listed":1,"syntology":null},{"url":"/paper/image-and-text-fusion-for-upmc-food-101-using","slug":"image-and-text-fusion-for-upmc-food-101-using","title":"Image and Text fusion for UPMC Food-101 \\\\using BERT and CNNs","date":"2020-12-17","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/detecting-video-game-player-burnout-with-the","slug":"detecting-video-game-player-burnout-with-the","title":"Detecting Video Game Player Burnout with the Use of Sensor Data and Machine Learning","date":"2020-11-29","arxiv_id":"2012.02299","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-learning-for-hateful-memes","slug":"multimodal-learning-for-hateful-memes","title":"Multimodal Learning for Hateful Memes Detection","date":"2020-11-25","arxiv_id":"2011.12870","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-emotion-recognition-with","slug":"multimodal-emotion-recognition-with","title":"Multimodal Emotion Recognition with Transformer-Based Self Supervised Feature Fusion","date":"2020-10-27","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/creation-and-validation-of-a-chest-x-ray","slug":"creation-and-validation-of-a-chest-x-ray","title":"Creation and Validation of a Chest X-Ray Dataset with Eye-tracking and Report Dictation for AI Development","date":"2020-09-15","arxiv_id":"2009.07386","repositories_listed":1,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":0,"n_honours":2,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/creation-and-validation-of-a-chest-x-ray#ran","syntology_url":"https://syntology.ai/paper/2009.07386","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2009.07386"}},"official":{"repos":["cxr-eye-gaze/eye-gaze-dataset"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/mmea-entity-alignment-for-multi-modal","slug":"mmea-entity-alignment-for-multi-modal","title":"MMEA: Entity Alignment for Multi-Modal Knowledge Graphs","date":"2020-08-20","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/jointly-fine-tuning-bert-like-self-supervised-1","slug":"jointly-fine-tuning-bert-like-self-supervised-1","title":"Jointly Fine-Tuning “BERT-like” Self Supervised Models to Improve Multimodal Speech Emotion Recognition","date":"2020-08-15","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/more-diverse-means-better-multimodal-deep","slug":"more-diverse-means-better-multimodal-deep","title":"More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification","date":"2020-08-12","arxiv_id":"2008.05457","repositories_listed":1,"syntology":null},{"url":"/paper/image-search-with-text-feedback-by","slug":"image-search-with-text-feedback-by","title":"Image Search With Text Feedback by Visiolinguistic Attention Learning","date":"2020-06-01","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/hydra-a-multimodal-deep-learning-framework","slug":"hydra-a-multimodal-deep-learning-framework","title":"HYDRA: A multimodal deep learning framework for malware classification","date":"2020-05-12","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/analysis-of-social-media-data-using","slug":"analysis-of-social-media-data-using","title":"Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response","date":"2020-04-14","arxiv_id":"2004.11838","repositories_listed":1,"syntology":null},{"url":"/paper/predicting-the-leading-political-ideology-of","slug":"predicting-the-leading-political-ideology-of","title":"Predicting the Leading Political Ideology of YouTube Channels Using Acoustic, Textual, and Metadata Information","date":"2019-10-20","arxiv_id":"1910.08948","repositories_listed":1,"syntology":null},{"url":"/paper/audio-conditioned-u-net-for-position","slug":"audio-conditioned-u-net-for-position","title":"Audio-Conditioned U-Net for Position Estimation in Full Sheet Images","date":"2019-10-16","arxiv_id":"1910.07254","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-age-and-gender-classification","slug":"multimodal-age-and-gender-classification","title":"Multimodal Age and Gender Classification Using Ear and Profile Face Images","date":"2019-07-23","arxiv_id":"1907.10081","repositories_listed":1,"syntology":null},{"url":"/paper/learn-to-combine-modalities-in-multimodal","slug":"learn-to-combine-modalities-in-multimodal","title":"Learn to Combine Modalities in Multimodal Deep Learning","date":"2018-05-29","arxiv_id":"1805.11730","repositories_listed":1,"syntology":null},{"url":"/paper/xflow-1d-2d-cross-modal-deep-neural-networks","slug":"xflow-1d-2d-cross-modal-deep-neural-networks","title":"XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification","date":"2017-09-02","arxiv_id":"1709.00572","repositories_listed":1,"syntology":null},{"url":null,"slug":"2506-04756","title":"Ontology-based knowledge representation for bone disease diagnosis: a foundation for safe and sustainable medical artificial intelligence systems","date":"2025-06-05","arxiv_id":"2506.04756","repositories_listed":0,"syntology":null},{"url":null,"slug":"unified-cross-modal-attention-mixer-based","title":"Unified Cross-Modal Attention-Mixer Based Structural-Functional Connectomics Fusion for Neuropsychiatric Disorder Diagnosis","date":"2025-05-21","arxiv_id":"2505.15139","repositories_listed":0,"syntology":null},{"url":null,"slug":"multimodal-fusion-of-glucose-monitoring-and","title":"Multimodal Fusion of Glucose Monitoring and Food Imagery for Caloric Content Prediction","date":"2025-05-13","arxiv_id":"2505.09018","repositories_listed":0,"syntology":null}],"syntology_records":21,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"key_notes":{"n_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"since_archive_judged":{"papers":0,"papers_placed":0,"judged_by":{"model":"a model","models_as_recorded":[],"from":"lb_rows_run_verified.jsonl: each record's `model` field (the extractor's call that wrote decisions[])"},"note":"a model's judgement from the paper's own tables that they report on this leaderboard, not a result: no value from a paper that was not placed is printed or carried here; papers counts each paper once over these leaderboards","backfill":"8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read)","backfill_complete":false,"leaderboards":[]},"papers_where_code_ran":21,"papers_where_code_ran_first_page":"/task/multimodal-deep-learning/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not run it on this task or check it against the task's benchmarks.","pages":[{"order":"archive","page":2,"of":3,"url":"/task/multimodal-deep-learning/papers/2","json":"/task/multimodal-deep-learning/papers/2.json","rows":100},{"order":"archive","page":3,"of":3,"url":"/task/multimodal-deep-learning/papers/3","json":"/task/multimodal-deep-learning/papers/3.json","rows":13},{"order":"ran","page":1,"of":1,"url":"/task/multimodal-deep-learning/papers/ran/1","json":"/task/multimodal-deep-learning/papers/ran/1.json","rows":21}],"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-10-05T00:36:11+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"row_route":"each syntology object on a benchmark's first row and on a paper carries sample_list (this site's paper page, #ran), syntology_url (syntology.ai's page for the paper, only when that route probed live) and mcp (the agent call that lists the paper's harvested samples; not a verdict on the paper)","developers":"https://syntology.ai/developers"},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"},"record_sha256":"4ba7e4e6371fb650a0534b5bdd38ef470b605d9ecbae36935f73fa9d014bb606","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}