{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/neural-sentence-embedding-models-for-semantic","title":"Neural sentence embedding models for semantic similarity estimation in the biomedical domain","arxiv_id":"2110.15708","date":"2021-10-01","proceeding":null,"authors":["Kathrin Blagec","Hong Xu","Asan Agibetov","Matthias Samwald"],"abstract":"BACKGROUND: In this study, we investigated the efficacy of current state-of-the-art neural sentence embedding models for semantic similarity estimation of sentences from biomedical literature. We trained different neural embedding models on 1.7 million articles from the PubMed Open Access dataset, and evaluated them based on a biomedical benchmark set containing 100 sentence pairs annotated by human experts and a smaller contradiction subset derived from the original benchmark set. RESULTS: With a Pearson correlation of 0.819, our best unsupervised model based on the Paragraph Vector Distributed Memory algorithm outperforms previous state-of-the-art results achieved on the BIOSSES biomedical benchmark set. Moreover, our proposed supervised model that combines different string-based similarity metrics with a neural embedding model surpasses previous ontology-dependent supervised state-of-the-art approaches in terms of Pearson's r (r=0.871) on the biomedical benchmark set. In contrast to the promising results for the original benchmark, we found our best models' performance on the smaller contradiction subset to be poor. CONCLUSIONS: In this study we highlighted the value of neural network-based models for semantic similarity estimation in the biomedical domain by showing that they can keep up with and even surpass previous state-of-the-art approaches for semantic similarity estimation that depend on the availability of laboriously curated ontologies when evaluated on a biomedical benchmark set. Capturing contradictions and negations in biomedical sentences, however, emerged as an essential area for further work.","url_abs":"https://arxiv.org/abs/2110.15708v1","url_pdf":"https://arxiv.org/pdf/2110.15708v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-sentence-embedding-models-for-semantic","repo_url":"https://github.com/kathrinblagec/neural-sentence-embedding-models-for-biomedical-applications","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"articles","task_name":"Articles"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-embedding","task_name":"Sentence Embedding"},{"task_slug":"sentence-embeddings-for-biomedical-texts","task_name":"Sentence Embeddings For Biomedical Texts"},{"task_slug":"sentence-embedding-1","task_name":"Sentence-Embedding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"Supervised combination of: Jaccard, Q-gram, sent2vec, Paragraph vector DM, skip-thoughts, fastText","rank_in_archive_order":1,"of":14,"metrics":{"Pearson Correlation":"0.871"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"Unsupervised combination (mean) of: Jaccard, q-gram, Paragraph vector (PV-DBOW) and sent2vec","rank_in_archive_order":2,"of":14,"metrics":{"Pearson Correlation":"0.846"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"Paragraph vector (PV-DM)","rank_in_archive_order":3,"of":14,"metrics":{"Pearson Correlation":"0.819"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"Paragraph vector (PV-DBOW)","rank_in_archive_order":5,"of":14,"metrics":{"Pearson Correlation":"0.804"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"Sent2vec","rank_in_archive_order":6,"of":14,"metrics":{"Pearson Correlation":"0.798"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"fastText (skip-gram, max pooling)","rank_in_archive_order":9,"of":14,"metrics":{"Pearson Correlation":"0.766"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"Q-gram (q = 3)","rank_in_archive_order":10,"of":14,"metrics":{"Pearson Correlation":"0.723"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"Skip-thoughts","rank_in_archive_order":11,"of":14,"metrics":{"Pearson Correlation":"0.485"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-embeddings-for-biomedical-texts-on","task":"Sentence Embeddings For Biomedical Texts","dataset":"BIOSSES","model":"fastText (CBOW, max pooling)","rank_in_archive_order":14,"of":14,"metrics":{"Pearson Correlation":"0.253"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2110.15708","atlas_url":"https://app.syntology.ai/?focus=2110.15708","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2110.15708"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2110.15708\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2110.15708"},"call":"get_concepts_for_paper(arxiv_id=\"2110.15708\")"}],"by_repo":[{"repo":"kathrinblagec/neural-sentence-embedding-models-for-biomedical-applications","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"16c5cdef31f1c982188ac45b68e720e916a7bc2019c1861fecf9f4944720c878","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}