{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/searching-for-legal-clauses-by-analogy-few","title":"Contract Discovery: Dataset and a Few-Shot Semantic Retrieval Challenge with Competitive Baselines","arxiv_id":"1911.03911","date":"2019-11-10","proceeding":"Findings of the Association for Computational Linguistics 2020","authors":["Łukasz Borchmann","Dawid Wiśniewski","Andrzej Gretkowski","Izabela Kosmala","Dawid Jurkiewicz","Łukasz Szałkiewicz","Gabriela Pałka","Karol Kaczmarek","Agnieszka Kaliska","Filip Graliński"],"abstract":"We propose a new shared task of semantic retrieval from legal texts, in which a so-called contract discovery is to be performed, where legal clauses are extracted from documents, given a few examples of similar clauses from other legal acts. The task differs substantially from conventional NLI and shared tasks on legal information extraction (e.g., one has to identify text span instead of a single document, page, or paragraph). The specification of the proposed task is followed by an evaluation of multiple solutions within the unified framework proposed for this branch of methods. It is shown that state-of-the-art pretrained encoders fail to provide satisfactory results on the task proposed. In contrast, Language Model-based solutions perform better, especially when unsupervised fine-tuning is applied. Besides the ablation studies, we addressed questions regarding detection accuracy for relevant text fragments depending on the number of examples available. In addition to the dataset and reference results, LMs specialized in the legal domain were made publicly available.","url_abs":"https://arxiv.org/abs/1911.03911v2","url_pdf":"https://arxiv.org/pdf/1911.03911v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"searching-for-legal-clauses-by-analogy-few","repo_url":"https://github.com/applicaai/contract-discovery","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"semantic-retrieval","task_name":"Semantic Retrieval"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"}],"methods":[{"method_slug":"k-nn","method_name":"k-NN"}],"datasets_introduced":[{"slug":"contract-discovery","name":"Contract Discovery","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-retrieval-on-contract-discovery","task":"Semantic Retrieval","dataset":"Contract Discovery","model":"Human baseline","rank_in_archive_order":1,"of":6,"metrics":{"Soft-F1":"0.84"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-retrieval-on-contract-discovery","task":"Semantic Retrieval","dataset":"Contract Discovery","model":"k-NN with sentence n-grams, GPT-2 embeddings, fICA","rank_in_archive_order":2,"of":6,"metrics":{"Soft-F1":"0.51"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-retrieval-on-contract-discovery","task":"Semantic Retrieval","dataset":"Contract Discovery","model":"LSA baseline","rank_in_archive_order":4,"of":6,"metrics":{"Soft-F1":"0.39"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-retrieval-on-contract-discovery","task":"Semantic Retrieval","dataset":"Contract Discovery","model":"Universal Sentence Encoder","rank_in_archive_order":5,"of":6,"metrics":{"Soft-F1":"0.38"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-retrieval-on-contract-discovery","task":"Semantic Retrieval","dataset":"Contract Discovery","model":"Sentence BERT","rank_in_archive_order":6,"of":6,"metrics":{"Soft-F1":"0.31"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"applicaai/contract-discovery","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"ce7ad6e3ed23bb50e57d319aeae31d882f275cba1f9b55f6279ebdd5629150b6","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}