{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/matching-with-text-data-an-experimental","title":"Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality","arxiv_id":"1801.00644","date":"2018-01-02","proceeding":null,"authors":["Reagan Mozer","Luke Miratrix","Aaron Russell Kaufman","L. Jason Anastasopoulos"],"abstract":"Matching for causal inference is a well-studied problem, but standard methods\nfail when the units to match are text documents: the high-dimensional and rich\nnature of the data renders exact matching infeasible, causes propensity scores\nto produce incomparable matches, and makes assessing match quality difficult.\nIn this paper, we characterize a framework for matching text documents that\ndecomposes existing methods into: (1) the choice of text representation, and\n(2) the choice of distance metric. We investigate how different choices within\nthis framework affect both the quantity and quality of matches identified\nthrough a systematic multifactor evaluation experiment using human subjects.\nAltogether we evaluate over 100 unique text matching methods along with 5\ncomparison methods taken from the literature. Our experimental results identify\nmethods that generate matches with higher subjective match quality than current\nstate-of-the-art techniques. We enhance the precision of these results by\ndeveloping a predictive model to estimate the match quality of pairs of text\ndocuments as a function of our various distance scores. This model, which we\nfind successfully mimics human judgment, also allows for approximate and\nunsupervised evaluation of new procedures. We then employ the identified best\nmethod to illustrate the utility of text matching in two applications. First,\nwe engage with a substantive debate in the study of media bias by using text\nmatching to control for topic selection when comparing news articles from\nthirteen news sources. We then show how conditioning on text data leads to more\nprecise causal inferences in an observational study examining the effects of a\nmedical intervention.","url_abs":"http://arxiv.org/abs/1801.00644v7","url_pdf":"http://arxiv.org/pdf/1801.00644v7.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"matching-with-text-data-an-experimental","repo_url":"https://github.com/reaganmozer/textmatch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"articles","task_name":"Articles"},{"task_slug":"causal-inference","task_name":"Causal Inference"},{"task_slug":"text-matching","task_name":"Text Matching"}],"methods":[{"method_slug":"causal-inference","method_name":"Causal inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1801.00644","atlas_url":"https://app.syntology.ai/?focus=1801.00644","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1801.00644"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1801.00644\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1801.00644"},"call":"get_concepts_for_paper(arxiv_id=\"1801.00644\")"}],"by_repo":[{"repo":"reaganmozer/textmatch","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"56140d7791cb34d8d431ef718457a7e643a97b7dd79beb90f65663f9f29336da","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}