{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/a-meta-analysis-of-the-anomaly-detection","title":"A Meta-Analysis of the Anomaly Detection Problem","arxiv_id":"1503.01158","date":"2015-03-03","proceeding":null,"authors":["Andrew Emmott","Shubhomoy Das","Thomas Dietterich","Alan Fern","Weng-Keen Wong"],"abstract":"This article provides a thorough meta-analysis of the anomaly detection\nproblem. To accomplish this we first identify approaches to benchmarking\nanomaly detection algorithms across the literature and produce a large corpus\nof anomaly detection benchmarks that vary in their construction across several\ndimensions we deem important to real-world applications: (a) point difficulty,\n(b) relative frequency of anomalies, (c) clusteredness of anomalies, and (d)\nrelevance of features. We apply a representative set of anomaly detection\nalgorithms to this corpus, yielding a very large collection of experimental\nresults. We analyze these results to understand many phenomena observed in\nprevious work. First we observe the effects of experimental design on\nexperimental results. Second, results are evaluated with two metrics, ROC Area\nUnder the Curve and Average Precision. We employ statistical hypothesis testing\nto demonstrate the value (or lack thereof) of our benchmarks. We then offer\nseveral approaches to summarizing our experimental results, drawing several\nconclusions about the impact of our methodology as well as the strengths and\nweaknesses of some algorithms. Last, we compare results against a trivial\nsolution as an alternate means of normalizing the reported performance of\nalgorithms. The intended contributions of this article are many; in addition to\nproviding a large publicly-available corpus of anomaly detection benchmarks, we\nprovide an ontology for describing anomaly detection contexts, a methodology\nfor controlling various aspects of benchmark creation, guidelines for future\nexperimental design and a discussion of the many potential pitfalls of trying\nto measure success in this field.","url_abs":"http://arxiv.org/abs/1503.01158v2","url_pdf":"http://arxiv.org/pdf/1503.01158v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-meta-analysis-of-the-anomaly-detection","repo_url":"https://github.com/yaroslav-moiseev/evidence-based-possibly-best-practices-in-classical-ML","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"experimental-design","task_name":"Experimental Design"},{"task_slug":"hypothesis-testing","task_name":"Two-sample testing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1503.01158","atlas_url":"https://app.syntology.ai/?focus=1503.01158","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1503.01158"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1503.01158\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1503.01158"},"call":"get_concepts_for_paper(arxiv_id=\"1503.01158\")"}],"by_repo":[{"repo":"yaroslav-moiseev/evidence-based-possibly-best-practices-in-classical-ML","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"7245888718ed061c9d5dba60de2ea64da5eb6ef40fe4ce5074d17e2d79586b5e","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}