{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/statistical-estimation-of-malware-detection","title":"Statistical Estimation of Malware Detection Metrics in the Absence of Ground Truth","arxiv_id":"1810.07260","date":"2018-09-24","proceeding":null,"authors":["Du Pang","Sun Zheyuan","Chen Huashan","Cho Jin-Hee","Xu Shouhuai"],"abstract":"The accurate measurement of security metrics is a critical research problem\nbecause an improper or inaccurate measurement process can ruin the usefulness\nof the metrics, no matter how well they are defined. This is a highly\nchallenging problem particularly when the ground truth is unknown or noisy. In\ncontrast to the well perceived importance of defining security metrics, the\nmeasurement of security metrics has been little understood in the literature.\nIn this paper, we measure five malware detection metrics in the {\\em absence}\nof ground truth, which is a realistic setting that imposes many technical\nchallenges. The ultimate goal is to develop principled, automated methods for\nmeasuring these metrics at the maximum accuracy possible. The problem naturally\ncalls for investigations into statistical estimators by casting the measurement\nproblem as a {\\em statistical estimation} problem. We propose statistical\nestimators for these five malware detection metrics. By investigating the\nstatistical properties of these estimators, we are able to characterize when\nthe estimators are accurate, and what adjustments can be made to improve them\nunder what circumstances. We use synthetic data with known ground truth to\nvalidate these statistical estimators. Then, we employ these estimators to\nmeasure five metrics with respect to a large dataset collected from VirusTotal.\nWe believe our study touches upon a vital problem that has not been paid due\nattention and will inspire many future investigations.","url_abs":"http://arxiv.org/abs/1810.07260v1","url_pdf":"http://arxiv.org/pdf/1810.07260v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"statistical-estimation-of-malware-detection","repo_url":"https://github.com/Chenutsa/trustworthiness","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"malware-detection","task_name":"Malware Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"Chenutsa/trustworthiness","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"7b48611f82655678acad0b4f7f4cde1b7b7d40decb48cdbe1815a28e3607b9b7","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}