{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/effects-of-sampling-skewness-of-the","title":"Effects of sampling skewness of the importance-weighted risk estimator on model selection","arxiv_id":"1804.07344","date":"2018-04-19","proceeding":null,"authors":["Wouter M. Kouw","Marco Loog"],"abstract":"Importance-weighting is a popular and well-researched technique for dealing\nwith sample selection bias and covariate shift. It has desirable\ncharacteristics such as unbiasedness, consistency and low computational\ncomplexity. However, weighting can have a detrimental effect on an estimator as\nwell. In this work, we empirically show that the sampling distribution of an\nimportance-weighted estimator can be skewed. For sample selection bias\nsettings, and for small sample sizes, the importance-weighted risk estimator\nproduces overestimates for datasets in the body of the sampling distribution,\ni.e. the majority of cases, and large underestimates for data sets in the tail\nof the sampling distribution. These over- and underestimates of the risk lead\nto suboptimal regularization parameters when used for importance-weighted\nvalidation.","url_abs":"http://arxiv.org/abs/1804.07344v1","url_pdf":"http://arxiv.org/pdf/1804.07344v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"effects-of-sampling-skewness-of-the","repo_url":"https://github.com/wmkouw/covshift-skewness","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"selection-bias","task_name":"Selection bias"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"wmkouw/covshift-skewness","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"b956c8b2dd08716a49d6677083c6d3f0f03edf74c9bbc9ffb0190ffabf668a37","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}