{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/estimating-mutual-information-for-discrete","title":"Estimating Mutual Information for Discrete-Continuous Mixtures","arxiv_id":"1709.06212","date":"2017-09-19","proceeding":"NeurIPS 2017 12","authors":["Weihao Gao","Sreeram Kannan","Sewoong Oh","Pramod Viswanath"],"abstract":"Estimating mutual information from observed samples is a basic primitive,\nuseful in several machine learning tasks including correlation mining,\ninformation bottleneck clustering, learning a Chow-Liu tree, and conditional\nindependence testing in (causal) graphical models. While mutual information is\na well-defined quantity in general probability spaces, existing estimators can\nonly handle two special cases of purely discrete or purely continuous pairs of\nrandom variables. The main challenge is that these methods first estimate the\n(differential) entropies of X, Y and the pair (X;Y) and add them up with\nappropriate signs to get an estimate of the mutual information. These\n3H-estimators cannot be applied in general mixture spaces, where entropy is not\nwell-defined. In this paper, we design a novel estimator for mutual information\nof discrete-continuous mixtures. We prove that the proposed estimator is\nconsistent. We provide numerical experiments suggesting superiority of the\nproposed estimator compared to other heuristics of adding small continuous\nnoise to all the samples and applying standard estimators tailored for purely\ncontinuous variables, and quantizing the samples and applying standard\nestimators tailored for purely discrete variables. This significantly widens\nthe applicability of mutual information estimation in real-world applications,\nwhere some variables are discrete, some continuous, and others are a mixture\nbetween continuous and discrete components.","url_abs":"http://arxiv.org/abs/1709.06212v3","url_pdf":"http://arxiv.org/pdf/1709.06212v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"estimating-mutual-information-for-discrete","repo_url":"https://github.com/alexandreguichet/MFS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"mutual-information-estimation","task_name":"Mutual Information Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1709.06212","atlas_url":"https://app.syntology.ai/?focus=1709.06212","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1709.06212"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1709.06212\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1709.06212"},"call":"get_concepts_for_paper(arxiv_id=\"1709.06212\")"}],"by_repo":[{"repo":"alexandreguichet/MFS","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"f6830b234bee0e1d647943464ac1985ac41606c60aa3214598361037cb11727f","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}