{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/on-distinguishability-criteria-for-estimating","title":"On distinguishability criteria for estimating generative models","arxiv_id":"1412.6515","date":"2014-12-19","proceeding":null,"authors":["Ian J. Goodfellow"],"abstract":"Two recently introduced criteria for estimation of generative models are both\nbased on a reduction to binary classification. Noise-contrastive estimation\n(NCE) is an estimation procedure in which a generative model is trained to be\nable to distinguish data samples from noise samples. Generative adversarial\nnetworks (GANs) are pairs of generator and discriminator networks, with the\ngenerator network learning to generate samples by attempting to fool the\ndiscriminator network into believing its samples are real data. Both estimation\nprocedures use the same function to drive learning, which naturally raises\nquestions about how they are related to each other, as well as whether this\nfunction is related to maximum likelihood estimation (MLE). NCE corresponds to\ntraining an internal data model belonging to the {\\em discriminator} network\nbut using a fixed generator network. We show that a variant of NCE, with a\ndynamic generator network, is equivalent to maximum likelihood estimation.\nSince pairing a learned discriminator with an appropriate dynamically selected\ngenerator recovers MLE, one might expect the reverse to hold for pairing a\nlearned generator with a certain discriminator. However, we show that\nrecovering MLE for a learned generator requires departing from the\ndistinguishability game. Specifically:\n  (i) The expected gradient of the NCE discriminator can be made to match the\nexpected gradient of\n  MLE, if one is allowed to use a non-stationary noise distribution for NCE,\n  (ii) No choice of discriminator network can make the expected gradient for\nthe GAN generator match that of MLE, and\n  (iii) The existing theory does not guarantee that GANs will converge in the\nnon-convex case.\n  This suggests that the key next step in GAN research is to determine whether\nGANs converge, and if not, to modify their training algorithm to force\nconvergence.","url_abs":"http://arxiv.org/abs/1412.6515v4","url_pdf":"http://arxiv.org/pdf/1412.6515v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-distinguishability-criteria-for-estimating","repo_url":"https://github.com/Stonesjtu/Pytorch-NCE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1412.6515","atlas_url":"https://app.syntology.ai/?focus=1412.6515","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1412.6515"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1412.6515\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1412.6515"},"call":"get_concepts_for_paper(arxiv_id=\"1412.6515\")"}],"by_repo":[{"repo":"Stonesjtu/Pytorch-NCE","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"a14742763dee1a415575a8d224ed12d27ee2655eba00f856ad269ac74f45a2a2","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}