{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/hierarchical-generative-modeling-for","title":"Hierarchical Generative Modeling for Controllable Speech Synthesis","arxiv_id":"1810.07217","date":"2018-10-16","proceeding":"ICLR 2019 5","authors":["Wei-Ning Hsu","Yu Zhang","Ron J. Weiss","Heiga Zen","Yonghui Wu","Yuxuan Wang","Yuan Cao","Ye Jia","Zhifeng Chen","Jonathan Shen","Patrick Nguyen","Ruoming Pang"],"abstract":"This paper proposes a neural sequence-to-sequence text-to-speech (TTS) model\nwhich can control latent attributes in the generated speech that are rarely\nannotated in the training data, such as speaking style, accent, background\nnoise, and recording conditions. The model is formulated as a conditional\ngenerative model based on the variational autoencoder (VAE) framework, with two\nlevels of hierarchical latent variables. The first level is a categorical\nvariable, which represents attribute groups (e.g. clean/noisy) and provides\ninterpretability. The second level, conditioned on the first, is a multivariate\nGaussian variable, which characterizes specific attribute configurations (e.g.\nnoise level, speaking rate) and enables disentangled fine-grained control over\nthese attributes. This amounts to using a Gaussian mixture model (GMM) for the\nlatent distribution. Extensive evaluation demonstrates its ability to control\nthe aforementioned attributes. In particular, we train a high-quality\ncontrollable TTS model on real found data, which is capable of inferring\nspeaker and style attributes from a noisy utterance and use it to synthesize\nclean speech with controllable speaking style.","url_abs":"http://arxiv.org/abs/1810.07217v2","url_pdf":"http://arxiv.org/pdf/1810.07217v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hierarchical-generative-modeling-for","repo_url":"https://github.com/lturing/Tools","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"hierarchical-generative-modeling-for","repo_url":"https://github.com/rarefin/TTS_VAE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"speech-synthesis","task_name":"Speech Synthesis"},{"task_slug":"text-to-speech","task_name":"Text to Speech"},{"task_slug":"text-to-speech-1","task_name":"text-to-speech"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1810.07217","atlas_url":"https://app.syntology.ai/?focus=1810.07217","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1810.07217"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1810.07217\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1810.07217"},"call":"get_concepts_for_paper(arxiv_id=\"1810.07217\")"}],"by_repo":[{"repo":"lturing/Tools","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"rarefin/TTS_VAE","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"2f5f91bd36b60ac10d6087081518347eb1c036124b99e5b451edfb20aa6d9211","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}