{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/vq-vae","slug":"vq-vae","name":"VQ-VAE","full_name":"VQ-VAE","full_name_withheld":false,"description_markdown":"**VQ-VAE** is a type of variational autoencoder that uses vector quantisation to obtain a discrete latent representation. It differs from [VAEs](https://paperswithcode.com/method/vae) in two key ways: the encoder network outputs discrete, rather than continuous, codes; and the prior is learnt rather than static. In order to learn a discrete latent representation, ideas from vector quantisation (VQ) are incorporated. Using the VQ method allows the model to circumvent issues of posterior collapse - where the latents are ignored when they are paired with a powerful autoregressive decoder - typically observed in the VAE framework. Pairing these representations with an autoregressive prior, the model can generate high quality images, videos, and speech as well as doing high quality speaker conversion and unsupervised learning of phonemes.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Neural Discrete Representation Learning","paper":"/paper/neural-discrete-representation-learning","first_author":"Aaron van den Oord","n_authors":3,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/neural-discrete-representation-learning"},"source":{"url":"http://arxiv.org/abs/1711.00937v2","title":"Neural Discrete Representation Learning","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Likelihood-Based Generative Models","url":"/methods/category/likelihood-based-generative-models","pwc_aliases":[]},{"area":"Computer Vision","area_id":"computer-vision","collection":"Generative Models","url":"/methods/category/generative-models","pwc_aliases":[]}],"n_papers_tagged":197,"archive_num_papers":197,"papers_newest_first":[{"paper":"/paper/duetgen-music-driven-two-person-dance","slug":"duetgen-music-driven-two-person-dance","title":"DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling","date":"2025-06-23","arxiv_id":"2506.18680","n_code_links":1,"syntology":null},{"paper":null,"slug":"policy-based-trajectory-clustering-in-offline","title":"Policy-Based Trajectory Clustering in Offline Reinforcement Learning","date":"2025-06-10","arxiv_id":"2506.09202","n_code_links":0,"syntology":null},{"paper":"/paper/star-learning-diverse-robot-skill","slug":"star-learning-diverse-robot-skill","title":"STAR: Learning Diverse Robot Skill Abstractions through Rotation-Augmented Vector Quantization","date":"2025-06-04","arxiv_id":"2506.03863","n_code_links":1,"syntology":null},{"paper":"/paper/vesselgpt-autoregressive-modeling-of-vascular","slug":"vesselgpt-autoregressive-modeling-of-vascular","title":"VesselGPT: Autoregressive Modeling of Vascular Geometry","date":"2025-05-19","arxiv_id":"2505.13318","n_code_links":1,"syntology":null},{"paper":null,"slug":"m3g-multi-granular-gesture-generator-for","title":"M3G: Multi-Granular Gesture Generator for Audio-Driven Full-Body Human Motion Synthesis","date":"2025-05-13","arxiv_id":"2505.08293","n_code_links":0,"syntology":null},{"paper":"/paper/towards-foundation-models-for-experimental","slug":"towards-foundation-models-for-experimental","title":"Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data","date":"2025-05-13","arxiv_id":"2505.08736","n_code_links":1,"syntology":null},{"paper":null,"slug":"ego4o-egocentric-human-motion-capture-and","title":"Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input","date":"2025-04-11","arxiv_id":"2504.08449","n_code_links":0,"syntology":null},{"paper":null,"slug":"instruction-guided-autoregressive-neural","title":"Instruction-Guided Autoregressive Neural Network Parameter Generation","date":"2025-04-02","arxiv_id":"2504.02012","n_code_links":0,"syntology":null},{"paper":null,"slug":"make-some-noise-towards-llm-audio-reasoning","title":"Make Some Noise: Towards LLM audio reasoning and generation using sound tokens","date":"2025-03-28","arxiv_id":"2503.22275","n_code_links":0,"syntology":null},{"paper":null,"slug":"agir-assessing-3d-gait-impairment-with","title":"AGIR: Assessing 3D Gait Impairment with Reasoning based on LLMs","date":"2025-03-23","arxiv_id":"2503.18141","n_code_links":0,"syntology":null},{"paper":null,"slug":"improving-autoregressive-image-generation","title":"Improving Autoregressive Image Generation through Coarse-to-Fine Token Prediction","date":"2025-03-20","arxiv_id":"2503.16194","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-foundation-model-for-patient-behavior","title":"A Foundation Model for Patient Behavior Monitoring and Suicide Detection","date":"2025-03-19","arxiv_id":"2503.15221","n_code_links":0,"syntology":null},{"paper":null,"slug":"genm-3-generative-pretrained-multi-path","title":"GenM$^3$: Generative Pretrained Multi-path Motion Model for Text Conditional Human Motion Generation","date":"2025-03-19","arxiv_id":"2503.14919","n_code_links":0,"syntology":null},{"paper":null,"slug":"bioserenity-e1-a-self-supervised-eeg-model","title":"BioSerenity-E1: a self-supervised EEG model for medical applications","date":"2025-03-13","arxiv_id":"2503.10362","n_code_links":0,"syntology":null},{"paper":null,"slug":"unigenx-unified-generation-of-sequence-and","title":"UniGenX: Unified Generation of Sequence and Structure with Autoregressive Diffusion","date":"2025-03-09","arxiv_id":"2503.06687","n_code_links":0,"syntology":null},{"paper":null,"slug":"enhancing-spoken-discourse-modeling-in","title":"Enhancing Spoken Discourse Modeling in Language Models Using Gestural Cues","date":"2025-03-05","arxiv_id":"2503.03474","n_code_links":0,"syntology":null},{"paper":null,"slug":"caps-context-aware-priority-sampling-for","title":"CAPS: Context-Aware Priority Sampling for Enhanced Imitation Learning in Autonomous Driving","date":"2025-03-03","arxiv_id":"2503.01650","n_code_links":0,"syntology":null},{"paper":null,"slug":"vevo-controllable-zero-shot-voice-imitation","title":"Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement","date":"2025-02-11","arxiv_id":"2502.07243","n_code_links":0,"syntology":null},{"paper":null,"slug":"token-assorted-mixing-latent-and-text-tokens","title":"Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning","date":"2025-02-05","arxiv_id":"2502.03275","n_code_links":0,"syntology":null},{"paper":null,"slug":"patch-aware-vector-quantized-codebook","title":"Patch-aware Vector Quantized Codebook Learning for Unsupervised Visual Defect Detection","date":"2025-01-15","arxiv_id":"2501.09187","n_code_links":0,"syntology":null},{"paper":null,"slug":"hoigpt-learning-long-sequence-hand-object","title":"HOIGPT: Learning Long-Sequence Hand-Object Interaction with Language Models","date":"2025-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"llm-driven-multimodal-and-multi-identity","title":"LLM-driven Multimodal and Multi-Identity Listening Head Generation","date":"2025-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"tar3d-creating-high-quality-3d-assets-via","title":"TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction","date":"2024-12-22","arxiv_id":"2412.16919","n_code_links":0,"syntology":null},{"paper":null,"slug":"csl-l2m-controllable-song-level-lyric-to","title":"CSL-L2M: Controllable Song-Level Lyric-to-Melody Generation Based on Conditional Transformer with Fine-Grained Lyric and Musical Controls","date":"2024-12-13","arxiv_id":"2412.09887","n_code_links":0,"syntology":null},{"paper":null,"slug":"adaptive-2-adaptive-domain-mining-for-fine","title":"Adaptive$^2$: Adaptive Domain Mining for Fine-grained Domain Adaptation Modeling","date":"2024-12-11","arxiv_id":"2412.08198","n_code_links":0,"syntology":null},{"paper":null,"slug":"sweettokenizer-semantic-aware-spatial","title":"SweetTokenizer: Semantic-Aware Spatial-Temporal Tokenizer for Compact Visual Discretization","date":"2024-12-11","arxiv_id":"2412.10443","n_code_links":0,"syntology":null},{"paper":null,"slug":"vqalattent-a-transparent-speech-generation","title":"VQalAttent: a Transparent Speech Generation Pipeline based on Transformer-learned VQ-VAE Latent Space","date":"2024-11-22","arxiv_id":"2411.14642","n_code_links":0,"syntology":null},{"paper":null,"slug":"augmenting-training-data-with-vector","title":"Augmenting Training Data with Vector-Quantized Variational Autoencoder for Classifying RF Signals","date":"2024-10-23","arxiv_id":"2410.18283","n_code_links":0,"syntology":null},{"paper":null,"slug":"gaussian-mixture-vector-quantization-with","title":"Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior","date":"2024-10-14","arxiv_id":"2410.10180","n_code_links":0,"syntology":null},{"paper":"/paper/intermask-3d-human-interaction-generation-via","slug":"intermask-3d-human-interaction-generation-via","title":"InterMask: 3D Human Interaction Generation via Collaborative Masked Modelling","date":"2024-10-13","arxiv_id":"2410.10010","n_code_links":1,"syntology":null},{"paper":"/paper/restructuring-vector-quantization-with-the","slug":"restructuring-vector-quantization-with-the","title":"Restructuring Vector Quantization with the Rotation Trick","date":"2024-10-08","arxiv_id":"2410.06424","n_code_links":3,"syntology":{"ran":7,"of":7,"n_ran_checked":5,"n_instrument":2,"unverified":0,"pointer_only":7,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 2 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["cfifty/rotation_trick","lucidrains/vector-quantize-pytorch"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/echopulse-ecg-controlled-echocardio-grams","slug":"echopulse-ecg-controlled-echocardio-grams","title":"ECHOPulse: ECG controlled echocardio-grams video generation","date":"2024-10-04","arxiv_id":"2410.03143","n_code_links":1,"syntology":{"ran":14,"of":14,"n_ran_checked":11,"n_instrument":3,"unverified":0,"pointer_only":14,"phrase":"14 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 4 honoured, 6 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","official":{"repos":["levyisthebest/echopulse_prelease"],"state":"official (archive's flag): 14 ran","n_ran":14,"n_constructed":0,"n_ran_no_instrument_failure":11,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"collage-collaborative-human-agent-interaction","title":"COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Models","date":"2024-09-30","arxiv_id":"2409.20502","n_code_links":0,"syntology":null},{"paper":"/paper/comparative-analysis-of-pretrained-audio","slug":"comparative-analysis-of-pretrained-audio","title":"Comparative Analysis of Pretrained Audio Representations in Music Recommender Systems","date":"2024-09-13","arxiv_id":"2409.08987","n_code_links":1,"syntology":null},{"paper":"/paper/probtalk3d-non-deterministic-emotion","slug":"probtalk3d-non-deterministic-emotion","title":"ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE","date":"2024-09-12","arxiv_id":"2409.07966","n_code_links":1,"syntology":null},{"paper":null,"slug":"an-end-to-end-approach-for-chord-conditioned","title":"An End-to-End Approach for Chord-Conditioned Song Generation","date":"2024-09-10","arxiv_id":"2409.06307","n_code_links":0,"syntology":null},{"paper":null,"slug":"latent-3d-brain-mri-counterfactual","title":"Latent 3D Brain MRI Counterfactual","date":"2024-09-09","arxiv_id":"2409.05585","n_code_links":0,"syntology":null},{"paper":null,"slug":"2408-01826","title":"GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer","date":"2024-08-03","arxiv_id":"2408.01826","n_code_links":0,"syntology":null},{"paper":null,"slug":"2408-00707","title":"Synthetic dual image generation for reduction of labeling efforts in semantic segmentation of micrographs with a customized metric function","date":"2024-08-01","arxiv_id":"2408.00707","n_code_links":0,"syntology":null},{"paper":null,"slug":"quantised-global-autoencoder-a-holistic","title":"Quantised Global Autoencoder: A Holistic Approach to Representing Visual Data","date":"2024-07-16","arxiv_id":"2407.11913","n_code_links":0,"syntology":null},{"paper":null,"slug":"balance-of-number-of-embedding-and-their","title":"Balance of Number of Embedding and their Dimensions in Vector Quantization","date":"2024-07-06","arxiv_id":"2407.04939","n_code_links":0,"syntology":null},{"paper":null,"slug":"from-audio-encoders-to-piano-judges","title":"From Audio Encoders to Piano Judges: Benchmarking Performance Understanding for Solo Piano","date":"2024-07-05","arxiv_id":"2407.04518","n_code_links":0,"syntology":null},{"paper":"/paper/meshanything-artist-created-mesh-generation","slug":"meshanything-artist-created-mesh-generation","title":"MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers","date":"2024-06-14","arxiv_id":"2406.10163","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["buaacyw/meshanything"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"single-codec-single-codebook-speech-codec","title":"Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech Generation","date":"2024-06-11","arxiv_id":"2406.07422","n_code_links":0,"syntology":null},{"paper":null,"slug":"t2lm-long-term-3d-human-motion-generation","title":"T2LM: Long-Term 3D Human Motion Generation from Multiple Sentences","date":"2024-06-02","arxiv_id":"2406.00636","n_code_links":0,"syntology":null},{"paper":null,"slug":"stratified-avatar-generation-from-sparse","title":"Stratified Avatar Generation from Sparse Observations","date":"2024-05-30","arxiv_id":"2405.20786","n_code_links":0,"syntology":null},{"paper":"/paper/raq-vae-rate-adaptive-vector-quantized","slug":"raq-vae-rate-adaptive-vector-quantized","title":"Rate-Adaptive Quantization: A Multi-Rate Codebook Adaptation for Vector Quantization-based Generative Models","date":"2024-05-23","arxiv_id":"2405.14222","n_code_links":1,"syntology":null},{"paper":"/paper/fadam-adam-is-a-natural-gradient-optimizer","slug":"fadam-adam-is-a-natural-gradient-optimizer","title":"FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information","date":"2024-05-21","arxiv_id":"2405.12807","n_code_links":1,"syntology":null},{"paper":null,"slug":"controllable-image-generation-with-composed","title":"Controllable Image Generation With Composed Parallel Token Prediction","date":"2024-05-10","arxiv_id":"2405.06535","n_code_links":0,"syntology":null},{"paper":"/paper/self-supervised-pre-training-of-text","slug":"self-supervised-pre-training-of-text","title":"Self-supervised Pre-training of Text Recognizers","date":"2024-05-01","arxiv_id":"2405.00420","n_code_links":1,"syntology":null},{"paper":null,"slug":"midget-music-conditioned-3d-dance-generation","title":"MIDGET: Music Conditioned 3D Dance Generation","date":"2024-04-18","arxiv_id":"2404.12062","n_code_links":0,"syntology":null},{"paper":"/paper/closely-interactive-human-reconstruction-with","slug":"closely-interactive-human-reconstruction-with","title":"Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption","date":"2024-04-17","arxiv_id":"2404.11291","n_code_links":1,"syntology":{"ran":4,"of":8,"n_ran_checked":2,"n_instrument":2,"unverified":4,"pointer_only":1,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified","official":{"repos":["boycehbz/humaninteraction"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":"/paper/taming-transformers-for-realistic-lidar-point","slug":"taming-transformers-for-realistic-lidar-point","title":"Taming Transformers for Realistic Lidar Point Cloud Generation","date":"2024-04-08","arxiv_id":"2404.05505","n_code_links":1,"syntology":null},{"paper":"/paper/visual-autoregressive-modeling-scalable-image","slug":"visual-autoregressive-modeling-scalable-image","title":"Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction","date":"2024-04-03","arxiv_id":"2404.02905","n_code_links":3,"syntology":{"ran":7,"of":11,"n_ran_checked":2,"n_instrument":5,"unverified":4,"pointer_only":0,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified","official":{"repos":["FoundationVision/VAR"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"a-novel-audio-representation-for-music-genre","title":"A Novel Audio Representation for Music Genre Identification in MIR","date":"2024-04-01","arxiv_id":"2404.01058","n_code_links":0,"syntology":null},{"paper":"/paper/parco-part-coordinating-text-to-motion","slug":"parco-part-coordinating-text-to-motion","title":"ParCo: Part-Coordinating Text-to-Motion Synthesis","date":"2024-03-27","arxiv_id":"2403.18512","n_code_links":1,"syntology":null},{"paper":null,"slug":"contact-aware-human-motion-generation-from","title":"Contact-aware Human Motion Generation from Textual Descriptions","date":"2024-03-23","arxiv_id":"2403.15709","n_code_links":0,"syntology":null},{"paper":"/paper/dyadic-interaction-modeling-for-social","slug":"dyadic-interaction-modeling-for-social","title":"Dyadic Interaction Modeling for Social Behavior Generation","date":"2024-03-14","arxiv_id":"2403.09069","n_code_links":2,"syntology":null},{"paper":null,"slug":"vector-quantization-for-deep-learning-based","title":"Vector Quantization for Deep-Learning-Based CSI Feedback in Massive MIMO Systems","date":"2024-03-12","arxiv_id":"2403.07355","n_code_links":0,"syntology":null},{"paper":null,"slug":"say-anything-with-any-style","title":"Say Anything with Any Style","date":"2024-03-11","arxiv_id":"2403.06363","n_code_links":0,"syntology":null},{"paper":"/paper/self-supervised-speech-quality-estimation-and","slug":"self-supervised-speech-quality-estimation-and","title":"Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech","date":"2024-02-26","arxiv_id":"2402.16321","n_code_links":1,"syntology":{"ran":5,"of":11,"n_ran_checked":2,"n_instrument":3,"unverified":6,"pointer_only":11,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 6 unverified","official":{"repos":["JasonSWFu/VQscore"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":6,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"structured-world-modeling-via-semantic-vector","title":"Neural Language of Thought Models","date":"2024-02-02","arxiv_id":"2402.01203","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamic-texture-transfer-using-patchmatch-and","title":"Dynamic Texture Transfer using PatchMatch and Transformers","date":"2024-02-01","arxiv_id":"2402.00606","n_code_links":0,"syntology":null},{"paper":null,"slug":"hq-vae-hierarchical-discrete-representation","title":"HQ-VAE: Hierarchical Discrete Representation Learning with Variational Bayes","date":"2023-12-31","arxiv_id":"2401.00365","n_code_links":0,"syntology":null},{"paper":"/paper/fast-gradient-free-activation-maximization","slug":"fast-gradient-free-activation-maximization","title":"Fast gradient-free activation maximization for neurons in spiking neural networks","date":"2023-12-28","arxiv_id":"2401.10748","n_code_links":1,"syntology":null},{"paper":null,"slug":"stylespeech-self-supervised-style-enhancing","title":"StyleSpeech: Self-supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesis","date":"2023-12-19","arxiv_id":"2312.12181","n_code_links":0,"syntology":null},{"paper":null,"slug":"t2m-hifigpt-generating-high-quality-human","title":"T2M-HiFiGPT: Generating High Quality Human Motion from Textual Descriptions with Residual Discrete Representations","date":"2023-12-17","arxiv_id":"2312.10628","n_code_links":0,"syntology":null},{"paper":"/paper/givt-generative-infinite-vocabulary","slug":"givt-generative-infinite-vocabulary","title":"GIVT: Generative Infinite-Vocabulary Transformers","date":"2023-12-04","arxiv_id":"2312.02116","n_code_links":2,"syntology":null},{"paper":null,"slug":"speechact-towards-generating-whole-body","title":"SpeechAct: Towards Generating Whole-body Motion from Speech","date":"2023-11-29","arxiv_id":"2311.17425","n_code_links":0,"syntology":null},{"paper":null,"slug":"tlcontrol-trajectory-and-language-control-for","title":"TLControl: Trajectory and Language Control for Human Motion Synthesis","date":"2023-11-28","arxiv_id":"2311.17135","n_code_links":0,"syntology":null},{"paper":null,"slug":"pre-trained-language-models-do-not-help-auto","title":"Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation","date":"2023-11-27","arxiv_id":"2311.16201","n_code_links":0,"syntology":null},{"paper":null,"slug":"advancements-in-generative-ai-a-comprehensive","title":"Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers","date":"2023-11-17","arxiv_id":"2311.10242","n_code_links":0,"syntology":null},{"paper":null,"slug":"utilizing-vq-vae-for-end-to-end-health","title":"Utilizing VQ-VAE for End-to-End Health Indicator Generation in Predicting Rolling Bearing RUL","date":"2023-11-17","arxiv_id":"2311.10525","n_code_links":0,"syntology":null},{"paper":null,"slug":"scbeacon-single-cell-biomarker-extraction-via","title":"scBeacon: single-cell biomarker extraction via identifying paired cell clusters across biological conditions with contrastive siamese networks","date":"2023-11-05","arxiv_id":"2311.02594","n_code_links":0,"syntology":null},{"paper":"/paper/composer-style-specific-symbolic-music","slug":"composer-style-specific-symbolic-music","title":"Composer Style-specific Symbolic Music Generation Using Vector Quantized Discrete Diffusion Models","date":"2023-10-21","arxiv_id":"2310.14044","n_code_links":1,"syntology":null},{"paper":"/paper/humantomato-text-aligned-whole-body-motion","slug":"humantomato-text-aligned-whole-body-motion","title":"HumanTOMATO: Text-aligned Whole-body Motion Generation","date":"2023-10-19","arxiv_id":"2310.12978","n_code_links":1,"syntology":{"ran":13,"of":18,"n_ran_checked":12,"n_instrument":1,"unverified":5,"pointer_only":18,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 1 honoured, 0 violated, 11 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified","official":{"repos":["IDEA-Research/HumanTOMATO"],"state":"official (archive's flag): 13 ran","n_ran":13,"n_constructed":0,"n_ran_no_instrument_failure":12,"n_unverified":5,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"action-quantized-offline-reinforcement","title":"Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning","date":"2023-10-18","arxiv_id":"2310.11731","n_code_links":0,"syntology":null},{"paper":null,"slug":"ll-vq-vae-learnable-lattice-vector","title":"LL-VQ-VAE: Learnable Lattice Vector-Quantization For Efficient Representations","date":"2023-10-13","arxiv_id":"2310.09382","n_code_links":0,"syntology":null},{"paper":"/paper/edvae-mitigating-codebook-collapse-with","slug":"edvae-mitigating-codebook-collapse-with","title":"EdVAE: Mitigating Codebook Collapse with Evidential Discrete Variational Autoencoders","date":"2023-10-09","arxiv_id":"2310.05718","n_code_links":1,"syntology":null},{"paper":"/paper/finite-scalar-quantization-vq-vae-made-simple","slug":"finite-scalar-quantization-vq-vae-made-simple","title":"Finite Scalar Quantization: VQ-VAE Made Simple","date":"2023-09-27","arxiv_id":"2309.15505","n_code_links":3,"syntology":{"ran":3,"of":3,"n_ran_checked":3,"n_instrument":0,"unverified":0,"pointer_only":2,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 2 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["google-research/google-research"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":null,"slug":"attentive-vq-vae","title":"Attentive VQ-VAE","date":"2023-09-20","arxiv_id":"2309.11641","n_code_links":0,"syntology":null},{"paper":"/paper/attt2m-text-driven-human-motion-generation-1","slug":"attt2m-text-driven-human-motion-generation-1","title":"AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism","date":"2023-09-02","arxiv_id":"2309.00796","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["zcymonkey/attt2m"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/few-shot-font-generation-via-transferring","slug":"few-shot-font-generation-via-transferring","title":"Few shot font generation via transferring similarity guided global style and quantization local style","date":"2023-09-02","arxiv_id":"2309.00827","n_code_links":1,"syntology":{"ran":14,"of":20,"n_ran_checked":10,"n_instrument":4,"unverified":6,"pointer_only":0,"phrase":"14 ran (of which 9 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 4 where Syntology's instrument failed) · 6 unverified","official":{"repos":["awei669/VQ-Font"],"state":"official (archive's flag): 14 ran","n_ran":14,"n_constructed":9,"n_ran_no_instrument_failure":10,"n_unverified":6,"ran_from_kinds":["official"]}}},{"paper":"/paper/qs-tts-towards-semi-supervised-text-to-speech","slug":"qs-tts-towards-semi-supervised-text-to-speech","title":"QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learning","date":"2023-08-31","arxiv_id":"2309.00126","n_code_links":1,"syntology":null},{"paper":null,"slug":"priority-centric-human-motion-generation-in","title":"Priority-Centric Human Motion Generation in Discrete Latent Space","date":"2023-08-28","arxiv_id":"2308.14480","n_code_links":0,"syntology":null},{"paper":null,"slug":"burnt-area-extraction-from-high-resolution","title":"Burnt area extraction from high-resolution satellite images based on anomaly detection","date":"2023-08-25","arxiv_id":"2308.13367","n_code_links":0,"syntology":null},{"paper":null,"slug":"can-language-models-learn-to-listen","title":"Can Language Models Learn to Listen?","date":"2023-08-21","arxiv_id":"2308.10897","n_code_links":0,"syntology":null},{"paper":null,"slug":"neural-categorical-priors-for-physics-based","title":"Neural Categorical Priors for Physics-Based Character Control","date":"2023-08-14","arxiv_id":"2308.07200","n_code_links":0,"syntology":null},{"paper":null,"slug":"controlling-character-motions-without","title":"Controlling Character Motions without Observable Driving Source","date":"2023-08-11","arxiv_id":"2308.06025","n_code_links":0,"syntology":null},{"paper":"/paper/minddiffuser-controlled-image-reconstruction-1","slug":"minddiffuser-controlled-image-reconstruction-1","title":"MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion","date":"2023-08-08","arxiv_id":"2308.04249","n_code_links":1,"syntology":{"ran":1,"of":2,"n_ran_checked":1,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["reedonepeck/minddiffuser"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/vqgraph-graph-vector-quantization-for","slug":"vqgraph-graph-vector-quantization-for","title":"VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs","date":"2023-08-04","arxiv_id":"2308.02117","n_code_links":1,"syntology":{"ran":23,"of":30,"n_ran_checked":10,"n_instrument":13,"unverified":7,"pointer_only":30,"phrase":"23 ran (of which 1 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 4 violated, 5 with no contract checked; 13 where Syntology's instrument failed) · 7 unverified","official":{"repos":["yangling0818/vqgraph"],"state":"official (archive's flag): 23 ran","n_ran":23,"n_constructed":1,"n_ran_no_instrument_failure":10,"n_unverified":7,"ran_from_kinds":["official"]}}},{"paper":"/paper/online-clustered-codebook","slug":"online-clustered-codebook","title":"Online Clustered Codebook","date":"2023-07-27","arxiv_id":"2307.15139","n_code_links":1,"syntology":{"ran":9,"of":10,"n_ran_checked":8,"n_instrument":1,"unverified":1,"pointer_only":2,"phrase":"9 ran (of which 2 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","official":{"repos":["lyndonzheng/cvq-vae"],"state":"official (archive's flag): 9 ran","n_ran":9,"n_constructed":2,"n_ran_no_instrument_failure":8,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/tokenization-with-factorized-subword-encoding","slug":"tokenization-with-factorized-subword-encoding","title":"Tokenization with Factorized Subword Encoding","date":"2023-06-13","arxiv_id":"2306.07764","n_code_links":1,"syntology":null},{"paper":null,"slug":"dualvae-controlling-colours-of-generated-and","title":"DualVAE: Controlling Colours of Generated and Real Images","date":"2023-05-30","arxiv_id":"2305.18769","n_code_links":0,"syntology":null},{"paper":null,"slug":"visual-affordance-prediction-for-guiding","title":"Visual Affordance Prediction for Guiding Robot Exploration","date":"2023-05-28","arxiv_id":"2305.17783","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-one-class-classifier-for-the-detection-of","title":"A One-Class Classifier for the Detection of GAN Manipulated Multi-Spectral Satellite Images","date":"2023-05-19","arxiv_id":"2305.11795","n_code_links":0,"syntology":null},{"paper":"/paper/qpgesture-quantization-based-and-phase-guided-1","slug":"qpgesture-quantization-based-and-phase-guided-1","title":"QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation","date":"2023-05-18","arxiv_id":"2305.11094","n_code_links":1,"syntology":{"ran":4,"of":5,"n_ran_checked":4,"n_instrument":0,"unverified":1,"pointer_only":5,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["youngseng/qpgesture"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/catch-missing-details-image-reconstruction","slug":"catch-missing-details-image-reconstruction","title":"Catch Missing Details: Image Reconstruction with Frequency Augmented Variational Autoencoder","date":"2023-05-04","arxiv_id":"2305.02541","n_code_links":1,"syntology":{"ran":13,"of":15,"n_ran_checked":11,"n_instrument":2,"unverified":2,"pointer_only":3,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 3 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","official":{"repos":["oppo-us-research/FA-VAE"],"state":"official (archive's flag): 13 ran","n_ran":13,"n_constructed":0,"n_ran_no_instrument_failure":11,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/tm2d-bimodality-driven-3d-dance-generation","slug":"tm2d-bimodality-driven-3d-dance-generation","title":"TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration","date":"2023-04-05","arxiv_id":"2304.02419","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":1,"n_instrument":0,"unverified":0,"pointer_only":1,"phrase":"1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result","official":{"repos":["Garfield-kh/TM2D"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":1,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"minddiffuser-controlled-image-reconstruction","title":"MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion","date":"2023-03-24","arxiv_id":"2303.14139","n_code_links":0,"syntology":null}],"papers_shown":100,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":197,"with_a_code_link":82,"where_syntology_ran_a_sample":35,"not_listed_spam_title":0,"listed":197,"listed_where_code_ran":35,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":32,"every_run_a_failure_of_syntologys_instrument":3,"listed_with_a_run_with_no_instrument_failure":32,"listed_every_run_a_failure_of_syntologys_instrument":3,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":35,"papers_where_code_ran_first_page":"/method/vq-vae/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"archive","page":2,"of":2,"url":"/method/vq-vae/papers/2","json":"/method/vq-vae/papers/2.json","rows":97},{"order":"ran","page":1,"of":1,"url":"/method/vq-vae/papers/ran/1","json":"/method/vq-vae/papers/ran/1.json","rows":35}],"tasks":[{"task":"/task/quantization","name":"Quantization","papers":30},{"task":"/task/decoder","name":"Decoder","papers":28},{"task":"/task/image-generation","name":"Image Generation","papers":23},{"task":"/task/representation-learning","name":"Representation Learning","papers":17},{"task":"/task/motion-generation","name":"Motion Generation","papers":13},{"task":"/task/language-modelling","name":"Language Modelling","papers":12},{"task":"/task/motion-synthesis","name":"Motion Synthesis","papers":11},{"task":"/task/image-reconstruction","name":"Image Reconstruction","papers":10},{"task":"/task/language-modeling","name":"Language Modeling","papers":10},{"task":"/task/speech-synthesis","name":"Speech Synthesis","papers":10},{"task":"/task/disentanglement","name":"Disentanglement","papers":7},{"task":"/task/video-generation","name":"Video Generation","papers":7},{"task":"/task/clustering","name":"Clustering","papers":6},{"task":"/task/denoising","name":"Denoising","papers":6},{"task":"/task/diversity","name":"Diversity","papers":6},{"task":"/task/music-generation","name":"Music Generation","papers":6},{"task":"/task/text-to-image-generation","name":"Text-to-Image Generation","papers":6},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":6},{"task":"/task/large-language-model","name":"Large Language Model","papers":5},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":5}],"tasks_shown":20,"n_tasks":171,"usage_by_year":[{"year":"2017","papers":1},{"year":"2018","papers":2},{"year":"2019","papers":6},{"year":"2020","papers":19},{"year":"2021","papers":29},{"year":"2022","papers":31},{"year":"2023","papers":46},{"year":"2024","papers":41},{"year":"2025","papers":22}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/vq-vae"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=VQ-VAE","read_at":"2026-10-05T01:03:47+00:00","query":"VQ-VAE","kind":"method","have":"no_code","tier":"paper_only","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"VQ-VAE","origin_arxiv_id":"2606.21970"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":"for_this_exact_name_only","absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"different_paper","archive_paper_arxiv_id":"1711.00937","repositories":{"listed":null,"in_graph":0,"list_truncated":null,"strongest_link":null},"retrieve_with":null,"name_competition":{"verdict":"outranked","may_be_wrong_sense":true,"bearers_found":2,"served":{"arxiv_id":"2606.21970","citations":null},"leading_other_bearer":{"arxiv_id":"1711.00937","title":"Neural Discrete Representation Learning","citations":7848,"named_by":["acronym_expansion"],"bearer_name":"Vector Quantised-Variational AutoEncoder (VQ-VAE)","ask_with":"have('1711.00937')"},"n_other_bearers_listed":1,"note":"'VQ-VAE' is ALSO the name of 'Vector Quantised-Variational AutoEncoder (VQ-VAE)', a method in 'Neural Discrete Representation Learning' (arXiv:1711.00937, 7,848 citations), which is cited FAR more than the paper this answer is about (citations not recorded). This answer may be about the wrong VQ-VAE. If you meant the better-known one, ask again with that entry of `also_named`."},"name_competition_text":"The same name also belongs to “Vector Quantised-Variational AutoEncoder (VQ-VAE)”, a method in the paper “Neural Discrete Representation Learning” (arXiv:1711.00937, 7,848 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name.","near_matches":null,"code_nearby":[{"name":"SAMS-VAE","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='SAMS-VAE')","origin_arxiv_id":"2311.02794"},{"name":"Chroma-VAE","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='Chroma-VAE')","origin_arxiv_id":"2211.15231"},{"name":"Disentangle Variational Autoencoder (D-VAE)","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='Disentangle Variational Autoencoder (D-VAE)')","origin_arxiv_id":"2405.01460"}],"code_nearby_caption":"Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.","origin_paper":{"arxiv_id":"2606.21970","page":null,"syntology_url":"https://syntology.ai/paper/2606.21970","in_graph_layer":false,"samples":0},"line":"about another paper, arXiv:2606.21970: no code under this exact name (no_code; paper_only). About another paper: this answer is about arXiv:2606.21970, while the archive credits arXiv:1711.00937, so it does not say whether this method has code.","level_legend":{"state":"quoted","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","card_sha256":"479785a84658dd682a11b10237b3620d651d7f87799f5dc7add970aaa83f6d39","note":"Syntology's served verification levels, quoted verbatim from its MCP server card; a row is split where the card separates clauses with a double hyphen, and the separator is not reproduced","levels":[{"level":"V0","quoted":["imports"]},{"level":"V1","quoted":["executes against a synthesized fixture and returns a value of the recorded type"]},{"level":"V2","quoted":["two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs","neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)"]},{"level":"V3","quoted":["paper-derived property tests pass on both of those same two generations."]}],"no_rung":"No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results."}},"syntology_have_state":"loaded","record_sha256":"8f06c2a35593c02ff364b54fef01f8b724dfe4c6cb94ccc5330a9156cfc4aa1e","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}