{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/identity-preserving-text-to-video-generation","title":"Identity-Preserving Text-to-Video Generation by Frequency Decomposition","arxiv_id":"2411.17440","date":"2024-11-26","proceeding":"CVPR 2025 1","authors":["Shenghai Yuan","Jinfa Huang","Xianyi He","Yunyuan Ge","Yujun Shi","Liuhan Chen","Jiebo Luo","Li Yuan"],"abstract":"Identity-preserving text-to-video (IPT2V) generation aims to create high-fidelity videos with consistent human identity. It is an important task in video generation but remains an open problem for generative models. This paper pushes the technical frontier of IPT2V in two directions that have not been resolved in literature: (1) A tuning-free pipeline without tedious case-by-case finetuning, and (2) A frequency-aware heuristic identity-preserving DiT-based control scheme. We propose ConsisID, a tuning-free DiT-based controllable IPT2V model to keep human identity consistent in the generated video. Inspired by prior findings in frequency analysis of diffusion transformers, it employs identity-control signals in the frequency domain, where facial features can be decomposed into low-frequency global features and high-frequency intrinsic features. First, from a low-frequency perspective, we introduce a global facial extractor, which encodes reference images and facial key points into a latent space, generating features enriched with low-frequency information. These features are then integrated into shallow layers of the network to alleviate training challenges associated with DiT. Second, from a high-frequency perspective, we design a local facial extractor to capture high-frequency details and inject them into transformer blocks, enhancing the model's ability to preserve fine-grained features. We propose a hierarchical training strategy to leverage frequency information for identity preservation, transforming a vanilla pre-trained video generation model into an IPT2V model. Extensive experiments demonstrate that our frequency-aware heuristic scheme provides an optimal control solution for DiT-based models. Thanks to this scheme, our ConsisID generates high-quality, identity-preserving videos, making strides towards more effective IPT2V. Code: https://github.com/PKU-YuanGroup/ConsisID.","url_abs":"https://arxiv.org/abs/2411.17440v3","url_pdf":"https://arxiv.org/pdf/2411.17440v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"identity-preserving-text-to-video-generation","repo_url":"https://github.com/PKU-YuanGroup/ConsisID","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"human-domain-subject-to-video","task_name":"Human-Domain Subject-to-Video"},{"task_slug":"image-to-video","task_name":"Image to Video Generation"},{"task_slug":"open-domain-subject-to-video","task_name":"Open-Domain Subject-to-Video"},{"task_slug":"text-to-video-generation","task_name":"Text-to-Video Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[{"slug":"consisid-preview-data","name":"ConsisID-preview-Data","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/open-domain-subject-to-video-on-opens2v-eval","task":"Open-Domain Subject-to-Video","dataset":"OpenS2V-Eval","model":"Kling 1.6","rank_in_archive_order":1,"of":10,"metrics":{"Aesthetics":"0.4460","FaceSim":"0.4010","GmeScore":"0.6620","Motion":"0.4160","NaturalScore":"0.7906","NexusScore":"0.4592","Total Score":"0.5446","Venue":"Close-Source"},"uses_additional_data":false},{"leaderboard":"/sota/open-domain-subject-to-video-on-opens2v-eval","task":"Open-Domain Subject-to-Video","dataset":"OpenS2V-Eval","model":"Pika 2.1","rank_in_archive_order":6,"of":10,"metrics":{"Aesthetics":"0.4687","FaceSim":"0.3080","GmeScore":"0.6921","Motion":"0.2470","NaturalScore":"0.6979","NexusScore":"0.4541","Total Score":"0.4888","Venue":"Close-Source"},"uses_additional_data":false},{"leaderboard":"/sota/open-domain-subject-to-video-on-opens2v-eval","task":"Open-Domain Subject-to-Video","dataset":"OpenS2V-Eval","model":"Vidu 2.0","rank_in_archive_order":8,"of":10,"metrics":{"Aesthetics":"0.4147","FaceSim":"0.3511","GmeScore":"0.6757","Motion":"0.1352","NaturalScore":"0.7144","NexusScore":"0.4355","Total Score":"0.4759","Venue":"Close-Source"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.17440","atlas_url":"https://app.syntology.ai/?focus=2411.17440","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2411.17440"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2411.17440\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2411.17440"},"call":"get_concepts_for_paper(arxiv_id=\"2411.17440\")"}],"by_repo":[{"repo":"PKU-YuanGroup/ConsisID","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2505.20292"],"harvested_for_calls":[{"arxiv_id":"2505.20292","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2505.20292"},"call":"get_harvested_code_for_paper(arxiv_id=\"2505.20292\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"2d93995bad1361c93014cb56a4c61c13e505ef77645a562f116448fda369a380","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}