{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/pointclip-point-cloud-understanding-by-clip","title":"PointCLIP: Point Cloud Understanding by CLIP","arxiv_id":"2112.02413","date":"2021-12-04","proceeding":"CVPR 2022 1","authors":["Renrui Zhang","Ziyu Guo","Wei zhang","Kunchang Li","Xupeng Miao","Bin Cui","Yu Qiao","Peng Gao","Hongsheng Li"],"abstract":"Recently, zero-shot and few-shot learning via Contrastive Vision-Language Pre-training (CLIP) have shown inspirational performance on 2D visual recognition, which learns to match images with their corresponding texts in open-vocabulary settings. However, it remains under explored that whether CLIP, pre-trained by large-scale image-text pairs in 2D, can be generalized to 3D recognition. In this paper, we identify such a setting is feasible by proposing PointCLIP, which conducts alignment between CLIP-encoded point cloud and 3D category texts. Specifically, we encode a point cloud by projecting it into multi-view depth maps without rendering, and aggregate the view-wise zero-shot prediction to achieve knowledge transfer from 2D to 3D. On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D. By just fine-tuning the lightweight adapter in the few-shot settings, the performance of PointCLIP could be largely improved. In addition, we observe the complementary property between PointCLIP and classical 3D-supervised networks. By simple ensembling, PointCLIP boosts baseline's performance and even surpasses state-of-the-art models. Therefore, PointCLIP is a promising alternative for effective 3D point cloud understanding via CLIP under low resource cost and data regime. We conduct thorough experiments on widely-adopted ModelNet10, ModelNet40 and the challenging ScanObjectNN to demonstrate the effectiveness of PointCLIP. The code is released at https://github.com/ZrrSkywalker/PointCLIP.","url_abs":"https://arxiv.org/abs/2112.02413v1","url_pdf":"https://arxiv.org/pdf/2112.02413v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pointclip-point-cloud-understanding-by-clip","repo_url":"https://github.com/zrrskywalker/pointclip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"pointclip-point-cloud-understanding-by-clip","repo_url":"https://github.com/pku-dair/hetu","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-open-vocabulary-instance-segmentation","task_name":"3D Open-Vocabulary Instance Segmentation"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"open-vocabulary-object-detection","task_name":"Open Vocabulary Object Detection"},{"task_slug":"training-free-3d-part-segmentation","task_name":"Training-free 3D Part Segmentation"},{"task_slug":"training-free-3d-point-cloud-classification","task_name":"Training-free 3D Point Cloud Classification"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"zero-shot-transfer-3d-point-cloud","task_name":"Zero-Shot Transfer 3D Point Cloud Classification"},{"task_slug":"zero-shot-3d-point-cloud-classification","task_name":"Zero-shot 3D Point Cloud Classification"},{"task_slug":"zero-shot-3d-classification","task_name":"Zero-shot 3D classification"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-open-vocabulary-instance-segmentation-on-3","task":"3D Open-Vocabulary Instance Segmentation","dataset":"STPLS3D","model":"PointCLIP","rank_in_archive_order":3,"of":3,"metrics":{"AP50":"02.6"},"uses_additional_data":false},{"leaderboard":"/sota/training-free-3d-part-segmentation-on","task":"Training-free 3D Part Segmentation","dataset":"ShapeNet-Part","model":"PointCLIP","rank_in_archive_order":3,"of":3,"metrics":{"Need 3D Data?":"No","mIoU":"31.0"},"uses_additional_data":true},{"leaderboard":"/sota/training-free-3d-point-cloud-classification","task":"Training-free 3D Point Cloud Classification","dataset":"ModelNet40","model":"PointCLIP","rank_in_archive_order":7,"of":7,"metrics":{"Accuracy (%)":"20.2","Need 3D Data?":"No"},"uses_additional_data":true},{"leaderboard":"/sota/training-free-3d-point-cloud-classification-1","task":"Training-free 3D Point Cloud Classification","dataset":"ScanObjectNN","model":"PointCLIP","rank_in_archive_order":6,"of":6,"metrics":{"Accuracy (%)":"15.4","Need 3D Data?":"No"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-transfer-3d-point-cloud-1","task":"Zero-Shot Transfer 3D Point Cloud Classification","dataset":"ModelNet10","model":"PointCLIP","rank_in_archive_order":4,"of":4,"metrics":{"Accuracy (%)":"30.23"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-transfer-3d-point-cloud","task":"Zero-Shot Transfer 3D Point Cloud Classification","dataset":"ModelNet40","model":"PointCLIP","rank_in_archive_order":16,"of":16,"metrics":{"Accuracy (%)":"20.18"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-transfer-3d-point-cloud-2","task":"Zero-Shot Transfer 3D Point Cloud Classification","dataset":"ScanObjectNN","model":"PointCLIP","rank_in_archive_order":10,"of":10,"metrics":{"OBJ_BG Accuracy(%)":"21.34","OBJ_ONLY Accuracy(%)":"19.28","PB_T50_RS Accuracy (%)":"15.38"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2112.02413","atlas_url":"https://app.syntology.ai/?focus=2112.02413","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2112.02413"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2112.02413\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2112.02413"},"call":"get_concepts_for_paper(arxiv_id=\"2112.02413\")"}],"by_repo":[{"repo":"zrrskywalker/pointclip","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"pku-dair/hetu","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2211.13878"],"harvested_for_calls":[{"arxiv_id":"2211.13878","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2211.13878"},"call":"get_harvested_code_for_paper(arxiv_id=\"2211.13878\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"3ce44d619488ef5a8914c61396926c5986b7eed07c2d3171d47e677cb655318b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}