{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/handsformer-keypoint-transformer-for","title":"Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation","arxiv_id":"2104.14639","date":"2021-04-29","proceeding":"CVPR 2022 1","authors":["Shreyas Hampali","Sayan Deb Sarkar","Mahdi Rad","Vincent Lepetit"],"abstract":"We propose a robust and accurate method for estimating the 3D poses of two hands in close interaction from a single color image. This is a very challenging problem, as large occlusions and many confusions between the joints may happen. State-of-the-art methods solve this problem by regressing a heatmap for each joint, which requires solving two problems simultaneously: localizing the joints and recognizing them. In this work, we propose to separate these tasks by relying on a CNN to first localize joints as 2D keypoints, and on self-attention between the CNN features at these keypoints to associate them with the corresponding hand joint. The resulting architecture, which we call \"Keypoint Transformer\", is highly efficient as it achieves state-of-the-art performance with roughly half the number of model parameters on the InterHand2.6M dataset. We also show it can be easily extended to estimate the 3D pose of an object manipulated by one or two hands with high performance. Moreover, we created a new dataset of more than 75,000 images of two hands manipulating an object fully annotated in 3D and will make it publicly available.","url_abs":"https://arxiv.org/abs/2104.14639v2","url_pdf":"https://arxiv.org/pdf/2104.14639v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"handsformer-keypoint-transformer-for","repo_url":"https://github.com/shreyashampali/kypt_transformer","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"3d-hand-pose-estimation","task_name":"3D Hand Pose Estimation"},{"task_slug":"3d-interacting-hand-pose-estimation","task_name":"3D Interacting Hand Pose Estimation"},{"task_slug":"3d-pose-estimation","task_name":"3D Pose Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"hand-object-pose","task_name":"hand-object pose"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"heatmap","method_name":"Heatmap"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-hand-pose-estimation-on-ho-3d","task":"3D Hand Pose Estimation","dataset":"HO-3D v2","model":"KPT-Transformer","rank_in_archive_order":18,"of":24,"metrics":{"AUC_J":"0.786","PA-MPJPE (mm)":"10.8"},"uses_additional_data":false},{"leaderboard":"/sota/3d-hand-pose-estimation-on-ho-3d-v3","task":"3D Hand Pose Estimation","dataset":"HO-3D v3","model":"KPT-Transformer","rank_in_archive_order":7,"of":8,"metrics":{"AUC_J":"0.785","PA-MPJPE":"10.9"},"uses_additional_data":false},{"leaderboard":"/sota/3d-interacting-hand-pose-estimation-on","task":"3D Interacting Hand Pose Estimation","dataset":"InterHand2.6M","model":"Keypoint Transformer","rank_in_archive_order":6,"of":9,"metrics":{"MPJPE Test":"12.78","MPVPE Test":"-","MRRPE Test":"29.63"},"uses_additional_data":false},{"leaderboard":"/sota/hand-object-pose-on-ho-3d","task":"hand-object pose","dataset":"HO-3D v2","model":"Keypoint-Trans","rank_in_archive_order":4,"of":9,"metrics":{"ADD-S":"21.4","Average MPJPE (mm)":"25.5","OME":"68.0","PA-MPJPE":"10.8","ST-MPJPE":"25.7"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2104.14639","atlas_url":"https://app.syntology.ai/?focus=2104.14639","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2104.14639"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2104.14639\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2104.14639"},"call":"get_concepts_for_paper(arxiv_id=\"2104.14639\")"}],"by_repo":[{"repo":"shreyashampali/kypt_transformer","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"5c39000aa0a80252a26fe0704e4ceb05d3ec5691e63e5b7b131cb278b2cf5646","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}