{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/recognition-of-instrument-tissue-interactions","title":"Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets","arxiv_id":"2007.05405","date":"2020-07-10","proceeding":null,"authors":["Chinedu Innocent Nwoye","Cristians Gonzalez","Tong Yu","Pietro Mascagni","Didier Mutter","Jacques Marescaux","Nicolas Padoy"],"abstract":"Recognition of surgical activity is an essential component to develop context-aware decision support for the operating room. In this work, we tackle the recognition of fine-grained activities, modeled as action triplets <instrument, verb, target> representing the tool activity. To this end, we introduce a new laparoscopic dataset, CholecT40, consisting of 40 videos from the public dataset Cholec80 in which all frames have been annotated using 128 triplet classes. Furthermore, we present an approach to recognize these triplets directly from the video data. It relies on a module called Class Activation Guide (CAG), which uses the instrument activation maps to guide the verb and target recognition. To model the recognition of multiple triplets in the same frame, we also propose a trainable 3D Interaction Space, which captures the associations between the triplet components. Finally, we demonstrate the significance of these contributions via several ablation studies and comparisons to baselines on CholecT40.","url_abs":"https://arxiv.org/abs/2007.05405v1","url_pdf":"https://arxiv.org/pdf/2007.05405v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recognition-of-instrument-tissue-interactions","repo_url":"https://github.com/camma-public/tripnet","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"recognition-of-instrument-tissue-interactions","repo_url":"https://github.com/CAMMA-public/cholect50","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"recognition-of-instrument-tissue-interactions","repo_url":"https://github.com/camma-public/attention-tripnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"recognition-of-instrument-tissue-interactions","repo_url":"https://github.com/camma-public/rendezvous","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"action-localization","task_name":"Action Localization"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"action-triplet-recognition","task_name":"Action Triplet Recognition"},{"task_slug":null,"task_name":"Triplet"},{"task_slug":"weakly-supervised-action-localization","task_name":"Weakly Supervised Action Localization"}],"methods":[{"method_slug":"cag","method_name":"CAG"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"heatmap","method_name":"Heatmap"}],"datasets_introduced":[{"slug":"cholect40","name":"CholecT40","full_name":"Cholecystectomy Action Triplet"}],"methods_introduced":[{"slug":"cag","name":"CAG","full_name":"Class activation guide"}],"results":[{"leaderboard":"/sota/action-triplet-recognition-on-cholect40","task":"Action Triplet Recognition","dataset":"CholecT40","model":"Tripnet","rank_in_archive_order":1,"of":1,"metrics":{"mAP":"18.95"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50","task":"Action Triplet Recognition","dataset":"CholecT50","model":"Tripnet (TensorFlow v1)","rank_in_archive_order":6,"of":6,"metrics":{"Mean AP":"20.0"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Tripnet (TensorFlow v1)","rank_in_archive_order":20,"of":27,"metrics":{"mAP":"23.4"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2007.05405","atlas_url":"https://app.syntology.ai/?focus=2007.05405","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2007.05405"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2007.05405\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2007.05405"},"call":"get_concepts_for_paper(arxiv_id=\"2007.05405\")"}],"by_repo":[{"repo":"camma-public/tripnet","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"CAMMA-public/cholect50","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"camma-public/attention-tripnet","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"camma-public/rendezvous","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"bc4f7573bbf298ea3bc03eee20b17c4b2dc63e0408714d76af8a80053e391262","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}