{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/video-relationship-reasoning-using-gated","title":"Video Relationship Reasoning using Gated Spatio-Temporal Energy Graph","arxiv_id":"1903.10547","date":"2019-03-25","proceeding":"CVPR 2019 6","authors":["Yao-Hung Hubert Tsai","Santosh Divvala","Louis-Philippe Morency","Ruslan Salakhutdinov","Ali Farhadi"],"abstract":"Visual relationship reasoning is a crucial yet challenging task for\nunderstanding rich interactions across visual concepts. For example, a\nrelationship 'man, open, door' involves a complex relation 'open' between\nconcrete entities 'man, door'. While much of the existing work has studied this\nproblem in the context of still images, understanding visual relationships in\nvideos has received limited attention. Due to their temporal nature, videos\nenable us to model and reason about a more comprehensive set of visual\nrelationships, such as those requiring multiple (temporal) observations (e.g.,\n'man, lift up, box' vs. 'man, put down, box'), as well as relationships that\nare often correlated through time (e.g., 'woman, pay, money' followed by\n'woman, buy, coffee'). In this paper, we construct a Conditional Random Field\non a fully-connected spatio-temporal graph that exploits the statistical\ndependency between relational entities spatially and temporally. We introduce a\nnovel gated energy function parametrization that learns adaptive relations\nconditioned on visual observations. Our model optimization is computationally\nefficient, and its space computation complexity is significantly amortized\nthrough our proposed parameterization. Experimental results on benchmark video\ndatasets (ImageNet Video and Charades) demonstrate state-of-the-art performance\nacross three standard relationship reasoning tasks: Detection, Tagging, and\nRecognition.","url_abs":"http://arxiv.org/abs/1903.10547v2","url_pdf":"http://arxiv.org/pdf/1903.10547v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"video-relationship-reasoning-using-gated","repo_url":"https://github.com/yaohungt/GSTEG_CVPR_2019","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"model-optimization","task_name":"Model Optimization"},{"task_slug":"video-relationship","task_name":"Video Relationship"},{"task_slug":"video-visual-relation-detection","task_name":"Video Visual Relation Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1903.10547","atlas_url":"https://app.syntology.ai/?focus=1903.10547","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1903.10547"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1903.10547\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1903.10547"},"call":"get_concepts_for_paper(arxiv_id=\"1903.10547\")"}],"by_repo":[{"repo":"yaohungt/GSTEG_CVPR_2019","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["1705.07750"],"harvested_for_calls":[{"arxiv_id":"1705.07750","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"1705.07750"},"call":"get_harvested_code_for_paper(arxiv_id=\"1705.07750\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"1f626a3a729434f72c9e1eba2df88a277cfed0757021bc0bb4105262a9e05a14","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}