{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/mapping-between-fmri-responses-to-movies-and","title":"Mapping Between fMRI Responses to Movies and their Natural Language Annotations","arxiv_id":"1610.03914","date":"2016-10-13","proceeding":null,"authors":["Kiran Vodrahalli","Po-Hsuan Chen","YIngyu Liang","Christopher Baldassano","Janice Chen","Esther Yong","Christopher Honey","Uri Hasson","Peter Ramadge","Ken Norman","Sanjeev Arora"],"abstract":"Several research groups have shown how to correlate fMRI responses to the\nmeanings of presented stimuli. This paper presents new methods for doing so\nwhen only a natural language annotation is available as the description of the\nstimulus. We study fMRI data gathered from subjects watching an episode of BBCs\nSherlock [1], and learn bidirectional mappings between fMRI responses and\nnatural language representations. We show how to leverage data from multiple\nsubjects watching the same movie to improve the accuracy of the mappings,\nallowing us to succeed at a scene classification task with 72% accuracy (random\nguessing would give 4%) and at a scene ranking task with average rank in the\ntop 4% (random guessing would give 50%). The key ingredients are (a) the use of\nthe Shared Response Model (SRM) and its variant SRM-ICA [2, 3] to aggregate\nfMRI data from multiple subjects, both of which are shown to be superior to\nstandard PCA in producing low-dimensional representations for the tasks in this\npaper; (b) a sentence embedding technique adapted from the natural language\nprocessing (NLP) literature [4] that produces semantic vector representation of\nthe annotations; (c) using previous timestep information in the featurization\nof the predictor data.","url_abs":"http://arxiv.org/abs/1610.03914v3","url_pdf":"http://arxiv.org/pdf/1610.03914v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mapping-between-fmri-responses-to-movies-and","repo_url":"https://github.com/asprout/CPSC490","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"scene-classification","task_name":"Scene Classification"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-embedding","task_name":"Sentence Embedding"},{"task_slug":"sentence-embedding-1","task_name":"Sentence-Embedding"}],"methods":[{"method_slug":"pca","method_name":"PCA"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1610.03914","atlas_url":"https://app.syntology.ai/?focus=1610.03914","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1610.03914"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1610.03914\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1610.03914"},"call":"get_concepts_for_paper(arxiv_id=\"1610.03914\")"}],"by_repo":[{"repo":"asprout/CPSC490","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"856c7f5d177f98f5657cb03449a1463d37367730f55ca197e66d893eb2fb17c5","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}