{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/arxiv-2604-12374","title":"Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning","arxiv_id":"2604.12374","date":"2026-04-14","proceeding":null,"authors":[" NVIDIA"," :","Aakshita Chandiramani","Aaron Blakeman","Abdullahi Olaoye","Abhibha Gupta","Abhilash Somasamudramath","Abhinav Khattar","Adeola Adesoba","Adi Renduchintala","Adil Asif","Aditya Agrawal","Aditya Vavre","Ahmad Kiswani","Aishwarya Padmakumar","Ajay Hotchandani","Akanksha Shukla","Akhiad Bercovich","Aleksander Ficek","Aleksandr Shaposhnikov","Alex Gronskiy","Alex Kondratenko","Alex Neefus","Alex Steiner","Alex Yang","Alexander Bukharin","Alexander Young","Ali Hatamizadeh","Ali Taghibakhshi","Alina Galiautdinova","Alisa Liu","Alok Kumar","Ameya Sunil Mahabaleshwarkar","Amir Klein","Amit Zuker","Amnon Geifman","Anahita Bhiwandiwalla","Ananth Subramaniam","Andrew Tao","Anjaney Shrivastava","Anjulie Agrusa","Ankur Srivastava","Ankur Verma","Ann Guan","Anna Shors","Annamalai Chockalingam","Anubhav Mandarwal","Aparnaa Ramani","Arham Mehta","Arti Jain","Arun Venkatesan","Asha Anoosheh","Ashwath Aithal","Ashwin Poojary","Asif Ahamed","Asit Mishra","Asli Sabanci Demiroz","Asma Kuriparambil Thekkumpate","Atefeh Sohrabizadeh","Avinash Kaur","Ayush Dattagupta","Barath Subramaniam Anandan","Bardiya Sadeghi","Barnaby Simkin","Ben Lanir","Benedikt Schifferer","Benjamin Chislett","Besmira Nushi","Bilal Kartal","Bill Thiede","Bita Darvish Rouhani","Bobby Chen","Boris Ginsburg","Brandon Norick","Branislav Kisacanin","Brian Yu","Bryan Catanzaro","Buvaneswari Mani","Carlo del Mundo","Chankyu Lee","Chanran Kim","Chantal Hwang","Chao Ni","Charles Wang","Charlie Truong","Cheng-Ping Hsieh","Chenhan Yu","Chenjie Luo","Cherie Wang","Chetan Mungekar","Chintan Patel","Chris Alexiuk","Chris Holguin","Chris Wing","Christian Munley","Christopher Parisien","Chuck Desai","Chunyang Sheng","Collin Neale","Cyril Meurillon","Dakshi Kumar","Dan Gil","Dan Su","Dane Corneil","Daniel Afrimi","Daniel Burkhardt Eliuth Triana","Daniel Egert","Daniel Fatade","Daniel Lo","Daniel Rohrer","Daniel Serebrenik","Daniil Sorokin","Daria Gitman","Daria Levy","Darko Stosic","David Edelsohn","David Messina","David Mosallanezhad","David Tamok","Deena Donia","Deepak Narayanan","Devin O'Kelly","Dheeraj Peri","Dhruv Nathawani","Di Wu","Dima Rekesh","Dina Yared","Divyanshu Kakwani","Dmitry Konyagin Brandon Tuttle","Dong Ahn","Dongfu Jiang","Dorrin Poorkay","Douglas O'Flaherty","Duncan Riach","Dusan Stosic","Dustin Van Stee","Edgar Minasyan","Edward Lin","Eileen Peters Long","Elad Segal","Elena Lantz","Elena Lewis","Ellie Evans","Elliott Ning","Eric Chung","Eric Harper","Eric Pham-Hung","Eric W. Tramel","Erick Galinkin","Erik Pounds","Esti Etrog","Evan Briones","Evan Wu","Evelina Bakhturina","Evgeny Tsykunov","Ewa Dobrowolska","Farshad Saberi Movahed","Farzan Memarian","Fay Wang","Fei Jia","Felipe Soares","Felipe Vieira Frujeri","Feng Chen","Fengguang Lin","Ferenc Galko","Fortuna Zhang","Frankie Siino","Frida Hou","Gantavya Bhatt","Gargi Prasad","Geethapriya Venkataramani","Geetika Gupta","George Armstrong","Gerald Shen","Giulio Borghesi","Gordana Neskovic","Gorkem Batmaz","Grace Lam","Grace Wu","Greg Pauloski","Greyson Davis","Grigor Nalbandyan","Guoming Zhang","Guy Farber","Guyue Huang","Haifeng Qian","Haran Kumar Shiv Kumar","Harry Kim","Harsh Sharma","Hayate Iso","Hayley Ross","Herbert Hum","Herman Sahota","Hexin Wang","Himanshu Soni","Hiren Upadhyay","Huy Nguyen","Iain Cunningham","Ido Galil","Ido Shahaf","Igino Padovani","Igor Gitman","Igor Shovkun","Ikroop Dhillon","Ilya Loshchilov","Ingrid Kelly","Itamar Schen","Itay Levy","Ivan Moshkov","Izik Golan","Izzy Putterman","Jain Tu","Jan Baczek","Jan Kautz","Jane Polak Scowcroft","Janica Rosenberg","Jared Casper","Jarrod Pflum","Jason Grant","Jason Sewall","Jatin Mitra","Jeffrey Glick","Jenny Chen","Jesse Oliver","Jiacheng Xu","Jiafan Zhu","Jialin Song","Jian Zhang","Jiaqi Zeng","Jie Lou","Jill Milton","Jim Chow","Jimmy Zhang","Jinhang Choi","Jining Huang","Jocelyn Huang","Joel Caruso","Joey Conway","Joey Guman","Johan Jatko","John Kamalu","Johnny Greco","Jonathan Cohen","Jonathan Raiman","Joseph Jennings","Joyjit Daw","Juan Yu","Julio Tapia","Junkeun Yi","Jupinder Parmar","Jyothi Achar","Kari Briski","Kartik Mattoo","Katherine Cheung","Katherine Luna","Keith Wyss","Kevin Shih","Kezhi Kong","Khanh Nguyen","Khushi Bhardwaj","Kirill Buryak","Kirthi Shankar Sivamani","Konstantinos Krommydas","Kris Murphy","Krishna C. Puvvada","Krzysztof Pawelec","Kumar Anik","Laikh Tewari","Laya Sleiman","Leo Du","Leon Derczynski","Li Ding","Lilach Ilan","Lingjie Wu","Lizzie Wei","Luis Vega","Lun Su","Maarten Van Segbroeck","Maer Rodrigues de Melo","Magaret Zhang","Mahan Fathi","Makesh Narsimhan Sreedhar","Makesh Sreedhar","Makesh Tarun Chandran","Manuel Reyes Gomez","Maor Ashkenazi","Marc Cuevas","Marc Romeijn","Margaret Zhang","Mark Cai","Mark Gabel","Markus Kliegl","Martyna Patelka","Maryam Moosaei","Matthew Varacalli","Matvei Novikov","Mauricio Ferrato","Mehrzad Samadi","Melissa Corpuz","Meng Xin","Mengdi Wang","Mengru Wang","Meredith Price","Micah Schaffer","Michael Andersch","Michael Boone","Michael Evans","Michael Z Wang","Miguel Martinez","Mikail Khona","Mike Chrzanowski","Mike Hollinger","Mingyuan Ma","Minseok Lee","Mohammad Dabbah","Mohammad Shoeybi","Mostofa Patwary","Nabin Mulepati","Nader Khalil","Najeeb Nabwani","Nancy Agarwal","Nanthini Balasubramaniam","Narimane Hennouni","Narsi Kodukula","Natalie Hereth","Nathaniel Pinckney","Nave Assaf","Negar Habibi","Nestor Qin","Neta Zmora","Netanel Haber","Nick Reamaroon","Nickson Quak","Nidhi Bhatia","Nikhil Jukar","Nikki Pope","Nikolai Ludwig","Nima Tajbakhsh","Nir Ailon","Nirmal Juluru","Nirmalya De","Nowel Pitt","Oleg Rybakov","Oleksii Hrinchuk","Oleksii Kuchaiev","Olivier Delalleau","Oluwatobi Olabiyi","Omer Ullman Argov","Omri Almog","Omri Puny","Oren Tropp","Otavio Padovani","Ouye Xie","Parth Chadha","Pasha Shamis","Paul Gibbons","Pavlo Molchanov","Peter Belcak","Peter Jin","Pinky Xu","Piotr Januszewski","Pooya Jannaty","Prachi Shevate","Pradeep Thalasta","Pranav Prashant Thombre","Prasoon Varshney","Prerana Gambhir","Pritam Gundecha","Przemek Tredak","Qing Miao","Qiyu Wan","Quan Tran Minh","Rabeeh Karimi Mahabadi","Rachel Oberman","Rachit Garg","Rahul Kandu","Raina Zhong","Ran El-Yaniv","Ran Zilberstein","Rasoul Shafipour","Renee Yao","Renjie Pi","Richard Mazzarese","Richard Wang","Rick Izzo","Ridhima Singla","Rima Shahbazyan","Rishabh Garg","Ritika Borkar","Ritu Gala","Riyad Islam","Robert Clark","Robert Hesse","Roger Waleffe","Rohit Varma Kalidindi","Rohit Watve","Roi Koren","Ron Fan","Ruchika Kharwar","Ruisi Cai","Ruoxi Zhang","Russell J. Hewett","Ryan Prenger","Ryan Timbrook","Ryota Egashira","Sadegh Mahdavi","Sagar Singh Ashutosh Joshi","Sahil Modi","Samuel Kriman","Sandeep Pombra","Sanjay Kariyappa","Sanjeev Satheesh","Santiago Pombo","Saori Kaji","Satish Pasumarthi","Saurav Mishra","Saurav Muralidharan","Scott Hara","Sean Narenthiran","Sebastian Rogawski","Seonjin Na","Seonmyeong Bak","Sepehr Sameni","Seth Poulos","Shahar Mor","Shantanu Acharya","Shaona Ghosh Adam Lord","Sharath Turuvekere Sreenivas","Shaun Kotek","Shaya Gharghabi","Shelby Thomas","Sheng-Chieh Lin","Shibani Likhite","Shiqing Fan","Shiyang Chen","Shreya Gopal","Shrimai Prabhumoye","Shubham Pachori","Shubham Toshniwal","Shuo Zhang","Shuoyang Ding","Shyam Renjith","Shyamala Prayaga","Siddhartha Jain","Simeng Sun","Sirisha Rella","Sirshak Das","Smita Ithape","Sneha Harishchandra S","Somshubra Majumdar","Soumye Singhal","Sri Harsha Singudasu","Sriharsha Niverty","Stas Sergienko","Stefana Gloginic","Stefania Alborghetti","Stephen Ge","Stephen McCullough","Sugam Dipak Devare","Suguna Varshini Velury","Sukrit Rao","Sumeet Kumar Barua","Sunny Gai","Suseella Panguluri","Sushil Koundinyan","Swathi Patnam","Sweta Priyadarshi","Swetha Bhendigeri","Syeda Nahida Akter","Sylendran Arunagiri","Tailling Yuan","Talor Abramovich","Tan Bui","Tan Yu","Terry Kong","Thanh Do","Thomas Gburek","Thorgane Marques","Tiffany Moore","Tijmen Blankevoort","Tim Moon","Timothy Ma","Tiyasa Mitra","Tomasz Grzegorzek","Tomer Asida","Tomer Bar Natan","Tomer Keren","Tomer Ronen","Traian Rebedea","Trenton Starkey","Tugrul Konuk","Twinkle Vashishth","Tyler Condensa","Udi Karpas","Ushnish De","Vahid Noorozi","Vahid Noroozi","Vanshil Atul Shah","Veena Vaidyanathan","Venkat Srinivasan","Venmugil Elango","Victor Cui","Vijay Korthikanti","Vikas Mehta","Virginia Adams","Virginia Wu","Vitaly Kurin","Vitaly Lavrukhin","Vladimir Anisimov","Wan Seo","Wanli Jiang","Wasi Uddin Ahmad","Wei Du","Wei Ping","Wei-Ming Chen","Wendy Quan","Wenliang Dai","Wenwen Gao","Will Jennings","William Zhang","Xiaowei Ren","Xiaowen Xin","Xin Li","Yang Yu","Yangyi Chen","Yaniv Galron","Yashaswi Karnati","Yejin Choi","Yev Meyer","Yi-Fu Wu","Yian Zhang","Ying Lin","Yonatan Geifman","Yonggan Fu","Yoshi Suhara","Youngeun Kwon","Yuan Zhang","Yuki Huang","Zach Moshe","Zhilin Wang","Zhiyu Cheng","Zhongbo Zhu","Zhuolin Yang","Zihan Liu","Zijia Chen","Zijie Yan","Zuhair Ahmed"],"abstract":"We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemotron 3 Super is the first model in the Nemotron 3 family to 1) be pre-trained in NVFP4, 2) leverage LatentMoE, a new Mixture-of-Experts architecture that optimizes for both accuracy per FLOP and accuracy per parameter, and 3) include MTP layers for inference acceleration through native speculative decoding. We pre-trained Nemotron 3 Super on 25 trillion tokens followed by post-training using supervised fine tuning (SFT) and reinforcement learning (RL). The final model supports up to 1M context length and achieves comparable accuracy on common benchmarks, while also achieving up to 2.2x and 7.5x higher inference throughput compared to GPT-OSS-120B and Qwen3.5-122B, respectively. Nemotron 3 Super datasets, along with the base, post-trained, and quantized checkpoints, are open-sourced on HuggingFace.","url_abs":"https://arxiv.org/abs/2604.12374","url_pdf":"https://arxiv.org/pdf/2604.12374","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2604.12374","atlas_url":"https://app.syntology.ai/?focus=2604.12374","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.12374"},"calls":[{"tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2604.12374"},"returns":"the samples harvested from this paper, each with its run record; it does not verify the paper"},{"tool":"get_code","arguments_in":"samples[].code_sha256","returns":"one stored sample by its code_sha256; not shown on this site"}]},"developers":"https://syntology.ai/developers","held":true,"reason":"in_graph_layer","checked_against":null,"agent_calls":[{"tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2604.12374"},"call":"get_harvested_code_for_paper(arxiv_id=\"2604.12374\")"},{"tool":"get_code","arguments_in":{"code_sha256":"syntology.samples[].code_sha256"},"call":"get_code(code_sha256=\"…\")"},{"tool":"get_citation_path","arguments":{"paper_1":"2604.12374"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2604.12374\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2604.12374"},"call":"get_concepts_for_paper(arxiv_id=\"2604.12374\")"}],"by_repo":[{"repo":"NVIDIA-NeMo/Evaluator","kind":"found_in_text","samples":9,"ran":8,"instrument":5,"unverified":1},{"repo":"NVIDIA-NeMo/Skills","kind":"found_in_text","samples":15,"ran":12,"instrument":7,"unverified":3}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code","read_at":"2026-10-05T00:36:11+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","ran_record":{"url":"https://syntology.ai/api/ran/2604.12374.json","schema":"syntology.ran_record/1","counts_crosswalk":{"n_samples":"syntology.counts.lifted","n_ran":"syntology.counts.executed","n_ran_checked":"syntology.counts.checked","n_instrument":"syntology.counts.instrument_failures","n_unverified":"syntology.counts.no_recorded_run"},"note":"live record, measured at request time; may be newer than this page's graph read","badge":"https://syntology.ai/api/ran/2604.12374.svg"},"claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/NVIDIA-NeMo/Evaluator","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/NVIDIA-NeMo/Skills","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":12,"ran":4,"ran_violates":3,"unverified":4},"n_samples":24,"n_ran":20,"n_constructed":0,"n_ran_checked":8,"n_instrument":12,"n_unverified":4,"n_instrument_is":"failures of Syntology's instrument, not of the code","every_run_is_an_instrument_failure":false,"key_notes":{"n_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"by_repo_kind":{"found_in_text":{"samples":24,"ran":20,"n_ran_checked":8,"n_instrument":12,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"inline_ok_is":"inline_ok is true only when both licence gates allow Syntology to serve this code's text: this paper's link (inline_ok_link, from the licence of this paper's copy) and the stored copy of the same code that Syntology serves, which may have been first harvested from another repository. withheld_clause is Syntology's own wording of which gate keeps the text a pointer and why, and is null when inline_ok is true. This site shows no code text either way; n_pointer_only_for_licence counts samples with inline_ok false (both gates), n_pointer_only_link_allows those of them whose inline_ok_link is true","n_pointer_only_link_allows":0,"fixture_fields_is":{"signature_and_fixture_out_type":"the arguments Syntology's fixture called the sample with, each as name:type/rank/element kind (? where none was recorded), and the type of what the fixture's run returned, exactly as the harness recorded them; the arguments and the returned type were recorded by two separate runs of the same fixture. No output value is shown, and a returned type says nothing about whether the output was right","level_recorded_at":"when Syntology's graph wrote the sample's verification level, not when the code executed; absent where the level was written before the date was kept, and never filled in with the build date","present_on":"samples that ran only; a key is absent, never empty, where the graph records nothing"},"n_ran_with_level_date":20,"n_ran_without_level_date":0,"n_ran_signature_not_shown":0,"levels_written_between":{"first":"2026-09-25","last":"2026-09-25","is":"the first and last UTC dates on which Syntology's graph wrote a verification level for a sample on this page that ran, not when the code executed"},"code_page_min_papers":5,"code_page_is":"a page for the entry name across every paper's harvested code; the name is a string, not an identity","samples":[{"code_sha256":"2e7aafbc230cb972","code_sha256_prefix":"2e7aafbc230cb972","entry":"calculate_chunk_indices","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/file_utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/file_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"num_samples:int/0/int, num_chunks:int/0/int, chunk_id:int/0/int","fixture_out_type":"tuple","level_recorded_at":"2026-09-25T06:02:23.992132+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"2e7aafbc230cb972"}},{"code_sha256":"dd507a16a8426263","code_sha256_prefix":"dd507a16a8426263","entry":"config_summary","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/run_store.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/run_store.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"config:EvalConfig/0/?","fixture_out_type":"str","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"dd507a16a8426263"}},{"code_sha256":"dfb61cd2ac2b564a","code_sha256_prefix":"dfb61cd2ac2b564a","entry":"determine_proof_status","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/code_execution/proof_utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/code_execution/proof_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"compiler_output:dict/0/?","fixture_out_type":"str","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"dfb61cd2ac2b564a"}},{"code_sha256":"b4bbfbf06bb92de6","code_sha256_prefix":"b4bbfbf06bb92de6","entry":"extract_code_output","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/code_execution/utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/code_execution/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"signature":"generation:str/0/str, code_output_begin:str/0/str, code_output_end:str/0/str, extract_all:bool/0/bool","fixture_out_type":"str","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","retrieve_with":{"tool":"get_code","code_sha256":"b4bbfbf06bb92de6"},"code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"b4bbfbf06bb92de6"}},{"code_sha256":"b75a1053974f21ce","code_sha256_prefix":"b75a1053974f21ce","entry":"extract_code_to_execute","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/code_execution/utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/code_execution/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"signature":"generation:str/0/str, code_begin:str/0/str, code_end:str/0/str, extract_all:bool/0/bool","fixture_out_type":"str","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","retrieve_with":{"tool":"get_code","code_sha256":"b75a1053974f21ce"},"code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"b75a1053974f21ce"}},{"code_sha256":"df5a88c92c8f57ce","code_sha256_prefix":"df5a88c92c8f57ce","entry":"extract_proof_only","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/code_execution/proof_utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/code_execution/proof_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"signature":"lean_code:str/0/str","fixture_out_type":"str","level_recorded_at":"2026-09-25T06:32:53.441431+00:00","retrieve_with":{"tool":"get_code","code_sha256":"df5a88c92c8f57ce"},"code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"df5a88c92c8f57ce"}},{"code_sha256":"b2186409cb2b7dfb","code_sha256_prefix":"b2186409cb2b7dfb","entry":"format_code_output","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/code_execution/utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/code_execution/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"execution_dict:dict/0/?, code_output_begin:str/0/str, code_output_end:str/0/str, code_output_format:str/0/str, remaining_code_executions:int/0/int","fixture_out_type":"str","level_recorded_at":"2026-09-25T06:32:53.441431+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"b2186409cb2b7dfb"}},{"code_sha256":"866f54c7c1ed4422","code_sha256_prefix":"866f54c7c1ed4422","entry":"generate_run_id","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/run_store.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/run_store.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"config:EvalConfig/0/?","fixture_out_type":"str","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"866f54c7c1ed4422"}},{"code_sha256":"364d75dfb259e939","code_sha256_prefix":"364d75dfb259e939","entry":"get_logger_name","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"signature":"file:str/0/str","fixture_out_type":"str","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","retrieve_with":{"tool":"get_code","code_sha256":"364d75dfb259e939"},"code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"364d75dfb259e939"}},{"code_sha256":"28ecc1878ccced7c","code_sha256_prefix":"28ecc1878ccced7c","entry":"import_score_module","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/evaluation/compute_group_score.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/evaluation/compute_group_score.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"score_module:str/0/str","fixture_out_type":"module","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"28ecc1878ccced7c"}},{"code_sha256":"a2afdfc356abe1d1","code_sha256_prefix":"a2afdfc356abe1d1","entry":"init_wandb","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"project:str/0/str, name:str/0/str, exp_dir:str/0/str, verbose:bool/0/bool","fixture_out_type":"bool","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":"/code/init-wandb","census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"a2afdfc356abe1d1"}},{"code_sha256":"ef8cafe3279feab7","code_sha256_prefix":"ef8cafe3279feab7","entry":"is_compaction_call_kind","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/adapters/call_kind.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/adapters/call_kind.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"value:str/0/str","fixture_out_type":"bool","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"ef8cafe3279feab7"}},{"code_sha256":"7ca72af77a32990e","code_sha256_prefix":"7ca72af77a32990e","entry":"is_text_completions_endpoint_type","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/completions_guard.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/completions_guard.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"endpoint_type:str/0/str","fixture_out_type":"bool","level_recorded_at":"2026-09-25T06:32:53.441431+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"7ca72af77a32990e"}},{"code_sha256":"34aaeffdd36d0505","code_sha256_prefix":"34aaeffdd36d0505","entry":"is_text_completions_path","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/completions_guard.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/completions_guard.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"path:str/0/str","fixture_out_type":"bool","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"34aaeffdd36d0505"}},{"code_sha256":"8c48543062658550","code_sha256_prefix":"8c48543062658550","entry":"jload","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/file_utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/file_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"filepath:str/0/str, mode:str/0/str, verbose:bool/0/bool","fixture_out_type":"list","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":"/code/jload","census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"8c48543062658550"}},{"code_sha256":"6f55869f2815d973","code_sha256_prefix":"6f55869f2815d973","entry":"load_metric_files","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/evaluation/compute_group_score.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/evaluation/compute_group_score.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"metric_files:list/1/str","fixture_out_type":"dict","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"6f55869f2815d973"}},{"code_sha256":"4b6af30dc8edc4af","code_sha256_prefix":"4b6af30dc8edc4af","entry":"load_run_meta","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/run_store.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/run_store.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"output_dir:PosixPath/0/?","fixture_out_type":"RunMeta","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"4b6af30dc8edc4af"}},{"code_sha256":"04ad34e04aa0fc45","code_sha256_prefix":"04ad34e04aa0fc45","entry":"merge_compaction_extra_headers","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/adapters/call_kind.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/adapters/call_kind.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"kwargs:dict/0/?","fixture_out_type":"dict","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"04ad34e04aa0fc45"}},{"code_sha256":"4261064eb4ec7756","code_sha256_prefix":"4261064eb4ec7756","entry":"setup_logging","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"disable_hydra_logs:bool/0/bool, log_level:int/0/int, use_rich:bool/0/bool","fixture_out_type":"Logger","level_recorded_at":"2026-09-25T04:30:57.282821+00:00","code_page":"/code/setup-logging","census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"4261064eb4ec7756"}},{"code_sha256":"fd60226680534b23","code_sha256_prefix":"fd60226680534b23","entry":"strip_call_kind_header","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/adapters/call_kind.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/adapters/call_kind.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"signature":"headers:dict/0/?","fixture_out_type":"str","level_recorded_at":"2026-09-25T06:02:23.992132+00:00","code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"fd60226680534b23"}},{"code_sha256":"5b3c40e3214faf87","code_sha256_prefix":"5b3c40e3214faf87","entry":"count_newlines","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/file_utils.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/file_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"5b3c40e3214faf87"}},{"code_sha256":"e74354657e62b51d","code_sha256_prefix":"e74354657e62b51d","entry":"load_config","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/conversion/hf_to_nemo_llama.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/conversion/hf_to_nemo_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"code_page":"/code/load-config","census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"e74354657e62b51d"}},{"code_sha256":"de5cce096268a65e","code_sha256_prefix":"de5cce096268a65e","entry":"load_config","repo":"NVIDIA-NeMo/Skills","repo_kind":"found_in_text","path":"nemo_skills/conversion/hf_to_nemo_qwen.py","file_url":"https://github.com/NVIDIA-NeMo/Skills/blob/HEAD/nemo_skills/conversion/hf_to_nemo_qwen.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"code_page":"/code/load-config","census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"de5cce096268a65e"}},{"code_sha256":"8dff29d46617515d","code_sha256_prefix":"8dff29d46617515d","entry":"normalize_adapter_config_for_endpoint","repo":"NVIDIA-NeMo/Evaluator","repo_kind":"found_in_text","path":"src/nemo_evaluator/completions_guard.py","file_url":"https://github.com/NVIDIA-NeMo/Evaluator/blob/HEAD/src/nemo_evaluator/completions_guard.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"code_page":null,"census":null,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"mcp_get_code":{"code_sha256":"8dff29d46617515d"}}]},"code_links_note":"this paper is newer than the archive, which holds no code links for it; its repositories, from Syntology's graph, are under syntology.repos","arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 8,886 of the 18,637 papers on this site that are newer than the archive (for 3,717 of them no archive leaderboard matched the paper's tables, so there was nothing further to check); 733 were read and have nothing to place (arXiv has no HTML version of the paper, or that version has no tables), 41 could not be read (the extractor's reply could not be parsed), and results from the other 8,977 appear after they are checked.","papers_newer_than_archive":18637,"papers_checked":8886,"papers_read_nothing_to_place":733,"papers_could_not_be_read":41},"entries":[],"judged_to_report_on":{"model":"Claude Sonnet 4.5","model_as_recorded":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","note":"a model's judgement from the paper's own tables, not a result; outcome is what Syntology's checks did with it; reason is the rules' plain words, only for refused_by_rule","boards":[{"leaderboard":"/sota/arithmetic-reasoning-on-gsm8k","slug":"arithmetic-reasoning-on-gsm8k","task":"Arithmetic Reasoning","dataset":"GSM8K","outcome":"refused_by_rule","reason":"the table's column header does not name the leaderboard's metric"},{"leaderboard":"/sota/common-sense-reasoning-on-arc-challenge","slug":"common-sense-reasoning-on-arc-challenge","task":"Common Sense Reasoning","dataset":"ARC (Challenge)","outcome":"refused_by_rule","reason":"the table's column header does not name the leaderboard's metric"},{"leaderboard":"/sota/common-sense-reasoning-on-winogrande","slug":"common-sense-reasoning-on-winogrande","task":"Common Sense Reasoning","dataset":"WinoGrande","outcome":"refused_by_rule","reason":"the table's column header does not name the leaderboard's metric"},{"leaderboard":"/sota/multi-task-language-understanding-on-mgsm","slug":"multi-task-language-understanding-on-mgsm","task":"Multi-task Language Understanding","dataset":"MGSM","outcome":"refused_by_rule","reason":"the table's column header does not name the leaderboard's metric"},{"leaderboard":"/sota/multi-task-language-understanding-on-mmlu-5-1","slug":"multi-task-language-understanding-on-mmlu-5-1","task":"Multi-task Language Understanding","dataset":"MMLU (5-Shot)","outcome":"refused_by_rule","reason":"the table's column header does not name the leaderboard's metric"},{"leaderboard":"/sota/reading-comprehension-on-race","slug":"reading-comprehension-on-race","task":"Reading Comprehension","dataset":"RACE","outcome":"refused_by_rule","reason":"the table's column header does not name the leaderboard's metric"}]},"not_placed":{"boards":6,"rejected_by_independent_check":0,"refused_by_a_rule":6,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}},"record_sha256":"fc3442c19b80ba85531e4de5dfa47dc9587e89704b2676a574bf7f27d54ce97b","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}