{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/task/rag","name":"RAG","slug":"rag","description_markdown":"Retrieval-Augmented Generation (RAG) is a task that combines the strengths of both retrieval-based models and generation-based models. In this approach, a retrieval system selects relevant documents or passages from a large corpus, and a generation model, typically a neural language model, uses the retrieved information to generate a response. This method enhances the accuracy and coherence of generated text, especially in tasks requiring detailed knowledge or long context handling.\r\n\r\nRAG is particularly useful in open-domain question answering, knowledge-grounded dialogue, and summarization tasks. The retrieval step helps the model to access and incorporate external information, making it less reliant on memorized knowledge and better suited for generating responses based on the latest or domain-specific information.\r\n\r\nThe performance of RAG systems is usually measured using metrics such as precision, recall, F1 score, BLEU score, and exact match. Some popular datasets for evaluating RAG models include Natural Questions, MS MARCO, TriviaQA, and SQuAD.","categories":[{"name":"Knowledge Base","url":"/area/knowledge-base"},{"name":"Methodology","url":"/area/methodology"},{"name":"Natural Language Processing","url":"/area/natural-language-processing"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":2111,"papers_with_code":758,"benchmarks":1,"benchmark_tables_in_archive":1,"benchmark_tables_shown":1,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":6,"subtasks":0,"parent_tasks":1},"benchmarks":[{"leaderboard":"/sota/rag-on-pubmedqa-corpus-with-metadata","slug":"rag-on-pubmedqa-corpus-with-metadata","dataset":"PubMedQA corpus with metadata","dataset_url":"/dataset/pubmedqa-corpus-with-metadata","rows_in_archive":1,"metrics":["ANS-EM"],"first_row_in_archive_order":{"model":"MetaGen Blended RAG","paper_title":"MetaGen Blended RAG: Higher Accuracy for Domain-Specific Q&A Without Fine-Tuning","paper_url":"/paper/metagen-blended-rag-higher-accuracy-for","paper_date":"2025-05-23","arxiv_id":"2505.18247","code_links":[{"title":"ibm-self-serve-assets/metagen-blended-rag","url":"https://github.com/ibm-self-serve-assets/metagen-blended-rag"}],"syntology":null}}],"datasets":[{"url":"/dataset/peerqa","name":"PeerQA","full_name":"","num_papers_in_archive":12},{"url":"/dataset/compmix-ir","name":"CompMix-IR","full_name":"","num_papers_in_archive":1},{"url":"/dataset/crsb","name":"CRSB","full_name":"Context Retrieval Supervision Benchmark","num_papers_in_archive":1},{"url":"/dataset/frames-part","name":"Frames (part)","full_name":"","num_papers_in_archive":1},{"url":"/dataset/pubmedqa-corpus-with-metadata","name":"PubMedQA corpus with metadata","full_name":"","num_papers_in_archive":1},{"url":"/dataset/riskdata","name":"RiskData","full_name":"","num_papers_in_archive":1}],"subtasks":[],"parent_tasks":[{"url":"/task/large-language-model","name":"Large Language Model"}],"papers":{"order":"repositories listed in the archive (most first), then date (newest first), then slug; the archive holds no stars","population":"every paper the archive tags with this task (distinct papers; spam titles not listed). 'of' counts the papers listed; until 2026-09-25 it counted only the papers with a code link, and that number is of_with_code","page":1,"pages_in_order":22,"rows_per_page":100,"shown":100,"of":2111,"of_with_code":758,"tagged_in_all":2111,"counts":{"archive_papers_tagged":2111,"with_a_code_link":758,"where_syntology_ran_a_sample":211,"not_listed_spam_title":0,"listed":2111,"listed_where_code_ran":211,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":163,"every_run_a_failure_of_syntologys_instrument":48,"listed_with_a_run_with_no_instrument_failure":163,"listed_every_run_a_failure_of_syntologys_instrument":48,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"items":[{"url":"/paper/retrieval-augmented-generation-for-knowledge","slug":"retrieval-augmented-generation-for-knowledge","title":"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks","date":"2020-05-22","arxiv_id":"2005.11401","repositories_listed":18,"syntology":{"n":6,"n_ran":6,"n_constructed":3,"n_ran_checked":5,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"6 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/retrieval-augmented-generation-for-knowledge#ran","syntology_url":"https://syntology.ai/paper/2005.11401","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.11401"}},"official":null}},{"url":"/paper/self-rag-learning-to-retrieve-generate-and","slug":"self-rag-learning-to-retrieve-generate-and","title":"Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection","date":"2023-10-17","arxiv_id":"2310.11511","repositories_listed":6,"syntology":{"n":14,"n_ran":11,"n_constructed":0,"n_ran_checked":3,"n_instrument":8,"n_unverified":3,"n_honours":0,"n_violates":1,"n_no_contract":2,"n_pointer_only":6,"phrase":"11 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 8 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/self-rag-learning-to-retrieve-generate-and#ran","syntology_url":"https://syntology.ai/paper/2310.11511","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.11511"}},"official":{"repos":["AkariAsai/self-rag"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":2,"ran_from_kinds":["listed","official","unlocated"]}}},{"url":"/paper/r1-searcher-incentivizing-the-search","slug":"r1-searcher-incentivizing-the-search","title":"R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning","date":"2025-03-07","arxiv_id":"2503.05592","repositories_listed":5,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":1,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/r1-searcher-incentivizing-the-search#ran","syntology_url":"https://syntology.ai/paper/2503.05592","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.05592"}},"official":null}},{"url":"/paper/colpali-efficient-document-retrieval-with","slug":"colpali-efficient-document-retrieval-with","title":"ColPali: Efficient Document Retrieval with Vision Language Models","date":"2024-06-27","arxiv_id":"2407.01449","repositories_listed":5,"syntology":{"n":4,"n_ran":4,"n_constructed":2,"n_ran_checked":4,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 2 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/colpali-efficient-document-retrieval-with#ran","syntology_url":"https://syntology.ai/paper/2407.01449","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.01449"}},"official":null}},{"url":"/paper/retrieval-augmented-generation-for-large","slug":"retrieval-augmented-generation-for-large","title":"Retrieval-Augmented Generation for Large Language Models: A Survey","date":"2023-12-18","arxiv_id":"2312.10997","repositories_listed":4,"syntology":null},{"url":"/paper/r1-searcher-incentivizing-the-dynamic","slug":"r1-searcher-incentivizing-the-dynamic","title":"R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning","date":"2025-05-22","arxiv_id":"2505.17005","repositories_listed":3,"syntology":{"n":10,"n_ran":9,"n_constructed":0,"n_ran_checked":8,"n_instrument":1,"n_unverified":1,"n_honours":1,"n_violates":0,"n_no_contract":7,"n_pointer_only":0,"phrase":"9 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/r1-searcher-incentivizing-the-dynamic#ran","syntology_url":"https://syntology.ai/paper/2505.17005","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.17005"}},"official":{"repos":["rucaibox/r1-searcher-plus"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/simpledeepsearcher-deep-information-seeking","slug":"simpledeepsearcher-deep-information-seeking","title":"SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis","date":"2025-05-22","arxiv_id":"2505.16834","repositories_listed":3,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":0,"n_instrument":2,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/simpledeepsearcher-deep-information-seeking#ran","syntology_url":"https://syntology.ai/paper/2505.16834","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.16834"}},"official":{"repos":["rucaibox/simpledeepsearcher"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/search-r1-training-llms-to-reason-and","slug":"search-r1-training-llms-to-reason-and","title":"Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning","date":"2025-03-12","arxiv_id":"2503.09516","repositories_listed":3,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":0,"n_instrument":2,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":1,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/search-r1-training-llms-to-reason-and#ran","syntology_url":"https://syntology.ai/paper/2503.09516","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.09516"}},"official":{"repos":["petergriffinjin/search-r1"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official","unlocated"]}}},{"url":"/paper/lrp4rag-detecting-hallucinations-in-retrieval","slug":"lrp4rag-detecting-hallucinations-in-retrieval","title":"LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation","date":"2024-08-28","arxiv_id":"2408.15533","repositories_listed":3,"syntology":null},{"url":"/paper/from-local-to-global-a-graph-rag-approach-to","slug":"from-local-to-global-a-graph-rag-approach-to","title":"From Local to Global: A Graph RAG Approach to Query-Focused Summarization","date":"2024-04-24","arxiv_id":"2404.16130","repositories_listed":3,"syntology":{"n":5,"n_ran":5,"n_constructed":0,"n_ran_checked":5,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/from-local-to-global-a-graph-rag-approach-to#ran","syntology_url":"https://syntology.ai/paper/2404.16130","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.16130"}},"official":null}},{"url":"/paper/retrieval-augmented-generation-for-ai","slug":"retrieval-augmented-generation-for-ai","title":"Retrieval-Augmented Generation for AI-Generated Content: A Survey","date":"2024-02-29","arxiv_id":"2402.19473","repositories_listed":3,"syntology":null},{"url":"/paper/the-power-of-noise-redefining-retrieval-for","slug":"the-power-of-noise-redefining-retrieval-for","title":"The Power of Noise: Redefining Retrieval for RAG Systems","date":"2024-01-26","arxiv_id":"2401.14887","repositories_listed":3,"syntology":{"n":5,"n_ran":5,"n_constructed":0,"n_ran_checked":0,"n_instrument":5,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":5,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/the-power-of-noise-redefining-retrieval-for#ran","syntology_url":"https://syntology.ai/paper/2401.14887","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.14887"}},"official":{"repos":["florin-git/The-Power-of-Noise"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/ragtruth-a-hallucination-corpus-for","slug":"ragtruth-a-hallucination-corpus-for","title":"RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models","date":"2023-12-31","arxiv_id":"2401.00396","repositories_listed":3,"syntology":{"n":5,"n_ran":3,"n_constructed":0,"n_ran_checked":1,"n_instrument":2,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/ragtruth-a-hallucination-corpus-for#ran","syntology_url":"https://syntology.ai/paper/2401.00396","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.00396"}},"official":{"repos":["particlemedia/ragtruth"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/ragas-automated-evaluation-of-retrieval","slug":"ragas-automated-evaluation-of-retrieval","title":"RAGAS: Automated Evaluation of Retrieval Augmented Generation","date":"2023-09-26","arxiv_id":"2309.15217","repositories_listed":3,"syntology":{"n":6,"n_ran":4,"n_constructed":0,"n_ran_checked":4,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/ragas-automated-evaluation-of-retrieval#ran","syntology_url":"https://syntology.ai/paper/2309.15217","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.15217"}},"official":null}},{"url":"/paper/robust-affine-feature-matching-via-quadratic","slug":"robust-affine-feature-matching-via-quadratic","title":"Robust affine point matching via quadratic assignment on Grassmannians","date":"2023-03-05","arxiv_id":"2303.02698","repositories_listed":3,"syntology":null},{"url":"/paper/arctic-long-sequence-training-scalable-and","slug":"arctic-long-sequence-training-scalable-and","title":"Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences","date":"2025-06-16","arxiv_id":"2506.13996","repositories_listed":2,"syntology":null},{"url":"/paper/a-survey-of-llm-times-data","slug":"a-survey-of-llm-times-data","title":"A Survey of LLM $\\times$ DATA","date":"2025-05-24","arxiv_id":"2505.18458","repositories_listed":2,"syntology":null},{"url":"/paper/relevance-isn-t-all-you-need-scaling-rag","slug":"relevance-isn-t-all-you-need-scaling-rag","title":"Relevance Isn't All You Need: Scaling RAG Systems With Inference-Time Compute Via Multi-Criteria Reranking","date":"2025-03-14","arxiv_id":"2504.07104","repositories_listed":2,"syntology":null},{"url":"/paper/benchmarking-retrieval-augmented-generation-1","slug":"benchmarking-retrieval-augmented-generation-1","title":"Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts","date":"2025-02-24","arxiv_id":"2502.17297","repositories_listed":2,"syntology":null},{"url":"/paper/lettucedetect-a-hallucination-detection","slug":"lettucedetect-a-hallucination-detection","title":"LettuceDetect: A Hallucination Detection Framework for RAG Applications","date":"2025-02-24","arxiv_id":"2502.17125","repositories_listed":2,"syntology":null},{"url":"/paper/infinite-retrieval-attention-enhanced-llms-in","slug":"infinite-retrieval-attention-enhanced-llms-in","title":"Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processing","date":"2025-02-18","arxiv_id":"2502.12962","repositories_listed":2,"syntology":null},{"url":"/paper/ras-retrieval-and-structuring-for-knowledge","slug":"ras-retrieval-and-structuring-for-knowledge","title":"RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generation","date":"2025-02-16","arxiv_id":"2502.10996","repositories_listed":2,"syntology":{"n":13,"n_ran":8,"n_constructed":0,"n_ran_checked":7,"n_instrument":1,"n_unverified":5,"n_honours":0,"n_violates":0,"n_no_contract":7,"n_pointer_only":2,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified","sample_list":"/paper/ras-retrieval-and-structuring-for-knowledge#ran","syntology_url":"https://syntology.ai/paper/2502.10996","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.10996"}},"official":{"repos":["pat-jj/ras","pat-jj/Retrieval-And-Structure"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":7,"n_unverified":4,"ran_from_kinds":["community","official"]}}},{"url":"/paper/autoagent-a-fully-automated-and-zero-code","slug":"autoagent-a-fully-automated-and-zero-code","title":"AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents","date":"2025-02-09","arxiv_id":"2502.05957","repositories_listed":2,"syntology":null},{"url":"/paper/webwalker-benchmarking-llms-in-web-traversal","slug":"webwalker-benchmarking-llms-in-web-traversal","title":"WebWalker: Benchmarking LLMs in Web Traversal","date":"2025-01-13","arxiv_id":"2501.07572","repositories_listed":2,"syntology":null},{"url":"/paper/eliza-a-web3-friendly-ai-agent-operating","slug":"eliza-a-web3-friendly-ai-agent-operating","title":"Eliza: A Web3 friendly AI Agent Operating System","date":"2025-01-12","arxiv_id":"2501.06781","repositories_listed":2,"syntology":null},{"url":"/paper/search-o1-agentic-search-enhanced-large","slug":"search-o1-agentic-search-enhanced-large","title":"Search-o1: Agentic Search-Enhanced Large Reasoning Models","date":"2025-01-09","arxiv_id":"2501.05366","repositories_listed":2,"syntology":null},{"url":"/paper/jasper-and-stella-distillation-of-sota","slug":"jasper-and-stella-distillation-of-sota","title":"Jasper and Stella: distillation of SOTA embedding models","date":"2024-12-26","arxiv_id":"2412.19048","repositories_listed":2,"syntology":{"n":4,"n_ran":4,"n_constructed":0,"n_ran_checked":4,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":1,"n_no_contract":3,"n_pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/jasper-and-stella-distillation-of-sota#ran","syntology_url":"https://syntology.ai/paper/2412.19048","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.19048"}},"official":{"repos":["NLPJCL/RAG-Retrieval"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/initial-nugget-evaluation-results-for-the","slug":"initial-nugget-evaluation-results-for-the","title":"Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework","date":"2024-11-14","arxiv_id":"2411.09607","repositories_listed":2,"syntology":null},{"url":"/paper/autorag-automated-framework-for-optimization","slug":"autorag-automated-framework-for-optimization","title":"AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline","date":"2024-10-28","arxiv_id":"2410.20878","repositories_listed":2,"syntology":null},{"url":"/paper/lorann-low-rank-matrix-factorization-for","slug":"lorann-low-rank-matrix-factorization-for","title":"LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search","date":"2024-10-24","arxiv_id":"2410.18926","repositories_listed":2,"syntology":{"n":20,"n_ran":17,"n_constructed":0,"n_ran_checked":6,"n_instrument":11,"n_unverified":3,"n_honours":3,"n_violates":0,"n_no_contract":3,"n_pointer_only":0,"phrase":"17 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 0 violated, 3 with no contract checked; 11 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/lorann-low-rank-matrix-factorization-for#ran","syntology_url":"https://syntology.ai/paper/2410.18926","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.18926"}},"official":{"repos":["ejaasaari/lorann","ejaasaari/lorann-experiments"],"state":"official (archive's flag): 17 ran","n_ran":17,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/faithbench-a-diverse-hallucination-benchmark","slug":"faithbench-a-diverse-hallucination-benchmark","title":"FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs","date":"2024-10-17","arxiv_id":"2410.13210","repositories_listed":2,"syntology":null},{"url":"/paper/swiftkv-fast-prefill-optimized-inference-with","slug":"swiftkv-fast-prefill-optimized-inference-with","title":"SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation","date":"2024-10-04","arxiv_id":"2410.03960","repositories_listed":2,"syntology":null},{"url":"/paper/colacare-enhancing-electronic-health-record","slug":"colacare-enhancing-electronic-health-record","title":"ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaboration","date":"2024-10-03","arxiv_id":"2410.02551","repositories_listed":2,"syntology":null},{"url":"/paper/l-citeeval-do-long-context-models-truly","slug":"l-citeeval-do-long-context-models-truly","title":"L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?","date":"2024-10-03","arxiv_id":"2410.02115","repositories_listed":2,"syntology":null},{"url":"/paper/data-prep-kit-getting-your-data-ready-for-llm","slug":"data-prep-kit-getting-your-data-ready-for-llm","title":"Data-Prep-Kit: getting your data ready for LLM application development","date":"2024-09-26","arxiv_id":"2409.18164","repositories_listed":2,"syntology":null},{"url":"/paper/fact-fetch-and-reason-a-unified-evaluation-of","slug":"fact-fetch-and-reason-a-unified-evaluation-of","title":"Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation","date":"2024-09-19","arxiv_id":"2409.12941","repositories_listed":2,"syntology":null},{"url":"/paper/2409-13694","slug":"2409-13694","title":"Multi-Source Knowledge Pruning for Retrieval-Augmented Generation: A Benchmark and Empirical Study","date":"2024-09-03","arxiv_id":"2409.13694","repositories_listed":2,"syntology":null},{"url":"/paper/writing-in-the-margins-better-inference","slug":"writing-in-the-margins-better-inference","title":"Writing in the Margins: Better Inference Pattern for Long Context Retrieval","date":"2024-08-27","arxiv_id":"2408.14906","repositories_listed":2,"syntology":null},{"url":"/paper/tc-rag-turing-complete-rag-s-case-study-on","slug":"tc-rag-turing-complete-rag-s-case-study-on","title":"TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systems","date":"2024-08-17","arxiv_id":"2408.09199","repositories_listed":2,"syntology":null},{"url":"/paper/w-rag-weakly-supervised-dense-retrieval-in","slug":"w-rag-weakly-supervised-dense-retrieval-in","title":"W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering","date":"2024-08-15","arxiv_id":"2408.08444","repositories_listed":2,"syntology":null},{"url":"/paper/2407-21439","slug":"2407-21439","title":"MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Training","date":"2024-07-31","arxiv_id":"2407.21439","repositories_listed":2,"syntology":{"n":4,"n_ran":3,"n_constructed":0,"n_ran_checked":0,"n_instrument":3,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/2407-21439#ran","syntology_url":"https://syntology.ai/paper/2407.21439","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.21439"}},"official":{"repos":["idea-finai/ragllava","idea-finai/ragvl"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/ragnarok-a-reusable-rag-framework-and","slug":"ragnarok-a-reusable-rag-framework-and","title":"Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track","date":"2024-06-24","arxiv_id":"2406.16828","repositories_listed":2,"syntology":{"n":23,"n_ran":19,"n_constructed":0,"n_ran_checked":12,"n_instrument":7,"n_unverified":4,"n_honours":0,"n_violates":1,"n_no_contract":11,"n_pointer_only":0,"phrase":"19 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 1 violated, 11 with no contract checked; 7 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/ragnarok-a-reusable-rag-framework-and#ran","syntology_url":"https://syntology.ai/paper/2406.16828","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.16828"}},"official":{"repos":["castorini/ragnarok"],"state":"official (archive's flag): 9 ran","n_ran":9,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":3,"ran_from_kinds":["listed","official"]}}},{"url":"/paper/trace-the-evidence-constructing-knowledge","slug":"trace-the-evidence-constructing-knowledge","title":"TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation","date":"2024-06-17","arxiv_id":"2406.11460","repositories_listed":2,"syntology":{"n":4,"n_ran":4,"n_constructed":0,"n_ran_checked":0,"n_instrument":4,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":4,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/trace-the-evidence-constructing-knowledge#ran","syntology_url":"https://syntology.ai/paper/2406.11460","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.11460"}},"official":{"repos":["jyfang6/trace"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/domainrag-a-chinese-benchmark-for-evaluating","slug":"domainrag-a-chinese-benchmark-for-evaluating","title":"DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation","date":"2024-06-09","arxiv_id":"2406.05654","repositories_listed":2,"syntology":{"n":12,"n_ran":12,"n_constructed":0,"n_ran_checked":6,"n_instrument":6,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":6,"n_pointer_only":12,"phrase":"12 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/domainrag-a-chinese-benchmark-for-evaluating#ran","syntology_url":"https://syntology.ai/paper/2406.05654","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.05654"}},"official":{"repos":["ShootingWong/DomainRAG"],"state":"official (archive's flag): 12 ran","n_ran":12,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/crag-comprehensive-rag-benchmark","slug":"crag-comprehensive-rag-benchmark","title":"CRAG -- Comprehensive RAG Benchmark","date":"2024-06-07","arxiv_id":"2406.04744","repositories_listed":2,"syntology":{"n":4,"n_ran":4,"n_constructed":0,"n_ran_checked":1,"n_instrument":3,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":4,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/crag-comprehensive-rag-benchmark#ran","syntology_url":"https://syntology.ai/paper/2406.04744","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.04744"}},"official":{"repos":["facebookresearch/crag"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/multi-head-rag-solving-multi-aspect-problems","slug":"multi-head-rag-solving-multi-aspect-problems","title":"Multi-Head RAG: Solving Multi-Aspect Problems with LLMs","date":"2024-06-07","arxiv_id":"2406.05085","repositories_listed":2,"syntology":null},{"url":"/paper/one-token-can-help-learning-scalable-and","slug":"one-token-can-help-learning-scalable-and","title":"One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models","date":"2024-05-30","arxiv_id":"2405.19670","repositories_listed":2,"syntology":null},{"url":"/paper/cacheblend-fast-large-language-model-serving","slug":"cacheblend-fast-large-language-model-serving","title":"CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion","date":"2024-05-26","arxiv_id":"2405.16444","repositories_listed":2,"syntology":{"n":14,"n_ran":10,"n_constructed":0,"n_ran_checked":7,"n_instrument":3,"n_unverified":4,"n_honours":0,"n_violates":0,"n_no_contract":7,"n_pointer_only":14,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/cacheblend-fast-large-language-model-serving#ran","syntology_url":"https://syntology.ai/paper/2405.16444","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.16444"}},"official":{"repos":["YaoJiayi/CacheBlend"],"state":"official (archive's flag): 10 ran","n_ran":10,"n_constructed":0,"n_ran_no_instrument_failure":7,"n_unverified":4,"ran_from_kinds":["official"]}}},{"url":"/paper/hipporag-neurobiologically-inspired-long-term","slug":"hipporag-neurobiologically-inspired-long-term","title":"HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models","date":"2024-05-23","arxiv_id":"2405.14831","repositories_listed":2,"syntology":{"n":5,"n_ran":5,"n_constructed":0,"n_ran_checked":5,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/hipporag-neurobiologically-inspired-long-term#ran","syntology_url":"https://syntology.ai/paper/2405.14831","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14831"}},"official":{"repos":["osu-nlp-group/hipporag"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/rag-and-rau-a-survey-on-retrieval-augmented","slug":"rag-and-rau-a-survey-on-retrieval-augmented","title":"RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing","date":"2024-04-30","arxiv_id":"2404.19543","repositories_listed":2,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/rag-and-rau-a-survey-on-retrieval-augmented#ran","syntology_url":"https://syntology.ai/paper/2404.19543","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.19543"}},"official":{"repos":["2471023025/ralm_survey"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/adaptive-rag-learning-to-adapt-retrieval","slug":"adaptive-rag-learning-to-adapt-retrieval","title":"Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity","date":"2024-03-21","arxiv_id":"2403.14403","repositories_listed":2,"syntology":{"n":5,"n_ran":4,"n_constructed":0,"n_ran_checked":2,"n_instrument":2,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":4,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/adaptive-rag-learning-to-adapt-retrieval#ran","syntology_url":"https://syntology.ai/paper/2403.14403","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.14403"}},"official":{"repos":["starsuzi/adaptive-rag"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/benchmarking-retrieval-augmented-generation","slug":"benchmarking-retrieval-augmented-generation","title":"Benchmarking Retrieval-Augmented Generation for Medicine","date":"2024-02-20","arxiv_id":"2402.13178","repositories_listed":2,"syntology":{"n":5,"n_ran":5,"n_constructed":0,"n_ran_checked":1,"n_instrument":4,"n_unverified":0,"n_honours":0,"n_violates":1,"n_no_contract":0,"n_pointer_only":5,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/benchmarking-retrieval-augmented-generation#ran","syntology_url":"https://syntology.ai/paper/2402.13178","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.13178"}},"official":{"repos":["teddy-xionggz/medrag","teddy-xionggz/mirage"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/the-finben-an-holistic-financial-benchmark","slug":"the-finben-an-holistic-financial-benchmark","title":"FinBen: A Holistic Financial Benchmark for Large Language Models","date":"2024-02-20","arxiv_id":"2402.12659","repositories_listed":2,"syntology":{"n":12,"n_ran":8,"n_constructed":0,"n_ran_checked":4,"n_instrument":4,"n_unverified":4,"n_honours":0,"n_violates":3,"n_no_contract":1,"n_pointer_only":7,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 3 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/the-finben-an-holistic-financial-benchmark#ran","syntology_url":"https://syntology.ai/paper/2402.12659","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.12659"}},"official":{"repos":["the-finai/pixiu"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/in-search-of-needles-in-a-10m-haystack","slug":"in-search-of-needles-in-a-10m-haystack","title":"In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss","date":"2024-02-16","arxiv_id":"2402.10790","repositories_listed":2,"syntology":null},{"url":"/paper/generative-representational-instruction","slug":"generative-representational-instruction","title":"Generative Representational Instruction Tuning","date":"2024-02-15","arxiv_id":"2402.09906","repositories_listed":2,"syntology":{"n":10,"n_ran":10,"n_constructed":0,"n_ran_checked":3,"n_instrument":7,"n_unverified":0,"n_honours":0,"n_violates":1,"n_no_contract":2,"n_pointer_only":5,"phrase":"10 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 7 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/generative-representational-instruction#ran","syntology_url":"https://syntology.ai/paper/2402.09906","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.09906"}},"official":{"repos":["contextualai/gritlm"],"state":"official (archive's flag): 10 ran","n_ran":10,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/g-retriever-retrieval-augmented-generation","slug":"g-retriever-retrieval-augmented-generation","title":"G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering","date":"2024-02-12","arxiv_id":"2402.07630","repositories_listed":2,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":1,"n_instrument":1,"n_unverified":0,"n_honours":1,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/g-retriever-retrieval-augmented-generation#ran","syntology_url":"https://syntology.ai/paper/2402.07630","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.07630"}},"official":{"repos":["xiaoxinhe/g-retriever"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/poisonedrag-knowledge-poisoning-attacks-to","slug":"poisonedrag-knowledge-poisoning-attacks-to","title":"PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models","date":"2024-02-12","arxiv_id":"2402.07867","repositories_listed":2,"syntology":{"n":17,"n_ran":12,"n_constructed":0,"n_ran_checked":10,"n_instrument":2,"n_unverified":5,"n_honours":3,"n_violates":2,"n_no_contract":5,"n_pointer_only":7,"phrase":"12 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 3 honoured, 2 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified","sample_list":"/paper/poisonedrag-knowledge-poisoning-attacks-to#ran","syntology_url":"https://syntology.ai/paper/2402.07867","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.07867"}},"official":{"repos":["sleeepeer/poisonedrag"],"state":"official (archive's flag): 10 ran","n_ran":10,"n_constructed":0,"n_ran_no_instrument_failure":9,"n_unverified":5,"ran_from_kinds":["listed","official"]}}},{"url":"/paper/realm-rag-driven-enhancement-of-multimodal","slug":"realm-rag-driven-enhancement-of-multimodal","title":"REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models","date":"2024-02-10","arxiv_id":"2402.07016","repositories_listed":2,"syntology":null},{"url":"/paper/corrective-retrieval-augmented-generation","slug":"corrective-retrieval-augmented-generation","title":"Corrective Retrieval Augmented Generation","date":"2024-01-29","arxiv_id":"2401.15884","repositories_listed":2,"syntology":{"n":5,"n_ran":5,"n_constructed":0,"n_ran_checked":0,"n_instrument":5,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":5,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/corrective-retrieval-augmented-generation#ran","syntology_url":"https://syntology.ai/paper/2401.15884","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.15884"}},"official":{"repos":["huskyinsalt/crag"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/multihop-rag-benchmarking-retrieval-augmented","slug":"multihop-rag-benchmarking-retrieval-augmented","title":"MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries","date":"2024-01-27","arxiv_id":"2401.15391","repositories_listed":2,"syntology":{"n":4,"n_ran":4,"n_constructed":0,"n_ran_checked":1,"n_instrument":3,"n_unverified":0,"n_honours":0,"n_violates":1,"n_no_contract":0,"n_pointer_only":4,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/multihop-rag-benchmarking-retrieval-augmented#ran","syntology_url":"https://syntology.ai/paper/2401.15391","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.15391"}},"official":{"repos":["yixuantt/MultiHop-RAG"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/dynamic-retrieval-augmented-generation-of","slug":"dynamic-retrieval-augmented-generation-of","title":"Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI)","date":"2023-12-18","arxiv_id":"2312.10904","repositories_listed":2,"syntology":null},{"url":"/paper/retrieval-augmented-generation-to-improve","slug":"retrieval-augmented-generation-to-improve","title":"Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference","date":"2023-10-04","arxiv_id":"2310.03184","repositories_listed":2,"syntology":{"n":20,"n_ran":12,"n_constructed":0,"n_ran_checked":12,"n_instrument":0,"n_unverified":8,"n_honours":0,"n_violates":0,"n_no_contract":12,"n_pointer_only":0,"phrase":"12 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 0 where Syntology's instrument failed) · 8 unverified","sample_list":"/paper/retrieval-augmented-generation-to-improve#ran","syntology_url":"https://syntology.ai/paper/2310.03184","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.03184"}},"official":{"repos":["digitalharborfoundation/rag-for-math-qa"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":7,"n_unverified":8,"ran_from_kinds":["listed","official"]}}},{"url":"/paper/retrieval-augmented-generation-and","slug":"retrieval-augmented-generation-and","title":"Retrieval Augmented Generation and Representative Vector Summarization for large unstructured textual data in Medical Education","date":"2023-08-01","arxiv_id":"2308.00479","repositories_listed":2,"syntology":null},{"url":"/paper/towards-populating-generalizable-engineering","slug":"towards-populating-generalizable-engineering","title":"Retrieval Augmented Generation using Engineering Design Knowledge","date":"2023-07-13","arxiv_id":"2307.06985","repositories_listed":2,"syntology":null},{"url":"/paper/fine-tune-the-entire-rag-architecture","slug":"fine-tune-the-entire-rag-architecture","title":"Fine-tune the Entire RAG Architecture (including DPR retriever) for Question-Answering","date":"2021-06-22","arxiv_id":"2106.11517","repositories_listed":2,"syntology":null},{"url":"/paper/zero-shot-slot-filling-with-dpr-and-rag","slug":"zero-shot-slot-filling-with-dpr-and-rag","title":"Zero-shot Slot Filling with DPR and RAG","date":"2021-04-17","arxiv_id":"2104.08610","repositories_listed":2,"syntology":null},{"url":"/paper/end-to-end-training-of-neural-retrievers-for","slug":"end-to-end-training-of-neural-retrievers-for","title":"End-to-End Training of Neural Retrievers for Open-Domain Question Answering","date":"2021-01-02","arxiv_id":"2101.00408","repositories_listed":2,"syntology":null},{"url":"/paper/developing-visual-augmented-q-a-system-using","slug":"developing-visual-augmented-q-a-system-using","title":"Developing Visual Augmented Q&A System using Scalable Vision Embedding Retrieval & Late Interaction Re-ranker","date":"2025-07-16","arxiv_id":"2507.12378","repositories_listed":1,"syntology":null},{"url":"/paper/orchestrator-agent-trust-a-modular-agentic-ai","slug":"orchestrator-agent-trust-a-modular-agentic-ai","title":"Orchestrator-Agent Trust: A Modular Agentic AI Visual Classification System with Trust-Aware Orchestration and RAG-Based Reasoning","date":"2025-07-09","arxiv_id":"2507.10571","repositories_listed":1,"syntology":null},{"url":"/paper/rag-r1-incentivize-the-search-and-reasoning","slug":"rag-r1-incentivize-the-search-and-reasoning","title":"RAG-R1 : Incentivize the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism","date":"2025-06-30","arxiv_id":"2507.02962","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_constructed":0,"n_ran_checked":3,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":3,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/rag-r1-incentivize-the-search-and-reasoning#ran","syntology_url":"https://syntology.ai/paper/2507.02962","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2507.02962"}},"official":null}},{"url":"/paper/erarag-efficient-and-incremental-retrieval","slug":"erarag-efficient-and-incremental-retrieval","title":"EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora","date":"2025-06-26","arxiv_id":"2506.20963","repositories_listed":1,"syntology":null},{"url":"/paper/psylite-technical-report","slug":"psylite-technical-report","title":"PsyLite Technical Report","date":"2025-06-26","arxiv_id":"2506.21536","repositories_listed":1,"syntology":null},{"url":"/paper/response-quality-assessment-for-retrieval","slug":"response-quality-assessment-for-retrieval","title":"Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality","date":"2025-06-26","arxiv_id":"2506.20978","repositories_listed":1,"syntology":{"n":3,"n_ran":2,"n_constructed":0,"n_ran_checked":1,"n_instrument":1,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":1,"n_pointer_only":3,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/response-quality-assessment-for-retrieval#ran","syntology_url":"https://syntology.ai/paper/2506.20978","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.20978"}},"official":{"repos":["n4feng/ResponseQualityAssessment"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/fine-tuning-and-prompt-engineering-of-llms","slug":"fine-tuning-and-prompt-engineering-of-llms","title":"Fine-Tuning and Prompt Engineering of LLMs, for the Creation of Multi-Agent AI for Addressing Sustainable Protein Production Challenges","date":"2025-06-25","arxiv_id":"2506.20598","repositories_listed":1,"syntology":null},{"url":"/paper/mmsearch-r1-incentivizing-lmms-to-search","slug":"mmsearch-r1-incentivizing-lmms-to-search","title":"MMSearch-R1: Incentivizing LMMs to Search","date":"2025-06-25","arxiv_id":"2506.20670","repositories_listed":1,"syntology":null},{"url":"/paper/rag-visualrec-an-open-resource-for-vision-and","slug":"rag-visualrec-an-open-resource-for-vision-and","title":"RAG-VisualRec: An Open Resource for Vision- and Text-Enhanced Retrieval-Augmented Generation in Recommendation","date":"2025-06-25","arxiv_id":"2506.20817","repositories_listed":1,"syntology":null},{"url":"/paper/cast-enhancing-code-retrieval-augmented","slug":"cast-enhancing-code-retrieval-augmented","title":"cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree","date":"2025-06-18","arxiv_id":"2506.15655","repositories_listed":1,"syntology":null},{"url":"/paper/lightweight-relevance-grader-in-rag","slug":"lightweight-relevance-grader-in-rag","title":"Lightweight Relevance Grader in RAG","date":"2025-06-17","arxiv_id":"2506.14084","repositories_listed":1,"syntology":null},{"url":"/paper/ltrr-learning-to-rank-retrievers-for-llms","slug":"ltrr-learning-to-rank-retrievers-for-llms","title":"LTRR: Learning To Rank Retrievers for LLMs","date":"2025-06-16","arxiv_id":"2506.13743","repositories_listed":1,"syntology":null},{"url":"/paper/simpledoc-multi-modal-document-understanding","slug":"simpledoc-multi-modal-document-understanding","title":"SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinement","date":"2025-06-16","arxiv_id":"2506.14035","repositories_listed":1,"syntology":{"n":11,"n_ran":11,"n_constructed":0,"n_ran_checked":7,"n_instrument":4,"n_unverified":0,"n_honours":1,"n_violates":1,"n_no_contract":5,"n_pointer_only":11,"phrase":"11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 1 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/simpledoc-multi-modal-document-understanding#ran","syntology_url":"https://syntology.ai/paper/2506.14035","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.14035"}},"official":{"repos":["ag2ai/simpledoc"],"state":"official (archive's flag): 11 ran","n_ran":11,"n_constructed":0,"n_ran_no_instrument_failure":7,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/tree-based-text-retrieval-via-hierarchical","slug":"tree-based-text-retrieval-via-hierarchical","title":"Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations","date":"2025-06-16","arxiv_id":"2506.13607","repositories_listed":1,"syntology":null},{"url":"/paper/flexrag-a-flexible-and-comprehensive","slug":"flexrag-a-flexible-and-comprehensive","title":"FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation","date":"2025-06-14","arxiv_id":"2506.12494","repositories_listed":1,"syntology":null},{"url":"/paper/ciir-liverag-2025-optimizing-multi-agent","slug":"ciir-liverag-2025-optimizing-multi-agent","title":"CIIR@LiveRAG 2025: Optimizing Multi-Agent Retrieval Augmented Generation through Self-Training","date":"2025-06-12","arxiv_id":"2506.10844","repositories_listed":1,"syntology":null},{"url":"/paper/constructing-and-evaluating-declarative-rag","slug":"constructing-and-evaluating-declarative-rag","title":"Constructing and Evaluating Declarative RAG Pipelines in PyTerrier","date":"2025-06-12","arxiv_id":"2506.10802","repositories_listed":1,"syntology":null},{"url":"/paper/llm-embedding-based-attribution-lea","slug":"llm-embedding-based-attribution-lea","title":"LLM Embedding-based Attribution (LEA): Quantifying Source Contributions to Generative Model's Response for Vulnerability Analysis","date":"2025-06-12","arxiv_id":"2506.12100","repositories_listed":1,"syntology":null},{"url":"/paper/tablerag-a-retrieval-augmented-generation","slug":"tablerag-a-retrieval-augmented-generation","title":"TableRAG: A Retrieval Augmented Generation Framework for Heterogeneous Document Reasoning","date":"2025-06-12","arxiv_id":"2506.10380","repositories_listed":1,"syntology":null},{"url":"/paper/bridging-the-gap-between-open-source-and","slug":"bridging-the-gap-between-open-source-and","title":"Bridging the Gap Between Open-Source and Proprietary LLMs in Table QA","date":"2025-06-11","arxiv_id":"2506.09657","repositories_listed":1,"syntology":null},{"url":"/paper/learning-efficient-and-generalizable-graph","slug":"learning-efficient-and-generalizable-graph","title":"Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering","date":"2025-06-11","arxiv_id":"2506.09645","repositories_listed":1,"syntology":null},{"url":"/paper/2506-10030","slug":"2506-10030","title":"Safeguarding Multimodal Knowledge Copyright in the RAG-as-a-Service Environment","date":"2025-06-10","arxiv_id":"2506.10030","repositories_listed":1,"syntology":null},{"url":"/paper/dragged-into-conflicts-detecting-and","slug":"dragged-into-conflicts-detecting-and","title":"DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs","date":"2025-06-10","arxiv_id":"2506.08500","repositories_listed":1,"syntology":null},{"url":"/paper/faithfulrag-fact-level-conflict-modeling-for","slug":"faithfulrag-fact-level-conflict-modeling-for","title":"FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation","date":"2025-06-10","arxiv_id":"2506.08938","repositories_listed":1,"syntology":null},{"url":"/paper/fedrag-a-framework-for-fine-tuning-retrieval","slug":"fedrag-a-framework-for-fine-tuning-retrieval","title":"FedRAG: A Framework for Fine-Tuning Retrieval-Augmented Generation Systems","date":"2025-06-10","arxiv_id":"2506.09200","repositories_listed":1,"syntology":null},{"url":"/paper/xgraphrag-interactive-visual-analysis-for","slug":"xgraphrag-interactive-visual-analysis-for","title":"XGraphRAG: Interactive Visual Analysis for Graph-based Retrieval-Augmented Generation","date":"2025-06-10","arxiv_id":"2506.13782","repositories_listed":1,"syntology":null},{"url":"/paper/hierarchical-lexical-graph-for-enhanced-multi","slug":"hierarchical-lexical-graph-for-enhanced-multi","title":"Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval","date":"2025-06-09","arxiv_id":"2506.08074","repositories_listed":1,"syntology":null},{"url":"/paper/llamarec-lkg-rag-a-single-pass-learnable","slug":"llamarec-lkg-rag-a-single-pass-learnable","title":"LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking","date":"2025-06-09","arxiv_id":"2506.07449","repositories_listed":1,"syntology":null},{"url":"/paper/slidecoder-layout-aware-rag-enhanced","slug":"slidecoder-layout-aware-rag-enhanced","title":"SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Design","date":"2025-06-09","arxiv_id":"2506.07964","repositories_listed":1,"syntology":null},{"url":"/paper/ar-rag-autoregressive-retrieval-augmentation","slug":"ar-rag-autoregressive-retrieval-augmentation","title":"AR-RAG: Autoregressive Retrieval Augmentation for Image Generation","date":"2025-06-08","arxiv_id":"2506.06962","repositories_listed":1,"syntology":null},{"url":"/paper/joint-gcg-unified-gradient-based-poisoning","slug":"joint-gcg-unified-gradient-based-poisoning","title":"Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systems","date":"2025-06-06","arxiv_id":"2506.06151","repositories_listed":1,"syntology":null},{"url":"/paper/when-to-use-graphs-in-rag-a-comprehensive","slug":"when-to-use-graphs-in-rag-a-comprehensive","title":"When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation","date":"2025-06-06","arxiv_id":"2506.05690","repositories_listed":1,"syntology":{"n":5,"n_ran":5,"n_constructed":0,"n_ran_checked":5,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/when-to-use-graphs-in-rag-a-comprehensive#ran","syntology_url":"https://syntology.ai/paper/2506.05690","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.05690"}},"official":{"repos":["graphrag-bench/graphrag-benchmark"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/ecorag-evidentiality-guided-compression-for","slug":"ecorag-evidentiality-guided-compression-for","title":"ECoRAG: Evidentiality-guided Compression for Long Context RAG","date":"2025-06-05","arxiv_id":"2506.05167","repositories_listed":1,"syntology":null}],"syntology_records":35,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"key_notes":{"n_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"since_archive_judged":{"papers":0,"papers_placed":0,"judged_by":{"model":"a model","models_as_recorded":[],"from":"lb_rows_run_verified.jsonl: each record's `model` field (the extractor's call that wrote decisions[])"},"note":"a model's judgement from the paper's own tables that they report on this leaderboard, not a result: no value from a paper that was not placed is printed or carried here; papers counts each paper once over these leaderboards","backfill":"8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read)","backfill_complete":false,"leaderboards":[]},"papers_where_code_ran":211,"papers_where_code_ran_first_page":"/task/rag/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not run it on this task or check it against the task's benchmarks.","pages":[{"order":"archive","page":2,"of":22,"url":"/task/rag/papers/2","json":"/task/rag/papers/2.json","rows":100},{"order":"archive","page":3,"of":22,"url":"/task/rag/papers/3","json":"/task/rag/papers/3.json","rows":100},{"order":"archive","page":4,"of":22,"url":"/task/rag/papers/4","json":"/task/rag/papers/4.json","rows":100},{"order":"archive","page":5,"of":22,"url":"/task/rag/papers/5","json":"/task/rag/papers/5.json","rows":100},{"order":"archive","page":6,"of":22,"url":"/task/rag/papers/6","json":"/task/rag/papers/6.json","rows":100},{"order":"archive","page":7,"of":22,"url":"/task/rag/papers/7","json":"/task/rag/papers/7.json","rows":100},{"order":"archive","page":8,"of":22,"url":"/task/rag/papers/8","json":"/task/rag/papers/8.json","rows":100},{"order":"archive","page":9,"of":22,"url":"/task/rag/papers/9","json":"/task/rag/papers/9.json","rows":100},{"order":"archive","page":10,"of":22,"url":"/task/rag/papers/10","json":"/task/rag/papers/10.json","rows":100},{"order":"archive","page":11,"of":22,"url":"/task/rag/papers/11","json":"/task/rag/papers/11.json","rows":100},{"order":"archive","page":12,"of":22,"url":"/task/rag/papers/12","json":"/task/rag/papers/12.json","rows":100},{"order":"archive","page":13,"of":22,"url":"/task/rag/papers/13","json":"/task/rag/papers/13.json","rows":100},{"order":"archive","page":14,"of":22,"url":"/task/rag/papers/14","json":"/task/rag/papers/14.json","rows":100},{"order":"archive","page":15,"of":22,"url":"/task/rag/papers/15","json":"/task/rag/papers/15.json","rows":100},{"order":"archive","page":16,"of":22,"url":"/task/rag/papers/16","json":"/task/rag/papers/16.json","rows":100},{"order":"archive","page":17,"of":22,"url":"/task/rag/papers/17","json":"/task/rag/papers/17.json","rows":100},{"order":"archive","page":18,"of":22,"url":"/task/rag/papers/18","json":"/task/rag/papers/18.json","rows":100},{"order":"archive","page":19,"of":22,"url":"/task/rag/papers/19","json":"/task/rag/papers/19.json","rows":100},{"order":"archive","page":20,"of":22,"url":"/task/rag/papers/20","json":"/task/rag/papers/20.json","rows":100},{"order":"archive","page":21,"of":22,"url":"/task/rag/papers/21","json":"/task/rag/papers/21.json","rows":100},{"order":"archive","page":22,"of":22,"url":"/task/rag/papers/22","json":"/task/rag/papers/22.json","rows":11},{"order":"ran","page":1,"of":3,"url":"/task/rag/papers/ran/1","json":"/task/rag/papers/ran/1.json","rows":100},{"order":"ran","page":2,"of":3,"url":"/task/rag/papers/ran/2","json":"/task/rag/papers/ran/2.json","rows":100},{"order":"ran","page":3,"of":3,"url":"/task/rag/papers/ran/3","json":"/task/rag/papers/ran/3.json","rows":11}],"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-10-05T00:36:11+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"row_route":"each syntology object on a benchmark's first row and on a paper carries sample_list (this site's paper page, #ran), syntology_url (syntology.ai's page for the paper, only when that route probed live) and mcp (the agent call that lists the paper's harvested samples; not a verdict on the paper)","developers":"https://syntology.ai/developers"},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"},"record_sha256":"d610ed74b80c9e5901a8b703ab57343bcb0f8cbef424fa2424415d93ef0a99af","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}