{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/rlaif","slug":"rlaif","name":"RLAIF","full_name":"Reinforcement Learning from AI Feedback","full_name_withheld":false,"description_markdown":null,"description_state":"absent","introduced_year":null,"introduced_by":{"title":"Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback","paper":"/paper/tuning-large-multimodal-models-for-videos","first_author":"Daechul Ahn","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/tuning-large-multimodal-models-for-videos"},"source":{"url":"https://arxiv.org/abs/2402.03746v3","title":"Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Reinforcement Learning Frameworks","url":"/methods/category/reinforcement-learning-frameworks","pwc_aliases":[]}],"n_papers_tagged":19,"archive_num_papers":19,"papers_newest_first":[{"paper":"/paper/toward-evaluative-thinking-meta-policy","slug":"toward-evaluative-thinking-meta-policy","title":"Toward Evaluative Thinking: Meta Policy Optimization with Evolving Reward Models","date":"2025-04-28","arxiv_id":"2504.20157","n_code_links":1,"syntology":{"ran":9,"of":15,"n_ran_checked":6,"n_instrument":3,"unverified":6,"pointer_only":15,"phrase":"9 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 6 unverified","official":{"repos":["minnesotanlp/mpo"],"state":"official (archive's flag): 9 ran","n_ran":9,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":6,"ran_from_kinds":["official"]}}},{"paper":"/paper/r2vul-learning-to-reason-about-software","slug":"r2vul-learning-to-reason-about-software","title":"R2Vul: Learning to Reason about Software Vulnerabilities with Reinforcement Learning and Structured Reasoning Distillation","date":"2025-04-07","arxiv_id":"2504.04699","n_code_links":1,"syntology":null},{"paper":null,"slug":"synthetic-data-generation-multi-step-rl-for","title":"Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use","date":"2025-04-07","arxiv_id":"2504.04736","n_code_links":0,"syntology":null},{"paper":null,"slug":"training-dialogue-systems-by-ai-feedback-for","title":"Training Dialogue Systems by AI Feedback for Improving Overall Dialogue Impression","date":"2025-01-22","arxiv_id":"2501.12698","n_code_links":0,"syntology":null},{"paper":null,"slug":"popalign-diversifying-contrasting-patterns","title":"PopAlign: Diversifying Contrasting Patterns for a More Comprehensive Alignment","date":"2024-10-17","arxiv_id":"2410.13785","n_code_links":0,"syntology":null},{"paper":null,"slug":"exploring-llm-based-data-annotation","title":"Exploring LLM-based Data Annotation Strategies for Medical Dialogue Preference Alignment","date":"2024-10-05","arxiv_id":"2410.04112","n_code_links":0,"syntology":null},{"paper":null,"slug":"generative-reward-models","title":"Generative Reward Models","date":"2024-10-02","arxiv_id":"2410.12832","n_code_links":0,"syntology":null},{"paper":"/paper/maferw-query-rewriting-with-multi-aspect","slug":"maferw-query-rewriting-with-multi-aspect","title":"MaFeRw: Query Rewriting with Multi-Aspect Feedbacks for Retrieval-Augmented Large Language Models","date":"2024-08-30","arxiv_id":"2408.17072","n_code_links":1,"syntology":null},{"paper":null,"slug":"applying-rlaif-for-code-generation-with-api","title":"Applying RLAIF for Code Generation with API-usage in Lightweight LLMs","date":"2024-06-28","arxiv_id":"2406.20060","n_code_links":0,"syntology":null},{"paper":null,"slug":"diminishing-stereotype-bias-in-image","title":"Diminishing Stereotype Bias in Image Generation Model using Reinforcemenlent Learning Feedback","date":"2024-06-27","arxiv_id":"2407.09551","n_code_links":0,"syntology":null},{"paper":"/paper/multi-objective-reinforcement-learning-from","slug":"multi-objective-reinforcement-learning-from","title":"Multi-objective Reinforcement learning from AI Feedback","date":"2024-06-11","arxiv_id":"2406.07295","n_code_links":1,"syntology":null},{"paper":null,"slug":"are-you-sure-rank-them-again-repeated-ranking","title":"Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets","date":"2024-05-29","arxiv_id":"2405.18952","n_code_links":0,"syntology":null},{"paper":"/paper/optimization-based-prompt-injection-attack-to","slug":"optimization-based-prompt-injection-attack-to","title":"Optimization-based Prompt Injection Attack to LLM-as-a-Judge","date":"2024-03-26","arxiv_id":"2403.17710","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":1,"n_instrument":1,"unverified":0,"pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["shijiawenwen/judgedeceiver"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/codeultrafeedback-an-llm-as-a-judge-dataset","slug":"codeultrafeedback-an-llm-as-a-judge-dataset","title":"CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences","date":"2024-03-14","arxiv_id":"2403.09032","n_code_links":2,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["martin-wey/codeultrafeedback"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"hrlaif-improvements-in-helpfulness-and","title":"HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedback","date":"2024-03-13","arxiv_id":"2403.08309","n_code_links":0,"syntology":null},{"paper":"/paper/a-critical-evaluation-of-ai-feedback-for","slug":"a-critical-evaluation-of-ai-feedback-for","title":"A Critical Evaluation of AI Feedback for Aligning Large Language Models","date":"2024-02-19","arxiv_id":"2402.12366","n_code_links":1,"syntology":{"ran":9,"of":11,"n_ran_checked":6,"n_instrument":3,"unverified":2,"pointer_only":1,"phrase":"9 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","official":{"repos":["architsharma97/dpo-rlaif"],"state":"official (archive's flag): 9 ran","n_ran":9,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/direct-large-language-model-alignment-through","slug":"direct-large-language-model-alignment-through","title":"Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation","date":"2024-02-19","arxiv_id":"2402.11907","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["exlaw/dlma"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"fintral-a-family-of-gpt-4-level-multimodal","title":"FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models","date":"2024-02-16","arxiv_id":"2402.10986","n_code_links":0,"syntology":null},{"paper":"/paper/tuning-large-multimodal-models-for-videos","slug":"tuning-large-multimodal-models-for-videos","title":"Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback","date":"2024-02-06","arxiv_id":"2402.03746","n_code_links":1,"syntology":null}],"papers_shown":19,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":19,"with_a_code_link":9,"where_syntology_ran_a_sample":5,"not_listed_spam_title":0,"listed":19,"listed_where_code_ran":5,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":3,"every_run_a_failure_of_syntologys_instrument":2,"listed_with_a_run_with_no_instrument_failure":3,"listed_every_run_a_failure_of_syntologys_instrument":2,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":5,"papers_where_code_ran_first_page":"/method/rlaif/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/rlaif/papers/ran/1","json":"/method/rlaif/papers/ran/1.json","rows":5}],"tasks":[{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":5},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":5},{"task":"/task/language-modelling","name":"Language Modelling","papers":4},{"task":"/task/large-language-model","name":"Large Language Model","papers":3},{"task":"/task/mathematical-reasoning","name":"Mathematical Reasoning","papers":3},{"task":"/task/decision-making","name":"Decision Making","papers":2},{"task":"/task/language-modeling","name":"Language Modeling","papers":2},{"task":"/task/question-answering","name":"Question Answering","papers":2},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":2},{"task":"/task/retrieval","name":"Retrieval","papers":2},{"task":"/task/code-generation","name":"Code Generation","papers":1},{"task":"/task/dataset-generation","name":"Dataset Generation","papers":1},{"task":"/task/denoising","name":"Denoising","papers":1},{"task":"/task/fairness","name":"Fairness","papers":1},{"task":"/task/financial-analysis","name":"Financial Analysis","papers":1},{"task":"/task/gsm8k","name":"GSM8K","papers":1},{"task":"/task/gender-classification","name":"Gender Classification","papers":1},{"task":"/task/hallucination","name":"Hallucination","papers":1},{"task":"/task/humaneval","name":"HumanEval","papers":1},{"task":"/task/image-generation","name":"Image Generation","papers":1}],"tasks_shown":20,"n_tasks":34,"usage_by_year":[{"year":"2024","papers":15},{"year":"2025","papers":4}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/rlaif"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=RLAIF","read_at":"2026-10-05T01:01:18+00:00","query":"RLAIF","kind":"paper","have":"no_code","tier":"paper_only","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"arxiv_id":"openreview_AAxIs3D2ZZ"},"resolution":"paper","resolved_name_differs":false,"absence_asserted":"for_this_paper_only","absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"unknown","archive_paper_arxiv_id":"2402.03746","repositories":{"listed":null,"in_graph":0,"list_truncated":null,"strongest_link":null},"retrieve_with":null,"name_competition":{"verdict":"outranked","may_be_wrong_sense":false,"bearers_found":3,"served":{"arxiv_id":"openreview_AAxIs3D2ZZ","citations":null},"leading_other_bearer":{"arxiv_id":"2603.16124","title":"SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding","citations":6,"named_by":["acronym_expansion"],"bearer_name":"Supervised Fine-Tuning (SFT) followed by Reinforcement Learning from AI Feedback (RLAIF)","ask_with":"have('2603.16124')"},"n_other_bearers_listed":2,"note":"'RLAIF' is ALSO the name of 'Supervised Fine-Tuning (SFT) followed by Reinforcement Learning from AI Feedback (RLAIF)', a method in 'SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding' (arXiv:2603.16124, 6 citations), which is cited more than the paper this answer is about (citations not recorded). The margin is narrow, so this is not a claim that the other one is what you meant -- check `served.arxiv_id` against the paper you had in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"openreview_AAxIs3D2ZZ","page":null,"syntology_url":null,"in_graph_layer":false,"samples":0},"line":"the paper matched by title, the paper Syntology's graph keys as openreview_AAxIs3D2ZZ, held, no code (no_code; paper_only). The lookup matched a paper by its title, the paper Syntology's graph keys as openreview_AAxIs3D2ZZ, not a method entry. The archive credits no paper with an arXiv id for this method, so whether this answer (about the paper Syntology's graph keys as openreview_AAxIs3D2ZZ) is about this method is not checked; it does not say whether this method has code.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"a3433ba552dc7b121d4284e07f20a3d8d2d2b72c33f20a83dceab5a36c2778e1","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}