{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/relic","slug":"relic","name":"ReLIC","full_name":"ReLIC","full_name_withheld":false,"description_markdown":"**ReLIC**, or **Representation Learning via Invariant Causal Mechanisms**, is a self-supervised learning objective that enforces invariant prediction of proxy targets across augmentations through an invariance regularizer which yields improved generalization guarantees. \r\n\r\nWe can write the objective as:\r\n\r\n$$\r\n\\underset{X}{\\mathbb{E}} \\underset{\\sim\\_{l k}, a\\_{q \\mathcal{A}}}{\\mathbb{E}} \\sum_{b \\in\\left\\(a\\_{l k}, a\\_{q t}\\right\\)} \\mathcal{L}\\_{b}\\left(Y^{R}, f(X)\\right) \\text { s.t. } K L\\left(p^{d o\\left(a\\_{l k}\\right)}\\left(Y^{R} \\mid f(X)\\right), p^{d o\\left(a\\_{q t}\\right)}\\left(Y^{R} \\mid f(X)\\right)\\right) \\leq \\rho\r\n$$\r\n\r\nwhere $\\mathcal{L}$ is the proxy task loss and $K L$ is the Kullback-Leibler (KL) divergence. Note that any distance measure on distributions can be used in place of the KL divergence.\r\n\r\nConcretely, as proxy task we associate to every datapoint $x\\_{i}$ the label $y\\_{i}^{R}=i$. This corresponds to the instance discrimination task, commonly used in contrastive learning. We take pairs of points $\\left(x\\_{i}, x\\_{j}\\right)$ to compute similarity scores and use pairs of augmentations $a\\_{l k}=\\left(a\\_{l}, a\\_{k}\\right) \\in$ $\\mathcal{A} \\times \\mathcal{A}$ to perform a style intervention. Given a batch of samples $\\left\\(x\\_{i}\\right\\)\\_{i=1}^{N} \\sim \\mathcal{D}$, we use\r\n\r\n$$\r\np^{d o\\left(a\\_{l k}\\right)}\\left(Y^{R}=j \\mid f\\left(x\\_{i}\\right)\\right) \\propto \\exp \\left(\\phi\\left(f\\left(x\\_{i}^{a\\_{l}}\\right), h\\left(x\\_{j}^{a\\_{k}}\\right)\\right) / \\tau\\right)\r\n$$\r\n\r\nwith $x^{a}$ data augmented with $a$ and $\\tau$ a softmax temperature parameter. We encode $f$ using a neural network and choose $h$ to be related to $f$, e.g. $h=f$ or as a network with an exponential moving average of the weights of $f$ (e.g. target networks). To compare representations we use the function $\\phi\\left(f\\left(x\\_{i}\\right), h\\left(x\\_{j}\\right)\\right)=\\left\\langle g\\left(f\\left(x\\_{i}\\right)\\right), g\\left(h\\left(x\\_{j}\\right)\\right)\\right\\rangle$ where $g$ is a fully-connected neural network often called the critic.\r\n\r\nCombining these pieces, we learn representations by minimizing the following objective over the full set of data $x\\_{i} \\in \\mathcal{D}$ and augmentations $a_{l k} \\in \\mathcal{A} \\times \\mathcal{A}$\r\n\r\n$$\r\n-\\sum_{i=1}^{N} \\sum\\_{a\\_{l k}} \\log \\frac{\\exp \\left(\\phi\\left(f\\left(x\\_{i}^{a_{l}}\\right), h\\left(x\\_{i}^{a\\_{k}}\\right)\\right) / \\tau\\right)}{\\sum\\_{m=1}^{M} \\exp \\left(\\phi\\left(f\\left(x\\_{i}^{a\\_{l}}\\right), h\\left(x\\_{m}^{a\\_{k}}\\right)\\right) / \\tau\\right)}+\\alpha \\sum\\_{a\\_{l k}, a\\_{q t}} K L\\left(p^{d o\\left(a\\_{l k}\\right)}, p^{d o\\left(a\\_{q t}\\right)}\\right)\r\n$$\r\n\r\nwith $M$ the number of points we use to construct the contrast set and $\\alpha$ the weighting of the invariance penalty. The shorthand $p^{d o(a)}$ is used for $p^{d o(a)}\\left(Y^{R}=j \\mid f\\left(x\\_{i}\\right)\\right)$. The Figure shows a schematic of the RELIC objective.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Representation Learning via Invariant Causal Mechanisms","paper":"/paper/representation-learning-via-invariant-causal-1","first_author":"Jovana Mitrovic","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/representation-learning-via-invariant-causal-1"},"source":{"url":"https://arxiv.org/abs/2010.07922v1","title":"Representation Learning via Invariant Causal Mechanisms","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Self-Supervised Learning","url":"/methods/category/self-supervised-learning","pwc_aliases":[]}],"n_papers_tagged":12,"archive_num_papers":12,"papers_newest_first":[{"paper":null,"slug":"relic-evaluating-compositional-instruction","title":"RELIC: Evaluating Compositional Instruction Following via Language Recognition","date":"2025-06-05","arxiv_id":"2506.05205","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-collaborative-jade-recognition-system-for","title":"A Collaborative Jade Recognition System for Mobile Devices Based on Lightweight and Large Models","date":"2025-02-20","arxiv_id":"2502.14332","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-rna-condensate-model-for-the-origin-of","title":"An RNA condensate model for the origin of life","date":"2024-12-06","arxiv_id":"2412.05396","n_code_links":0,"syntology":null},{"paper":null,"slug":"mrife-a-mask-recovering-and-interactive","title":"MRIFE: A Mask-Recovering and Interactive-Feature-Enhancing Semantic Segmentation Network For Relic Landslide Detection","date":"2024-11-26","arxiv_id":"2411.17167","n_code_links":0,"syntology":null},{"paper":null,"slug":"machine-learning-analysis-of-radiative-decays","title":"Machine-Learning Analysis of Radiative Decays to Dark Matter at the LHC","date":"2024-10-17","arxiv_id":"2410.13799","n_code_links":0,"syntology":null},{"paper":"/paper/relic-a-recipe-for-64k-steps-of-in-context","slug":"relic-a-recipe-for-64k-steps-of-in-context","title":"ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI","date":"2024-10-03","arxiv_id":"2410.02751","n_code_links":1,"syntology":{"ran":5,"of":7,"n_ran_checked":4,"n_instrument":1,"unverified":2,"pointer_only":7,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified","official":{"repos":["aielawady/relic"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"symbolic-regression-for-beyond-the-standard","title":"Symbolic Regression for Beyond the Standard Model Physics","date":"2024-05-28","arxiv_id":"2405.18471","n_code_links":0,"syntology":null},{"paper":"/paper/self-supervised-learning-through-efference","slug":"self-supervised-learning-through-efference","title":"Self-Supervised Learning Through Efference Copies","date":"2022-10-17","arxiv_id":"2210.09224","n_code_links":1,"syntology":null},{"paper":null,"slug":"mixed-anhydrides-at-the-intersection-between","title":"Mixed Anhydrides at the Intersection Between Peptide and RNA Autocatalytic Sets: Evolution of Biological Coding","date":"2022-05-25","arxiv_id":"2208.01491","n_code_links":0,"syntology":null},{"paper":"/paper/pushing-the-limits-of-self-supervised-resnets","slug":"pushing-the-limits-of-self-supervised-resnets","title":"Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?","date":"2022-01-13","arxiv_id":"2201.05119","n_code_links":1,"syntology":{"ran":13,"of":14,"n_ran_checked":13,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["google-deepmind/relicv2"],"state":"official (archive's flag): 13 ran","n_ran":13,"n_constructed":0,"n_ran_no_instrument_failure":13,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/coberl-contrastive-bert-for-reinforcement","slug":"coberl-contrastive-bert-for-reinforcement","title":"CoBERL: Contrastive BERT for Reinforcement Learning","date":"2021-07-12","arxiv_id":"2107.05431","n_code_links":2,"syntology":{"ran":3,"of":3,"n_ran_checked":1,"n_instrument":2,"unverified":0,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["deepmind/dm_control"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/representation-learning-via-invariant-causal-1","slug":"representation-learning-via-invariant-causal-1","title":"Representation Learning via Invariant Causal Mechanisms","date":"2020-10-15","arxiv_id":"2010.07922","n_code_links":2,"syntology":{"ran":2,"of":2,"n_ran_checked":0,"n_instrument":2,"unverified":0,"pointer_only":1,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":null}}],"papers_shown":12,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":12,"with_a_code_link":5,"where_syntology_ran_a_sample":4,"not_listed_spam_title":0,"listed":12,"listed_where_code_ran":4,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":3,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":3,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":4,"papers_where_code_ran_first_page":"/method/relic/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/relic/papers/ran/1","json":"/method/relic/papers/ran/1.json","rows":4}],"tasks":[{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":3},{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":2},{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":2},{"task":"/task/representation-learning","name":"Representation Learning","papers":2},{"task":"/task/self-supervised-image-classification","name":"Self-Supervised Image Classification","papers":2},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":2},{"task":"/task/image-classification","name":"image-classification","papers":2},{"task":"/task/few-shot-imitation-learning","name":"Few-Shot Imitation Learning","papers":1},{"task":"/task/imitation-learning","name":"Imitation Learning","papers":1},{"task":"/task/in-context-learning","name":"In-Context Learning","papers":1},{"task":null,"name":"In-Context Reinforcement Learning","papers":1},{"task":"/task/instruction-following","name":"Instruction Following","papers":1},{"task":"/task/linear-evaluation","name":"Linear evaluation","papers":1},{"task":"/task/object-detection","name":"Object Detection","papers":1},{"task":"/task/out-of-distribution-generalization","name":"Out-of-Distribution Generalization","papers":1},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":1},{"task":"/task/semi-supervised-image-classification","name":"Semi-Supervised Image Classification","papers":1},{"task":"/task/symbolic-regression","name":"Symbolic Regression","papers":1},{"task":"/task/model","name":"model","papers":1}],"tasks_shown":20,"n_tasks":23,"usage_by_year":[{"year":"2020","papers":1},{"year":"2021","papers":1},{"year":"2022","papers":3},{"year":"2024","papers":5},{"year":"2025","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/relic"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=ReLIC","read_at":"2026-10-05T01:01:04+00:00","query":"ReLIC","kind":"method","have":"no_code","tier":"paper_only","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"Relic","origin_arxiv_id":"2609.32965"},"resolution":"case_insensitive","resolved_name_differs":false,"absence_asserted":"for_this_exact_name_only","absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"different_paper","archive_paper_arxiv_id":"2010.07922","repositories":{"listed":null,"in_graph":0,"list_truncated":null,"strongest_link":null},"retrieve_with":null,"name_competition":{"verdict":"outranked","may_be_wrong_sense":true,"bearers_found":8,"served":{"arxiv_id":"2609.32965","citations":null},"leading_other_bearer":{"arxiv_id":"2311.16842","title":"RELIC: Investigating Large Language Model Responses using Self-Consistency","citations":60,"named_by":["title_prefix"],"bearer_name":"RELIC: Investigating Large Language Model Responses using Self-Consistency","ask_with":"have('2311.16842')"},"n_other_bearers_listed":5,"note":"'ReLIC' is ALSO the name of the paper 'RELIC: Investigating Large Language Model Responses using Self-Consistency' (arXiv:2311.16842, 60 citations), which is cited FAR more than the paper this answer is about (citations not recorded). This answer may be about the wrong ReLIC. If you meant the better-known one, ask again with that entry of `also_named`."},"name_competition_text":"The same name also belongs to the paper “RELIC: Investigating Large Language Model Responses using Self-Consistency” (arXiv:2311.16842, 60 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name.","near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"2609.32965","page":null,"syntology_url":null,"in_graph_layer":false,"samples":0},"line":"about another paper, arXiv:2609.32965: no code under this name (no_code; paper_only). The lookup matched 'Relic' ignoring case. About another paper: this answer is about arXiv:2609.32965, while the archive credits arXiv:2010.07922, so it does not say whether this method has code.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"aeddbafb1a98e47a47d882a2fd82cf5fa5bea41e49dac569446dfef1f27894bf","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}