{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/mean-shifted-contrastive-loss-for-anomaly","title":"Mean-Shifted Contrastive Loss for Anomaly Detection","arxiv_id":"2106.03844","date":"2021-06-07","proceeding":null,"authors":["Tal Reiss","Yedid Hoshen"],"abstract":"Deep anomaly detection methods learn representations that separate between normal and anomalous images. Although self-supervised representation learning is commonly used, small dataset sizes limit its effectiveness. It was previously shown that utilizing external, generic datasets (e.g. ImageNet classification) can significantly improve anomaly detection performance. One approach is outlier exposure, which fails when the external datasets do not resemble the anomalies. We take the approach of transferring representations pre-trained on external datasets for anomaly detection. Anomaly detection performance can be significantly improved by fine-tuning the pre-trained representations on the normal training images. In this paper, we first demonstrate and analyze that contrastive learning, the most popular self-supervised learning paradigm cannot be naively applied to pre-trained features. The reason is that pre-trained feature initialization causes poor conditioning for standard contrastive objectives, resulting in bad optimization dynamics. Based on our analysis, we provide a modified contrastive objective, the Mean-Shifted Contrastive Loss. Our method is highly effective and achieves a new state-of-the-art anomaly detection performance including $98.6\\%$ ROC-AUC on the CIFAR-10 dataset.","url_abs":"https://arxiv.org/abs/2106.03844v2","url_pdf":"https://arxiv.org/pdf/2106.03844v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mean-shifted-contrastive-loss-for-anomaly","repo_url":"https://github.com/talreiss/Mean-Shifted-Anomaly-Detection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"mean-shifted-contrastive-loss-for-anomaly","repo_url":"https://github.com/talreiss/PANDA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/anomaly-detection-on-anomaly-detection-on","task":"Anomaly Detection","dataset":"Anomaly Detection on Unlabeled CIFAR-10 vs LSUN (Fix)","model":"MeanShifted","rank_in_archive_order":5,"of":8,"metrics":{"Network":"ResNet-152","ROC-AUC":"92.6"},"uses_additional_data":true},{"leaderboard":"/sota/anomaly-detection-on-mvtec-ad","task":"Anomaly Detection","dataset":"MVTec AD","model":"Mean-Shifted Contrastive Loss","rank_in_archive_order":117,"of":148,"metrics":{"Detection AUROC":"87.2"},"uses_additional_data":false},{"leaderboard":"/sota/anomaly-detection-on-one-class-cifar-10","task":"Anomaly Detection","dataset":"One-class CIFAR-10","model":"Mean-Shifted Contrastive Loss","rank_in_archive_order":6,"of":36,"metrics":{"AUROC":"98.6"},"uses_additional_data":true},{"leaderboard":"/sota/anomaly-detection-on-one-class-cifar-100","task":"Anomaly Detection","dataset":"One-class CIFAR-100","model":"Mean-Shifted Contrastive Loss","rank_in_archive_order":4,"of":15,"metrics":{"AUROC":"96.5"},"uses_additional_data":true},{"leaderboard":"/sota/anomaly-detection-on-unlabeled-cifar-10-vs","task":"Anomaly Detection","dataset":"Unlabeled CIFAR-10 vs CIFAR-100","model":"MeanShifted","rank_in_archive_order":5,"of":13,"metrics":{"AUROC":"90.0","Network":"ResNet-152"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.03844","atlas_url":"https://app.syntology.ai/?focus=2106.03844","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2106.03844"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2106.03844\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2106.03844"},"call":"get_concepts_for_paper(arxiv_id=\"2106.03844\")"}],"by_repo":[{"repo":"talreiss/Mean-Shifted-Anomaly-Detection","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"talreiss/PANDA","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"674a2ea58eedfedfe1bb9991b912fbdc4d37110a9093327618a448311e08ce9a","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}