{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/natural-adversarial-objects","title":"Natural Adversarial Objects","arxiv_id":"2111.04204","date":"2021-11-07","proceeding":null,"authors":["Felix Lau","Nishant Subramani","Sasha Harrison","Aerin Kim","Elliot Branson","Rosanne Liu"],"abstract":"Although state-of-the-art object detection methods have shown compelling performance, models often are not robust to adversarial attacks and out-of-distribution data. We introduce a new dataset, Natural Adversarial Objects (NAO), to evaluate the robustness of object detection models. NAO contains 7,934 images and 9,943 objects that are unmodified and representative of real-world scenarios, but cause state-of-the-art detection models to misclassify with high confidence. The mean average precision (mAP) of EfficientDet-D7 drops 74.5% when evaluated on NAO compared to the standard MSCOCO validation set. Moreover, by comparing a variety of object detection architectures, we find that better performance on MSCOCO validation set does not necessarily translate to better performance on NAO, suggesting that robustness cannot be simply achieved by training a more accurate model. We further investigate why examples in NAO are difficult to detect and classify. Experiments of shuffling image patches reveal that models are overly sensitive to local texture. Additionally, using integrated gradients and background replacement, we find that the detection model is reliant on pixel information within the bounding box, and insensitive to the background context when predicting class labels. NAO can be downloaded at https://drive.google.com/drive/folders/15P8sOWoJku6SSEiHLEts86ORfytGezi8.","url_abs":"https://arxiv.org/abs/2111.04204v1","url_pdf":"https://arxiv.org/pdf/2111.04204v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[{"slug":"nao","name":"NAO","full_name":"Natural Adversarial Object"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-detection-on-nao","task":"Object Detection","dataset":"NAO","model":"Mask RCNN R50","rank_in_archive_order":1,"of":7,"metrics":{"mAP":"15.2","mAP w/o OOD":"24.6","mAR":"43.8"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-nao","task":"Object Detection","dataset":"NAO","model":"EfficientDet-D4","rank_in_archive_order":2,"of":7,"metrics":{"mAP":"15.0","mAP w/o OOD":"29.6","mAR":"42.7"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-nao","task":"Object Detection","dataset":"NAO","model":"EfficientDet-D7","rank_in_archive_order":3,"of":7,"metrics":{"mAP":"13.6","mAP w/o OOD":"26.6","mAR":"40.8"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-nao","task":"Object Detection","dataset":"NAO","model":"Faster RCNN","rank_in_archive_order":4,"of":7,"metrics":{"mAP":"13.5","mAP w/o OOD":"22.8","mAR":"41.4"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-nao","task":"Object Detection","dataset":"NAO","model":"EfficientDet-D2","rank_in_archive_order":5,"of":7,"metrics":{"mAP":"12.8","mAP w/o OOD":"25.4","mAR":"40.2"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-nao","task":"Object Detection","dataset":"NAO","model":"RetinaNet-R50","rank_in_archive_order":6,"of":7,"metrics":{"mAP":"11.1","mAP w/o OOD":"19.5","mAR":"37.2"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-nao","task":"Object Detection","dataset":"NAO","model":"YOLOv3","rank_in_archive_order":7,"of":7,"metrics":{"mAP":"10.0","mAP w/o OOD":"17.5","mAR":"28.4"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2111.04204","atlas_url":"https://app.syntology.ai/?focus=2111.04204","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2111.04204"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2111.04204\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2111.04204"},"call":"get_concepts_for_paper(arxiv_id=\"2111.04204\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"6fda5131d5114abcc64e52336e60693ded32ca67707d10fa41b969b17d0819df","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}