{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/hs3-bench-a-benchmark-and-strong-baseline-for","title":"HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios","arxiv_id":"2409.11205","date":"2024-09-17","proceeding":null,"authors":["Nick Theisen","Robin Bartsch","Dietrich Paulus","Peer Neubert"],"abstract":"Semantic segmentation is an essential step for many vision applications in order to understand a scene and the objects within. Recent progress in hyperspectral imaging technology enables the application in driving scenarios and the hope is that the devices perceptive abilities provide an advantage over RGB-cameras. Even though some datasets exist, there is no standard benchmark available to systematically measure progress on this task and evaluate the benefit of hyperspectral data. In this paper, we work towards closing this gap by providing the HyperSpectral Semantic Segmentation benchmark (HS3-Bench). It combines annotated hyperspectral images from three driving scenario datasets and provides standardized metrics, implementations, and evaluation protocols. We use the benchmark to derive two strong baseline models that surpass the previous state-of-the-art performances with and without pre-training on the individual datasets. Further, our results indicate that the existing learning-based methods benefit more from leveraging additional RGB training data than from leveraging the additional hyperspectral channels. This poses important questions for future research on hyperspectral imaging for semantic segmentation in driving scenarios. Code to run the benchmark and the strong baseline approaches are available under https://github.com/nickstheisen/hyperseg.","url_abs":"https://arxiv.org/abs/2409.11205v1","url_pdf":"https://arxiv.org/pdf/2409.11205v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hs3-bench-a-benchmark-and-strong-baseline-for","repo_url":"https://github.com/nickstheisen/hyperseg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"hyperspectral-image-segmentation","task_name":"Hyperspectral Image Segmentation"},{"task_slug":"hyperspectral-semantic-segmentation","task_name":"Hyperspectral Semantic Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on-hsi","task":"Hyperspectral Semantic Segmentation","dataset":"HSI-Drive v2.0","model":"RU-Net","rank_in_archive_order":1,"of":3,"metrics":{"Accuracy":"96.08","Average Accuracy":"79.82","Avg. F1":"82.34","Jaccard (Mean)":"72.18"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on-hsi","task":"Hyperspectral Semantic Segmentation","dataset":"HSI-Drive v2.0","model":"U-Net","rank_in_archive_order":2,"of":3,"metrics":{"Accuracy":"94.95","Average Accuracy":"74.74","Avg. F1":"76.08","Jaccard (Mean)":"64.95"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on-hsi","task":"Hyperspectral Semantic Segmentation","dataset":"HSI-Drive v2.0","model":"DeepLabV3+","rank_in_archive_order":3,"of":3,"metrics":{"Accuracy":"92.51","Average Accuracy":"65.58","Avg. F1":"67.86","Jaccard (Mean)":"56.63"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on-hyko2","task":"Hyperspectral Semantic Segmentation","dataset":"HyKo2-VIS","model":"RU-Net","rank_in_archive_order":1,"of":3,"metrics":{"Accuracy":"86.72","Average Accuracy":"68.79","Average Jaccard":"58.64","Avg. F1":"69.19"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on-hyko2","task":"Hyperspectral Semantic Segmentation","dataset":"HyKo2-VIS","model":"U-Net","rank_in_archive_order":2,"of":3,"metrics":{"Accuracy":"85.36","Average Accuracy":"68.15","Average Jaccard":"57.39","Avg. F1":"68.55"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on-hyko2","task":"Hyperspectral Semantic Segmentation","dataset":"HyKo2-VIS","model":"DeepLabV3+","rank_in_archive_order":3,"of":3,"metrics":{"Accuracy":"84.10","Average Accuracy":"63.01","Average Jaccard":"53.22","Avg. F1":"64.90"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on","task":"Hyperspectral Semantic Segmentation","dataset":"Hyperspectral City","model":"RU-Net","rank_in_archive_order":1,"of":3,"metrics":{"Accuracy ":"87.63","Average Accuracy":"54.14","Avg. F1":"53.26","Jaccard (Mean)":"43.33"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on","task":"Hyperspectral Semantic Segmentation","dataset":"Hyperspectral City","model":"DeepLabV3+","rank_in_archive_order":2,"of":3,"metrics":{"Accuracy ":"86.60","Average Accuracy":"53.15","Avg. F1":"51.83","Jaccard (Mean)":"40.79"},"uses_additional_data":false},{"leaderboard":"/sota/hyperspectral-semantic-segmentation-on","task":"Hyperspectral Semantic Segmentation","dataset":"Hyperspectral City","model":"U-Net","rank_in_archive_order":3,"of":3,"metrics":{"Accuracy ":"85.25","Average Accuracy":"48.62","Avg. F1":"48.18","Jaccard (Mean)":"37.73"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"nickstheisen/hyperseg","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"d223794213f13e1a1c3fb8d7d4b597032076434fe55d9dddad9708d0b3037266","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}