{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/mix-ffn","slug":"mix-ffn","name":"Mix-FFN","full_name":"Mix-FFN","full_name_withheld":false,"description_markdown":"**Mix-FFN** is a feedforward layer used in the [SegFormer](https://paperswithcode.com/method/segformer) architecture. [ViT](https://www.paperswithcode.com/method/vision-transformer) uses [positional encoding](https://paperswithcode.com/methods/category/position-embeddings) (PE) to introduce the location information. However, the resolution of $\\mathrm{PE}$ is fixed. Therefore, when the test resolution is different from the training one, the positional code needs to be interpolated and this often leads to dropped accuracy. To alleviate this problem, [CPVT](https://www.paperswithcode.com/method/cpvt) uses $3 \\times 3$ Conv together with the PE to implement a data-driven PE. The authors of Mix-FFN argue that positional encoding is actually not necessary for semantic segmentation. Instead, they use Mix-FFN which considers the effect of zero padding to leak location information, by directly using a $3 \\times 3$ Conv in the feed-forward network (FFN). Mix-FFN can be formulated as:\r\n\r\n$$\r\n\\mathbf{x}\\_{\\text {out }}=\\operatorname{MLP}\\left(\\operatorname{GELU}\\left(\\operatorname{Conv}\\_{3 \\times 3}\\left(\\operatorname{MLP}\\left(\\mathbf{x}\\_{i n}\\right)\\right)\\right)\\right)+\\mathbf{x}\\_{i n}\r\n$$\r\n\r\nwhere $\\mathbf{x}\\_{i n}$ is the feature from a self-attention module. Mix-FFN mixes a $3 \\times 3$ convolution and an MLP into each FFN.","description_state":"present","introduced_year":null,"introduced_by":{"title":"SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers","paper":"/paper/segformer-simple-and-efficient-design-for","first_author":"Enze Xie","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/segformer-simple-and-efficient-design-for"},"source":{"url":"https://arxiv.org/abs/2105.15203v3","title":"SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Feedforward Networks","url":"/methods/category/feedforward-networks","pwc_aliases":[]}],"n_papers_tagged":47,"archive_num_papers":47,"papers_newest_first":[{"paper":"/paper/from-pixels-to-damage-severity-estimating","slug":"from-pixels-to-damage-severity-estimating","title":"From Pixels to Damage Severity: Estimating Earthquake Impacts Using Semantic Segmentation of Social Media Images","date":"2025-07-03","arxiv_id":"2507.02781","n_code_links":0,"syntology":null},{"paper":null,"slug":"leveraging-modified-ex-situ-tomography-data","title":"Leveraging Modified Ex Situ Tomography Data for Segmentation of In Situ Synchrotron X-Ray Computed Tomography","date":"2025-04-27","arxiv_id":"2504.19200","n_code_links":0,"syntology":null},{"paper":null,"slug":"sar-object-detection-with-self-supervised","title":"SAR Object Detection with Self-Supervised Pretraining and Curriculum-Aware Sampling","date":"2025-04-17","arxiv_id":"2504.13310","n_code_links":0,"syntology":null},{"paper":"/paper/improving-underwater-semantic-segmentation","slug":"improving-underwater-semantic-segmentation","title":"Improving underwater semantic segmentation with underwater image quality attention and muti-scale aggregation attention","date":"2025-03-30","arxiv_id":"2503.23422","n_code_links":1,"syntology":null},{"paper":null,"slug":"automated-retinal-layer-and-fluid","title":"Comprehensive Evaluation of OCT-based Automated Segmentation of Retinal Layer, Fluid and Hyper-Reflective Foci: Impact on Diabetic Retinopathy Severity Assessment","date":"2025-03-03","arxiv_id":"2503.01248","n_code_links":0,"syntology":null},{"paper":"/paper/cross-model-transferability-of-adversarial","slug":"cross-model-transferability-of-adversarial","title":"Cross-Model Transferability of Adversarial Patches in Real-time Segmentation for Autonomous Driving","date":"2025-02-22","arxiv_id":"2502.16012","n_code_links":1,"syntology":null},{"paper":null,"slug":"global-average-feature-augmentation-for","title":"Global Average Feature Augmentation for Robust Semantic Segmentation with Transformers","date":"2024-12-02","arxiv_id":"2412.01941","n_code_links":0,"syntology":null},{"paper":null,"slug":"syndiff-ad-improving-semantic-segmentation","title":"SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Models","date":"2024-11-25","arxiv_id":"2411.16776","n_code_links":0,"syntology":null},{"paper":"/paper/habaek-high-performance-water-segmentation","slug":"habaek-high-performance-water-segmentation","title":"Habaek: High-performance water segmentation through dataset expansion and inductive bias optimization","date":"2024-10-21","arxiv_id":"2410.15794","n_code_links":1,"syntology":null},{"paper":null,"slug":"risk-assessment-for-autonomous-landing-in","title":"Risk Assessment for Autonomous Landing in Urban Environments using Semantic Segmentation","date":"2024-10-16","arxiv_id":"2410.12988","n_code_links":0,"syntology":null},{"paper":null,"slug":"adapting-segment-anything-model-to-melanoma","title":"Adapting Segment Anything Model to Melanoma Segmentation in Microscopy Slide Images","date":"2024-10-03","arxiv_id":"2410.02207","n_code_links":0,"syntology":null},{"paper":null,"slug":"semantic-segmentation-of-unmanned-aerial","title":"Semantic Segmentation of Unmanned Aerial Vehicle Remote Sensing Images using SegFormer","date":"2024-10-01","arxiv_id":"2410.01092","n_code_links":0,"syntology":null},{"paper":null,"slug":"amber-advanced-segformer-for-multi-band-image","title":"AMBER -- Advanced SegFormer for Multi-Band Image Segmentation: an application to Hyperspectral Imaging","date":"2024-09-14","arxiv_id":"2409.09386","n_code_links":0,"syntology":null},{"paper":null,"slug":"sharp-net-a-refined-pyramid-network-for","title":"SHARP-Net: A Refined Pyramid Network for Deficiency Segmentation in Culverts and Sewer Pipes","date":"2024-08-02","arxiv_id":"2408.08879","n_code_links":0,"syntology":null},{"paper":"/paper/magicbathynet-a-multimodal-remote-sensing","slug":"magicbathynet-a-multimodal-remote-sensing","title":"MagicBathyNet: A Multimodal Remote Sensing Dataset for Bathymetry Prediction and Pixel-based Classification in Shallow Waters","date":"2024-05-24","arxiv_id":"2405.15477","n_code_links":1,"syntology":null},{"paper":"/paper/segformer-efficient-token-merging-strategies","slug":"segformer-efficient-token-merging-strategies","title":"Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation","date":"2024-05-23","arxiv_id":"2405.14467","n_code_links":2,"syntology":null},{"paper":"/paper/wmh-seg-transformer-based-u-net-for-robust","slug":"wmh-seg-transformer-based-u-net-for-robust","title":"wmh_seg: Transformer based U-Net for Robust and Automatic White Matter Hyperintensity Segmentation across 1.5T, 3T and 7T","date":"2024-02-20","arxiv_id":"2402.12701","n_code_links":1,"syntology":null},{"paper":null,"slug":"kitchen-food-waste-image-segmentation-and","title":"Kitchen Food Waste Image Segmentation and Classification for Compost Nutrients Estimation","date":"2024-01-26","arxiv_id":"2401.15175","n_code_links":0,"syntology":null},{"paper":"/paper/cabuar-california-burned-areas-dataset-for","slug":"cabuar-california-burned-areas-dataset-for","title":"CaBuAr: California Burned Areas dataset for delineation","date":"2024-01-21","arxiv_id":"2401.11519","n_code_links":1,"syntology":null},{"paper":"/paper/u-mixformer-unet-like-transformer-with-mix","slug":"u-mixformer-unet-like-transformer-with-mix","title":"U-MixFormer: UNet-like Transformer with Mix-Attention for Efficient Semantic Segmentation","date":"2023-12-11","arxiv_id":"2312.06272","n_code_links":1,"syntology":null},{"paper":null,"slug":"enhancing-transformer-based-segmentation-for","title":"Enhancing Transformer-Based Segmentation for Breast Cancer Diagnosis using Auto-Augmentation and Search Optimisation Techniques","date":"2023-11-18","arxiv_id":"2311.11065","n_code_links":0,"syntology":null},{"paper":null,"slug":"depth-guided-free-space-segmentation-for-a","title":"Depth-guided Free-space Segmentation for a Mobile Robot","date":"2023-11-03","arxiv_id":"2311.01966","n_code_links":0,"syntology":null},{"paper":"/paper/minesegsat-an-automated-system-to-evaluate","slug":"minesegsat-an-automated-system-to-evaluate","title":"MineSegSAT: An automated system to evaluate mining disturbed area extents from Sentinel-2 imagery","date":"2023-11-03","arxiv_id":"2311.01676","n_code_links":1,"syntology":null},{"paper":"/paper/robust-burned-area-delineation-through","slug":"robust-burned-area-delineation-through","title":"Robust Burned Area Delineation through Multitask Learning","date":"2023-09-15","arxiv_id":"2309.08368","n_code_links":2,"syntology":null},{"paper":"/paper/dacl10k-benchmark-for-semantic-bridge-damage","slug":"dacl10k-benchmark-for-semantic-bridge-damage","title":"dacl10k: Benchmark for Semantic Bridge Damage Segmentation","date":"2023-09-01","arxiv_id":"2309.00460","n_code_links":1,"syntology":null},{"paper":"/paper/fosp-focus-and-separation-network-for-early","slug":"fosp-focus-and-separation-network-for-early","title":"FoSp: Focus and Separation Network for Early Smoke Segmentation","date":"2023-06-07","arxiv_id":"2306.04474","n_code_links":1,"syntology":null},{"paper":null,"slug":"sam-for-poultry-science","title":"SAM for Poultry Science","date":"2023-05-17","arxiv_id":"2305.10254","n_code_links":0,"syntology":null},{"paper":"/paper/concurrent-misclassification-and-out-of","slug":"concurrent-misclassification-and-out-of","title":"Concurrent Misclassification and Out-of-Distribution Detection for Semantic Segmentation via Energy-Based Normalizing Flow","date":"2023-05-16","arxiv_id":"2305.09610","n_code_links":1,"syntology":null},{"paper":null,"slug":"radious-unveiling-the-enigma-of-dental","title":"Radious: Unveiling the Enigma of Dental Radiology with BEIT Adaptor and Mask2Former in Semantic Segmentation","date":"2023-05-10","arxiv_id":"2305.06236","n_code_links":0,"syntology":null},{"paper":"/paper/implications-of-the-convergence-of-language","slug":"implications-of-the-convergence-of-language","title":"Do Vision and Language Models Share Concepts? A Vector Space Alignment Study","date":"2023-02-13","arxiv_id":"2302.06555","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":2,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["jiaangli/vlca"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"buildseg-a-general-framework-for-the","title":"BuildSeg: A General Framework for the Segmentation of Buildings","date":"2023-01-15","arxiv_id":"2301.06190","n_code_links":0,"syntology":null},{"paper":"/paper/head-free-lightweight-semantic-segmentation","slug":"head-free-lightweight-semantic-segmentation","title":"Head-Free Lightweight Semantic Segmentation with Linear Transformer","date":"2023-01-11","arxiv_id":"2301.04648","n_code_links":1,"syntology":null},{"paper":"/paper/efficientvit-lightweight-multi-scale","slug":"efficientvit-lightweight-multi-scale","title":"EfficientViT: Lightweight Multi-Scale Attention for High-Resolution Dense Prediction","date":"2023-01-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"semi-supervised-semantic-segmentation-methods","title":"Semi-Supervised Semantic Segmentation Methods for UW-OCTA Diabetic Retinopathy Grade Assessment","date":"2022-12-27","arxiv_id":"2212.13486","n_code_links":0,"syntology":null},{"paper":null,"slug":"enabling-and-accelerating-dynamic-vision","title":"Vision Transformer Computation and Resilience for Dynamic Inference","date":"2022-12-06","arxiv_id":"2212.02687","n_code_links":0,"syntology":null},{"paper":"/paper/fsanet-frequency-self-attention-for-semantic","slug":"fsanet-frequency-self-attention-for-semantic","title":"FsaNet: Frequency Self-attention for Semantic Segmentation","date":"2022-11-28","arxiv_id":"2211.15595","n_code_links":1,"syntology":null},{"paper":"/paper/1st-place-solution-of-the-robust-vision","slug":"1st-place-solution-of-the-robust-vision","title":"1st Place Solution of The Robust Vision Challenge 2022 Semantic Segmentation Track","date":"2022-10-23","arxiv_id":"2210.12852","n_code_links":1,"syntology":null},{"paper":null,"slug":"the-outcome-of-the-2022-landslide4sense","title":"The Outcome of the 2022 Landslide4Sense Competition: Advanced Landslide Detection from Multi-Source Satellite Imagery","date":"2022-09-06","arxiv_id":"2209.02556","n_code_links":0,"syntology":null},{"paper":"/paper/siamixformer-a-siamese-transformer-network","slug":"siamixformer-a-siamese-transformer-network","title":"SiamixFormer: a fully-transformer Siamese network with temporal Fusion for accurate building detection and change detection in bi-temporal remote sensing images","date":"2022-08-01","arxiv_id":"2208.00657","n_code_links":0,"syntology":null},{"paper":null,"slug":"improving-semantic-segmentation-in","title":"Improving Semantic Segmentation in Transformers using Hierarchical Inter-Level Attention","date":"2022-07-05","arxiv_id":"2207.02126","n_code_links":0,"syntology":null},{"paper":"/paper/cv-3315-is-all-you-need-semantic-segmentation","slug":"cv-3315-is-all-you-need-semantic-segmentation","title":"CV 3315 Is All You Need : Semantic Segmentation Competition","date":"2022-06-25","arxiv_id":"2206.12571","n_code_links":1,"syntology":null},{"paper":"/paper/efficientvit-enhanced-linear-attention-for","slug":"efficientvit-enhanced-linear-attention-for","title":"EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction","date":"2022-05-29","arxiv_id":"2205.14756","n_code_links":6,"syntology":{"ran":4,"of":4,"n_ran_checked":1,"n_instrument":3,"unverified":0,"pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","official":{"repos":["mit-han-lab/efficientvit","rwightman/pytorch-image-models"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/a-transformer-based-siamese-network-for","slug":"a-transformer-based-siamese-network-for","title":"A Transformer-Based Siamese Network for Change Detection","date":"2022-01-04","arxiv_id":"2201.01293","n_code_links":3,"syntology":{"ran":4,"of":6,"n_ran_checked":2,"n_instrument":2,"unverified":2,"pointer_only":3,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","official":{"repos":["wgcban/changeformer"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/fbnetv5-neural-architecture-search-for","slug":"fbnetv5-neural-architecture-search-for","title":"FBNetV5: Neural Architecture Search for Multiple Tasks in One Run","date":"2021-11-19","arxiv_id":"2111.10007","n_code_links":0,"syntology":null},{"paper":null,"slug":"dynamically-pruning-segformer-for-efficient","title":"Dynamically pruning segformer for efficient semantic segmentation","date":"2021-11-18","arxiv_id":"2111.09499","n_code_links":0,"syntology":null},{"paper":"/paper/panoptic-segformer","slug":"panoptic-segformer","title":"Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers","date":"2021-09-08","arxiv_id":"2109.03814","n_code_links":3,"syntology":null},{"paper":"/paper/segformer-simple-and-efficient-design-for","slug":"segformer-simple-and-efficient-design-for","title":"SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers","date":"2021-05-31","arxiv_id":"2105.15203","n_code_links":28,"syntology":{"ran":66,"of":86,"n_ran_checked":52,"n_instrument":14,"unverified":20,"pointer_only":20,"phrase":"66 ran (of which 22 constructed an object rather than computing a result; 52 with no instrument failure: 1 honoured, 2 violated, 49 with no contract checked; 14 where Syntology's instrument failed) · 20 unverified","official":{"repos":["NVlabs/SegFormer"],"state":"official: no sample here; runs from other or unrecorded repositories","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["community","listed","unlocated"]}}}],"papers_shown":47,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":47,"with_a_code_link":23,"where_syntology_ran_a_sample":4,"not_listed_spam_title":0,"listed":47,"listed_where_code_ran":4,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":4,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":4,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":4,"papers_where_code_ran_first_page":"/method/mix-ffn/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/mix-ffn/papers/ran/1","json":"/method/mix-ffn/papers/ran/1.json","rows":4}],"tasks":[{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":33},{"task":"/task/segmentation","name":"Segmentation","papers":22},{"task":"/task/decoder","name":"Decoder","papers":7},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":4},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":4},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":3},{"task":"/task/diversity","name":"Diversity","papers":3},{"task":null,"name":"GPU","papers":3},{"task":"/task/image-classification","name":"Image Classification","papers":3},{"task":"/task/instance-segmentation","name":"Instance Segmentation","papers":3},{"task":"/task/object-detection","name":"Object Detection","papers":3},{"task":"/task/object-detection-1","name":"object-detection","papers":3},{"task":"/task/2d-semantic-segmentation","name":"2D Semantic Segmentation","papers":2},{"task":"/task/burned-area-delineation","name":"Burned Area Delineation","papers":2},{"task":null,"name":"CPU","papers":2},{"task":"/task/change-detection","name":"Change Detection","papers":2},{"task":"/task/disaster-response","name":"Disaster Response","papers":2},{"task":"/task/land-cover-classification","name":"Land Cover Classification","papers":2},{"task":"/task/object","name":"Object","papers":2},{"task":"/task/prediction","name":"Prediction","papers":2}],"tasks_shown":20,"n_tasks":57,"usage_by_year":[{"year":"2021","papers":4},{"year":"2022","papers":10},{"year":"2023","papers":14},{"year":"2024","papers":13},{"year":"2025","papers":6}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/mix-ffn"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Mix-FFN","read_at":"2026-10-05T00:59:08+00:00","query":"Mix-FFN","kind":"method","have":"unknown_name","tier":null,"verified_implementation":null,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":null,"resolution":null,"resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":null,"archive_paper_arxiv_id":"2105.15203","repositories":null,"retrieve_with":null,"name_competition":{"verdict":"none","may_be_wrong_sense":false,"bearers_found":0,"served":null,"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing in this corpus is named 'Mix-FFN' -- no method node, no paper announcing it in its title, no expanded form declaring it as an acronym."},"name_competition_text":null,"near_matches":["e-Mix","i-Mix","MB-MIX","NP-Mix"],"code_nearby":[],"code_nearby_caption":null,"origin_paper":null,"line":"name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"145c8949e19138d87ae667309cf6fc8697e06b8b6d199180fb7095939d543417","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}