Papers › Polarized Self-Attention: Towards High-quality Pixel-wise Regression

Polarized Self-Attention: Towards High-quality Pixel-wise Regression

2 Jul 2021arXiv preprint 2021 7arXiv:2107.00782archive 2025-07-28

Huajun Liu, Fuqiang Liu, Xinyi Fan, Dong Huang

Pixel-wise regression is probably the most common problem in fine-grained computer vision tasks, such as estimating keypoint heatmaps and segmentation masks. These regression problems are very challenging particularly because they require, at low computation overheads, modeling long-range dependencies on high-resolution inputs/outputs to estimate the highly nonlinear pixel-wise semantics. While attention mechanisms in Deep Convolutional Neural Networks(DCNNs) has become popular for boosting long-range dependencies, element-specific attention, such as Nonlocal blocks, is highly complex and noise-sensitive to learn, and most of simplified attention hybrids try to reach the best compromise among multiple types of tasks. In this paper, we present the Polarized Self-Attention(PSA) block that incorporates two critical designs towards high-quality pixel-wise regression: (1) Polarized filtering: keeping high internal resolution in both channel and spatial attention computation while completely collapsing input tensors along their counterpart dimensions. (2) Enhancement: composing non-linearity that directly fits the output distribution of typical fine-grained regression, such as the 2D Gaussian distribution (keypoint heatmaps), or the 2D Binormial distribution (binary segmentation masks). PSA appears to have exhausted the representation capacity within its channel-only and spatial-only branches, such that there is only marginal metric differences between its sequential and parallel layouts. Experimental results show that PSA boosts standard baselines by 2-4 points, and boosts state-of-the-arts by 1-2 points on 2D pose estimation and semantic segmentation benchmarks.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

DeLightCMU/PSA officialmentioned in papermentioned on GitHubpytorchSyntology: not harvested report
sithu31296/pose-estimation mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Human Pose Regression with Residual Log-likelihood Estimation (arXiv:2107.11291; that paper's own run record: 10 ran (of which 2 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
sithu31296/semantic-segmentation mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (arXiv:2105.15203; that paper's own run record: 66 ran (of which 22 constructed an object rather than computing a result; 52 with no instrument failure: 1 honoured, 2 violated, 49 with no contract checked; 14 where Syntology's instrument failed) · 20 unverified), FaPN: Feature-aligned Pyramid Network for Dense Image Prediction (arXiv:2108.07058; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified) and A ConvNet for the 2020s (arXiv:2201.03545; that paper's own run record: 57 ran (of which 39 constructed an object rather than computing a result; 49 with no instrument failure: 2 honoured, 0 violated, 47 with no contract checked; 8 where Syntology's instrument failed) · 23 unverified); nothing checks that they implement this paper's method. report
PaddlePaddle/PaddleSeg paddleSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow (arXiv:2109.09406; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
mindspore-courses/External-Attention-MindSpore mindsporeSyntology: not harvested report
xmu-xiaoma666/External-Attention-pytorch pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Attention Is All You Need (arXiv:1706.03762; that paper's own run record: 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified), RepVGG: Making VGG-style ConvNets Great Again (arXiv:2101.03697; that paper's own run record: 13 ran (of which 7 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified), Involution: Inverting the Inherence of Convolution for Visual Recognition (arXiv:2103.06255; that paper's own run record: 7 ran (of which 7 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 7 samples that ran constructed an object rather than computing a result) and 4 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

2D Pose EstimationKeypoint DetectionPose EstimationSegmentationSemantic SegmentationVocal Bursts Intensity Predictionregression

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Keypoint Detection COCO (Common Objects in Context) UDP-Pose-PSA(384x288) Validation AP 79.5 #20 of 24 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(384x288) AP 79.5 #3 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(384x288) AP50 93.6 #3 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(384x288) AP75 85.9 #3 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(384x288) APL 84.3 #3 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(384x288) APM 76.3 #3 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(384x288) AR 81.9 #3 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(256x192) AP 78.9 #7 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(256x192) AP50 93.6 #7 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(256x192) AP75 85.8 #7 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(256x192) APL 83.6 #7 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(256x192) APM 76.1 #7 of 47 Archive leaderboard report
Pose Estimation COCO test-dev UDP-Pose-PSA(256x192) AR 81.4 #7 of 47 Archive leaderboard report
Semantic Segmentation Cityscapes val HRNetV2-OCR+PSA mIoU 86.93 #5 of 99 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections