Papers › CapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routing

CapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routing

30 Sep 2019ICCV 2019 10arXiv:1910.00132archive 2025-07-28

Kevin Duarte, Yogesh S Rawat, Mubarak Shah

In this work we propose a capsule-based approach for semi-supervised video object segmentation. Current video object segmentation methods are frame-based and often require optical flow to capture temporal consistency across frames which can be difficult to compute. To this end, we propose a video based capsule network, CapsuleVOS, which can segment several frames at once conditioned on a reference frame and segmentation mask. This conditioning is performed through a novel routing algorithm for attention-based efficient capsule selection. We address two challenging issues in video object segmentation: 1) segmentation of small objects and 2) occlusion of objects across time. The issue of segmenting small objects is addressed with a zooming module which allows the network to process small spatial regions of the video. Apart from this, the framework utilizes a novel memory module based on recurrent networks which helps in tracking objects when they move out of frame or are occluded. The network is trained end-to-end and we demonstrate its effectiveness on two benchmark video object segmentation datasets; it outperforms current offline approaches on the Youtube-VOS dataset while having a run-time that is almost twice as fast as competing methods. The code is publicly available at https://github.com/KevinDuarte/CapsuleVOS.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

KevinDuarte/CapsuleVOS officialmentioned in papermentioned on GitHubtfSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

ObjectOne-shot visual object segmentationOptical Flow EstimationSegmentationSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object SegmentationVideo Semantic SegmentationVisual Object Tracking

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) CapsuleVOS F-measure (Mean) 55.2 #54 of 59 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) CapsuleVOS F-measure (Recall) 64.6 #54 of 59 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) CapsuleVOS J&F 51.3 #54 of 59 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) CapsuleVOS Jaccard (Mean) 47.4 #54 of 59 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) CapsuleVOS Jaccard (Recall) 54.1 #54 of 59 Archive leaderboard report
Semi-Supervised Video Object Segmentation YouTube-VOS 2018 CapsuleVOS F-Measure (Seen) 68.1 #49 of 53 Archive leaderboard report
Semi-Supervised Video Object Segmentation YouTube-VOS 2018 CapsuleVOS F-Measure (Unseen) 59.9 #49 of 53 Archive leaderboard report
Semi-Supervised Video Object Segmentation YouTube-VOS 2018 CapsuleVOS Jaccard (Seen) 67.3 #49 of 53 Archive leaderboard report
Semi-Supervised Video Object Segmentation YouTube-VOS 2018 CapsuleVOS Jaccard (Unseen) 53.7 #49 of 53 Archive leaderboard report
Semi-Supervised Video Object Segmentation YouTube-VOS 2018 CapsuleVOS Overall 62.3 #49 of 53 Archive leaderboard report
Semi-Supervised Video Object Segmentation YouTube-VOS 2018 CapsuleVOS Speed (FPS) 13.5 #49 of 53 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections