Papers › PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning

PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning

28 Apr 2020ECCV 2020 8arXiv:2004.13513archive 2025-07-28

Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, Eduardo Valle

Lifelong learning has attracted much attention, but existing works still struggle to fight catastrophic forgetting and accumulate knowledge over long stretches of incremental learning. In this work, we propose PODNet, a model inspired by representation learning. By carefully balancing the compromise between remembering the old classes and learning new ones, PODNet fights catastrophic forgetting, even over very long runs of small incremental tasks --a setting so far unexplored by current works. PODNet innovates on existing art with an efficient spatial-based distillation-loss applied throughout the model and a representation comprising multiple proxy vectors for each class. We validate those innovations thoroughly, comparing PODNet with three state-of-the-art models on three datasets: CIFAR100, ImageNet100, and ImageNet1000. Our results showcase a significant advantage of PODNet over existing art, with accuracy gains of 12.10, 6.51, and 2.85 percentage points, respectively. Code is available at https://github.com/arthurdouillard/incremental_learning.pytorch

PaperPDFConference PDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

arthurdouillard/incremental_learning.pytorch officialmentioned in papermentioned on GitHubpytorchSyntology: not harvested report
g-u-n/pycil mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to RMM: Reinforced Memory Management for Class-Incremental Learning (arXiv:2301.05792; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Class Incremental LearningContinual LearningIncremental LearningLifelong learningRepresentation Learning

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Incremental Learning CIFAR-100 - 50 classes + 10 steps of 5 classes PODNet (CNN) Average Incremental Accuracy 63.19 #9 of 13 Archive leaderboard report
Incremental Learning CIFAR-100 - 50 classes + 25 steps of 2 classes PODNet Average Incremental Accuracy 60.72 #4 of 5 Archive leaderboard report
Incremental Learning CIFAR-100 - 50 classes + 5 steps of 10 classes PODNet (CNN) Average Incremental Accuracy 64.83 #9 of 15 Archive leaderboard report
Incremental Learning CIFAR-100 - 50 classes + 50 steps of 1 class PODNet Average Incremental Accuracy 57.98 #1 of 2 Archive leaderboard report
Incremental Learning CIFAR-100-B0(5steps of 20 classes) PODNet Average Incremental Accuracy 66.70 #9 of 10 Archive leaderboard report
Incremental Learning ImageNet - 500 classes + 10 steps of 50 classes PODNet Average Incremental Accuracy 64.13 #2 of 4 Archive leaderboard report
Incremental Learning ImageNet - 500 classes + 5 steps of 100 classes PODNet Average Incremental Accuracy 66.95 #3 of 4 Archive leaderboard report
Incremental Learning ImageNet-100 - 50 classes + 10 steps of 5 classes PODNet Average Incremental Accuracy 73.14 #5 of 5 Archive leaderboard report
Incremental Learning ImageNet-100 - 50 classes + 25 steps of 2 classes PODNet Average Incremental Accuracy 67.28 #3 of 3 Archive leaderboard report
Incremental Learning ImageNet-100 - 50 classes + 5 steps of 10 classes PODNet Average Incremental Accuracy 75.82 #4 of 5 Archive leaderboard report
Incremental Learning ImageNet-100 - 50 classes + 50 steps of 1 class PODNet Average Incremental Accuracy 62.08 #1 of 2 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections