Papers › A Lightweight Optical Flow CNN -- Revisiting Data Fidelity and Regularization

A Lightweight Optical Flow CNN -- Revisiting Data Fidelity and Regularization

15 Mar 2019arXiv:1903.07414archive 2025-07-28

Tak-Wai Hui, Xiaoou Tang, Chen Change Loy

Over four decades, the majority addresses the problem of optical flow estimation using variational methods. With the advance of machine learning, some recent works have attempted to address the problem using convolutional neural network (CNN) and have showed promising results. FlowNet2, the state-of-the-art CNN, requires over 160M parameters to achieve accurate flow estimation. Our LiteFlowNet2 outperforms FlowNet2 on Sintel and KITTI benchmarks, while being 25.3 times smaller in the model size and 3.1 times faster in the running speed. LiteFlowNet2 is built on the foundation laid by conventional methods and resembles the corresponding roles as data fidelity and regularization in variational methods. We compute optical flow in a spatial-pyramid formulation as SPyNet but through a novel lightweight cascaded flow inference. It provides high flow estimation accuracy through early correction with seamless incorporation of descriptor matching. Flow regularization is used to ameliorate the issue of outliers and vague flow boundaries through feature-driven local convolutions. Our network also owns an effective structure for pyramidal feature extraction and embraces feature warping rather than image warping as practiced in FlowNet2 and SPyNet. Comparing to LiteFlowNet, LiteFlowNet2 improves the optical flow accuracy on Sintel Clean by 23.3%, Sintel Final by 12.8%, KITTI 2012 by 19.6%, and KITTI 2015 by 18.8%, while being 2.2 times faster. Our network protocol and trained models are made publicly available on https://github.com/twhui/LiteFlowNet2.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

twhui/LiteFlowNet2 officialmentioned in papermentioned on GitHubtfNOASSERTIONSyntology: not harvested report
rogerhcheng/LiteFlowNet2-TF2 mentioned on GitHubtfGPL-3.0Syntology: not harvested report
open-mmlab/mmflow pytorchApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Learning to Estimate Hidden Motions with Global Motion Aggregation (arXiv:2104.02409; that paper's own run record: 4 ran (of which 1 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Optical Flow Estimation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Optical Flow Estimation KITTI 2012 LiteFlowNet2-ft Average End-Point Error 1.4 #6 of 12 Archive leaderboard report
Optical Flow Estimation KITTI 2012 LiteFlowNet2-ft Out-Noc 2.63 #6 of 12 Archive leaderboard report
Optical Flow Estimation KITTI 2015 LiteFlowNet2-ft Fl-all 7.62 #12 of 18 Archive leaderboard report
Optical Flow Estimation Sintel-clean LiteFlowNet2-ft Average End-Point Error 3.48 #20 of 29 Archive leaderboard report
Optical Flow Estimation Sintel-final LiteFlowNet2-ft Average End-Point Error 4.69 #22 of 28 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections