Papers › Value Prediction Network

Value Prediction Network

11 Jul 2017NeurIPS 2017 12arXiv:1707.03497archive 2025-07-28

Junhyuk Oh, Satinder Singh, Honglak Lee

This paper proposes a novel deep reinforcement learning (RL) architecture, called Value Prediction Network (VPN), which integrates model-free and model-based RL methods into a single neural network. In contrast to typical model-based RL methods, VPN learns a dynamics model whose abstract states are trained to make option-conditional predictions of future values (discounted sum of rewards) rather than of future observations. Our experimental results show that VPN has several advantages over both model-free and model-based baselines in a stochastic environment where careful planning is required but building an accurate observation-prediction model is difficult. Furthermore, VPN outperforms Deep Q-Network (DQN) on several Atari games even with short-lookahead planning, demonstrating its potential as a new way of learning a good state representation.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

junhyukoh/value-prediction-network officialmentioned in papermentioned on GitHubtfSyntology: not harvested report
geohot/twitchcoq mentioned on GitHubSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Atari GamesDeep Reinforcement LearningPredictionReinforcement LearningReinforcement Learning (RL)Value prediction

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Atari Games Atari 2600 Alien VPN Score 1429 #34 of 49 Archive leaderboard report
Atari Games Atari 2600 Amidar VPN Score 641 #30 of 48 Archive leaderboard report
Atari Games Atari 2600 Crazy Climber VPN Score 54119 #42 of 49 Archive leaderboard report
Atari Games Atari 2600 Enduro VPN Score 382 #36 of 48 Archive leaderboard report
Atari Games Atari 2600 Frostbite VPN Score 3811 #22 of 53 Archive leaderboard report
Atari Games Atari 2600 Krull VPN Score 15930 #9 of 45 Archive leaderboard report
Atari Games Atari 2600 Ms. Pacman VPN Score 2689 #30 of 47 Archive leaderboard report
Atari Games Atari 2600 Q*Bert VPN Score 14517 #29 of 57 Archive leaderboard report
Atari Games Atari 2600 Seaquest VPN Score 5628 #31 of 57 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections