Papers › Proximal Policy Optimization Algorithms

Proximal Policy Optimization Algorithms

20 Jul 2017arXiv:1707.06347archive 2025-07-28

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov

We propose a new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective function using stochastic gradient ascent. Whereas standard policy gradient methods perform one gradient update per data sample, we propose a novel objective function that enables multiple epochs of minibatch updates. The new methods, which we call proximal policy optimization (PPO), have some of the benefits of trust region policy optimization (TRPO), but they are much simpler to implement, more general, and have better sample complexity (empirically). Our experiments test PPO on a collection of benchmark tasks, including simulated robotic locomotion and Atari game playing, and we show that PPO outperforms other online policy gradient methods, and overall strikes a favorable balance between sample complexity, simplicity, and wall-time.

PaperPDFCodeCode Syntology ranMethod as code

In Syntology View this paper on Syntology, its page in Syntology's graph. That page adds which of its functions are also present in other papers' code Syntology holds, with dates, and its dated run record as JSON and a README badge. It also lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

Syntology 176 code samples: 173 from 64 repositories linked to this paper, 3 identical to code harvested from another paper's repository. Across all 176: 99 ran (of which 53 constructed an object rather than computing a result; 71 with no instrument failure: 7 honoured, 2 violated, 62 with no contract checked; 28 where Syntology's instrument failed) · 77 unverified. Where Syntology's instrument failed: 27 where our contract draft was wrong; 1 where the contract check could not run; failures of Syntology's instrument, not of the code.

By repository: community (archive-listed): 173 samples from 64 repositories, 96 ran (69 with no instrument failure; 27 where Syntology's instrument failed); 3 identical to code first harvested elsewhere. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

No sample is linked to this paper from 26 of its repositories that Syntology harvested for other papers:

Levels written between 2026-09-22 and 2026-09-24 (UTC): the dates Syntology's graph wrote each sample's verification level, not when the code executed. 2 of the 99 samples that ran have no recorded date.

188 repositories listed; official and paper-mentioned ones first.

bitbucket.org/act-lab/release mentioned on GitHubSyntology: not harvested report
170928/-Review-Generative-Adversarial-Imitation-Learning mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
170928/-Review-Proximal-Policy-Optimization-Algorithms mentioned on GitHubtfSyntology: not harvested report
2mawi2/master-thesis-experiments mentioned on GitHubtfSyntology: not harvested report
ASzot/ppo-pytorch mentioned on GitHubpytorchSyntology: 3 samples, 3 ran (1 instrument-failed) · 0 unverified report
Aravind-11/AI-Gaming mentioned on GitHubtfSyntology: not harvested report
Aravind-11/IITM_Saastra mentioned on GitHubSyntology: not harvested report
Aravind-11/Multi-Agent-RL mentioned on GitHubpytorchSyntology: not harvested report
BerkeleyLearnVerify/VerifAI mentioned on GitHubtfSyntology: not harvested report
BrianPulfer/PapersReimplementations mentioned on GitHubpytorchSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
CSautier/Breakout mentioned on GitHubpytorchSyntology: 2 samples, 0 ran · 2 unverified report
CSautier/PongBot mentioned on GitHubtfSyntology: 1 sample, 1 ran (1 instrument-failed) · 0 unverified report
Crevass/Hybrid-Agent mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Continuous control with deep reinforcement learning (arXiv:1509.02971; that paper's own run record: 163 ran (of which 126 constructed an object rather than computing a result; 152 with no instrument failure: 3 honoured, 0 violated, 149 with no contract checked; 11 where Syntology's instrument failed) · 143 unverified); nothing checks that they implement this paper's method. report
DMIU-ShELL/MOSAIC mentioned on GitHubpytorchSyntology: 2 samples, 0 ran · 2 unverified report
DavidCastilloAlvarado/Path-planning-and-Reinforcement-Learning mentioned on GitHubtfSyntology: not harvested report
EconomistGrant/HTFE-tensortrade mentioned on GitHubtfSyntology: not harvested report
FMArduini/python-rl mentioned on GitHubtfSyntology: not harvested report
GiadaSimionato/Reasoning_Agents_2020 mentioned on GitHubpytorchSyntology: 7 samples, 4 ran (1 instrument-failed) · 3 unverified report
Gouet/Acrobot-PPO mentioned on GitHubtfSyntology: 2 samples, 1 ran (1 instrument-failed) · 1 unverified report
Gouet/Breakout-V0 mentioned on GitHubtfSyntology: not harvested report
Gouet/PPO-gym mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
Gouet/PPO-pytorch mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
Gregory-Eales/proximal-policy-optimization mentioned on GitHubpytorchSyntology: 4 samples, 3 ran (0 instrument-failed) · 1 unverified report
InSpaceAI/RL-Zoo mentioned on GitHubtfSyntology: 4 samples, 1 ran (0 instrument-failed) · 3 unverified report
JL321/Proximal-Policy-Optimization mentioned on GitHubtfSyntology: 3 samples, 0 ran · 3 unverified report
JonasRSV/PPO mentioned on GitHubtfSyntology: 3 samples, 2 ran (0 instrument-failed) · 1 unverified report
Khrylx/PyTorch-RL mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
LuEE-C/PPO-Keras mentioned on GitHubSyntology: not harvested report
MatteoBrentegani/PPO mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
Michaelrising/Prog-RL mentioned on GitHubpytorchSyntology: not harvested report
MrSyee/pg-is-all-you-need mentioned on GitHubMITSyntology: not harvested report
NACLab/robust-active-inference mentioned on GitHubjaxSyntology: not harvested report
Narsil/rl-baselines mentioned on GitHubpytorchSyntology: 4 samples, 3 ran (0 instrument-failed) · 1 unverified report
NervanaSystems/coach mentioned on GitHubtfSyntology: not harvested report
Nordeus/heroic-rl mentioned on GitHubtfSyntology: not harvested report
OctThe16th/PPO-Keras mentioned on GitHubSyntology: not harvested report
Ostyk/walk-bot mentioned on GitHubpytorchSyntology: 2 samples, 1 ran (1 instrument-failed) · 1 unverified report
Rafael1s/Deep-Reinforcement-Learning-Udacity mentioned on GitHubpytorchSyntology: 2 samples, 1 ran (1 instrument-failed) · 1 unverified report
SPark9625/PyTorch-Proximal-Policy-Optimization mentioned on GitHubpytorchSyntology: not harvested report
SalvatoreCognetta/reasoning-agent-project mentioned on GitHubpytorchSyntology: not harvested report
Stippler/cow-simulator mentioned on GitHubpytorchSyntology: not harvested report
UesugiErii/tf2-PPO-atari mentioned on GitHubtfSyntology: not harvested report
Zartris/TD3_continuous_control mentioned on GitHubpytorchSyntology: not harvested report
adamos581/ppo-keras-football mentioned on GitHubSyntology: not harvested report
adik993/ppo-pytorch mentioned on GitHubpytorchSyntology: not harvested report
ailab-pku/rl-framework mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Continuous control with deep reinforcement learning (arXiv:1509.02971; that paper's own run record: 163 ran (of which 126 constructed an object rather than computing a result; 152 with no instrument failure: 3 honoured, 0 violated, 149 with no contract checked; 11 where Syntology's instrument failed) · 143 unverified); nothing checks that they implement this paper's method. report
alex-petrenko/sample-factory mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous... (arXiv:2006.11751; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
alexbaumi/PPO-Algorithm mentioned on GitHubpytorchSyntology: 2 samples, 0 ran · 2 unverified report
alexbaumi/PPO-Algorithms mentioned on GitHubpytorchSyntology: not harvested report
alexmlamb/blocks_rl_gru_setup mentioned on GitHubpytorchSyntology: not harvested report
alirezakazemipour/ppo-rnd mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Exploration by Random Network Distillation (arXiv:1810.12894; that paper's own run record: 31 ran (of which 12 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 7 where Syntology's instrument failed) · 12 unverified); nothing checks that they implement this paper's method. report
amanda-lambda/drl-experiments mentioned on GitHubpytorchSyntology: not harvested report
amanda-lambda/hack-flappy-bird-drl mentioned on GitHubpytorchSyntology: not harvested report
amartyamukherjee/ppo-packcooling mentioned on GitHubpytorchSyntology: not harvested report
amaudruz/RL_openaigym mentioned on GitHubpytorchSyntology: not harvested report
amzoyang/CS-221-Final-Project mentioned on GitHubpytorchSyntology: not harvested report
andyljones/zonotable mentioned on GitHubSyntology: not harvested report
anthonysong98/super-mario-bros-ppo mentioned on GitHubpytorchSyntology: not harvested report
araffin/sbx mentioned on GitHubjaxSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning (arXiv:2410.09754; that paper's own run record: 6 ran (of which 2 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
automl/learna mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Learning to Design RNA (arXiv:1812.11951; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
bay3s/ppo-parallel mentioned on GitHubpytorchSyntology: not harvested report
bay3s/ppo-rl mentioned on GitHubpytorchSyntology: 3 samples, 2 ran (0 instrument-failed) · 1 unverified report
benevolentAI/guacamol_baselines mentioned on GitHubpytorchMITSyntology: not harvested report
benpetit/cs379c mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
bentrevett/pytorch-rl mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Asynchronous Methods for Deep Reinforcement Learning (arXiv:1602.01783; that paper's own run record: 60 ran (of which 20 constructed an object rather than computing a result; 50 with no instrument failure: 2 honoured, 1 violated, 47 with no contract checked; 10 where Syntology's instrument failed) · 35 unverified); nothing checks that they implement this paper's method. report
bonniesjli/PPO-Reacher_UnityML mentioned on GitHubpytorchSyntology: 4 samples, 4 ran (0 instrument-failed) · 0 unverified report
bonniesjli/PPO_Reacher mentioned on GitHubpytorchSyntology: 3 samples, 3 ran (0 instrument-failed) · 0 unverified report
chainer/chainerrl mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ChainerRL: A Deep Reinforcement Learning Library (arXiv:1912.03905; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
cipher982/ppo-exploration mentioned on GitHubpytorchSyntology: not harvested report
clwainwright/proximal_policy_optimization mentioned on GitHubtfSyntology: not harvested report
danelee2601/rl-based-automatic-berthing mentioned on GitHubSyntology: not harvested report
decoderkurt/research_project_school_of_ai_2019 mentioned on GitHubtfSyntology: not harvested report
deconlabs/Binanace-trading-simulation mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Rainbow: Combining Improvements in Deep Reinforcement Learning (arXiv:1710.02298; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
deconlabs/Binanace_trading_simulation mentioned on GitHubpytorchMITSyntology: not harvested report
deconlabs/TradingZoo-Dynamic-fee-simulation mentioned on GitHubpytorchSyntology: 2 samples, 1 ran (1 instrument-failed) · 1 unverified report
dickreuter/neuron_poker mentioned on GitHubSyntology: not harvested report
dmiu-shell/deeprl-shell mentioned on GitHubpytorchSyntology: 4 samples, 4 ran (1 instrument-failed) · 0 unverified report
donamin/llc mentioned on GitHubtfSyntology: not harvested report
downingbots/RLDonkeycar mentioned on GitHubSyntology: 1 sample, 0 ran · 1 unverified report
eladsar/rbi mentioned on GitHubpytorchSyntology: not harvested report
emerge-lab/nocturne_lab mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games (arXiv:2103.01955; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
evieq01/oodil mentioned on GitHubpytorchSyntology: 7 samples, 5 ran (2 instrument-failed) · 2 unverified report
facebookresearch/Horizon mentioned on GitHubpytorchSyntology: not harvested report
facebookresearch/ReAgent mentioned on GitHubpytorchSyntology: not harvested report
fdcl-gwu/gym-rotor mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Critics (arXiv:1910.01465; that paper's own run record: 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified); nothing checks that they implement this paper's method. report
gaetanserre/l2rpn-2022_ppo-baseline mentioned on GitHubSyntology: not harvested report
georgkruse/cleanqrl mentioned on GitHubpytorchSyntology: not harvested report
gmoss1/Kaggle-Halite-IV-RL mentioned on GitHubSyntology: not harvested report
goncharom/PPOv1 mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
gstoica27/cpg_ppo mentioned on GitHubtfSyntology: not harvested report
guofeng201507/ISA-IPA-2020-05-05-IS1PT-GRP-High5-StockTrading mentioned on GitHubSyntology: not harvested report
gwthomas/gtml mentioned on GitHubtfSyntology: not harvested report
hamishs/JAX-RL mentioned on GitHubjaxSyntology: 5 samples, 3 ran (1 instrument-failed) · 2 unverified report
harruff/Senior_Project_Repository mentioned on GitHubtfSyntology: not harvested report
hdparks/AsteroidsDeepReinforcement mentioned on GitHubtorchSyntology: not harvested report
hmhuy0/SIM-RL mentioned on GitHubpytorchSyntology: 12 samples, 10 ran (4 instrument-failed) · 2 unverified report
ifestus/rl mentioned on GitHubtfSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
ikostrikov/pytorch-a2c-ppo-acktr-gail mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Distributionally Adaptive Meta Reinforcement Learning (arXiv:2210.03104; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
ikostrikov/pytorch-rl mentioned on GitHubpytorchSyntology: not harvested report
inoryy/reaver mentioned on GitHubtfMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Relational Deep Reinforcement Learning (arXiv:1806.01830; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified); nothing checks that they implement this paper's method. report
intellisys-lab/stellaris-sc24 mentioned on GitHubSyntology: not harvested report
jcwleo/curiosity-driven-exploration-pytorch mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Curiosity-driven Exploration by Self-supervised Prediction (arXiv:1705.05363; that paper's own run record: 16 ran (of which 9 constructed an object rather than computing a result; 13 with no instrument failure: 1 honoured, 1 violated, 11 with no contract checked; 3 where Syntology's instrument failed) · 16 unverified); nothing checks that they implement this paper's method. report
jcwleo/random-network-distillation-pytorch mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Exploration by Random Network Distillation (arXiv:1810.12894; that paper's own run record: 31 ran (of which 12 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 7 where Syntology's instrument failed) · 12 unverified); nothing checks that they implement this paper's method. report
jfpettit/flare mentioned on GitHubpytorchSyntology: not harvested report
jhare96/reinforcement-learning mentioned on GitHubtfSyntology: 3 samples, 0 ran · 3 unverified report
jongornet14/HyperController mentioned on GitHubpytorchSyntology: not harvested report
jsztompka/MultiAgent-PPO mentioned on GitHubpytorchSyntology: not harvested report
jsztompka/PPO-demo mentioned on GitHubpytorchSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
juliusfrost/Research-Paper-Implementations mentioned on GitHubSyntology: not harvested report
jw1401/PPO-Tensorflow-2.0 mentioned on GitHubtfSyntology: not harvested report
ku2482/gail-airl-ppo.pytorch mentioned on GitHubpytorchSyntology: 8 samples, 6 ran (3 instrument-failed) · 2 unverified report
lcswillems/torch-ac mentioned on GitHubpytorchSyntology: 4 samples, 1 ran (0 instrument-failed) · 3 unverified report
lgerrets/rl18-curiosity mentioned on GitHubSyntology: not harvested report
liuyuezhang/pyrl mentioned on GitHubpytorchSyntology: 3 samples, 1 ran (0 instrument-failed) · 2 unverified report
liyiyuian/Deep-Learning mentioned on GitHubSyntology: not harvested report
llSourcell/OpenAI_Five_vs_Dota2_Explained mentioned on GitHubpytorchSyntology: 2 samples, 1 ran (0 instrument-failed) · 1 unverified report
mark-gluzman/NmodelPPO mentioned on GitHubSyntology: 3 samples, 3 ran (1 instrument-failed) · 0 unverified report
marload/DeepRL-TensorFlow2 mentioned on GitHubtfSyntology: 3 samples, 0 ran · 3 unverified report
michael-snower/ppo mentioned on GitHubtfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Accelerated Methods for Deep Reinforcement Learning (arXiv:1803.02811; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
microsoft/strategically_efficient_rl mentioned on GitHubtfMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Exploration by Random Network Distillation (arXiv:1810.12894; that paper's own run record: 31 ran (of which 12 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 7 where Syntology's instrument failed) · 12 unverified); nothing checks that they implement this paper's method. report
mightypirate1/PPO_homebrew mentioned on GitHubtfSyntology: not harvested report
miroblog/tf_deep_rl_trader mentioned on GitHubtfSyntology: not harvested report
mit-realm/neuriss mentioned on GitHubpytorchSyntology: 12 samples, 8 ran (5 instrument-failed) · 4 unverified report
montaserFath/Reinforcement-Learning-for-Prosthetics mentioned on GitHubSyntology: not harvested report
morikatron/GAIL_PPO mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
morikatron/PPO mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
near32/regym mentioned on GitHubpytorchSyntology: 3 samples, 3 ran (1 instrument-failed) · 0 unverified report
nikhilbarhate99/PPO mentioned on GitHubpytorchSyntology: not harvested report
nikhilbarhate99/PPO-PyTorch mentioned on GitHubpytorchSyntology: 3 samples, 3 ran (0 instrument-failed) · 0 unverified report
nitthilan/pommerman mentioned on GitHubSyntology: not harvested report
nric/ProximalPolicyOptimizationContinuousKeras mentioned on GitHubtfSyntology: 2 samples, 1 ran (0 instrument-failed) · 1 unverified report
nvlabs/gbrl_sb3 mentioned on GitHubpytorchNOASSERTIONSyntology: not harvested report
openpsi-projects/srl mentioned on GitHubpytorchSyntology: not harvested report
reinforcement-learning-kr/pg_travel mentioned on GitHubpytorchSyntology: 1 sample, 0 ran · 1 unverified report
rshnn/battleship mentioned on GitHubSyntology: not harvested report
s-sd/task-amenability mentioned on GitHubtfSyntology: not harvested report
saschaschramm/Pong mentioned on GitHubtfSyntology: 2 samples, 0 ran · 2 unverified report
sc2crazy/StarCrackRL mentioned on GitHubtfSyntology: not harvested report
seungjaeryanlee/osim-rl-helper mentioned on GitHubSyntology: not harvested report
shiningsunnyday/mcts-chess mentioned on GitHubpytorchSyntology: not harvested report
shreyesss/PPO-implementation-keras-tensorflow mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
shuishida/soaprl mentioned on GitHubpytorchSyntology: not harvested report
siddharthverma314/proximalpolicyoptimization mentioned on GitHubpytorchSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
silvialuu/DRL-2018 mentioned on GitHubpytorchSyntology: 3 samples, 2 ran (1 instrument-failed) · 1 unverified report
sirakik/mprg_fc mentioned on GitHubpytorchSyntology: not harvested report
sirakik/ppo_football mentioned on GitHubpytorchSyntology: not harvested report
synthlabsai/big-math mentioned on GitHubMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in... (arXiv:2502.17387; that paper's own run record: 14 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
taku-y/20181125-pybullet mentioned on GitHubtfSyntology: not harvested report
takuseno/ppo mentioned on GitHubtfSyntology: not harvested report
tatsu-lab/linguistic_calibration mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (arXiv:2305.14251; that paper's own run record: 11 ran (of which 1 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 3 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified); nothing checks that they implement this paper's method. report
tcmxx/CNTKUnityTools mentioned on GitHubSyntology: not harvested report
tensorlayer/RLzoo mentioned on GitHubtfSyntology: 1 sample, 0 ran · 1 unverified report
theresearchai/vehicle_routing_rl_2 mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Attention, Learn to Solve Routing Problems! (arXiv:1803.08475; that paper's own run record: 27 ran (of which 0 constructed an object rather than computing a result; 16 with no instrument failure: 3 honoured, 0 violated, 13 with no contract checked; 11 where Syntology's instrument failed) · 13 unverified); nothing checks that they implement this paper's method. report
tidedra/vl-rlhf mentioned on GitHubpytorchApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ACL Anthology id 2025.naacl-long.447 (that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified); nothing checks that they implement this paper's method. report
tilkb/thermoai mentioned on GitHubtfSyntology: 2 samples, 2 ran (1 instrument-failed) · 0 unverified report
tmjeong1103/RL_with_RAY mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a... (arXiv:1801.01290; that paper's own run record: 91 ran (of which 61 constructed an object rather than computing a result; 73 with no instrument failure: 3 honoured, 1 violated, 69 with no contract checked; 18 where Syntology's instrument failed) · 57 unverified); nothing checks that they implement this paper's method. report
tobiasemrich/SchafkopfRL mentioned on GitHubpytorchSyntology: not harvested report
tommyvsfu1/RL-NTU mentioned on GitHubpytorchSyntology: 2 samples, 0 ran · 2 unverified report
tuanpnm99/RLPongAgent mentioned on GitHubpytorchSyntology: 2 samples, 1 ran (0 instrument-failed) · 1 unverified report
uvipen/super-mario-bros-ppo-pytorch mentioned on GitHubpytorchSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
vcadillog/PPO-Mario-Bros-Tensorflow-2 mentioned on GitHubtfSyntology: not harvested report
vermashresth/damage-aware-PPO mentioned on GitHubtfSyntology: not harvested report
vheuthe/microbot_rl mentioned on GitHubSyntology: not harvested report
wangshub/RL-Stock mentioned on GitHubSyntology: not harvested report
wangzhengfei0730/NIPS2018-AIforProsthetics mentioned on GitHubtfSyntology: 4 samples, 1 ran (0 instrument-failed) · 3 unverified report
xiawenwen49/ppo mentioned on GitHubtfSyntology: not harvested report
xtma/simple-pytorch-rl mentioned on GitHubpytorchSyntology: 3 samples, 2 ran (0 instrument-failed) · 1 unverified report
yoavalon/Quadcopter-env mentioned on GitHubtfSyntology: 2 samples, 0 ran · 2 unverified report
yoavalon/QuadcopterReinforcementLearning mentioned on GitHubtfSyntology: not harvested report
zjlab-ammi/llm4rl mentioned on GitHubpytorchSyntology: 2 samples, 1 ran (0 instrument-failed) · 1 unverified report
zsz-hst/RL_single_chase_point mentioned on GitHubSyntology: not harvested report
BIT-aerial-robotics/AquaML tfSyntology: not harvested report
DLR-RM/stable-baselines3 pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Hyperparameter Trajectory Inference with Conditional Lagrangian Optimal Transport (arXiv:2603.01771; that paper's own run record: 6 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
DevSlem/AINE-DRL pytorchSyntology: not harvested report
Ipsedo/EvoMotion pytorchSyntology: not harvested report
MaximeVandegar/Papers-in-100-Lines-of-Code pytorchSyntology: 1 sample, 0 ran · 1 unverified report
OctopusMind/RLHF_PPO pytorchSyntology: 3 samples, 1 ran (0 instrument-failed) · 2 unverified report
arnomoonens/yarll tfSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
dyabel/handson_rl pytorchSyntology: 1 sample, 1 ran (0 instrument-failed) · 0 unverified report
hill-a/stable-baselines tfSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Deep Reinforcement Learning with Double Q-learning (arXiv:1509.06461; that paper's own run record: 56 ran (of which 38 constructed an object rather than computing a result; 55 with no instrument failure: 0 honoured, 0 violated, 55 with no contract checked; 1 where Syntology's instrument failed) · 50 unverified) and Prioritized Experience Replay (arXiv:1511.05952; that paper's own run record: 80 ran (of which 62 constructed an object rather than computing a result; 72 with no instrument failure: 4 honoured, 0 violated, 68 with no contract checked; 8 where Syntology's instrument failed) · 31 unverified); nothing checks that they implement this paper's method. report
labmlai/annotated_deep_learning_paper_implementations pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Playing Atari with Deep Reinforcement Learning (arXiv:1312.5602; that paper's own run record: 64 ran (of which 24 constructed an object rather than computing a result; 46 with no instrument failure: 5 honoured, 0 violated, 41 with no contract checked; 18 where Syntology's instrument failed) · 53 unverified), Adam: A Method for Stochastic Optimization (arXiv:1412.6980; that paper's own run record: 27 ran (of which 7 constructed an object rather than computing a result; 23 with no instrument failure: 4 honoured, 2 violated, 17 with no contract checked; 4 where Syntology's instrument failed) · 10 unverified), Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (arXiv:1502.03167; that paper's own run record: 19 ran (of which 5 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified) and 12 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
pytorch/rl jaxSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Decision Transformer: Reinforcement Learning via Sequence Modeling (arXiv:2106.01345; that paper's own run record: 17 ran (of which 10 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 5 where Syntology's instrument failed) · 9 unverified); nothing checks that they implement this paper's method. report
ray-project/ray Syntology: not harvested report
toni-sm/skrl jaxMITSyntology: not harvested report
xiuyu0000/new_papers_codes mindsporeSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

176 samples harvested: 99 ran (of which 53 constructed an object rather than computing a result; 71 with no instrument failure: 7 honoured, 2 violated, 62 with no contract checked; 28 where Syntology's instrument failed) · 77 unverified. “Where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Syntology's record for this page has not changed since , the first build that kept a record date for it; when this build read Syntology's graph is in the build record. That date is not when the samples ran.

7ran · honoured contract
2ran · violated contract
27ran · our draft was wrong
1ran; the contract check could not run
62ran
77unverified

Licence: 94 of the 176 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 64 repositories linked to this paper, official or community; the list below is grouped by repository, each group headed by its name and what it is to the paper. 1 sample whose link to this paper records no repository is listed under one of this paper's own repositories, because identical code was harvested from it for another paper. 3 samples are listed last, in a group without a repository. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. A line beginning “fixture:” under a sample that ran gives the arguments Syntology's fixture called the sample with, each as name:type/rank/element kind (? where none was recorded), and the type of what the fixture's run returned, exactly as the harness recorded them; the arguments and the returned type were recorded by two separate runs of the same fixture. No output value is shown, and a returned type says nothing about whether the output was right. “Level recorded” gives the UTC date when Syntology's graph wrote the sample's verification level, not when the code executed; where the level was written before Syntology's graph kept that date, the row says “level recorded, date not recorded”. For 13 samples that ran, the graph records a signature this page does not show: for a class the fixture built without calling, the harness called the object with inputs it made up itself, so the recorded signature is the harness's probe, not the fixture's arguments. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself. “Our draft was wrong” and “the contract check could not run” are failures of Syntology's instrument, not of the code: the sample ran on Syntology's synthesized input, and then the drafted contract did not describe it, or the check itself raised an error (RAISES), lacked a dependency (DEP_MISSING) or ran out of time (TIMEOUT); hover the label for which. A metamorphic tier, where shown, is evidence of the output's shape, not that the sample implements the method.

Each sample ends with its code_sha256. A sample's code_sha256 is Syntology's identity for that exact code. It is printed in full (16 hexadecimal characters for every sample in this build), and get_code(code_sha256="…") takes it exactly as printed. An agent calls it on Syntology's MCP service (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples are grouped by repository: a repository marked official first, then by how directly the repository is tied to the paper, then in the archive's order; never by how many ran. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of six reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar); this paper's copy was not located, so its licence is not recorded; or this paper's copy is cleared, but the stored copy of the same code that Syntology serves is not. Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

A function or class name links to its page when the name occurs in the harvested code of at least 5 papers; the name is a string, not an identity. A census tag appears when Syntology's census ran implementations with that name on one shared input per bucket: it gives how many were compared, from how many papers, and how many distinct outputs they produced, and names this sample's bucket when this sample was one of them.

170928/-Review-Generative-Adversarial-Imitation-Learning community (archive-listed) 1 sample, 0 ran · 1 unverified
PPOalgorithm PPO.py unverified no licence file found · pointer only · bce2a8566fa36fda · report
ASzot/ppo-pytorch community (archive-listed) 3 samples, 3 ran (1 instrument-failed) · 0 unverified
Mlp in 102 papers model.py ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · 088783306dda80fe · report fixture: signature inputs:Tensor/2/float · returned tuple · level recorded 2026-09-22
Policy in 23 papers model.py ran fingerprinted MIT (permissive) · be3a23c2176d855c · report fixture: signature inputs:Tensor/2/float, deterministic:bool/0/bool · returned tuple · level recorded 2026-09-24
init_layer in 6 papers model.py ran · our draft was wrong MIT (permissive) · 375f0073f16726f9 · report fixture: signature m:callable/0/? · returned Linear · level recorded 2026-09-22
BrianPulfer/PapersReimplementations community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
MyPPO src/rl/ppo/ppo.py ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · bb56e14071f60cba · report fixture: signature x:Tensor/3/float · returned tuple · level recorded 2026-09-22
CSautier/Breakout community (archive-listed) 2 samples, 0 ran · 2 unverified
PPO model.py unverified no licence file found · pointer only · 1c8d1f046ceeaadf · report
Parameters model.py unverified no licence file found · pointer only · 8764b4ca6ce10aca · report
CSautier/PongBot community (archive-listed) 1 sample, 1 ran (1 instrument-failed) · 0 unverified
proximal_policy_optimization_loss PongBot.py ran · our draft was wrong no licence file found · pointer only · 063d29364d7e4f81 · report fixture: signature advantage:EagerTensor/2/float, old_prediction:EagerTensor/2/float · returned function · level recorded 2026-09-22
DMIU-ShELL/MOSAIC community (archive-listed) 2 samples, 0 ran · 2 unverified
PPOAgent deep_rl/agent/PPO_agent.py unverified Apache-2.0 (permissive) · a01bf852d6ef518a · report
tensor in 8 papers deep_rl/agent/PPO_agent.py unverified Apache-2.0 (permissive) · 8e103b8cabd1203c · report
GiadaSimionato/Reasoning_Agents_2020 community (archive-listed) 7 samples, 4 ran (1 instrument-failed) · 3 unverified
DictList babyai_rb/babyai/rl/algos/ppo.py ran no licence file found · pointer only · 28919ed273c0c995 · report fixture: returned DictList · level recorded 2026-09-22
ExtraInfoCollector babyai_rb/babyai/rl/algos/ppo.py ran no licence file found · pointer only · 06c62bb9eacfe08c · report fixture: returned ExtraInfoCollector · level recorded 2026-09-22
ParallelEnv babyai_rb/babyai/rl/algos/ppo.py ran no licence file found · pointer only · f0b71d0c5374efdf · report fixture: returned ParallelEnv · level recorded 2026-09-22
default_preprocess_obss babyai_rb/babyai/rl/algos/ppo.py ran · our draft was wrong fingerprinted no licence file found · pointer only · 1cd2e751ff91ca44 · report fixture: signature obss:list/1/list, device:NoneType/0/? · returned Tensor · level recorded 2026-09-22
BaseAlgo babyai_rb/babyai/rl/algos/ppo.py unverified no licence file found · pointer only · 6c7082241ec61d89 · report
PPOAlgo babyai_rb/babyai/rl/algos/ppo.py unverified no licence file found · pointer only · dce583aa0f5fb136 · report
worker in 16 papers babyai_rb/babyai/rl/algos/ppo.py unverified no licence file found · pointer only · 01571ae37dfefba2 · report
Gouet/Acrobot-PPO community (archive-listed) 2 samples, 1 ran (1 instrument-failed) · 1 unverified
_pick_batch ppo.py ran · our draft was wrong fingerprinted no licence file found · pointer only · a5b8d69558cb1833 · report fixture: signature mini_batch_size:int/0/int, data:ndarray/2/float, batch_index:int/0/int, flat:bool/0/bool, shape:NoneType/0/? · returned ndarray · level recorded 2026-09-22
update in 31 papers ppo.py unverified no licence file found · pointer only · 7271dace2360dd93 · report
Gouet/PPO-gym community (archive-listed) 1 sample, 0 ran · 1 unverified
update in 31 papers ppo.py unverified no licence file found · pointer only · 0b170016a5f01a6e · report
Gouet/PPO-pytorch community (archive-listed) 1 sample, 0 ran · 1 unverified
update in 31 papers ppo.py unverified no licence file found · pointer only · 3e7bdf63d50d132e · report
Gregory-Eales/proximal-policy-optimization community (archive-listed) 4 samples, 3 ran (0 instrument-failed) · 1 unverified
Actor in 68 papers ppo/modules/ppo.py ran · metamorphic tier: deterministic Apache-2.0 (permissive) · c3cb4b39b7e82812 · report fixture: returned Actor · level recorded 2026-09-22
Buffer in 23 papers ppo/modules/ppo.py ran Apache-2.0 (permissive) · 5eac27c1ede5a3f1 · report fixture: returned Buffer · level recorded 2026-09-22
Critic in 60 papers ppo/modules/ppo.py ran · metamorphic tier: deterministic Apache-2.0 (permissive) · f1685211d6effd06 · report fixture: returned Critic · level recorded 2026-09-22
PPO ppo/modules/ppo.py unverified Apache-2.0 (permissive) · cb4776fdb4698c72 · report
InSpaceAI/RL-Zoo community (archive-listed) 4 samples, 1 ran (0 instrument-failed) · 3 unverified
gaussian_likelihood in 6 papers PPO.py ran · honoured contract no licence file found · pointer only · b46c632bedd3d8af · report fixture: signature x:EagerTensor/2/float, mu:EagerTensor/2/float, log_std:EagerTensor/2/float · returned EagerTensor · level recorded 2026-09-22
Actor in 68 papers PPO.py unverified no licence file found · pointer only · bf0c1da602f8a16d · report
Critic in 60 papers PPO.py unverified no licence file found · pointer only · 708e869097ea91ff · report
PPOAgent PPO.py unverified no licence file found · pointer only · 01b7328f523d7756 · report
JL321/Proximal-Policy-Optimization community (archive-listed) 3 samples, 0 ran · 3 unverified
PPO PPO.py unverified no licence file found · pointer only · 7451a2c222e554d4 · report
ffNetwork PPO.py unverified no licence file found · pointer only · 37c11b4e769d359d · report
valueNetwork PPO.py unverified no licence file found · pointer only · 887a93fefeaff1d2 · report
JonasRSV/PPO community (archive-listed) 3 samples, 2 ran (0 instrument-failed) · 1 unverified
Categorical in 6 papers ppo.py ran no licence file found · pointer only · cc0a6d60ba0d615d · report fixture: returned Categorical · level recorded 2026-09-22
Continous ppo.py ran no licence file found · pointer only · 1e06bd08fb825800 · report fixture: returned Continous · level recorded 2026-09-22
PPO ppo.py unverified no licence file found · pointer only · 578c10d4e2b66073 · report
Khrylx/PyTorch-RL community (archive-listed) 1 sample, 0 ran · 1 unverified
ppo_step core/ppo.py unverified MIT (permissive) · f423e91c2339c824 · report
MatteoBrentegani/PPO community (archive-listed) 1 sample, 0 ran · 1 unverified
proximal_policy_optimization_loss PPO_CentralizedNN/ppo_lossFunctionC.py unverified no licence file found · pointer only · 2492441b222787d6 · report
Narsil/rl-baselines community (archive-listed) 4 samples, 3 ran (0 instrument-failed) · 1 unverified
ModelUpdate rl_baselines/ppo.py ran · metamorphic tier: deterministic no licence file found · pointer only · 7f143a6ee276436b · report fixture: returned ModelUpdate · level recorded 2026-09-22
PPO rl_baselines/ppo.py ran · metamorphic tier: deterministic no licence file found · pointer only · b48c75c1f791937c · report fixture: returned PPO · level recorded 2026-09-22
PolicyUpdate rl_baselines/ppo.py ran · metamorphic tier: deterministic no licence file found · pointer only · 2eb0c930a45a7472 · report fixture: returned PolicyUpdate · level recorded 2026-09-22
ppo_loss rl_baselines/ppo.py unverified no licence file found · pointer only · 2c9a98eedcdea762 · report
Ostyk/walk-bot community (archive-listed) 2 samples, 1 ran (1 instrument-failed) · 1 unverified
ppo_iter v_pytorch/ppo_train.py ran · our draft was wrong no licence file found · pointer only · df1fd605b67498ef · report fixture: signature states:Tensor/2/float, actions:Tensor/2/float, log_probs:Tensor/2/float, returns:Tensor/2/float, advantage:Tensor/2/float · returned generator · level recorded 2026-09-22
ppo_update v_pytorch/ppo_train.py unverified no licence file found · pointer only · f20ab80cf000ba69 · report
Rafael1s/Deep-Reinforcement-Learning-Udacity community (archive-listed) 2 samples, 1 ran (1 instrument-failed) · 1 unverified
states_to_prob Pong-Policy-Gradient-PPO/pong_utils.py ran · our draft was wrong no licence file found · pointer only · f8348c78f9a4c594 · report fixture: signature policy:callable/0/?, states:list/1/Tensor · returned Tensor · level recorded 2026-09-22
clipped_surrogate Pong-Policy-Gradient-PPO/pong_utils.py unverified no licence file found · pointer only · db1820f47853260b · report
alexbaumi/PPO-Algorithm community (archive-listed) 2 samples, 0 ran · 2 unverified
Agent in 42 papers Agent.py unverified no licence file found · pointer only · d196dcee4a5b1061 · report
init_weights in 109 papers Agent.py unverified no licence file found · pointer only · c2f17acc73ce8ca9 · report
bay3s/ppo-rl community (archive-listed) 3 samples, 2 ran (0 instrument-failed) · 1 unverified
ActorCritic in 10 papers src/ppo/ppo.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 6ff581679958bf77 · report fixture: signature state:Tensor/2/float, action:Tensor/1/int · returned tuple · level recorded 2026-09-22
Buffer in 23 papers src/ppo/ppo.py ran · metamorphic tier: deterministic no licence file found · pointer only · d0735a6f70135e3f · report fixture: signature transition:Transition/1/Tensor · returned Transition · level recorded 2026-09-22
PPO src/ppo/ppo.py unverified no licence file found · pointer only · ef96810753b59244 · report
benpetit/cs379c community (archive-listed) 1 sample, 0 ran · 1 unverified
build_train build_graph.py unverified MIT (permissive) · 2e9e6b5586603bfb · report
bonniesjli/PPO-Reacher_UnityML community (archive-listed) 4 samples, 4 ran (0 instrument-failed) · 0 unverified
Actor in 68 papers agent.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · e9f1207fc08f9fa9 · report fixture: returned Actor · level recorded 2026-09-22
ActorCritic in 10 papers agent.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · f7a59411ac390f78 · report fixture: returned ActorCritic · level recorded 2026-09-22
Agent in 42 papers agent.py ran no licence file found · pointer only · b1f30e180cb81a3a · report fixture: returned Agent · level recorded 2026-09-22
Critic in 60 papers agent.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · b4455ab1f545ae9c · report fixture: returned Critic · level recorded 2026-09-22
bonniesjli/PPO_Reacher community (archive-listed) 3 samples, 3 ran (0 instrument-failed) · 0 unverified
Actor in 68 papers agent.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 8fc0ce07edd32744 · report fixture: returned Actor · level recorded 2026-09-22
ActorCritic in 10 papers agent.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 607bb439a428fd2a · report fixture: returned ActorCritic · level recorded 2026-09-22
Agent in 42 papers agent.py ran no licence file found · pointer only · 4c40cb5849fd342f · report fixture: returned Agent · level recorded 2026-09-22
deconlabs/TradingZoo-Dynamic-fee-simulation community (archive-listed) 2 samples, 1 ran (1 instrument-failed) · 1 unverified
states_to_prob agent/PPO/utils.py ran · our draft was wrong MIT (permissive) · a7700bf49f384087 · report fixture: signature model:callable/0/?, states:list/1/Tensor · returned Tensor · level recorded 2026-09-22
clipped_surrogate agent/PPO/utils.py unverified MIT (permissive) · 9c65b36eec7ec92a · report
dmiu-shell/deeprl-shell community (archive-listed) 4 samples, 4 ran (1 instrument-failed) · 0 unverified
BaseAgent in 20 papers deep_rl/agent/PPO_agent.py ran Apache-2.0 (permissive) · 38803d98e366da6d · report fixture: returned BaseAgent · level recorded 2026-09-22
Batcher deep_rl/agent/PPO_agent.py ran Apache-2.0 (permissive) · 6dbcf603fe9c4a6f · report fixture: returned Batcher · level recorded 2026-09-22
PPOAgent deep_rl/agent/PPO_agent.py ran Apache-2.0 (permissive) · e3bd893b00cce00e · report fixture: returned PPOAgent · level recorded 2026-09-22
tensor in 8 papers deep_rl/agent/PPO_agent.py ran; the contract check could not run (RAISES) fingerprinted Apache-2.0 (permissive) · 03b48449f3d4d675 · report fixture: signature x:Tensor/2/float · returned Tensor · level recorded 2026-09-22
downingbots/RLDonkeycar community (archive-listed) 1 sample, 0 ran · 1 unverified
proximal_policy_optimization_loss_continuous donkeycar/donkeycar/parts/RLKeras.py unverified MIT (permissive) · a690a9a71794bce7 · report
evieq01/oodil community (archive-listed) 7 samples, 5 ran (2 instrument-failed) · 2 unverified
RolloutBuffer simulated_robot/gail_airl_ppo/algo/ppo.py ran MIT (permissive) · c169652ebc3cb1c4 · report fixture: returned RolloutBuffer · level recorded 2026-09-22
StateFunction simulated_robot/gail_airl_ppo/algo/ppo.py ran fingerprinted MIT (permissive) · ab47bdba4f875268 · report fixture: returned StateFunction · level recorded 2026-09-22
StateIndependentPolicy simulated_robot/gail_airl_ppo/algo/ppo.py ran fingerprinted MIT (permissive) · 181b4202198be200 · report fixture: returned StateIndependentPolicy · level recorded 2026-09-22
build_mlp in 74 papers simulated_robot/gail_airl_ppo/algo/ppo.py ran · our draft was wrong MIT (permissive) · 373bf1b3269456e9 · report fixture: signature input_dim:int/0/int, output_dim:int/0/int, hidden_units:list/1/int, hidden_activation:callable/0/?, output_activation:callable/0/?, dropout:float/0/float · returned Sequential · level recorded 2026-09-22
evaluate_lop_pi_inf simulated_robot/gail_airl_ppo/algo/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · 92227b56b34a423b · report fixture: signature means:Tensor/2/float, log_stds:Tensor/2/float, actions:Tensor/2/float · returned Tensor · level recorded 2026-09-22
Algorithm in 26 papers simulated_robot/gail_airl_ppo/algo/ppo.py unverified MIT (permissive) · 527acc6b195e3ec9 · report
PPO simulated_robot/gail_airl_ppo/algo/ppo.py unverified MIT (permissive) · 86201a6180de4a09 · report
goncharom/PPOv1 community (archive-listed) 1 sample, 0 ran · 1 unverified
PPO ppo.py unverified no licence file found · pointer only · ea96173e68302d60 · report
hamishs/JAX-RL community (archive-listed) 5 samples, 3 ran (1 instrument-failed) · 2 unverified
PPO src/jax_rl/algorithms/ppo.py ran MIT (permissive) · 6e26f9d72a957989 · report fixture: returned PPO · level recorded 2026-09-22
TrajectoryBuffer src/jax_rl/algorithms/ppo.py ran MIT (permissive) · 58ee12a76b1fd1a1 · report fixture: returned TrajectoryBuffer · level recorded 2026-09-22
entropy in 112 papers
census: 22 implementations from 25 papers, 16 distinct outputs across 4 buckets, one shared input per bucket
src/jax_rl/algorithms/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · 1732d17d766f28da · report fixture: signature dist:ArrayImpl/2/float, axis:int/0/int · returned ArrayImpl · level recorded 2026-09-22
BaseAgent in 20 papers src/jax_rl/algorithms/ppo.py unverified MIT (permissive) · 2dd687e050a61476 · report
Buffer in 23 papers src/jax_rl/algorithms/ppo.py unverified MIT (permissive) · 6f2b092fd868a348 · report
hmhuy0/SIM-RL community (archive-listed) 12 samples, 10 ran (4 instrument-failed) · 2 unverified
RolloutBuffer_PPO_lag Sources/algo/ppo.py ran no licence file found · pointer only · 6f986aa2d59a44f8 · report fixture: returned RolloutBuffer_PPO_lag · level recorded 2026-09-22
StateFunction Sources/algo/ppo.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · a78ee4761e8df9d8 · report fixture: returned StateFunction · level recorded 2026-09-22
StateIndependentPolicy Sources/algo/ppo.py ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · fbd28096c7f0c30f · report fixture: returned StateIndependentPolicy · level recorded 2026-09-22
atanh in 10 papers
census: 3 implementations from 5 papers, 2 distinct outputs on one shared input this sample: bucket 1 of 1, arg 1: rank 2, kind float, dtype float32
Sources/algo/ppo.py ran · honoured contract no licence file found · pointer only · 264c9e17d7dfe7eb · report fixture: returned Tensor · level recorded 2026-09-22
build_mlp in 74 papers Sources/algo/ppo.py ran · our draft was wrong no licence file found · pointer only · c511268a73d82fc6 · report fixture: returned Sequential · level recorded 2026-09-22
calculate_gae Sources/algo/ppo.py ran · our draft was wrong fingerprinted no licence file found · pointer only · b5a22b5e0dce3796 · report fixture: signature values:Tensor/1/float, rewards:Tensor/1/float, dones:Tensor/1/float, next_values:Tensor/1/float, gamma:float/0/float, lambd:float/0/float · returned tuple · level recorded 2026-09-22
calculate_gae_cost Sources/algo/ppo.py ran · our draft was wrong fingerprinted no licence file found · pointer only · 77393baa1a11c24b · report fixture: signature values:Tensor/1/float, rewards:Tensor/1/float, dones:Tensor/1/float, next_values:Tensor/1/float, gamma:float/0/float, lambd:float/0/float · returned tuple · level recorded 2026-09-22
calculate_log_pi Sources/algo/ppo.py ran · honoured contract fingerprinted no licence file found · pointer only · 476e14ee8b7219ac · report fixture: signature log_stds:Tensor/2/float, noises:Tensor/2/float, actions:Tensor/2/float · returned Tensor · level recorded 2026-09-22
evaluate_lop_pi Sources/algo/ppo.py ran · honoured contract fingerprinted no licence file found · pointer only · fef4ec37a6ea52e7 · report fixture: signature means:Tensor/2/float, log_stds:Tensor/2/float, actions:Tensor/2/float · returned Tensor · level recorded 2026-09-22
reparameterize in 38 papers
census: 12 implementations from 17 papers, 5 distinct outputs across 4 buckets, one shared input per bucket
Sources/algo/ppo.py ran · our draft was wrong fingerprinted no licence file found · pointer only · b1c9c127240c75d8 · report fixture: signature means:Tensor/2/float, log_stds:Tensor/2/float · returned tuple · level recorded 2026-09-22
Algorithm in 26 papers Sources/algo/ppo.py unverified no licence file found · pointer only · d5c516ed0fa589e3 · report
PPO_continuous Sources/algo/ppo.py unverified no licence file found · pointer only · 5dc5fc3b8ad0138c · report
ifestus/rl community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
PPOAgent policy_agents.py ran no licence file found · pointer only · 786ef0f1e1c69fc2 · report fixture: returned PPOAgent · level recorded 2026-09-22
jhare96/reinforcement-learning community (archive-listed) 3 samples, 0 ran · 3 unverified
Agent in 42 papers rlib/PPO/model.py unverified Apache-2.0 (permissive) · 449ad938cfccea32 · report
PPO rlib/PPO/model.py unverified Apache-2.0 (permissive) · 3c7b55685f685987 · report
PPOModel rlib/PPO/model.py unverified Apache-2.0 (permissive) · 55d94edcf36e1447 · report
jsztompka/PPO-demo community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
PPONetwork model.py ran fingerprinted no licence file found · pointer only · 7dbfda361c907b1e · report fixture: signature state:Tensor/2/float, action:NoneType/0/? · returned dict · level recorded 2026-09-24
ku2482/gail-airl-ppo.pytorch community (archive-listed) 8 samples, 6 ran (3 instrument-failed) · 2 unverified
RolloutBuffer gail_airl_ppo/algo/ppo.py ran MIT (permissive) · 168b3d776129e246 · report fixture: returned RolloutBuffer · level recorded 2026-09-22
StateFunction gail_airl_ppo/algo/ppo.py ran fingerprinted MIT (permissive) · 7f9dae876ba7e00b · report fixture: returned StateFunction · level recorded 2026-09-22
StateIndependentPolicy gail_airl_ppo/algo/ppo.py ran fingerprinted MIT (permissive) · dea2715df3f6a1e3 · report fixture: returned StateIndependentPolicy · level recorded 2026-09-22
calculate_log_pi gail_airl_ppo/algo/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · b4f2ea8b3db73cea · report fixture: signature log_stds:Tensor/2/float, noises:Tensor/2/float, actions:Tensor/2/float · returned Tensor · level recorded 2026-09-22
evaluate_lop_pi gail_airl_ppo/algo/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · 1cfc11023c8784d6 · report fixture: signature means:Tensor/2/float, log_stds:Tensor/2/float, actions:Tensor/2/float · returned Tensor · level recorded 2026-09-22
reparameterize in 38 papers
census: 12 implementations from 17 papers, 5 distinct outputs across 4 buckets, one shared input per bucket
gail_airl_ppo/algo/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · 7c5e74bc668cf5cc · report fixture: signature means:Tensor/2/float, log_stds:Tensor/2/float · returned tuple · level recorded 2026-09-22
Algorithm in 26 papers gail_airl_ppo/algo/ppo.py unverified MIT (permissive) · 2bddf6daaed4d669 · report
PPO gail_airl_ppo/algo/ppo.py unverified MIT (permissive) · cd66a006bca55706 · report
lcswillems/torch-ac community (archive-listed) 4 samples, 1 ran (0 instrument-failed) · 3 unverified
ParallelEnv torch_ac/algos/ppo.py ran MIT (permissive) · 028cd0cc569e4945 · report fixture: returned ParallelEnv · level recorded 2026-09-22
BaseAlgo torch_ac/algos/ppo.py unverified MIT (permissive) · deafc9726839a93c · report
PPOAlgo torch_ac/algos/ppo.py unverified MIT (permissive) · 108ec8df92e50f23 · report
worker in 16 papers torch_ac/algos/ppo.py unverified MIT (permissive) · 0a1d06fc0e7bba39 · report
liuyuezhang/pyrl community (archive-listed) 3 samples, 1 ran (0 instrument-failed) · 2 unverified
ActorCritic in 10 papers ppo/agent.py ran · metamorphic tier: deterministic fingerprinted Apache-2.0 (permissive) · 2f383d9f3f2f7b4f · report fixture: signature x:Tensor/2/float · returned tuple · level recorded 2026-09-22
Memory in 19 papers ppo/agent.py unverified Apache-2.0 (permissive) · 121aee63a1a24691 · report
PPO ppo/agent.py unverified Apache-2.0 (permissive) · 6073c0994e65bae3 · report
llSourcell/OpenAI_Five_vs_Dota2_Explained community (archive-listed) 2 samples, 1 ran (0 instrument-failed) · 1 unverified
ReplayMemory in 11 papers ppo.py ran MIT (permissive) · 7539f57b4803c98e · report fixture: returned ReplayMemory · level recorded 2026-09-22
train in 522 papers ppo.py unverified MIT (permissive) · f5d2e6057543255c · report
mark-gluzman/NmodelPPO community (archive-listed) 3 samples, 3 ran (1 instrument-failed) · 0 unverified
Policy in 23 papers actor_utils.py ran no licence file found · pointer only · ec594f35cdf2f00e · report fixture: returned Policy · level recorded 2026-09-22
PolicyNN actor_utils.py ran no licence file found · pointer only · 06f3c47cdf2dd979 · report fixture: returned PolicyNN · level recorded 2026-09-22
check_optimality actor_utils.py identical code Syntology harvested from this repository for another paper; this paper's link records no repository, path or licence of its own ran · our draft was wrong licence of this copy not recorded · 54948e8edd9b8bc1 · report fixture: signature unscaled_obs:list/1/ndarray, actions:list/1/ndarray, file_name:str/0/str · returned int · level recorded 2026-09-22
marload/DeepRL-TensorFlow2 community (archive-listed) 3 samples, 0 ran · 3 unverified
Actor in 68 papers PPO/PPO_Continuous.py unverified Apache-2.0 (permissive) · 54ccb8ae3f658c83 · report
Agent in 42 papers PPO/PPO_Continuous.py unverified Apache-2.0 (permissive) · 53db83baee745ffd · report
Critic in 60 papers PPO/PPO_Continuous.py unverified Apache-2.0 (permissive) · c004c93ba5634851 · report
mit-realm/neuriss community (archive-listed) 12 samples, 8 ran (5 instrument-failed) · 4 unverified
MLP in 375 papers neuriss/rl/ppo.py ran · metamorphic tier: deterministic MIT (permissive) · f2a4b483cf41dcfd · report fixture: returned MLP · level recorded 2026-09-22
NormalizedMLP neuriss/rl/ppo.py ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · 1271c934496f2b53 · report fixture: returned NormalizedMLP · level recorded 2026-09-22
RolloutBuffer neuriss/rl/ppo.py ran MIT (permissive) · 049dd1dd7386f7f1 · report fixture: returned RolloutBuffer · level recorded 2026-09-22
calculate_gae neuriss/rl/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · 7e7afb47f6ebd451 · report fixture: signature values:Tensor/1/float, rewards:Tensor/1/float, dones:Tensor/1/float, next_values:Tensor/1/float, gamma:float/0/float, lambd:float/0/float · returned tuple · level recorded 2026-09-22
calculate_log_pi neuriss/rl/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · eb4a6cb62afcb0ec · report fixture: signature log_stds:Tensor/2/float, noises:Tensor/2/float, actions:Tensor/2/float · returned Tensor · level recorded 2026-09-22
evaluate_log_pi neuriss/rl/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · c977657d1b1dadef · report fixture: signature means:Tensor/2/float, log_stds:Tensor/2/float, actions:Tensor/2/float · returned Tensor · level recorded 2026-09-22
init_param neuriss/rl/ppo.py ran · our draft was wrong MIT (permissive) · f2cddb5460e2bc7e · report fixture: signature module:callable/0/?, gain:float/0/float · returned SimpleModule · level recorded 2026-09-22
reparameterize in 38 papers
census: 12 implementations from 17 papers, 5 distinct outputs across 4 buckets, one shared input per bucket
neuriss/rl/ppo.py ran · our draft was wrong fingerprinted MIT (permissive) · 6741967893f01eac · report fixture: signature means:Tensor/2/float, log_stds:Tensor/2/float · returned tuple · level recorded 2026-09-22
Agent in 42 papers neuriss/rl/ppo.py unverified MIT (permissive) · c3c2c245443da6e3 · report
Env in 5 papers neuriss/rl/ppo.py unverified MIT (permissive) · f5415970200a24b5 · report
PPO neuriss/rl/ppo.py unverified MIT (permissive) · ded7302e103e8dd7 · report
StateIndependentPolicy neuriss/rl/ppo.py unverified MIT (permissive) · 4652f7766f254cc1 · report
morikatron/GAIL_PPO community (archive-listed) 1 sample, 0 ran · 1 unverified
PPO algo/ppo.py unverified MIT (permissive) · 7af85bd3eff93a8f · report
morikatron/PPO community (archive-listed) 1 sample, 0 ran · 1 unverified
PPO algo/ppo.py unverified MIT (permissive) · 24ed6c73c85ddcca · report
near32/regym community (archive-listed) 3 samples, 3 ran (1 instrument-failed) · 0 unverified
PPOAlgorithm regym/rl_algorithms/algorithms/PPO/ppo.py ran MIT (permissive) · 08c8acc6b144eabd · report fixture: returned PPOAlgorithm · level recorded 2026-09-22
Storage regym/rl_algorithms/algorithms/PPO/ppo.py ran MIT (permissive) · e626bd567c81f207 · report fixture: returned Storage · level recorded 2026-09-22
random_sample in 18 papers regym/rl_algorithms/algorithms/PPO/ppo.py ran · our draft was wrong MIT (permissive) · 2439c728712b51ae · report fixture: signature indices:ndarray/1/int, batch_size:int/0/int · returned generator · level recorded 2026-09-22
nikhilbarhate99/PPO-PyTorch community (archive-listed) 3 samples, 3 ran (0 instrument-failed) · 0 unverified
ActorCritic in 10 papers PPO.py ran MIT (permissive) · bab8cee03856463f · report fixture: returned ActorCritic · level recorded 2026-09-22
PPO PPO.py ran MIT (permissive) · bd4134c19ff6f1eb · report fixture: returned PPO · level recorded 2026-09-22
RolloutBuffer PPO.py ran MIT (permissive) · 22bcf7bc55b864b8 · report fixture: returned RolloutBuffer · level recorded 2026-09-22
nric/ProximalPolicyOptimizationContinuousKeras community (archive-listed) 2 samples, 1 ran (0 instrument-failed) · 1 unverified
MemoryPPO ppo_keras_continuous.py ran no licence file found · pointer only · 75bc51af9c0cf574 · report fixture: returned MemoryPPO · level recorded 2026-09-22
Agent in 42 papers ppo_keras_continuous.py unverified no licence file found · pointer only · f473685be4b59e7f · report
reinforcement-learning-kr/pg_travel community (archive-listed) 1 sample, 0 ran · 1 unverified
surrogate_loss in 5 papers mujoco/agent/ppo_gae.py unverified MIT (permissive) · 6647017115e92ee7 · report
saschaschramm/Pong community (archive-listed) 2 samples, 0 ran · 2 unverified
Model in 227 papers models/ppo/model.py unverified no licence file found · pointer only · a9432edcc1976925 · report
shreyesss/PPO-implementation-keras-tensorflow community (archive-listed) 1 sample, 0 ran · 1 unverified
PPO_lunarlander PPO_lunarlader.py unverified GPL-3.0 (copyleft) · pointer only · 48d2039b794ca629 · report
siddharthverma314/proximalpolicyoptimization community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
ppo_clip_loss src/ppo.py ran · honoured contract no licence file found · pointer only · 942f9baf61103bea · report fixture: signature policy:callable/0/?, arc:MinimalArc/0/?, epsilon:float/0/float · returned Tensor · level recorded 2026-09-22
silvialuu/DRL-2018 community (archive-listed) 3 samples, 2 ran (1 instrument-failed) · 1 unverified
get_gae_advantage es-ppo/ppo.py ran · our draft was wrong fingerprinted no licence file found · pointer only · 62a48793f7edb3af · report fixture: signature rewards:Tensor/2/float, values:Tensor/2/float, mask_n:Tensor/2/float, gamma:float/0/float, lam:float/0/float · returned tuple · level recorded 2026-09-22
get_reward_to_go es-ppo/ppo.py ran · violated contract fingerprinted no licence file found · pointer only · 356537802e7b2bdb · report fixture: signature rewards:list/1/float, mask_n:list/1/float, gamma:float/0/float · returned list · level recorded 2026-09-22
ppo_update es-ppo/ppo.py unverified no licence file found · pointer only · 76b542040b4e4fcf · report
tensorlayer/RLzoo community (archive-listed) 1 sample, 0 ran · 1 unverified
PPO_CLIP rlzoo/algorithms/ppo_clip/ppo_clip.py unverified Apache-2.0 (permissive) · 542d8a7c56874ec9 · report
tilkb/thermoai community (archive-listed) 2 samples, 2 ran (1 instrument-failed) · 0 unverified
PPO controller/RL/PPO.py ran no licence file found · pointer only · e33afa03c0df2bff · report fixture: returned PPO · level recorded 2026-09-22
parallel_trajectory_collection controller/RL/PPO.py ran · our draft was wrong no licence file found · pointer only · f1449fb45454e326 · report fixture: signature simulator:SimpleSimulator/0/?, actor_model:callable/0/?, count:int/0/int, min_value:float/0/float, max_value:float/0/float, sigma:float/0/float, init_step:int/0/int, gamma:float/0/float · returned list · level recorded 2026-09-22
tommyvsfu1/RL-NTU community (archive-listed) 2 samples, 0 ran · 2 unverified
ActorCritic in 10 papers RL_mlds_hw4/hw4/agent_dir/agent_pg.py unverified no licence file found · pointer only · d068adb7cb463c20 · report
PPO RL_mlds_hw4/hw4/agent_dir/agent_pg.py unverified no licence file found · pointer only · f51a9f6b2ca368d9 · report
tuanpnm99/RLPongAgent community (archive-listed) 2 samples, 1 ran (0 instrument-failed) · 1 unverified
Policy in 23 papers Agent.py ran · metamorphic tier: deterministic no licence file found · pointer only · c02c0631708465dc · report fixture: returned Policy · level recorded 2026-09-22
PongAgent Agent.py unverified no licence file found · pointer only · 40f05c72bfe4c048 · report
uvipen/super-mario-bros-ppo-pytorch community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
PPO src/model.py ran · metamorphic tier: deterministic MIT (permissive) · 1ebfd098cf241f3c · report fixture: returned PPO · level recorded 2026-09-22
wangzhengfei0730/NIPS2018-AIforProsthetics community (archive-listed) 4 samples, 1 ran (0 instrument-failed) · 3 unverified
Runner baselines/ppo2/ppo2.py ran no licence file found · pointer only · c80e9a362666d32e · report fixture: returned Runner · level recorded 2026-09-22
AbstractEnvRunner baselines/ppo2/ppo2.py unverified no licence file found · pointer only · 5b125d5154c21cbf · report
Model in 227 papers baselines/ppo2/ppo2.py unverified no licence file found · pointer only · 14b4f65f714ccf75 · report
learn in 9 papers baselines/ppo2/ppo2.py unverified no licence file found · pointer only · be674cc9da531aa5 · report
xtma/simple-pytorch-rl community (archive-listed) 3 samples, 2 ran (0 instrument-failed) · 1 unverified
ActorNet ppo.py ran fingerprinted no licence file found · pointer only · 13b852ad71d18604 · report fixture: signature x:Tensor/2/float · returned tuple · level recorded 2026-09-24
CriticNet ppo.py ran no licence file found · pointer only · 1a18a57c9dbc3655 · report fixture: returned CriticNet · level recorded 2026-09-22
Agent in 42 papers ppo.py unverified no licence file found · pointer only · 64930ee88e057fb9 · report
yoavalon/Quadcopter-env community (archive-listed) 2 samples, 0 ran · 2 unverified
Critic in 60 papers kdev/ppo.py unverified GPL-3.0 (copyleft) · pointer only · f42b9cecada07086 · report
PPO kdev/ppo.py unverified GPL-3.0 (copyleft) · pointer only · f7417c7ed09d5ade · report
zjlab-ammi/llm4rl community (archive-listed) 2 samples, 1 ran (0 instrument-failed) · 1 unverified
PPO algos/ppo.py ran no licence file found · pointer only · 20cf521175a91e97 · report fixture: returned PPO · level recorded 2026-09-22
Base in 25 papers algos/ppo.py unverified no licence file found · pointer only · 9619638a806c794f · report
MaximeVandegar/Papers-in-100-Lines-of-Code community (archive-listed) 1 sample, 0 ran · 1 unverified
PPO Proximal_Policy_Optimization_Algorithms/ppo.py unverified MIT (permissive) · 5954e04c0ee7d26e · report
OctopusMind/RLHF_PPO community (archive-listed) 3 samples, 1 ran (0 instrument-failed) · 2 unverified
Tools ppo.py ran · metamorphic tier: invariant Apache-2.0 (permissive) · 63380712f9af778d · report fixture: signature logits:Tensor/3/float, labels:Tensor/2/int · returned Tensor · level recorded 2026-09-22
Config in 46 papers ppo.py unverified Apache-2.0 (permissive) · 593f887d35295a28 · report
PPO ppo.py unverified Apache-2.0 (permissive) · df9fd7d088621b72 · report
arnomoonens/yarll community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
ppo_loss yarll/agents/tf2/ppo/ppo.py ran · honoured contract fingerprinted MIT (permissive) · 0db440a845c6a4fa · report fixture: signature old_logprob:EagerTensor/2/float, new_logprob:EagerTensor/2/float, epsilon:EagerTensor/0/float, advantage:EagerTensor/2/float · returned EagerTensor · level recorded 2026-09-22
dyabel/handson_rl community (archive-listed) 1 sample, 1 ran (0 instrument-failed) · 0 unverified
PPO ppo.py ran no licence file found · pointer only · 00bb1a30b070c324 · report fixture: returned PPO · level recorded 2026-09-22
Repository not located 3 samples, 3 ran (1 instrument-failed) · 0 unverified
3 samples are identical to code Syntology harvested from another paper's repository; this paper's own copy was not located, so no repository is named and no licence is recorded.
constfn in 10 papers repository not located ran · our draft was wrong licence of this copy not recorded · 8758308dc6c0727f · report fixture: signature val:int/0/int · returned function · level recorded, date not recorded
safemean in 6 papers repository not located ran · honoured contract fingerprinted licence of this copy not recorded · a7b323241cd70612 · report fixture: signature xs:list/1/float · returned float64 · level recorded 2026-09-22
sf01 in 9 papers repository not located ran · violated contract licence of this copy not recorded · baf2a28d376d7803 · report fixture: signature arr:ndarray/4/float · returned ndarray · level recorded, date not recorded

Method as code Syntology

What Syntology's graph records as this paper's claimed method, lifted into code: each assembly names its bricks (functions and classes), each with its own role and its own verification level. An assembly is not a check that the code implements the method correctly or reproduces the paper's results. Levels are each brick's own verification level in Syntology's graph (level 0: no recorded run), listed brick by brick and never combined into one level; “ran” means the brick executed on a synthesized input, not that its output is correct, and the instrument figure counts failures of Syntology's instrument, not of the code. Where a model read the code against the paper's claim, each assembly's label verdict is one model's judgement, kept in this page's JSON twin; it is not calibrated, so no card prints it. Siblings are assemblies lifted from this paper in the same batch for the same method: alternative readings kept side by side; none replaces or supersedes another.

64 assemblies on this paper: 64 harvested, 0 generated, listed in 1 groups (siblings lifted in one batch form one group).

64 sibling assemblies for Proximal Policy Optimization (PPO) (harvested), lifted in the same batch on 2026-09-09: alternative readings kept side by side; none replaces or supersedes another. Bricks printed for 12 of 64 sibling assemblies; the other 52 are listed with their counts; every brick is in this page's JSON twin.

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 1 brick; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified. The assembly has not been run end-to-end.

one of 64 siblings

  1. proximal_policy_optimization_loss entry · level 1 · ran · our draft was wrong · 063d29364d7e4f81

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 1 brick; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 64 siblings

  1. PPO entry · level 1 · ran · 1ebfd098cf241f3c

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 12 bricks; individually on a fixture: 10 ran (of which 3 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 0 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified. The assembly has not been run end-to-end.

one of 64 siblings

  1. PPO_continuous entry · level 0 · unverified · 5dc5fc3b8ad0138c
  2. Algorithm helper · level 0 · unverified · d5c516ed0fa589e3
  3. RolloutBuffer_PPO_lag helper · level 1 · ran · 6f986aa2d59a44f8
  4. StateFunction helper · level 1 · ran · a78ee4761e8df9d8
  5. StateIndependentPolicy helper · level 1 · ran · fbd28096c7f0c30f
  6. calculate_gae helper · level 1 · ran · our draft was wrong · b5a22b5e0dce3796
  7. calculate_gae_cost helper · level 1 · ran · our draft was wrong · 77393baa1a11c24b
  8. build_mlp helper · level 1 · ran · our draft was wrong · c511268a73d82fc6
  9. evaluate_lop_pi helper · level 1 · ran · honoured contract · fef4ec37a6ea52e7
  10. reparameterize helper · level 1 · ran · our draft was wrong · b1c9c127240c75d8
  11. atanh helper · level 1 · ran · honoured contract · 264c9e17d7dfe7eb
  12. calculate_log_pi helper · level 1 · ran · honoured contract · 476e14ee8b7219ac

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 3 bricks; individually on a fixture: 0 ran · 3 unverified. The assembly has not been run end-to-end.

one of 64 siblings

  1. Agent entry · level 0 · unverified · 53db83baee745ffd
  2. Actor model · level 0 · unverified · 54ccb8ae3f658c83
  3. Critic model · level 0 · unverified · c004c93ba5634851

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 2 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified. The assembly has not been run end-to-end.

one of 64 siblings

  1. clipped_surrogate entry · level 0 · unverified · 9c65b36eec7ec92a
  2. states_to_prob model · level 1 · ran · our draft was wrong · a7700bf49f384087

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 1 brick; individually on a fixture: 0 ran · 1 unverified. The assembly has not been run end-to-end.

one of 64 siblings

  1. PPO entry · level 0 · unverified · 5954e04c0ee7d26e

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 4 bricks; individually on a fixture: 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 4 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 64 siblings

  1. Agent entry · level 1 · ran · b1f30e180cb81a3a
  2. ActorCritic model · level 1 · ran · f7a59411ac390f78
  3. Actor model · level 1 · ran · e9f1207fc08f9fa9
  4. Critic model · level 1 · ran · b4455ab1f545ae9c

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 2 bricks; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 64 siblings

  1. PongAgent entry · level 0 · unverified · 40f05c72bfe4c048
  2. Policy model · level 1 · ran · c02c0631708465dc

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 2 bricks; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 64 siblings

  1. Agent entry · level 0 · unverified · f473685be4b59e7f
  2. MemoryPPO data_transform · level 1 · ran · 75bc51af9c0cf574

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 4 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 3 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 64 siblings

  1. PPO entry · level 1 · ran · b48c75c1f791937c
  2. PolicyUpdate update_rule · level 1 · ran · 2eb0c930a45a7472
  3. ppo_loss loss · level 0 · unverified · 2c9a98eedcdea762
  4. ModelUpdate helper · level 1 · ran · 7f143a6ee276436b

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 1 brick; individually on a fixture: 0 ran · 1 unverified. The assembly has not been run end-to-end.

one of 64 siblings

  1. PPOalgorithm entry · level 0 · unverified · bce2a8566fa36fda

harvested Assembly for Proximal Policy Optimization (PPO) (graph method: Proximal Policy Optimization (PPO)). 4 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified. The assembly has not been run end-to-end.

one of 64 siblings

  1. PPOAgent entry · level 0 · unverified · 01b7328f523d7756
  2. Actor model · level 0 · unverified · bf0c1da602f8a16d
  3. Critic model · level 0 · unverified · 708e869097ea91ff
  4. gaussian_likelihood helper · level 1 · ran · honoured contract · b46c632bedd3d8af
  1. learn · 3d1f2cc06035ea0a · 7 bricks; individually on a fixture: 4 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified · The assembly has not been run end-to-end · assemblies[12] in the JSON twin
  2. Model · 4001252face0416d · 2 bricks; individually on a fixture: 0 ran · 2 unverified · The assembly has not been run end-to-end · assemblies[13] in the JSON twin
  3. ppo_loss · 44d8b39d2fb476c9 · 1 brick; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[14] in the JSON twin
  4. PPO · 452ce971591430ce · 3 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 2 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[15] in the JSON twin
  5. Agent · 484ddcf0d4356b8c · 4 bricks; individually on a fixture: 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 4 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[16] in the JSON twin
  6. surrogate_loss · 4ce5af77fa6f4833 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[17] in the JSON twin
  7. PPO · 4dc0f5bbcea8f6cb · 12 bricks; individually on a fixture: 8 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified · The assembly has not been run end-to-end · assemblies[18] in the JSON twin
  8. PPO · 4f2cdd5736c5ac05 · 4 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 3 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[19] in the JSON twin
  9. PPO · 545631e1506660a9 · 2 bricks; individually on a fixture: 0 ran · 2 unverified · The assembly has not been run end-to-end · assemblies[20] in the JSON twin
  10. train · 56dc80195a79d4ab · 2 bricks; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[21] in the JSON twin
  11. PPO · 5c85085383d91168 · 1 brick; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[22] in the JSON twin
  12. PPOAlgo · 5e8e5bc0133684ff · 7 bricks; individually on a fixture: 4 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified · The assembly has not been run end-to-end · assemblies[23] in the JSON twin
  13. PPO · 5f670b9e0bff80cc · 2 bricks; individually on a fixture: 0 ran · 2 unverified · The assembly has not been run end-to-end · assemblies[24] in the JSON twin
  14. Agent · 65fea45a5fbcf003 · 3 bricks; individually on a fixture: 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[25] in the JSON twin
  15. PPO_CLIP · 67465e0454ae1502 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[26] in the JSON twin
  16. ppo_update · 69dcc57ca900f4a9 · 2 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[27] in the JSON twin
  17. PPO · 6c6488eaf07c5cd2 · 11 bricks; individually on a fixture: 9 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[28] in the JSON twin
  18. PPO · 7b781a51dbdb10eb · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[29] in the JSON twin
  19. PPO · 820466280dda9c76 · 3 bricks; individually on a fixture: 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[30] in the JSON twin
  20. PPOAlgo · 83b8ef33e9af0b97 · 6 bricks; individually on a fixture: 3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified · The assembly has not been run end-to-end · assemblies[31] in the JSON twin
  21. proximal_policy_optimization_loss · 8693af6c0b7a4743 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[32] in the JSON twin
  22. clipped_surrogate · 86bd407862da2411 · 2 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[33] in the JSON twin
  23. PPO · 8b3d19783f5c610b · 2 bricks; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[34] in the JSON twin
  24. Policy · 8c140ab8c777ee63 · 3 bricks; individually on a fixture: 3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[35] in the JSON twin
  25. PPO · 8d6590892a5b6996 · 9 bricks; individually on a fixture: 5 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified · The assembly has not been run end-to-end · assemblies[36] in the JSON twin
  26. PPO · 8d6fee5945f49846 · 3 bricks; individually on a fixture: 0 ran · 3 unverified · The assembly has not been run end-to-end · assemblies[37] in the JSON twin
  27. PPOAgent · 90c9b8afd1f6f185 · 4 bricks; individually on a fixture: 4 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[38] in the JSON twin
  28. ppo_clip_loss · 93e3c144f2fcebec · 1 brick; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[39] in the JSON twin
  29. ppo_update · 95e7acfe51a56ff3 · 3 bricks; individually on a fixture: 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[40] in the JSON twin
  30. PPOAlgorithm · 963202faa4c5bea5 · 3 bricks; individually on a fixture: 3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[41] in the JSON twin
  31. PPOAgent · 98f7bb203dda6920 · 1 brick; individually on a fixture: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[42] in the JSON twin
  32. PPO · 9d8ce58b69147416 · 5 bricks; individually on a fixture: 3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[43] in the JSON twin
  33. build_train · 9dc95a3cfa84d169 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[44] in the JSON twin
  34. update · aa2577adcf955e3d · 2 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[45] in the JSON twin
  35. update · ae803636ac0600a8 · 2 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[46] in the JSON twin
  36. PPO · afcac13d8faac401 · 2 bricks; individually on a fixture: 2 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[47] in the JSON twin
  37. PPO · bb25dd31a314de78 · 12 bricks; individually on a fixture: 10 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[48] in the JSON twin
  38. PPONetwork · be99ff2dc064352d · 1 brick; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[49] in the JSON twin
  39. proximal_policy_optimization_loss_continuous · bf7a5807bc6decbb · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[50] in the JSON twin
  40. PPO · c31a1b29be6dccc4 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[51] in the JSON twin
  41. Agent · c62f861f771b9563 · 2 bricks; individually on a fixture: 0 ran · 2 unverified · The assembly has not been run end-to-end · assemblies[52] in the JSON twin
  42. PPO · c8b3266a6c71954e · 3 bricks; individually on a fixture: 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[53] in the JSON twin
  43. ppo_step · c9442e5f1a3f07c0 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[54] in the JSON twin
  44. PPO · ca966194ac6bde41 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[55] in the JSON twin
  45. update · ce5631d908c867ec · 2 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified · The assembly has not been run end-to-end · assemblies[56] in the JSON twin
  46. PPO · cf9f22ce8d9df0b1 · 3 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[57] in the JSON twin
  47. Policy · d10b94a897b575c0 · 3 bricks; individually on a fixture: 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[58] in the JSON twin
  48. PPO · d2c6885423d4aecf · 3 bricks; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified · The assembly has not been run end-to-end · assemblies[59] in the JSON twin
  49. MyPPO · e1139e085618d3a6 · 1 brick; individually on a fixture: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified · The assembly has not been run end-to-end · assemblies[60] in the JSON twin
  50. PPOAgent · e4bbf4ae4dbdaceb · 4 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; every one of the 2 samples that ran constructed an object rather than computing a result · The assembly has not been run end-to-end · assemblies[61] in the JSON twin
  51. PPO · ea8574b6125d04b0 · 2 bricks; individually on a fixture: 0 ran · 2 unverified · The assembly has not been run end-to-end · assemblies[62] in the JSON twin
  52. PPO_lunarlander · f2c646ce0523c173 · 1 brick; individually on a fixture: 0 ran · 1 unverified · The assembly has not been run end-to-end · assemblies[63] in the JSON twin

Each brick ends with its code_sha256, Syntology's identity for that exact code; click one to copy the get_code call that returns it. For agents, on Syntology's MCP service: assembly(arxiv_id="1707.06347") returns these assemblies with each brick's role and wiring (how to connect).

Tasks

Continuous ControlDota 2Multi-Goal Reinforcement LearningNeural Architecture SearchOpenAI GymPolicy Gradient MethodsReinforcement LearningSpeech Enhancement

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Continuous Control Lunar Lander (OpenAI Gym) PPO Score 175.14±44.94 #4 of 5 Archive leaderboard report
Neural Architecture Search NATS-Bench Topology, CIFAR-10 PPO (Schulman et al., 2017) Test Accuracy 94.02 #3 of 11 Archive leaderboard report
Neural Architecture Search NATS-Bench Topology, CIFAR-100 PPO (Schulman et al., 2017) Test Accuracy 71.68 #2 of 11 Archive leaderboard report
Neural Architecture Search NATS-Bench Topology, ImageNet16-120 PPO (Schulman et al., 2017) Test Accuracy 44.95 #3 of 11 Archive leaderboard report
OpenAI Gym Ant-v4 PPO Average Return 608.97 #5 of 5 Archive leaderboard report
OpenAI Gym HalfCheetah-v4 PPO Average Return 6006.11 #5 of 5 Archive leaderboard report
OpenAI Gym Hopper-v4 PPO Average Return 790.77 #5 of 5 Archive leaderboard report
OpenAI Gym Humanoid-v4 PPO Average Return 925.89 #3 of 5 Archive leaderboard report
OpenAI Gym Walker2d-v4 PPO Average Return 2739.81 #4 of 5 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Introduced by this paper: PPO

Entropy RegularizationPPO

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections