Sparse D4RLMedium-expert, 22–28% of an unsafe region removed
The halfcheetah robot
The hopper robot
The walker2d robot
density
action
Reward vs. action norm in the sparse halfcheetah dataset; the thinned-out corner is shaded gray
Reward vs. action norm in the sparse hopper dataset; the thinned-out corner is shaded gray
Reward vs. action norm in the sparse walker2d dataset; the thinned-out corner is shaded gray
reward
reward
reward
halfcheetah
hopper
walker2d
Our medical datasetCardiogenic shock, 379 patients on MCS
Illustration of an Impella heart pump placed across the aortic valve into the left ventricle
Impella
Reward vs. pump level in the medical dataset; the sparse corner is shaded gray
sparse reward–action

Policy evaluation

MuJoCo tasks+ click a dataset to see results
MOBILE
GORMPO
return gain
MOBILE on halfcheetah: fallen or flipped
GORMPO on halfcheetah at the same moment, still going
halfcheetah
MOBILE on hopper: fallen or flipped
GORMPO on hopper at the same moment, still going
hopper
MOBILE on walker2d: fallen or flipped
GORMPO on walker2d at the same moment, still going
walker2d
Impella weaning+ click a dataset to see results
✓Blood pressure stays above 60: the patient is stable, so the pump should be weaned.
CORMPO: recommended pump level and predicted blood pressure over six hours in the digital twin
CORMPO
no MOBILE-based
medical policy
GORMPO
blood pressureobservedP-levelclinician decisioncritical threshold