Much of our ethical and regulatory focus on medical AI is on how to prevent it from going wrong. These concerns are crucial, but some of the most important consequences of medical AI may emerge when it works as intended.