Learning to Be still and Say no