and the base model was excellently calibrated. https://openai.com/research/gpt-4
"Interestingly, the base pre-trained model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct). However, through our current post-training process, the calibration is reduced."
Next ?