yes, still it boils down to the question of 'how well is the output of the chat bot aligned with reality' ? if you want to automate this, then you will likely need some system that is kind of censoring the output of the LLM, and that system should have a better model of what is real.