What you have described is something akin to what meta tags are for. Do we need another method at a domain or subdomain level? Plus, robots.txt, etc. is limited to domain and subdomain managers.
ai.txt is useful, but I am not sure we have nailed down what it can be used for. One use is to tell AI not to train on the content found within because it could be an AI generation.