zlacker

[parent] [thread] 1 comments
1. jefftk+(OP)[view] [source] 2023-05-10 13:38:33
> It's not a system that we should copy for preventing the use of content in training AI

I don't see the OP saying anything about "ai.txt" being for that? They're advocating it as a way that AIs could use fewer tokens to understand what a site is about.

(Which I also don't think is a good idea, since we already have lots of ways of including structured metadata in pages, but the main problem is not that crawlers would ignore it.)

replies(1): >>kmoser+Sw
2. kmoser+Sw[view] [source] 2023-05-10 15:57:03
>>jefftk+(OP)
Not only do we already have lots of ways of including structured metadata, but if you want to include directives about what should/shouldn't be scraped and by whom, we already have robots.txt.

In other words, there's no need to create an ai.txt when the robots.txt standard can just be extended.

[go to top]