if you put content on internet and accessible to humans, why do you want to now say to people that if it's a machine that does it, suddenly you do not agree ? i am free to write code or design a machine to go get that data, and do whatever i want with it (as long as i don't do something illegal like stealing content under copyright)
and i don't give a F about the "terms of use" those morons put online, because those have NO value. there is either a contract signed by two parties, or there is not. and content you put on internet, and accessible to everyone that sends you a GET, is like writing stuff on a page, and putting that page outside on the street.
we could use humans to go read all those pages, and create new content from it from the knowledge gained on those various subjects. machine are here to reproduce what humans can do, to free us time for more interesting things. those servers that send data back from a GET, it is the same request when it's done by me, a human, or a machine. and those morons did put that data there, accessible to all, so now to see them cry foul makes me laugh.