Robots.txt exists because shop photographers want to be allowed to take pictures rather than be blocked tout court.