It's a lot of work but for a small fraction of 20m a year you can hire a team who only work on that.
Didn't Digg primarily die from a redesign? Why has the web become so forgiving these days? People don't seem to give a shit anymore. "Oh OK, I'll use the official reddit app filled with ads, OK I will watch your 30 second pre-roll. OK I will give you may data for free". /s
I tried doing something like this for scraping corporate profiles. It's a lot harder than it seems. Most websites try to block bots that do this kind of scraping (unless you're Google/Facebook/etc...), so you end up having to setup a boatload of VPNs to make it seem like the bot is coming from multiple sources, but the VPNs end up being very unreliable and they too get banned as well.
For my own small scale it did manage to work but it was very challenging. I'd imagine doing it at the scale of Apollo would be all but impossible to get away with.