zlacker

[parent] [thread] 1 comments
1. Kranar+(OP)[view] [source] 2023-06-01 14:01:59
>It's a lot of work but for a small fraction of 20m a year you can hire a team who only work on that.

I tried doing something like this for scraping corporate profiles. It's a lot harder than it seems. Most websites try to block bots that do this kind of scraping (unless you're Google/Facebook/etc...), so you end up having to setup a boatload of VPNs to make it seem like the bot is coming from multiple sources, but the VPNs end up being very unreliable and they too get banned as well.

For my own small scale it did manage to work but it was very challenging. I'd imagine doing it at the scale of Apollo would be all but impossible to get away with.

replies(1): >>sschue+95
2. sschue+95[view] [source] 2023-06-01 14:25:16
>>Kranar+(OP)
They will block bots and large scrapers.

What you have to do is have the app do it directly. This is more work if you have to do it in the app but it makes life for reddit a living hell as they can not just filter high requests nunbers as the traffic is coming from individuals.

[go to top]