Like every other industry, there's a growth period where things are new and prices are reasonable, and then there's the "squeeze" where bean counters come in, make charts that are likely bs, and explain how much easier it'd be if we charged 4x as much for half the customer base.
Twitter was one of the first to give access to cheap mass data, and now they're one of the first to charge through the nose for that. The move is going to be that if you're not enterprise level you're not getting this data anymore, and I doubt it stops with reddit.
This can't just be laid on the feet of faceless bean counters or old men in the executive suite.