zlacker

As someone who works at a platform company that operates several (very) large log ingestion systems, if you're not indexing the logs usefully, having stored them on SSDs isn't doing much for you. It's just a weird comparison to make, is all I'm saying.

>>tptace+(OP)
Indexing that is 75x the original uncompressed data volume?

Because then I might accept the cost!

Realistically all of these systems use some type of data compression such as Parquet files, so the data on disk is likely smaller than the ingested data.

>>jiggaw+bw
My point is that you're not really just paying for "storage".

>>tptace+hD
What am I paying for?

I worked out that the markup on CPU, network bandwidth, and storage for the default logging products from the major clouds is on the order of 25x to 500x.

Okay, sure, there's some people that need to be paid, the back-end software may have some licensed components, etc, etc...

But still, comparing this to any other cloud service, the gross profit margin is just ridiculous!

It's the typical IT marketing trick of selling the commodity (VMs) at competitive prices, and then clawing back the profits via the "enterprise add-ons".

>>jiggaw+a41
I mean: we faced the same buy/build issue, and we went "build", and while I wouldn't say it was our worst decision, it was not one of our best either. Certainly the problem doesn't simply boil down to NVME gigabyte costs!

That said: we both agree, log ingestion services are extremely expensive.