zlacker

>>laurex+(OP)
These perverse incentives run at the heart of almost all Developer Software as a Service tooling. Using someone else's hosted model incentivizes increasing token usage, but it's nothing special about AI.

Consider Database-as-a-service companies: They're not incentivized to optimize on CPU usage, they charge per cpu. They're not incentivized to improve disk compression, they charge for disk-usage. There are several DB vendors who explicitly disable disk compression and happily charge for storage capacity.

When you run the software yourself, or the model yourself, the incentives aligned: use less power, use less memory, use less disk, etc.

>>erulab+zj
My favourite example of this is the recent trend towards “wide events” replacing logs and metrics… spearheaded and popularised by companies that charge by the gigabytes ingested.

>>jiggaw+gs
Companies that ingest logs generally rip their customers faces off with their pricing. At least oTel spans can be tail-sampled.

>>tptace+Qv
I worked out that it's cheaper to write logs to high-end Samsung SSDs and then throw them away every month than to retain them in the log analytics systems of some cloud services for the same period of time.

Wait, no, sorry... that doesn't quite "paint the right picture".

The "single use" SSDs are 75 times cheaper than storing the data in the cloud.

>>jiggaw+m81
As someone who works at a platform company that operates several (very) large log ingestion systems, if you're not indexing the logs usefully, having stored them on SSDs isn't doing much for you. It's just a weird comparison to make, is all I'm saying.

>>tptace+Fw2
Indexing that is 75x the original uncompressed data volume?

Because then I might accept the cost!

Realistically all of these systems use some type of data compression such as Parquet files, so the data on disk is likely smaller than the ingested data.