zlacker

I serve ~300tk/s of Mistral 7B for $0.60/hr by renting a cloud 3090. That's a lot cheaper than GPT-4-Turbo, though the quality is closer to GPT-3.5.

Mixtral 8x7b is closer to GPT-4 quality though and only 2x the compute requirement of Mistral 7B.