zlacker

>>convex+(OP)
Makes me wonder whether to keep building upon OpenAI? Given that they have an API and it takes effort to build on that vs. something else. I am small fry but maybe other people are wondering the same? Can they give reassurances about their products going into the future?

>>quickt+U4
I’d recommend trying to build out your systems to work across LLMs where you can. Create an interface layer and for now maybe use OpenAI and Vertex as a couple of options. Vertex is handy as while not always as good you may find it works well for some tasks and it can be a lot cheaper for those.

If you build out this way then when the next greatest LLM comes out you can plug that into your interface and switch the tasks it’s best at over.

>>mebutn+S7
The problem is swapping LLMs can require rework of all your prompts, and you may be relying on specific features of OpenAI. If you don't then you are at a disadvantage or at least slowing down your work.

>>quickt+aa
I have a hierarchy of templates, where I can automatically swap out parts of the prompt based on which LLM I am using. And also have a set of benchmarking tests to compare relative performance. I treat LLMs like a commodity and keep switching between them to compare performance.

>>discon+gb
Just curious are you using something specific for the tests?