zlacker

Readers should, charitably, interpret this as "the sequence of events which need to happen in order for OpenAI to justify the inflow of capital necessary to survive".

Your daily vibe coding challenge: Get GPT-4o to output functional code which uses Google Vertex AI to generate a text embedding. If they can solve that one by July, then maybe we're on track for "curing all disease and aging, brain uploading, and colonizing the solar system" by 2030.

>>827a+(OP)
You’ve intentionally hamstrung your test by choosing an inferior model though.

>>827a+(OP)
Haven't tested this (cbf setting up Google Cloud), but the output looks consistent with the docs it cites: https://chatgpt.com/share/67efd449-ce34-8003-bd37-9ec688a11b...

You may consider using search to be cheating, but we do it, so why shouldn't LLMs?

>>Philpa+yM1
I should have specified "nodejs", as that has been my most recent difficulty. The challenge, specifically, with that prompt is that Google has at least four nodejs libraries that are all seem at least reasonably capable of accessing text embedding models on vertex ai (@google-ai/generativelanguage, @google-cloud/vertexai, @google-cloud/aiplatform, and @google/genai), and they've also published breaking changes multiple times to all of them. So, in my experience, GPT not only will confuse methods from one of their libraries with the other, but will also sometimes hallucinate answers only applicable to older versions of the library, without understanding which version its giving code for. Once it has struggled enough, it'll sometimes just give up and tell you to use axios, but the APIs it recommends axios calls for are all their protobuf APIs; so I'm not even sure if that would work.

Search is totally reasonable, but in this case: Even Google's own documentation on these libraries is exceedingly bad. Nearly all the examples they give for them are for accessing the language models, not text embedding models; so GPT will also sometimes generate code that is perfectly correct for accessing one of the generative language models, but will swap e.g the "model: gemini-2.0" parameter for "model: text-embedding-005"; which also does not work.

>>slater+hC
o1 fails at this, likely because it does not seem to have access to search, so it is operating on outdated information. It recommends the usage of methods that have been removed by Google in later versions of the library. This is also, to be fair, a mistake gpt-4o can make if you don't explicitly tell it to search.

o3-mini-high's output might work, but it isn't ideal: It immediately jumps to recommending avoiding all google cloud libraries and directly issuing a request to their API with fetch.