zlacker

>>goranm+(OP)
I’ve been attempting to use the “official” function calling API for every new version of GPT they put out but it’s always a dead end. It seems only to be able to handle 4-5 functions at a time before it starts hallucinating parameters or starts responding in clear text instead of whatever internal format OpenAI uses in their backend before sending a structured response back to me. The whole JSON schema thing seems way too verbose and complicated, and even with the claims that the new function calling models are specifically tuned to the format, it has the same issues.

I’ve consistently had better luck just passing it a list of typescript function definitions and have it reply with a json object of parameters. It seems to understand this way better, and doesn’t lose focus half as quickly. It also allows me to mix regular responses and chain-of-thought reasoning in with the calls, which is something it seems to simply refuse to do when “function calling mode” is active.

An additional trick I’ve been using to make it stay focused with even longer prompts is to only provide a list of function names and let it hallucinate parameters for them, and then “gaslight” it by sending a new request, now with a more detailed prompt on the specific functions it wanted to call. More costly, but I haven’t found any other way of keeping it focused. Anyone know any additional tricks?

>>sippea+9n
Do you have any examples of this? Curious how this looks in practice. We are just adding function calling ourselves but not sure how add CoT