zlacker

[return to "Pushing ChatGPT's Structured Data Support to Its Limits"]
1. JoshMa+Dg[view] [source] 2023-12-27 16:42:33
>>goranm+(OP)
FWIW, I've seen stronger performance from gpt-4-1106-preview when I use `response_format: { type: "json_object" },` (providing a target typescript interface in context), vs the "tools" API.

More flexible, and (evaluating non-scientifically!) qualitatively better answers & instruction following -- particularly for deeply nested or complex schemas, which typescript expresses very clearly and succinctly.

Example from a hack week project earlier this month (using a TS-ish schema description that's copy/pasted from healthcare's FHIR standard): https://github.com/microsoft-healthcare-madison/hackweek-202...

Or a more complex example with one model call to invent a TS schema on-the-fly and another call to abstract clinical data into it: https://github.com/microsoft-healthcare-madison/hackweek-202...

◧◩
2. buggle+iu[view] [source] 2023-12-27 17:57:57
>>JoshMa+Dg
Something’s been broken with their JSON mode/function calling since the Dev Day launch. I have a bunch of scripts that stopped consistently returning JSON when I swapped in the gpt-4-1106 preview and gpt-3.5-turbo has gotten similarly erratic. They really need to pause all the “move fast and break things” stuff because their API and services like ChatGPT are becoming increasingly unreliable.
[go to top]