zlacker

> And I hope the future holds some kind of synthesis of these strategies.

Recently I’ve been involved in discussions about using an LLM to generate JSON according to a schema, as in OpenAI’s function calling or Jsonformer-LLMs do okay for generating code in mainstream languages like SQL or Python, but what if you have some proprietary query language? Maybe have a JSON schema for the AST, have the LLM generate JSON conforming to that schema, then serialise the JSON to the proprietary query language syntax?

And it makes me think - what if one used an LLM to generate or evaluate assertions in a Cyc-style ontology language? And that might be a bridge between the logic/ontology approach and the statistical/neural approach

>>skissa+(OP)
This is similar to what people are trying for mathematical theorem proving. Using LLMs to generate theorems that can be validated in Lean.

>>skissa+(OP)
I had the same idea last year. But it's difficult. To encode knowledge in CycL required intensive training, mostly in how their KB encoded very abstract concepts and "obvious" knowledge. They used to boast about how they had more philosophy PhDs than anywhere else.

It's possible that an LLM that's been trained on enough examples, and that's smart enough, could actually do this. But I'm not sure how you'd review the output to know if it's right. The LLM doesn't have to be much faster than you to overwhelm the capacity of reviewing the results.

>>skissa+(OP)
This might work; you can view it as distilling the common knowledge out of the LLM.

You’d need to provide enough examples of CycL for it to learn the syntax.

But in my experience LLMs are not great at authoring code with no ground truth to test against. So the LLM might hallucinate some piece of common knowledge, and it could be hard to detect.

But at the highest level, this sounds exactly how the WolframAlpha ChatGPT plug-in works; the LLM knows how to call the plugin and can use this to generate graphs or compute numerical functions for domains where it cannot compute the result directly.