zlacker

[parent] [thread] 3 comments
1. gdiamo+(OP)[view] [source] 2023-04-27 22:58:55
How can you connect the results from the LLM into the planner? Do you build a parser? Do you fine-tune the LLM to adhere to a format that the planner understands?
replies(2): >>mhink+D2 >>kordle+EB1
2. mhink+D2[view] [source] 2023-04-27 23:18:04
>>gdiamo+(OP)
> Do you fine-tune the LLM to adhere to a format that the planner understands?

This one. It looks like they're using GPT3 to translate the natural-language problem context and goal into a format called PDDL (planning domain definition language), then feeding the result into a separate program that generates a plan based on the context and goal.

With that in mind, the thing they're really testing here is how well GPT3 can translate the natural-language prompt into PDDL, evaluated on the basis of whether the generated PDDL can actually solve the problem and how long the resulting solution takes.

Naturally, I could be wrong but that's at least what it looks like.

replies(1): >>YeGobl+Nw1
◧◩
3. YeGobl+Nw1[view] [source] [discussion] 2023-04-28 14:10:08
>>mhink+D2
There is no fine-tuning. They simply use prompt engineering. See Section 3 for "Method", they have a short, easy-to grok motivating example, I don't think you need to be an expert in planning to see what they do.

To summarise, they assume a human expert can provide a domain description, specifying all actions that can be taken at each situation, and their effects. Then it looks like they include that domain description to the prompt, along with an example of the kind of planning task they want it to solve, and get the LLM to generate PDDL in the context of the prompt.

4. kordle+EB1[view] [source] 2023-04-28 14:35:15
>>gdiamo+(OP)
> How does GPT-4 achieve in-context learning without finetuning its parameters?

GPT-4 can use its ability to encode problems in PDDL and in-context learning to infer the problem PDDL file corresponding to a given problem (P). This can be done by providing the model with a minimal example that demonstrates what a correct problem PDDL looks like for a simple problem in the domain, as well as a problem description in natural language and its corresponding problem PDDL. This allows the model to leverage its ability to perform unseen downstream tasks without having to finetune its parameters.

[go to top]