zlacker

LLM with Planning

submitted by mercat+(OP) on 2023-04-27 22:34:16 | 81 points 16 comments
[view article] [source] [go to bottom]

NOTE: showing posts with links only show all posts
◧◩
4. willia+65[view] [source] [discussion] 2023-04-27 23:12:36
>>behnam+64
Here’s an example of a question-and-answer augmentation written in TypeScript that uses the exact approach used by PAL, et al:

https://github.com/williamcotton/transynthetical-engine

7. coders+td[view] [source] 2023-04-28 00:32:24
>>mercat+(OP)
Summary:

The paper introduces LLM+P, a framework that combines the strengths of classical planners with large language models (LLMs) to solve long-horizon planning problems. LLM+P takes in a natural language description of a planning problem, converts it into a PDDL file, leverages classical planners to find a solution, and then translates the solution back into natural language. The authors provide a set of benchmark problems and find that LLM+P is able to provide optimal solutions for most problems, while LLMs fail to provide even feasible plans for most problems. The paper suggests that LLM+P can be used as a natural language interface for giving tasks to robot systems. The authors also propose that classical planners can be another useful external module for improving the performance of downstream tasks of LLMs. The paper highlights the importance of providing context (i.e., an example problem and its corresponding problem PDDL) for in-context learning, and suggests future research directions to further extend the LLM+P framework.

PPDL: https://en.wikipedia.org/wiki/Planning_Domain_Definition_Lan...

◧◩
12. YeGobl+Vw1[view] [source] [discussion] 2023-04-28 13:50:08
>>i-use-+Bf
>> Planning, yes, but that’s a verb that casts a very wide net.

The article uses "planning" to mean "classical planning", which is a very specific thing, although it's such a fundamental concept in AI research that it is very difficult to find a simple definition (there's a lot of useless stuff on the internet about it, like tutorials that don't explain what it is they're tutorial-ing, or slides that don't give much context).

Even the Wikipedia article is not very well written. I followed this link to one of its references though and there's an entire textbook, available as a free pdf:

https://projects.laas.fr/planning/

In general, classical planning is one of those domains where GOFAI approaches continue to dominate over nouveau AI, statistical machine learning-based approaches. You'll have to take my word for that, though, because that's what I know from experience, and I don't have any references to back that up. On the other hand, if it wasn't the case, you wouldn't see papers like the one linked above, I suppose.

To clarify, the paper above makes it clear that LLMs, for one, are useless for planning but at least they can translate between natural language and PDDL, so that a planning problem can be handed off to a classical planning engine, that can actually do the job. How useful is that, I don't know. A human expert would probably do a better job of writing PDDL from scratch, but that's never explored in the linked article.

◧◩
13. YeGobl+Vx1[view] [source] [discussion] 2023-04-28 13:57:19
>>PaulHo+81
Yes, it seems like casting story-generation as a planning problem was a standard approach, at least in the recent past (I'm guessing everyone is turning to LLMs now):

Story planners start with the premise that the story generation process is a goal-driven process and apply some form of symbolic planner to the problem of generating a fabula. The plan is the story.

https://thegradient.pub/an-introduction-to-ai-story-generati...

As an aside, it is obvious from that The Gradient article I link above that story generation was doing just fine until LLMs came along and claimed they can do it right for the first time ever. I can see that the earlier approaches took some careful hand-engineering, but they also seemed to more reliably generate coherent stories that made sense (although it looks like maybe they didn't have very rich themes and development etc). But then, that's the trade-off you get between classical approaches and big machine learning: either you have to roll up those sleeves and use some elbow grease, or you have to label giant reams of data and pay the giant price of compute needed to train on them. In a sense, the claimed advance of deep learning is that domain experts can be replaced by cheaply paid inexpert labellers, plus some very big GPU clusters.

16. sitkac+CO8[view] [source] 2023-05-01 05:03:15
>>mercat+(OP)
There is a link in the paper to an introduction in PDDL which is incorrect, it should point to https://www.cs.toronto.edu/~sheila/2542/s14/A1/introtopddl2....
[go to top]