I do see LLMs as potentially more useful for “fanciful” queries, like “what can I make with kale, tomatoes, and mushrooms?”
I think that encyclopedias are cool, but I also feel like the Internet was hopefully going to be a slightly better version of that and that it's a little frustrating to be going in the opposite direction. I'm not sure how to articulate that other than that encyclopedias are in some ways a compromise around the fact that we very often don't have good search, so we accept human beings trying to pre-aggregate data for us with the hope that they are better than Google is at aggregating that data.
And I think Wikipedia is valuable not so much because of the summaries, and more because it's obsessively curated and has (or attempts to have) a very specific, predictable set of rules that it (tries) to adhere to about sources and coverage. The text-portion of Wikipedia isn't really the part that I think is most impressive about it. If GPT-3 was being used to aggressively curate search results and remove low-quality content, then yeah, that would be potentially interesting (although I'm not sure how well it could handle that task).
----
> what can I make with kale, tomatoes, and mushrooms?
I sort of see it, it's one of the more... it's fine. It wouldn't be a strict downgrade over existing search, maybe it would save time in some situations. But if I'm being completely honest, what I want as an answer to that question isn't a paragraph of text explaining multiple recipes, it's a bullet-pointed list of recipes with links to the original sites they're listed on, so I can check to see if they're worth making.
Bing's results (to its credit) seem like they're sort of headed in that direction, which, nice. Yeah, I could use that. But a bullet-pointed list of recipes is also... search results. So how much time and effort have we put into reinventing a cleaner search interface where at best it solves the same problems that search already solved, and where we in the meanwhile haven't made any progress on the really hard fundamental problem of "how do we get a good list of recipes to display in the first place and what does 'good' mean in that context?"
----
To be less cynical, one way I could see this genuinely being useful would be completely behind the scenes in a non-user-facing role just as a way of applying "tags" to webpages or doing filtering. I would love to be able to search "is spinach poisonous for rats who are pets and just because I used the word 'poisonous' does not mean I want 2 pages of links about exterminators or getting rid of wild rats."
An LLM would be a great fit for that, because we've done such a garbage job of categorizing the web that it's very difficult to know which words to type to exclude "categories" of information from a search query. So maybe an LLM helps standardize that a bit? But then I want a normal page of search results. I don't want to have a conversation with the LLM, I want the LLM's role to be exclusively "I think this webpage should be additionally included/excluded in your query. I think this webpage is about extermination even if it doesn't use that specific word."
If Bing's service ends up being able to do that kind of thing well, then yeah, that's useful. I'm a little skeptical they can, because... gestures to the current quality of search results, but maybe their integration with GPT gives them way more capabilities and ratchets up their quality.
But similar to above, it feels a little bit like reinventing the wheel. I can refine GPT queries during a conversation, great. Could I have that feature for regular search? Why do I need to do it as part of a conversation? That seems like a good thing tied to a bad UX (although again, I might be atypical in thinking that natural language conversations are often bad UX).
And I do want to couch that by saying that maybe Bing will surprise me and it will have easy options to do that kind of thing. I'm just not currently seeing it presented in a way that looks useful.