zlacker

>>meetpa+(OP)
My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried.

There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person does. But I'm finding that these Gemini models are finally able to replace searches and stackoverflow for a lot of my day-to-day programming.

>>segpha+J4
Making LLMs know what they don't know is a hard problem. Many attempts at making them refuse to answer what they don't know caused them to refuse to answer things they did in fact know.

>>redox9+na
> Many attempts at making them refuse to answer what they don't know caused them to refuse to answer things they did in fact know.

Are we sure they know these things as opposed to being able to consistently guess correctly? With LLMs I'm not sure we even have a clear definition of what it means for it to "know" something.

>>Volund+hk
> Are we sure they know these things as opposed to being able to consistently guess correctly?

What is the practical difference you're imagining between "consistently correct guess" and "knowledge"?

LLMs aren't databases. We have databases. LLMs are probabilistic inference engines. All they do is guess, essentially. The discussion here is about how to get the guess to "check itself" with a firmer idea of "truth". And it turns out that's hard because it requires that the guessing engine know that something needs to be checked in the first place.

>>ajross+Ar
> What is the practical difference you're imagining between "consistently correct guess" and "knowledge"?

Knowing it's correct. You've just instructed it not to guess remember? With practice people can get really good at guessing all sorts of things.

I think people have a serious misunderstanding about how these things work. They don't have their training set sitting around for reference. They are usually guessing. Most of the time with enough consistency that it seems like they "know'. Then when they get it wrong we call it "hallucinations". But instructing then not to guess means suddenly they can't answer much. There no guessing vs not with an LLM, it's all the same statistical process, the difference is just if it gives the right answer or not.

>>Volund+vZ
Maybe LLM's know so much that it makes it difficult to feel the absence. When someone asks me about the history of the Ethiopian region, I can at most recall very few pieces of information, and critically, there is an absence of feelings of familiarity. In memory research, familiarity signal is can prompt continued retrieval attempts, but importantly absence of that signal can suggest that further retrieval attempts would be fruitless, and that you know that you don't know. Maybe knowing so much means that there is near saturation for stop tokens...or that llms need to produce a familiarity like scoring of the key response of interest.