There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person does. But I'm finding that these Gemini models are finally able to replace searches and stackoverflow for a lot of my day-to-day programming.
so it's a great tool in the hands of a creative architect, but it is not one in and by itself and I don't see yet how it can be.
my pet theory is that the human brain can't understand and formalize its creativity because you need a higher order logic to fully capture some other logic. I've been contested that the second Gödel incompleteness theorem "can't be applied like this to the brain" but I stubbornly insist yes, the brain implements _some_ formal system and it can't understand how that system works. tongue in cheek, somewhat, maybe.
but back to earth I agree llms are a great tool for a creative human mind.
I would argue that the second incompleteness theorem doesn't have much relevance to the human brain, because it is trying to prove a falsehood. The brain is blatantly not a consistent system. It is, however, paraconsistent: we are perfectly capable of managing a set of inconsistent premises and extracting useful insight from them. That's a good thing.
It's also true that we don't understand how our own brain works, of course.