The sad truth is that ChatGPT is about as good an AI as ELIZA was in 1966, it's just better (granted: much better) at hiding its total lack of actual human understanding. It's nothing more than an expensive parlor trick, IMHO.
Github CoPilot? Great, now I have to perform the most mentally taxing part of developing software, namely understanding other people's code (or my own from 6 months ago...) while writing new code. I'm beyond thrilled ...
So, no, I don't have an AI fatigue, because we absolutely have no AI anywhere. But I have a massive bullshit and hype fatigue that is getting worse all the time.
If writing boilerplate becomes effortless, then you'll write more of it, instead of feeling the pain of writing it and then trying to reduce it, because you don't want to spend time writing it.
And since Copilot was accepted as a way to help the developers on the teams, the increase of boilerplate have been immersive.
I'm borderline pissed, but mostly at our own development processes, not at Copilot per se. But damn if I didn't wish it existed somehow, although it was inevitable it would at one point.
What I have seen about it ranged from things that can be nearly just as well handled by your $EDITOR's snippet functionality to things where my argument kicked in - I have to verify this generated code does what I want, ergo I have to read and understand something not written by me. Paired with the at least somewhat legally and ethical questionable source of the training data, this is not for me.
However, in this case, it does seem that there is a level of fraudulence and deception. Given that “fake” often is used exactly the way you say, maybe “fake intelligence” would indeed be a more appropriate term.
[1] Synonyms of artificial has "faked" : https://www.thesaurus.com/browse/artificial
[2] Synonyms of fake has "artificial": https://www.thesaurus.com/browse/fake
But defining "intelligence" is a philosopical question that doesn't necessarily have one answer for everything and everyone.
I think there's an argument to be made that AI is being used here to help you tackle the more trivial tasks so you have more time to focus on the more important, and challenging tasks. Albeit I recognise GitHub CoPilot is legally questionable.
But yes, I agree with your overall point that AI has still not been able to 'think' like a human but rather can only still pretend to think like a human, and history has shown that users are often fooled by this.
Hype or not, it's incredibly useful and has increased my productivity by at least 20%. Worth every penny.
Lack of actual human understanding? Of course, by definition a machine will always lack human understanding. Why does that matter so much if it's a helpful tool?
For what it's worth, I do agree that there is a lot of hype. But contrary to blockchain, NFTs, web3, etc., this is actually useful for many people in many everyday use cases.
I see it as more similar to the dot com hype - buying a domain and creating a silly generic website didn't really multiply the value of your company as some people thought in that era, but that doesn't mean that websites weren't a useful technology with staying power, as time has shown.
But yes indeed, there are many, many AI products launched during this era of rapid progress. Even kind of shoddy products can be monetized if they provide value over what we had before. I think the crowded market and all the bullshit and all the awesome, all at once, is a sign of very rapid progress in this space. It will probably not always be like this and who knows what we are approaching.
Has it really? Or are you worried that this is something that will happen?
Of course I don't know how other people use it but I find that it's very much like having a fairly skilled pair programmer on board. I still need to do a lot of work but I get genuine help. I don't find that I personally write more boilerplate code than before, every programming principle applies as it always has.
ad.: Code review takes less time than writing code for the same reason reading a book takes less time than writing one. Distillation and organization of ideas requires expertise gained through experience and long thought. Reading a book requires reading ability.
Understanding a book (and the intricacies underlying it) takes effort on the order of the original writing, but most people don't seek that level of understanding. The same is true of code.
ChatGPT, when provided with a synthetic prompt is reliably a synthesizer, or to use the loaded term, a bullshiter.
When provided with an analytic prompt, it is reliably a translator.
Terms, etc: https://www.williamcotton.com/articles/chatgpt-and-the-analy...
In terms of closing the gap between AI hype and useful general purpose AI tools, no one can reasonably deny that it's an absolute quantum leap.
It's just not a daily driver for technical experts yet.
Why not get some of the freed up, Copilot augmented developer labor budget moved to testing and do more there or build more tools to make your personal, boilerplate, repetitive tasks more efficient?
If the coders are truly just dumping bad code your way, that's an externality and the cost should be called out.
Often I have times where I'm think about a specific piece of code that I need and I have it partially in my head and github copilot "just completes" it. I press tab and that's it.
I'm not talking about writing entire functions where you have to mentally strain yourself to understand what it wrote.
But I've never seen any autocompleter do it so good then github copilot. Even for documentation purposes like JSdoc and related commenting system it's amazing.
It's a tool I pay for now since it's proven to be a tool that increases my productivity.
Is it gonna replace us? I hope not, but it does look promising as one of those tools people will talk about in the future.
sounds like most people tbf
We're about six minutes away from "AI bros" becoming a thing.
The same kind of grifters who always latch onto the latest thing and hype it up in order to make a quick buck are already knocking on AI's door.
See also: Cryptocurrency, and Beanie Babies.
ChatGPT isn't as good as a human who puts in a lot of effort, but in many jobs it can easily outperform humans who don't care very much.
I suppose it makes sense though. Denial is the default response when we face threats to our identity and sense of self worth.
Copilot is amazing for reducing the tedium of typing obvious but lengthy code (and strings!). And it’s inline and passive; it’s not like you go edit -> insert -> copilot function and it dumps in 100 lines of code you have to debug. Which is what it sounds like parent is mistaking it for.
I’m reminded of 1995, when an elderly relative told me everything wrong with the internet based on TV news and not having ever actually seen the internet.
It you ask it to go through and comment code it does a pretty good job of that.
some things better than others(not that great at CSS)
need a basic definition of something. got it.
tell it to write a function it's not bad.
As a BA just tell it what your trying to do and what questions it should ask users. It will get some good ideas for you.
Want it to be a PM have create a loop asking every 10 minutes if your done yet.
Is it a senior engineer? no. can it pass a senior engineering interview? quite possibly.
debug code hit or miss.
I think the big thing it's not that great at front end code. It can't see so that probably makes sense. a fine-tuned version of clip that interacted with a browser would probably be pretty scary.
I agree.
And the worst thing is that the bullshit hype comes round every decade or so, and people run around like headless chickens insisting that "this time its different", and "this time its the REAL THING".
As you say, first(ish) there was ELIZA. Than this that and everything else. Then Autonomy and all that dot-com era jazz. Now with compute becoming more powerful and more compact, any man and his dog can stuff some AI bullshit where it doesn't belong.
I have seen comments below on this thread where people talk about "well, it's closing the gap". The thing you have to understand is that the gap will always exist. Ultimately you will always be asking a computer to do something. And computers are dumb. They are and will always be beholden to the humans that program them and the information that you feed them. The human will always have the upper hand at any tasks that require actual intelligence (i.e. thoughtful reasoning, adapting to rapidly changing events etc.).
I don't think there is any because there is no functional model for what organic intelligence is or how it operates. There are plethora of fascinating attempts / models but only a subset implore that it is solely "statistical". And even if it was statistical, the implementation of the wet system is absolutely not like a gigantic list of vectorized (stripped of their essence) tokens
Time will tell, I certainly can’t predict.
Think about it.
What's the most expressive medium we have which is also absolutely inundated with data?
To broadly be able to predict human speech you need to broadly be able to predict the human mind. To broadly predict a human mind requires you build a model of it, and to have a model of a human mind? Welcome to general intelligence.
We won't realize we've created an AGI until someone makes a text model, starts throwing random problems at it, and discovers that it's able to solve them.
See Scott Alexander for attempts to explain what is apparently impenetrable papers on it.
It's important to note that this is your assumption which I believe to be wrong (for most people here).
Co-pilot has been semi-useful. It's faster than search SO, but like you said, I still have to review all the code and it's often wrong in subtle ways.
Before this point in history we accepted 'I am that I am' because there wasn't any challenger to the title. Now that we are putting this to question we realize our definitions may not work well.
One simple example that I've had to reject more than once.
- Function 1 does something
- Developer needs something like Function 1 but minor change
- Developer starts typing name of function which has a similar name to Function 1, but again, minor difference
- Copilot helpfully suggests copy-pasting Function 1 but with the small change incorporated
- Developer accepts it, commits and sends the patch my way
Rather than extracting the common behavior into it's own function and call that from both of them, refactors which Copilot doesn't suggest, the developers is fine with just copy-pasting the function.
Now we have to maintain two full slightly different functions, rather than 1 full functions + 2 minor ones.
Obviously a small example, and it wouldn't be worth extracting it the first time it happens or on a smaller scale. But once you have entire teams doing something like this, it becomes a bit harder to justify copy-paste approach, especially when you want the codebase not to evolve to complete spaghetti.
And finally, I'm not blaming the tool, it's not Copilots fault. But it does seem to have made developers who rely on it think less, compared to the ones that don't.
We don't really understand what intelligence means -- in humans or our creations -- but ChatGPT gives us a little more insight (just like ELIZA, and the psychological research behind it, did).
At the very least, ChatGPT helps us build increasingly better Turing tests.
> These arguments take the form, "I grant you that you can make machines do all the things you have mentioned but you will never be able to make one to do X."
> [...]
> The criticisms that we are considering here are often disguised forms of the argument from consciousness, Usually if one maintains that a machine can do one of these things, and describes the kind of method that the machine could use, one will not make much of an impression.
Every time "learning machines" are able to do a new thing, there's a "wait, it is just mechanical, _real_ intelligence is the goalpost".
[0] https://www.espace-turing.fr/IMG/pdf/Computing_Machinery_and...
For _what purpose_, tho? It's a good party trick, but its tendency to be confidently wrong makes using it for anything important a bit fraught.
Given we are not talking about state changes in electrons, there is nothing wrong with this description of ChatGPT - it truly does feel like a massive advance to anyone who has even cursorily played with it.
For example, you can ask it questions like "Who was born first, Margaret Thatcher or George Bush?" and "Who was born first, Tony Blair or George Bush?" and in each instance it infers which George Bush you are talking about.
I honestly couldn't imagine something like this being this good only three years ago.
Which it occasionally mistypes. Then you're off to chase a small piece of error in a tub of boilerplate. Great stuff! For actual example, see [0]
[0] https://blog.ploeh.dk/2022/12/05/github-copilot-preliminary-...
That's about the only purpose I've found so far, but it seems a big one?
I'm sorry, what sort of bullshit argument is that ?
Flight and engines are both natural evolution using natural physics and mechanics.
Artificial Intelligence is nothing but a square-peg-round-hole, when you have a sledgehammer everything looks like a nut scenario.
I'll also throw random programming questions into it, and it's been hit and miss. SO is probably still faster, and I like seeing the discussion. The problem with chatGPT right now is it gives an answer like it's certainty when it's often wrong.
I can see the benefits of this interaction model (basically summarizing all the things from a search into what feels like a person talking back), but I don't see change the world level hype at the moment.
I also wonder if LLMs will get worse over time through propagation error as content is generated by other LLMs.
- Embedding free text data on safety observations, clustering them together, using text completion to automatically label the clusters, and identifying trends
- Embedding free text data on equipment failures. Some of our equipment failures have been classified manually by humans into various categories. I use the embeddings to train a model to predict those categories for uncategorized failures.
- Analyzing employee development goals and locating common themes. Then using this to identify where there are gaps we can fill in training offerings.
Intelligence may be a fuzzily defined word in everyday usage, but I don't think it's the mystery you present it to be. Joe public may argue against any and all definitions of the word that they personally disagree with (maybe just dislike), but it's nonetheless quite easy to come up with a straightforward and reductive definition if you actually want to!
It will turn out to be a useful tool for those who know what they’re asking about so they can check the answer quickly; but it will be USED by tons of people who don’t have a way of verifying the answers given.
I can't really imagine asking it a question about anything I cared about and not verifying via a second source, though, given its accuracy issues. This makes it feel a lot less useful.
start_value = get_*start_value(user_input)*
self.log.d*ebug(‘got start_value {start_value}’)*
And if it’s wrong, you just. . . keep typing. It’s autocomplete, just like IDEs have for other things. I’m kind of astounded that people have such an emotional reaction to an optional, low-key, passive, easily-ignored tool that sometimes saves a bunch of typing. Yes, if you always accept the suggestions you’ll have problems. Just like literally every other coding assistance tool.
This. To answer the OPs question, this is what I'm fatigued about.
I'm glad we're making progress. It's a hell of a parlor trick. But the hype around it is astounding considering how often it's answers are completely wrong. People think computers are magic boxes, and so we must be just a few lever pulls away from making it correct all the time.
Or maybe my problem is that I've overestimated the average human's intelligence. If you can't tell ChatGPT apart from a good con-man, can we consider the Turing test passed? It's likely time for a redefinition of the Turing test.
Instead of AI making machines smarter, it seems that computers are making humans dumber. Perhaps the AI revolution is about dropping the level of average human intelligence to match the level of a computer. A mental race to the bottom?
I'm reminded of the old Rod Serling quote: We're developing a new citizenry. One that will be very selective about cereals and automobiles, but won't be able to think.
It’s also plain that many people are very interested in looking inside the black box and think the contents of the black box are relevant and important. This fact doesn’t change just by your saying so.
Respectfully, that reads as needlessly combative within the context. It sounds like the blockchain proponents who say that the only people who are against cryptocurrencies are the ones who are “bitter for having missed the boat”.¹
It is possible and perfectly reasonable to identify problems in ChatGPT and similar technologies without feeling threatened. Simple example: someone who is retired and monetarily well off, whose way of living and sense of self worth are in no way affected by developments in AI, can still be critical and express valid concerns when these models tell you that it’s safe to boil a baby² or give other confident but absurdly wrong answers to important questions.
¹ I’m not saying that’s your intention, but consider that type of rhetoric may be counterproductive if you’re trying to make another understand your point of view.
² I passed by that specific example on Mastodon but I’m not finding it now.
Language is way, way far removed from intelligence. This is well-known in cognitive psychology. You'll find plenty of examples of stroke victims who are still intelligent but have lost the ability to produce coherent sentences, and (though much rarer) examples of people who can produce clear, eloquent prose, yet are so learning and mentally challenged that they can't even tell the difference between fantasy and reality.
ChatGPT is good at making up stories.
My point was that “consciousness” and “intelligence” are very different things. One does not imply the other.
Consciousness is about self reflection. Intelligence is about insight and/or problem solving. The two are often correlated, especially in animals, especially in humans, but they’re not the same thing at all.
“Is chatgpt consciousness” is a totally different question than “is chatgpt intelligent”.
We will know chatgpt is intelligent when it passes our tests of intelligence, which are imperfect but at least directionally correct.
I have no idea if/when we we know whether chatgpt is conscious, because we don’t really have good definitions of consciousness, let along tests, as you note.
Consciousness is a subjective experience (regardless of what you believe/understand to be responsible for that experience), so discussing "consciousness/intelligence" is rather like discussing "cabbages/automobiles".
Helping write boilerplate is to Copilot what cropping is to Photoshop.
Some of the ways I've found Copilot a powerful tool in my toolbox: Writing missing comments (especially unfamiliar code bases), "translating" parts of unfamiliar code to a more familiar language, suggesting ideas for how to implement a feature (!) in comments.
I cannot but think that this approach of "Strong Opinions, Weakly Held" is a much stronger path forward towards AGI than what we had before.
It is also obvious that we are in the middle of a shift of some kind. Very hard to see from within, but clearly we will look back at 2022 as the beginning of something…
Just because people shift the goalposts doesn't mean that the new position of the goalposts isn't closer to being correct than the old position. You can criticise the people for being inconsistent or failing to anticipate certain developments, but that doesn't tell you anything about where the goalposts should be.
This is a non sequitur. The human mind does a whole lot more than string words together. Being able to predict which word would logically follow another does not require the ability to predict anything other than just that.
The writing is on the wall. Programming as we know it is going to end. We should be embracing these tools and should start moving from software developers to software architects role.
Being able to define what you want to achieve isn't generally the same as knowing HOW to achieve it (except in this case the definition of intelligence rather does suggest the right path).
I came here to make this comment. Thank you for doing it for me.
I remember feeling shocked when this article appeared in the Atlantic in 2008, "Is Google Making Us Stupid?": https://www.theatlantic.com/magazine/archive/2008/07/is-goog...
The existence of the article broke Betteridge's law for me. The fact that this phenomenon it is not more widely discussed describes the limit of human intelligence. Which brings me back around to the other side... perhaps we were never as intelligent as we suspected?
(2) Counter. I asked it the other day "how many movies were Tom Hanks and Meg Ryan in together" and the answer ChatGPT gave was 2 ... not only is that wrong it is astonishingly wrong (IMO). You could be forgiven for forgetting Ithaca from 2015. I could forgive ChatGPT for forgetting that one. But You've Got Mail? That's a very odd omission. So much so I'm genuinely curious how it could possible get the answer wrong in that way. And for the record, Google presents the correct answer (4) in a cut out segment right at the top, a result and presentation very close to what one would expect from ChatGPT.
I don't know about other use cases like generating stories (or tangentially art of any kind) for inspiration, etc. But as a search engine things like ChatGPT NEED to have attributions. If I ask the question "Does a submarine appear in the movie Battlefield Earth?" it will confidently answer "no". I _think_ that answer is right, but I'm not really all that confident it is right. It needs to present the reasons it thinks that is right. Something like "No. I believe this because (1) the keyword submarine doesn't appear in the IMDb keywords (<source>), (2) the word submarine doesn't appear in the wikipedia plot synopsis (<source>), (3) the film takes place in Denver (<source>) which is landlocked making it unlikely a submarine would be found in that location during the course of the film."
The Tom Hanks / Meg Ryan question/answer would at least more interesting if it explained how it managed to be so uniquely incorrect. That question will haunt me though ... there's some rule about this right? Asking about something you have above average knowledge in and watching someone confidently answer it incorrectly. How am I supposed to ever trust ChatGPT again about movie queries?
I think "human level intelligence" is an emergent phenomenon arising from a variety of smaller cognitive subsystems working together to solve a problem. It does seem that ChatGPT and similar models have at least partially automated one of the subsystems in this model. Still, it can't reason, doesn't know it's wrong, and can't lie because it doesn't understand what a lie is. So it has a long way to go. But it's still real progress in the sense that it's allowing us to better see the dividing lines between the subsystems that make up general intelligence.
I think that we'll need to build a better systems level model of what general intelligence is and the pieces it's built out of. With a better defined model, we can come up with better tests for each subsystem. These tests will replace the Turing test.
One of major problems of modern computer-based work is that there are too many people already in those roles, doing work that isn't needed. Case in point: the culling of tens of thousands of software engineers, people who would consider themselves to be doing 'bullshit jobs'.
> I was thinking more like:
That example is straight up from any of those "programming is not bound by typing speed" essays of yore.
> people have such an emotional reaction to an optional, low-key, passive, easily-ignored tool that sometimes saves a bunch of typing.
Maybe because it's not generally advertised by proponents as "an optional, low-key, passive, easily-ignored tool that sometimes saves a bunch of typing"? Just look at the rest of the thread, it's pronounced as a game-changer in productivity.
That reminds me how in my youth many were planning on vacations to Mars resorts and unlimited fusion energy) Stars looked so close, only a matter of time!
It is often frustrating that English has words with such different (but clearly related) definitions, as it can make it far too easy to end up talking past each other.
I can see how someone who’s always working on sophisticated, mentally challenging code would get less benefit and would see more frequent errors.
Yeah, I think you're right. Intelligence is just something our species has evolved as a strategy for survival. It isn't about intelligence, it's about survival.
The cognitive skills needed to survive/navigate/thrive in the digital era are very different than the cognitive skills required to survive in the pre-digital era.
We're biologically programmed through millions of years of evolution to survive in a world of scarcity. Intelligence used to be about tying together small bits of scarce information to find larger patterns so that we can better predict outcomes.
Those skills are being rendered more and more irrelevant in a world of information abundance. Perhaps the "best fit" humans of the future are those that possess new form of "intelligence", relying less on reason and more on the ability to quickly digest the firehose of data thrown at them 24-7.
If so, then the AI we were trying to build in the 1950s would necessarily be different than the AI that our grandchilden would find helpful.
> Being able to predict which word would logically follow another does not require the ability to predict anything other than just that.
Why? Wouldn't you expect that technique to generally fail if it isn't intelligent enough to know what's happening in the sentence?
This seems to have been the rallying cry of AI-ish stuff for the past 30 years, tho. At a certain point you have to ask "but how much time"? Like, a lot of people were confidently predicting speech recognition as good as a human's from the 90s on, for instance. It's 2023, and the state of the art in speech recognition is a fair bit better than Dragon Dictate in the 90s, but you still wouldn't trust it for anything important.
That's not to say AI is useless, but historically there's been a strong tendency to say, of AI-ish things "it's 95% of the way there, how hard could the last 5% be?" The answer appears to be "quite hard, actually", based on the last few decades.
As this AI hype cycle ramps up, we're actually simultaneously in the down ramp of _another_ AI hype cycle; the 5% for self-driving cars is going _very slowly indeed_, and people seem to have largely accepted that, while still predicting that the 5% for generative language models will be easy. It's odd.
(Though, also, I'm not convinced that it _is_ just a case of making a better ChatGPT; you could argue that if you want correct results, a generative language model just isn't the way to go at all, and that the future of these things mostly lies in being more convincingly wrong...)
I forgot to add something to my original post. >>"I remember feeling shocked when this article appeared in the Atlantic in 2008..."
At the time I was shocked that the question was even being asked!
The expressiveness of language lets this be true of almost everything.
Well, I'm no fan of chatGPT. But it appears most people are worse than chatGPT, because just regurgitate what they hear with no thought or contemplation. So you can't really blame average folks who struggle with the concepts of intelligence/understanding that you mention.
https://twitter.com/marvinvonhagen/status/162365814434901197...