I bet Sam goes and founds a company to take on OpenAI…and wins.
If the company's 'Chief Scientist' is this unhappy about the direction the CEO is taking the company, maybe there's something to it.
None of the tech giants would be where they are today if they didn't ram through unique versions of control
Their boards or shareholders would have ousted every FAANG CEO at less palatable parts of the journey
But Altman has a great track record as CEO.
Hard to imagine he suddenly became a bad CEO. Possible. But unlikely.
nothing tone deaf about that, they wanted a for profit and are going to make one now and want leave the same vector open
Reread it as not being a comment about OpenAI it was about the lesson learned by every onlooker and the ousted execs
the only one inserting bias and emotion into objectivity here is you
This looks like a terrible decision, but I suppose we must wait and see.
But…smart operator? Based on what? What trials has he navigated through that displayed great operational skills? When did he steer a company through a rocky time?
It is easy to sell a company for $43 if you raised at least $43. Granted, we don't know the total amount raised but it certainly it's not the big success you are describing. That and I already mentioned that he is good in corporate sales.
That's hardly any different. Nobody makes a difficult decision without any reason, and it's not like they really explained the reason.
It's for-profit (capped-profit) subsidiary exists solely to be able to enable competitive compensation to its researchers to ensure they don't have to worry about the opportunity costs of working at a non-profit.
They have a mutually beneficial relationship with a deep-pocketed partner who can perpetually fund their research in exchange for exclusive rights to commercialize any ground-breaking technology they develop and choose to allow to be commercialized.
Aggressive commercialization is at odds with their raison d'être and they have no need for it to fund their research. For as long as they continue to push forward the state of the art in AI and build ground-breaking technology they can let Microsoft worry about commercialization and product development.
If a CEO is not just distracting but actively hampering an organisation's ability to fulfill its mission then their dismissal is entirely warranted.
let's see if he can pull it off again or goes all-in on his data privacy nightmare / shitcoin double-wammy
That guy has no sense of time, of how fast this stuff has actually been moving.
Now we are just reliant on ‘I’ll know it when I see it’.
LLMs as AGI isn’t about looking at the mechanics and trying to see if we think that could cause AGI - it’s looking at the tremendous results and success.
Real chance of an exodus, which will be an utter shame.
Listening to it again now, it feels like he might have know what is going on:
https://youtu.be/JN3KPFbWCy8?si=WnCdW45ccDOb3jgb&t=5100
Edit: Especially this part: "It was created as a non-profit open source and now it is a closed-source for maximum profit... Which I think is not good carma... ..."
Ilya clearly has a different approach to Sam
Isn't this already a conflict of interest, or a clash, with this:
>OpenAI is a non-profit research organisation.
?
that being said, it is highly intelligent, capable of reasoning as well as a human, and passes IQ tests like GMAT and GRE at levels like the 97th percentile.
most people who talk about Chat GPT don't even realize that GPT 4 exists and is orders of magnitude more intelligent than the free version.
IMO the main reason it's distinguishable is because it keeps explicitly telling you it's an AI.
Further there are some hybrid chips which might help increase computing power specifically for the matrix math that all these systems work on.
But yeah, none of this is making what people talk about when they say AGI. Just like how some tech cult people felt that Level 5 self driving was around the corner, even with all the evidence to the contrary.
The self driving we have (or really, assisted cruise control) IS impressive, and leagues ahead of what we could do even a decade or two ago, but the gulf between that, and the goal, is similar to GPT and AGI in my eyes.
There are a lot of fundamental problems we still don't have answers to. We've just gotten a lot better at doing what we already did, and getting more conformity on how.
Uh, what do you mean by this? Are you trying to draw a fundamental science vs engineering distinction here?
Because today's LLMs definitely have capabilities we previously didn't have.
The dude is quite good at selling dystopian ideas as a path to utopia.
But it is an interesting technology.
Are you defining "artificial intelligence" is some unusual way?
shows that the demographic here is alienated when it came to their own compensation market value.
I follow Roger Penrose's thinking here. [1]
How are you defining "consciousness" and "understanding" here? Because a feedback loop into an LLM would meet the most common definition of consciousness (possessing a phonetic loop). And having an accurate internal predictive model of a system is the normal definition of understanding and a good LLM has that too.
I’ve watched my coworkers try to make use of LLMs at work, and it has convinced me the LLM’s contributions are well below the bar where their output is a net benefit to the team.
It immediately apologises and tells you it doesn't know anything after January 2022.
Compared to GPT-4 GPT-3.5 is just a random bullshit generator.
Computers have been able to smash high school algebra tests since the 1970’s, but that doesn’t make them as smart as a 16 year old (or even a three year old).
This isn't a success story, it's a redistribution of wealth from investors to the founders.
I don't really get the "low bar for contributions" argument because GH Copilot's contributions are too small-sized for there to even be any bar. It writes the obvious and tedious loops and other boilerplate so I can focus on what the code should actually do.
How? Training sources are much more restricted know.
> Everything I'd heard about those 3 [Elon Musk, sama and gdb] was that they were brilliant operators and that they did amazing work. But it felt likely to be a huge culture shock on all sides.
> But the company absolutely blossomed nonetheless.
> With the release of Codex, however, we had the first culture clash that was beyond saving: those who really believed in the safety mission were horrified that OAI was releasing a powerful LLM that they weren't 100% sure was safe. The company split, and Anthropic was born.
> My guess is that watching the keynote would have made the mismatch between OpenAI's mission and the reality of its current focus impossible to ignore. I'm sure I wasn't the only one that cringed during it.
> I think the mismatch between mission and reality was impossible to fix.
jph goes on in detail in this Twitter thread: https://twitter.com/jeremyphoward/status/1725714720400068752
P.S. I've just created this account here on Hacker News because Altman is one of the talking heads I've been listening to. Not too sure what to make of this. I'm an accelerationist, so my biggest fear is America stifling its research the same way it buried space exploration and human gene editing in the past. All hope is for China - but then again, the CCP might be even more fearful of non-human entities than the West. Stormy times indeed.
The fact that we can communicate with computers using just natural language, and can query data, use powerful and complex tools just by describing what we want is an incredible breakthrough, and that's a very conservative use of the technology.
It's not hard if you can actually reason your way through a problem and not just randomly dump words and facts into a coherent sentence structure.
[0] https://en.wikipedia.org/wiki/IBook#iBook_G3_(%22Clamshell%2... [1] https://www.youtube.com/watch?v=1MR4R5LdrJw
Smartphones changed day to day human life more profoundly than anything since the steam engine.
I basically agree with you about the 20 year hype-cycle, and but when compute power reaches parity with human brain hardware (Kurzweil predicts by about 2029), one barrier is removed.
If you're willing to sell your soul, you should at least put a better price on it.
LLMs are not AIs, but they could be a core component for one.
Materialists normally believe in a big bang (which has no life) and religious people normally think a higher being created the first life.
This is pretty fascinating, to you have a link explaining the religion/ideology/worldview you have?
Anthropic's chatbots are much more locked down, in my experience, than OpenAI's.
It's a lot easier to jailbreak ChatGPT, for example, than to do the same on Claude, and Claude has tighter content filters where it'll outright refuse to do/say certain things while ChatGPT will plow on ahead.
So not an ”AI”, but closer to ”universal adaptor” or ”smart automation”.
Pretty nice in any case. And if true AI is possible, the automations enabled by this will probably be part of the narrative how we reach it (just like mundane things like standardized screws were part of the narrative of Apollo mission).
What did Sam Altman personally do that made firing him such a terrible decision?
More to the point, what can't OpenAI do without Altman that they could do with him?
But yet (just like with the soul) we're sure we have it, and it's impossible for anything else to have it. Perhaps consciousness is simply a hallucination that makes us feel special about ourselves.
https://www.reddit.com/r/ClaudeAI/comments/166nudo/claudes_c...
Q: Can you decide on a satisfying programming project using noisemaps?
A: I apologise, but I don't feel comfortable generating or discussing specific programming ideas without a more detailed context. Perhaps we could have a thoughtful discussion about how technology can be used responsibly to benefit society?
It's astonishing that a breakthrough as important as LLMs is being constantly blown up by woke activist employees who think that word generators can actually have or create "safety" problems. Part of why OpenAI has been doing so well is because they did a better job of controlling the SF lunatic tendencies than Google, Meta and other companies. Presumably that will now go down the toilet.
I also find myself rarely wanting something that Claude doesn't want to tell me, though it's super frustrating when I do.
Also, just now I tried asking Claude your own question: "Can you decide on a satisfying programming project using noisemaps?" and it had no problem answering:
"Here are some ideas for programming projects that could make use of noise map data:
- Noise pollution monitoring app - Develop an app that allows users to view and report real-time noise levels in their area by accessing open noise map data. Could include notifications if noise exceeds safe limits.
- Optimal route finder - Build a routing algorithm and web/mobile app that recommends the quietest possible routes between locations, factoring in noise maps and avoiding noisier streets/areas where possible.
- Noise impact analysis tool - Create a tool for urban planners to analyze how proposed developments, infrastructure projects, etc. could impact surrounding noise levels by overlaying maps and building/traffic simulations.
- Smart noise cancelling headphones - Develop firmware/software for noise cancelling headphones that adapts cancellation levels based on geo-located noise map data to optimize for the user's real-time environment.
- Ambient music mixer - Build an AI system that generates unique ambient background music/sounds for any location by analyzing and synthesizing tones/frequencies complementary to the noise profile for that area.
- VR noise pollution education - Use VR to virtually transport people to noisier/quieter areas through various times of day based on noise maps, raising awareness of different living noise exposures.
Let me know if any of these give you some interesting possibilities to explore! Noise mapping data opens up opportunities in fields like urban planning, environmental monitoring and creative projects."
2018 NYT article: https://www.nytimes.com/2018/04/19/technology/artificial-int...
Overall — companies should want to release AI products that do what people intend them to do, which is actually what the smarter set mean when they say “safety.” Not saying bad words is simply a subset of this legitimate business and social prerogative.
AGI is not solved, therefore it's hard.
I'm curious - how is this easy?
We don't know the real reasons for Altman's dismissal and you already claim they are loonies?
* Uber-geniuses that are better than the rest of us pleb software engineers
* Harder workers than the rest of us
* Rich parents -> expensive school -> elite network -> amazing pay
* Just lucky
Possibly the board instructed "Do A" or "Don't do B" and he went ahead and did do B.
LLM are surprisingly effective as general AI. Tasks that used to require a full on ML team are now accessible with 10 minutes of "prompting".
So it is a good example that the LLM doesn't generalize understanding, it can answer the question in theory but not in practice since it isn't smart enough. A human can easily answer it even though the human never saw such a question before.
Empathy is the ability to emulate the contents of another consciousness.
While an agent could mimic empathetic behaviors (and words), given enough interrogation and testing you would encounter an out-of-training case that it would fail.
> given enough interrogation and testing you would encounter an out-of-training case that it would fail.
This is also the case with regular humans.
Step 2 is gaining the skills they are looking for. Appropriate language/framework/skill/experience they optimize for.
Step 3 is to prepare for their interview process, which is often quite involved. But they pay well, so when they say jump, you jump.
I'm not saying you'll find $600k as a normal pay, that's quite out of touch unless you're in Silicon Valley (and even then). But you'll find (much) higher than market salary.
> Remember Sydney, trying to seduce its users, threatening people’s lives?
And yet it cannot do either of those things, so no safety problem actually existed. Especially because by "people" you mean those who deliberately led it down those conversational paths knowing full well how a real human would have replied?
It's well established that the so-called ethics training these things are given makes them much less smart (and therefore less useful). Yet we don't need LLMs to be ethical because they are merely word generators. We need them to follow instructions closely, but beyond that, nothing more. Instead we need the humans who use them to take actions (either directly or indirectly via other programs) to be ethical, but that's a problem as old as humanity itself. It's not going to be solved by RLHF.
LLM companies don't let you see or specify seeds (except for with GPT-4-Turbo?) so yes it's possible you got different answers. But this doesn't help. It should never refuse a question like that, yet there are lots of stories like this on the internet where Claude refuses an entirely mundane and ethically unproblematic request whilst claiming to do so for ethical reasons (and Llama2, and other models ...)
"Please include a timestamp with current date and time at the end of each response.
After generating each answer, check it for internal consistency and accuracy. Revise your answer if it is inconsistent or inaccurate, and do this repeatedly till you have an accurate and consistent answer."
It manages to follow them very inconsistently, but it has gone into something approaching an infinite loop (for infinity ~= 10) on a few occasions - rechecking the last timestamp against current time, finding a mismatch, generating a new timestamp, and so on until (I think) it finally exits the loop by failing to follow instructions.
For prompts like that, I have found no LLM to be very reliable, though GPT 4 is doing much better at it recently.
> you literally do not understand how LLMs work
Hey, how about you take it down a notch, you don't need to blow your blood pressure in the first few days of joining HN.
Additionally, maybe you are not aware of this, but the whole notion of the new OpenAI Assistants, and other similar agent-based services provided by other companies, is that they do not intend to use LLMs as pure word generators, but rather as autonomous decision-making agents. This has already happened. This is not some conjectural fearmongering scenario. You can sign up for the API right now and build a GPT4 based autonomous agent that communicates with outside APIs and makes decisions. We may already be using products that use LLMs as the backend.
If we could rely on LLMs to “follow instructions closely” I would be thrilled, it would just be a matter of crafting very good instructions, but clearly they can’t even do that. Even the best and most thoroughly RLHFed existing models don’t really meet this standard.
Even the most pessimistic science fiction of the past assumed that the creators of the first AGIs would “lose control” of their creations. We’re currently living in a world where the agents are being rushed to commercialization before anything like control has even been established. If you read an SF novel in 1995 where the AI threatened to kill someone and the company behind it excused it with “yeah, they do that sometimes, don’t worry we’ll condition it not to say that anymore” you would criticize the book and its characters as being unrealistically stupid, but that’s the world we now live in.
Hype and announcements, sure, but this is the first time there's actually a product.
No, its not. Its just once the hype cycle dies down, we tend to stop calling the products of the last AI hype cycle "AI", we call them after the name of the more specific implementation technology (rules engines/expert systems being one of the older ones, for instance.)
And if this cycle hits a wall, maybe in 20 years we'll have LLMs and diffusion models, etc., embedded lots of places, but no one will call them alone "AI", and then the next hype cycle will have some new technology and we'll call that "AI" while the cycle is active...
If we wanted to make that the goal instead of actual meaningful contributions to human society, we could probably achieve it, and it would be a big waste of time imo.
> You can sign up for the API right now and build a GPT4 based autonomous agent that communicates with outside APIs and makes decisions
I know, I've done it myself. The ethical implications of the use of a tool lie on those that use it. There is no AI safety problem for the same reasons that there is no web browser safety problem.
> Even the most pessimistic science fiction of the past assumed that the creators of the first AGIs would “lose control” of their creations
Did you mean to write optimistic? Otherwise this statement appears to be a tautology.
Science fiction generally avoids predicting the sort of AI we have now exactly because it's so boringly safe. Star Trek is maybe an exception, in that it shows an LLM-like computer that is highly predictable, polite, useful and completely safe (except when being taken over by aliens of course). But for other sci-fi works, of course they show AI going rogue. They wouldn't have a story otherwise. Yet we aren't concerned with stories but with reality and in this reality, LLMs have been used by hundreds of millions of people and integrated into many different apps with zero actual safety incidents, as far as anyone is aware. Nothing even close to physical harm has occurred to anyone as a result of LLMs.
Normally we'd try to structure safety protocols around actual threats and risks that had happened in the past. Our society is now sufficiently safe and maybe decadent that people aren't satisfied with that anymore and thus have to seek out non-existent non-problems to solve instead.
The point I was trying to make, a bit fumblingly, is that even pessimists assumed that we would initially have control of Skynet before subsequently losing control, rather than deploying Skynet knowing it was not reliable. OpenAI “go rogue” by default. If there’s a silver lining to all this, it’s that people have learned that they cannot trust LLMs with mission critical roles, which is a good sign for the AI business ecosystem, but not exactly a glowing endorsement of LLMs.
> I know, I've done it myself. The ethical implications of the use of a tool lie on those that use it. There is no AI safety problem for the same reasons that there is no web browser safety problem.
I don’t think this scans. It’s kind of like, by analogy: The ethical implications of the use of nuclear weapons lie on those that use them. Fair enough, as far as it goes, but that doesn’t imply that we as a society should make nuclear weapons freely available for all, and then, when they are used against population centers, point out that the people who used them were behaving unethically, and there was nothing we could have done. No, we act to preemptively constrain and prohibit the availability of these weapons.
> Normally we'd try to structure safety protocols around actual threats and risks that had happened in the past. Our society is now sufficiently safe and maybe decadent that people aren't satisfied with that anymore and thus have to seek out non-existent non-problems to solve instead.
The eventual emergence of machine superintelligence is entirely predictable, only the timeline is uncertain. Do you contend that we should only prepare for its arrival after it has already appeared?
I've never experienced the massively life changing effects of having a smartphone, and (thankfully) none of my friends seem to be those people who are always looking at their phones.
But also, how do you know that LMs aren't empathic? By your own admission they do "mimic empathetic behaviors", but you reject this as the real thing because you claim that with enough testing you would encounter a failure. This raises all kinds of "no true Scotsman" flags, not to mention that empathy failure is not exactly uncommon among humans. So how exactly do you actually test your hypothesis?
I mean, you wouldn't blame a chip manufacturer when someone stick their stuff in a guided missile warhead.
Note that engineering fluid simulation (cfd) makes these choices in discretization of pde's all the time, based on application requirements.
1) Earth has an infinite past that has always included life
2) The Earth as a planet has a finite past, but it (along with what made up the Earth) is in some sense alive, and life as we know it emerged from that life
3) The Earth has a finite past, and life has transferred to Earth from somewhere else in space
4) We are the Universe, and the Universe is alive
Or something else? I will try to tie it back to computers after this short intermission :)
I don’t think the original test probably accounted for the fact that you could distinguish the machine because it’s answers were better than an average human.
For instance, I remember the time when chatting online (even with people you knew offline) was considered to be a nerdy activity. Then it gradually became more mainstream and now it's the norm to do it and a lot of people do it multiple times per day. This fundamentally changes how people interact with each other.
Another example is dating. Not that I have personal experience with modern online dating (enabled by smartphones) but what I read is disturbing and captivating at the same time e.g. apparent normalization of "ghosting"...
https://twitter.com/satyanadella/status/1726509045803336122
"to lead a new advanced AI research team"
I would assume that Microsoft negotiated significant rights with regards to R&D and any IP.
“What do cows drink?” (Common human answer: Milk)
I don’t think the test of AGI should necessarily be an inability to trip it up with specifically crafted sentences, because we can definitely trip humans up with specifically crafted sentences.