Idk. Folks much smarter than I seem worried so maybe I should be too but it just seems like such a long shot.
So yes, the insiders very likely know a thing or two that the rest of us don’t.
The most obvious reason is costs - if it costs many millions to train foundation models, they don't have a ton of experiments sitting around on a shelf waiting to be used. They may only get 1 shot at the base-model training. Sure productization isn't instant, but no one is throwing out that investment or delaying it longer than necessary. I cannot fathom that you can train an LLM at like 1% size/tokens/parameters to experiment on hyper parameters, architecture, etc and have a strong idea on end-performance or marketability.
Additionally, I've been part of many product launches - both hyped up big-news-events and unheard of flops. Every time, I'd say that 25-50% of the product is built/polished in the mad rush between press event and launch day. For an ML Model, this might be different, but again see above point.
Sure products may be planned month/years out, but OpenAI didn't even know LLMs were going to be this big a deal in May 2022. They had GPT-2 and GPT-3 and thought they were fun toys at that time, and had an idea for a cool tech demo. I think that OpenAI (and Google, etc) are entirely living day-to-day with this tech like those of us on the outside.
The vast majority of datacenters currently in production will be entirely powered by carbon free energy. From best to worst:
1. Meta: 100% renewable
2. AWS: 90% renewable
3. Google: 64% renewable with 100% renewable energy credit matching
4. Azure: 100% carbon neutral
[1]: https://sustainability.fb.com/energy/
[2]: https://sustainability.aboutamazon.com/products-services/the...
[3]: https://sustainability.google/progress/energy/
[4]: https://azure.microsoft.com/en-us/explore/global-infrastruct...
IMO, we should pause this for now and put these resources (human and capital) towards reducing the impact of global warming.
What are your timelines here? "Catastrophic" is vague but I'd put the climate change meaningfully affecting the quality of life of average westerner at end of century, while AGI could be before the middle of the century.
If imaginary cloud provider "ZFQ" uses 10MW of electricity on a grid and pays for it to magically come from green generation, that means 10MW of other loads on the grid were not powered by green energy, or 10MW of non-green power sources likely could have been throttled down/shut down.
There is no free lunch here; "we buy our electricity from green sources" is greenwashing bullshit.
Even if they install solar on the roofs and wind turbines nearby - that's still electrical generation capacity that could have been used for existing loads. By buying so many solar panels in such quantities, they affect availability and pricing of all those components.
The US, for example, has about 5GW of solar manufacturing capacity per year. NVIDIA sold half a million H100 chips in one quarter, each of which uses ~350W, which means in a year they're selling enough chips to use 700MW of power. That does not include power conversion losses, distribution, cooling, and the power usage of the host systems, storage, networking, etc.
And that doesn't even get into the water usage and carbon impact of manufacturing those chips; the IC industry uses a massive amount of water and generates a substantial amount of toxic waste.
It's hilarious how HN will wring its hands over how much rare earth metals a Prius has and shipping it to the US from Japan, but ask about the environmental impacts of AI and it's all "pshhtt, whatever".
If you've been working on AI, you've seen everything go up and to the right for a while - who really benefits from pointing out that a slowdown is occurring? Who is incentivized to talk about how the benefits from scaling are slowing down or the publicly available internet-scale corpuses are running out? Not anyone who trains models and needs compute, I can tell you that much. And not anyone who has a financial interest in these companies either.
No. Renewable energy capacity is often built out specifically for datacenters.
> Even if they install solar on the roofs and wind turbines nearby - that's still electrical generation capacity that could have been used for existing loads.
No. This capacity would never never have been built out to begin with if it was not for the data center.
> By buying so many solar panels in such quantities, they affect availability and pricing of all those components.
No. Renewable energy gets cheaper with scale, not more expensive.
> which means in a year they're selling enough chips to use 700MW of power.
There are contracts for renewal capacity to be built out or well into the gigawatts. Furthermore, solar is not the only source of renewable energy. Finally, nuclear energy is also often used.
> the IC industry uses a massive amount of water
A figurative drop in the bucket.
> It's hilarious how HN will wring its hands
HN is not a monolith.
That's easy, we just need to make meatspace people stupider. Seems to be working great so far.
Honestly? I'm not too worried
We've seen how the google employee that was "seeing a conscience" (in what was basically GPT-2 lol) was a nothing burger
We've seen other people in "AI Safety" overplay their importance and hype their CV more than actually do any relevant work. (Usually also playing the diversity card)
So, no, AI safety is important but I see it attracting the least helpful and resourceful people to the area.
That being said, the GP you’re talking about made no such statement whatsoever.
We do not have that. The cost of energy is mis-priced, although we are limping our way to fixing that.
Paying the likely fair cost for our goods, will probably kill a lot of current industries - while others which are currently viable, will become viable.
I agree with a majority of points you made. Exception is to this
> A figurative drop in the bucket.
Fresh water sources are limited. Fabs water demands and pollution are high impact.
Calling a drop in the bucket comes in the weasel words category.
We still need fabs, because we need chips. Harm will be done here. However, that is a cost we, as a society, will choose to pay.
Not fully accurate. Indeed there is renewable energy that is produced exclusively for the datacenter. But it is challenging to rely only on renewable energy (because it is intermittent and electricity is hard to store at scale so often you need to consume electricity when produced). So what happens in practice is that the electricity that does not come from dedicated renewable capacity is coming from the grid/network. What companies do is that they invest in renewable capacity in the network so that "the non renewable energy that they consume at time t (because not enough renewable energy available at that moment) is offsetted by someone else consuming renewable energy later". What I am saying here is not pure speculation, look at the link to meta website, they are saying themselves that this is what they are doing
Weather is not climate, as everyone is so careful to point out during cold waves.
Given the model is probabilistic and does many things in parallel, its output can be understood as a mixture, e.g. 30% trash, 60% rehashed training material, 10% reasoning.
People probe model in different ways, they see different results, and they make different conclusions.
E.g. somebody who assumes AI should have impeccable logic will find "trash" content (e.g. incorrectly retrieved memory) and will declare that the whole AI thing is overhyped bullshit.
Other people might call model a "stochastic parrot" as they recognize it basically just interpolates between parts of the training material.
Finally, people who want to probe reasoning capabilities might find it among the trash. E.g. people found that LLMs can evaluate non-trivial Python code as long as it sends intermediate results to output: https://x.com/GrantSlatton/status/1600388425651453953
I interpret "feel the AGI" (Ilya Sutskever slogan, now repeated by Jan Leike) as a focus on these capabilities, rather than on mistakes it makes. E.g. if we go from 0.1% reasoning to 1% reasoning it's a 10x gain in capabilities, while to an outsider it might look like "it's 99% trash".
In any case, I'd rather trust intuition of people like Ilya Sutskever and Jan Leike. They aren't trying to sell something, and overhyping the tech is not in their interest.
Regarding "missing something really critical", it's obvious that human learning is much more efficient than NN learning. So there's some algorithm people are missing. But is it really required for AGI?
And regarding "It cannot reason" - I've seen LLMs doing rather complex stuff which is almost certainly not in the training set, what is it if not reasoning? It's hard to take "it cannot reason" seriously from people
That has proven to be a mistake
The whole industry at this point is acting like the tobacco industry back when they first started getting in hot water. No doubt the prophecies about imminent AGI will one day look to our descendents exactly like filters on cigarettes. A weak attempt to prevent imminent regulation and reduced profitability as governments force an out of control industry to deal with the externalities involved in the creation of their products.
If it wasn't abundantly clear...I agree with you that AGI is infinitely far away. Its the damage that's going to be caused by sociopaths (Sam Altman at the top of the list) in attempting to justify the real things they want (money) in their march towards that impossible goal that concerns me.
What about Geoffrey Hinton? Stuart Russell? Dario Amodei?
Also exceptions to your model?
Who gets decide what the real impact price of energy is? That is not easily defined and well debated.
"Meanwhile what they have created is just a very impressive hot water bottle that turns a crank."
"Meanwhile what they have created is just a very impressive rock where neutrons hit other neutrons."
The point isn't how it works, the point is what it does.
What we're going to see over next year seems mostly pretty obvious - a lot of productization (tool use, history, etc), and a lot of efforts with multimodality, synthetic data, and post-training to add knowledge, reduce brittleness, and increase benchmark scores. None of which will do much to advance core intelligence.
The major short-term unknown seems to be how these companies will be attempting to improve planning/reasoning, and how successful that will be. OpenAI's Schulman just talked about post-training RL over longer (multi-reasoning steps) time horizons, and another approach is external tree-of-thoughts type scaffolding. These both seem more about maximizing what you can get out of the base model rather than fundamentally extending it's capabilities.
We have surpassed the 1.5°C goal and are on track towards 3.5°C to 5°C. This accelerates the climate change timeline so that we'll see effects postulated for the end of the century in about ~20 years.
I agree, and they are also living in a group-think bubble of AI/AGI hype. I don't think you'd be too welcome at OpenAI as a developer if you didn't believe they are on the path to AGI.
I’m pretty sure if Jan came to believe safety research wasn’t needed he would’ve just said that. Instead he said the actual opposite of that.
Why don’t you just answer the question? It’s a question about how these datapoints fit into your model.
Markets are our super computers. Human behavior is the empirical evidence of the choices people will make Given specific incentives.
Nobody defines what they’re trying to do as “useful AI” since that’s a much more weasily target, isn’t it?
Likewise, the cloud seeding they seem to be doing nearly worldwide now - the cloud formations from whatever they're spraying - are artificially changing weather patterns, and so a lot of the weather "anomalies" or unexpected-unusual weather-temperatures could very easily be because of those shenanigans; it could very easily be as a method to manufacture consent with the general population.
Similarly with the arson forest fires in Canada last summer, something like 90%+ of them were arson + a few years prior some of the governments in the prairie provinces (e.g. hottest and dryest) gutted their forest firefighting budgets; interesting behaviour considering if they're expecting more things to get hotter-dryer, you'd add to the budget, not take away from it, right?
Dane Wiginton (https://www.instagram.com/DaneWigington) is the founder of GeoengineerWatch.org as a very deep resource.
They have a free documentary called "The Dimming" you can watch on YouTube: https://www.youtube.com/watch?v=rf78rEAJvhY
In the documentary it includes credible witness testimonies such as politicians including a previous Minister of Defense for Canada; multiple states in the US have ban the spraying now - with more to follow, and the testimony and data provided there will be arguably be the most recent.
Here's a video on a "comedy" show from 5 years ago - there is a more recent appearance but I can't find it - in attempt to make light of it, without having an actual discussion with critical thinking or debate so people can be enlightened with the actual problems and potential problems and harms it can cause, to keep them none the wiser - it's just propaganda while trying to minimize: https://www.youtube.com/watch?v=wOfm5xYgiK0
A few of the problems cloud seeding will cause: - flooding in regions due to rain pattern changes - drought in areas due to rain pattern changes - cloud cover (amount of sun) changes crop yields - this harms local economies of farmers, impacting smaller farming operations more who's risk isn't spread out - potentially forcing them to sell or go into savings or go bankrupt, etc.
There are also very serious concerns/claims made of what exactly they are spraying - which includes aluminium nanoparticles, which can/would mean: - at a certain soil concentration of aluminium plants stop bearing fruit, - aluminium is a fire accelerant and so forest fires will then 1) more easily catch, and 2) more easily-quickly spread due to their increased intensity
Of course discussion on this is heavily suppressed in the mainstream, instead of having deep-thorough conversation with actual experts to present their cases - the label of conspiracy theorists or the idea of "detached from reality" are people's knee-jerk reactions often; and where propaganda can convince them of the "save the planet" narrative, which could also be a cover story for those toeing the line following orders supporting potentially very nefarious plans - doing it blindly because they think they're helping fight "climate change."
There are plenty of accounts on social media that are keeping track of and posting daily of the cloud seeding operations: https://www.instagram.com/p/CjNjAROPFs0/ - a couple testimonies.
1. When do you predict catastrophic global warming/climate change? How do you define "catastrophic"? (Are you pegging to an average temperature increase? [1])
2. When do you predict AGI?
How much uncertainty do you have in each estimate? When you stop and think about it, are you really willing to wager that (1) will happen before (2)? You think you have enough data to make that bet?
[1] I'm not an expert in the latest recommendations, but I see that a +2.7°F increase over preindustrial levels by 2100 is a target by some: https://news.mit.edu/2023/explained-climate-benchmark-rising...
I'm sorry, do you have a source for that claim? You seem to dismiss the video without any evidence.
If there is a top secret Manhattan Project for "climate change" - then someone's very likely pulling a fast one over everyone toeing that line, someone who has ulterior motives, misleading people to do their bidding.
But sure, fair question - a public discussion would allow actual experts to discuss the merits of what they're doing, and perhaps find a better solution than what has gained traction.
How much airspace of geographic area do you need access to in order to cloud seeds in other parts of the world though?
I haven't looked but perhaps GeoengineeringWatch.org has resources and has kept track of that?