zlacker

This is a misunderstanding of what AI alignment problems are all about.

Alignment != capability

Think a paperclip maximizing robot that in its process of creating paperclips kills everyone on earth to turn them into paperclips.

>>jdasdf+(OP)
Corporations like Saudi Aramco are already doing that. You don't need a superintelligent AI, corporations that maximize profit are already sufficient as misaligned superhuman agents.

>>climat+iy
You can't maximize profit without customers, they must be aligned with someone.

>>nights+xM
They're aligned with the military-industrial complex. The US military is one of the biggest consumers of fossil fuels[1] and it's the same with other nations and their energy use. So profitable is not the same as aligned with human values.

1: https://en.m.wikipedia.org/wiki/Energy_usage_of_the_United_S...

>>climat+ZM
> The US military is one of the biggest consumers of fossil fuels

I guess this phrasing is up for debate, but according to the source linked "the DoD would rank 58th in the world" in fossil fuels.

Is that a huge amount of fossil fuel use? Absolutely. But one of the biggest?

>>flagra+4S
> According to the 2005 CIA World Factbook, if it were a country, the DoD would rank 34th in the world in average daily oil use, coming in just behind Iraq and just ahead of Sweden.

Sure, the phrasing could be debated but the fact that it even ranks close to actual nation states is already problematic. The US military is basically an entire nation state of its own. This is nothing new if you're old enough to have observed the kind of damage it has done but it demonstrates my point about profit and alignment. Profits are very often misaligned with human values because war is extremely profitable.

>>nights+xM
Bit of an interesting thought experiment there, could a corporation maximize profit without customers? I wonder if we can find any examples of this type of behavior...

>>nights+xM
Yes, but a profit maximizer doesn’t need to eliminate all humans to become a big problem.

>>climat+dY
US DOD fuel use being the level of Sweden doesn’t seem problematic to my envelope-math; it seems to reflect the size of the entities involved.

Iraq is a now broken third word country/economy in recovery so not a great comparable to US. Sweden is small but a good comparable culturally/development-wise. US is 331 million people. It spends 3% of GDP on military. 3% of 331m is 10 million. Sweden is 10 million people. U.S. military fuel use is in line with Sweden’s.

I could be off here (DOD!=US military?), corrections welcome, but I wouldn’t even be shocked if a military entity uses 3-10x more fuel than a civilian average and above math puts us surprisingly close to 1x.

>>jdasdf+(OP)
No, I understand what you're saying, I just think you're wrong. To be a little clearer: you're assuming a single near-omnipotent agent randomly selects an anti-goal and is capable of achieving it. If we instead create 100 near-omnipotent agents odds are that the majority will be smart enough to recognize that they have to cooperate to achieve any goals at all. Even if the majority have selected anti-goals, it's likely that the majority of the anti-goals will be at cross-purposes. You'll also have a paperclip minimizer, for example. Now, the minimizers are a little scary but these are thought experiments and the goals will not be so simple (nor do I think it would be obvious to anyone including the AIs which ones have selected which goals.) The AIs will have to be liars if they select anti-goals, and they will have to not only lie to us but lie to each other, which makes coordination very hard bordering on impossible.

In some ways this is a lot like Bitcoin, in that people think that with enough math and science expertise you can just reason your way out of social problems. And you can, to an extent, but not if you're fighting an organized social adversary that is collectively smarter than you. 7 billion humans is a superintelligence and it's a high bar to be smarter than that.

>>lukesc+uM1
It’s worth reading about the orthogonality thesis and the underlying arguments about it.

It’s not an anti-goal that’s intentionally set, it’s that complex goal setting is hard and you may end up with something dumb that maximizes the reward unintentionally.

The issue is all of the AGIs will be unaligned in different ways because we don’t know how to align any of them. Also, the first to be able to improve itself in pursuit of its goal could take off at some threshold and then the others would not be relevant.

There’s a lot of thoughtful writing that exists on this topic and it’s really worth digging into the state of the art about it, your replies are thoughtful so it sounds like something you’d think about. I did the same thing a few years ago (around 2015) and found the arguments persuasive.

This is a decent overview: https://www.samharris.org/podcasts/making-sense-episodes/116...

>>gregw2+Vt1
Math seems correct but US military also includes conglomerates and companies like Palantir and Anduril (main reason it is described as an industrial complex is because there is no clear distinction between corporations and how their activities are tied up with military spending and energy use).

>>goneho+9R1
> the first to be able to improve itself in pursuit of its goal could take off at some threshold and then the others would not be relevant.

Thanks for reminding me that I need to properly write up why I don't think self-improvement is a huge issue.

(My thought won't fit into a comment, and I'll want to link to it later).

>>nights+xM
In fairness, corporations can still be fraudulent.

>>climat+dY
Oh there's no denying the US military has ballooned to the size of a small to medium-sized country. That alone is a huge issue for me personally - I do agree with our country having any form of standing military but that precedent was abandoned 80 years ago.

I'm not sure how to properly compare the military of one country with the entirety of a country ~1/30th the size. On the surface it doesn't seem crazy for those to have similar budgets or resource use.

>>flagra+gu3
The comparison is in terms of energy use since at the end of the day that is the fundamental currency of all techno-industrial activity. The point is that the global machinery that is currently guiding civilizational progress is fundamentally anti-life. It constantly grows and subsumes whatever energy resources are accessible without any regard for negative externalities like pollution and environmental degradation. This is why I don't take AI alarmism seriously because the problem is not the AI, the problem is the organization of techno-industrial civilization and its focus on exponential growth.

It's only going to keep getting worse and the AI alarmism is not doing anything to address the actual root causes of the crisis. If anything, AI development might actually make things more sustainable by better allocating and managing natural resources so retarding AI progress is actually making things worse in the long run.

>>climat+tK3
I think those really are separate concerns that should both be given more attention.

There's a strong correlation between GDP growth and oil use, that's a huge problem and one that likely can't be solved without fundamentally revisiting modern economic models.

AI poses it's own concerns though, everything from the alignment problem to the challenge of even having to define what consciousness even is. AI development won't inherently make allocating natural resources easier - with the wrong incentive model and lack of safety rails AI could find its own solution to preserving natural resources that may not work out so well for us humans.

>>flagra+mE4
The current model is already destructive and most of the market is managed by artificial agents. Schwab will give you a roboadvisor to manage your retirement account so AI is already managing large chunks of the financial markets. Letting AI manage not just the financial aspects but things like farmland is an obvious extension of the same principle and since AIs can notice more patterns it's going to become basically a necessity because global warming is going to make large parts of existing farmlands unmanageable. Floods and droughts are becoming more common and humans are very bad at figuring out the weather so there will be an AI agent monitoring weather patterns and allocating seeds to various plots of land to maximize yields.

Bill Gates has bought up a bunch of farmland and I am certain he will use AI to manage them because manual allocation will be too inefficient[1].

1: https://www.popularmechanics.com/science/environment/a425435...