zlacker

Like Emmett Shear says[0], in this discourse we crucially need to separate out the "AI ethics" (like output moderation) from the real problem of "AI NotKillEveryoneism". People are pattern matching "safety" as "those people concerned about AI racism and censoring" when they're orthogonal.

I encourage everyone who is skeptical of superalignment (there's a reason very smart people like Ilya were heading that team) watch Emmett Shear (former OpenAI and Twitch CEOs) talk about this problem in the discourse.

https://youtu.be/jZ2xw_1_KHY?si=Jog7ULmjjNW21ZIf&t=800

>>reduce+(OP)
100%. I see them conflated a lot, particularly outside of HN, and I think it really damages the discourse. Censorship for political or brand reasons (and I'm not making a moral judgement by my choice of language) is very different to aligning for safety reasons.

>>AlexAn+q5
Yep. And it's lazy to think that the pro-cryogenics, pro-nuclear, pro-engineering people are the same as the ones storming a Tesla factory or causing wildly unexpected Gemini output.

"No no no, this thing though actually, may be a problem. If you're rejecting it because we sound like a bunch of crazies, Just notice that some number of people who are worried about this, I'm on your team, I'm on the techno-optimist team. Its not obvious why its true. It takes a good deal of engagement with the material to see why its true. Because at first, it seems like it shouldn't be that big of a deal, it shouldn't be that big of a problem. but then the more you dig in, actually, oh wait a second, and it is." -Emmett

>>reduce+d7
"People who support cryonics are sober-minded, reasonable individuals who would surely never be misled by unscientific techno-mysticism."

I am amazed at the Dunning-Kruger syndrome on display here.

>>nickle+yZ
A large part of that Emmett Shear interview is about the fallacy of pattern-matching beliefs like you're doing right now. "oh those people have X belief, which is stupid, so I just tune the rest of them out." I guarantee you haven't done an in-depth study of the merits of cryogenics. Neither have I yet, but there are many smart people who do think the tradeoff is worth it. The point I and Emmett make is that AGI X-risk people are dismissed prematurely based on vaguely sounding like other stuff the subject already believes isn't worth investigating. Whether that's mistakenly believing they're not pro-tech along most axes, "dismissed", they generally have some other quirky belief like cryonics, "dismissed", or they think we could be headed for a very dangerous place, "sounds like other doomsday people, dismissed."

>>reduce+D11
> I guarantee you haven't done an in-depth study of the merits of cryogenics. Neither have I yet, but there are many smart people who do think the tradeoff is worth it.

This is literally the strategy of conspiracy theorists! "You didn't do the research, but Emmett did and he's smarter than you." The scientific consensus is that cryonics is pseudoscience. It does not matter if I have personally done a study. It is entirely rational for me to trust the Wikipedia article on cryonics + common sense. And lots of people way smarter than Shear believed in stranger things.

Emmett Shear - and yourself! - uncritically described cryonics as being a rational, techno-optimist thing to support, and proof of Shear's credentials as a reasonable, grounded thinker, not like these Tesla-smashing wackos. That strongly implies that Shear is a bad judge of what counts as reasonable, because cryonics is ridiculous quackery. Hence why he has Dunning-Kruger syndrome. He's operating in an ignorant Silicon Valley bubble where sci-fi and reality are interchangeable as long as you have a thoughtful affectation.

One thing ChatGPT has made very obvious is that the tech community is vulnerable to the tactics of psychics and conspiracy theorists.

>>nickle+a71
> uncritically described cryonics as being a rational, techno-optimist thing to support

No cryonics is posited as on the "pro-tech" spectrum, not that it's trivially true, and only mentioned so that people pattern match less on "these anti-tech'ers." I fear this is fruitless as you're unable to discuss the meta here instead of the object.

Again, the point is to discuss the actual topic, not the dismissive association cop-out that many sides lazily use. Then we can see more well-reasoned discussion instead of "these decels," "those doomsday cultists," "those cryogenic quacks."