Credibility and Checking. We have ways of checking suggestions. Without passing such checks, for anything new, in simple terms. there is no, none, zero credibility. Current AI does not fundamentally change this situation: The AI output starts with no, none, zero credibility and to be taken seriously needs to be checked by traditional means.
AI is smart or soon will be? Maybe so, but I don't believe it. Whatever, to be taken seriously, e.g., as more than just wild suggestions to get credibility from elsewhere, AI results still need to be checked by traditional means.
Our society has long checked nearly ALL claims from nearly ALL sources before taking the claims seriously, and AI needs to pass the same checks.
I checked the credibility of ChatGPT for being smart by asking
(i) Given triangle ABC, construct D on AB and E on BC so that the lengths AD = DE = EC.
Results: Grade of flat F. Didn't make any progress at all.
(ii) Solve the initial value problem of ordinary differential equation
y'(t) = k y(t) ( b - y(t) )
Results: Grade of flat F. Didn't make any progress at all.
So, the AI didn't actually learn either high school plane geometry or freshman college calculus.
For the hearings today, we have from Senator Blumenthal
(1) "... this apparent reasoning ..."
(2) "... the promise of curing cancer, of developing new understandings of physics and biology ..."
Senator, you have misunderstood:
For (1), the AI is not "reasoning", e.g., can't reason with plane geometry or calculus. Instead, as in example you gave with a clone of your voice and based on your Senate floor speeches, the AI just rearranged some of your words.
For (2), the AI is not going to cure cancer or "develop new" anything.
If some researcher does find a cure for a cancer and publishes the results in a paper and AI reads the paper, there is still no expectation that the AI will understand any of it -- recall, the AI does NOT "understand" either high school plane geometry or freshman college calculus. And without some input with a recognized cure for the cancer, the AI won't know how to cure the cancer. If the cure for cancer is already in the training data, then the AI might be able to regurgitate the cure.
Again, the AI does NOT "understand" either high school plane geometry or freshman college calculus and, thus, there is no reasonable hope that the AI will cure cancer or contribute anything new and correct about physics or biology.
Or, Springer Verlag uses printing presses to print books on math, but the presses have no understanding of the math. And AI has no real understanding of high school plane geometry, freshman college calculus, cancer, physics, or biology.
The dangers? To me, Senator Blumenthal starts with no, none, zero understanding of AI. To take his claims seriously, I want to check out the claims with traditional means. Now I've done that. His claims fail. His opinions have no credibility. For AI, I want to do the same -- check the output with traditional means before taking the output seriously.
This checking defends me from statements from politicians AND from AI. AI dangerous? Same as for politicians, not if do the checking.