zlacker

>>tim_sw+(OP)
> How do we ensure AI systems much smarter than humans follow human intent?

You can't, by definition.

>>User23+Sj
You can if you are the one controlling their resource allocation and surrounding environment. Similar to how kings kept smartest people in their Kingdom in line.

>>crop_r+Zm
Only works for so long. A smart enough serf could easily find a way to socially engineer and slaughter the king.

>>tester+mr
I'm not convinced. Omniscience isn't the same as intelligence.

There's diminishing returns to intelligence and inherent unknowns to all moves the serf can make. The serf somehow has to evade detection, which might appear to be effectively impossible given the unknowns of how detection may take place.