Of course, there will be no shortage of overly enthusiastic (and absolutely confident) defeatism "We 'can't' do it because x, y, z" (complexities with security, ensuring the person casting the vote is indeed the actual person, excess amount of uninformed populism, etc.) So how about this: for the first <x> years, make it non-binding and simply observe the results. If the votes have no power, so much for the disingenuous claims that "we don't dare try it, and it won't work anyways", because it completely derisks the situation.
So then, when you have people still guaranteeing doom, I reckon there's a pretty good chance that would make a good shortlist of people who should no longer be allowed anywhere near the political process.
I would love to know why people are adamantly opposed to having a honest, transparent, and fact-checked public conversation on the idea.