The biggest problem with VR is headset bulk, and space. Lighter headsets will make a huge difference. Finding a way to give people more raw space to play in will make a huge difference.
On the other hand, if the controls were actually like "reach out and touch the thing" (which they just aren't because you're holding things that only signify interaction), I might have felt better about it. Not sure, because so far all VR has made me feel completely helpless wrt the controls.