But, they literally aren't. The object is in virtual space, and your hands are interacting with two controllers (wands) that you cannot see and this interaction is then translated to control the virtual space.
I suppose, with practice they would become more of an extension of yourself? But that's not what I assume what was meant with "having to press the action button instead of just reaching out and touch the thing". Maybe the VR I tried just had really shitty controllers?
It's similar to tool use. Tools become an extension of your body and you tend to perceive them as such. Musicians, sportsmen etc are familiar with this feeling.
Which is certainly an argument that people get used to VR controllers. But I think it's also an argument against VR being particularly special in terms of immersion.
Or rather, it's an argument against 6DOF controllers being central to VR immersion. I think they make a difference albeit a small one.
Actual physical hand tracking is wonderfully immersive but hits other snags. No haptic feedback and tracking limitations. For some scenarious however it's a step forward.
(You need to design interactions around the controller limitations. Current VR experiences are too enamoured of the novelty and give the user too much freedom. Immersion comes from carefully stage managing the experience to avoid those things that sign-post the artificiality)