It’s not necessarily cheaper energetically to get stuff from an asteroid than from Earth. You’d have to accelerate stuff from a wildly different orbit, and then steer it and slow it down. Metric tonnes of stuff. It’s not physically impossible, but it is wildly expensive (in pure energy terms, not even talking about money) and completely impractical with current technology. We just don’t have engines capable of doing this outside the atmosphere.
Delta V from just about anywhere in the solar system is lower than launching from the surface of Earth. You could launch stuff from Mars and bring it back to Earth orbit with less energy than launching it from Earth. The rocket equation is really punishing.