To create an analogy, my car doesn't have bullet proof glass, someone could easily shoot it up and i'd be dead. But nobody really goes around shooting up cars, so is it an issue?
Exact same story with error oracle attacks in cryptography.
Attackers go after the low hanging fruit first, and then they move up the tree.