In the naive sense, there is a lot of value in knowing location. Restraining orders, for example, could become effective. "How did your cell phone get to the bank that was robbed if you weren't there?" stuff like that. Furthermore, politicians pander. being able to answer how many people showed up for that protest is valuable. what were their demographics? Perhaps this is an issue that matters.
But the asymmetry is horrible. You want real time access to my location? ok, but make the location public for the police. and NSA employees. and senators. Having a record, that's made public after a few weeks or months seems pretty reasonable to me. Having a record that's secret and controlled by a handful of powerful people, which is what i think we have now, is much more frightening.