He also relates these issues to Christian philosophy, which I find very interesting. In particular, he claims that the a priori belief in the objects defined by Peano Arithmetic, is equivalent to worshipping numbers, as the Pythagoreans did.
I think this is the best starting point if you're interested in reading about his ideas: https://web.math.princeton.edu/~nelson/papers/warn.pdf
1) Not unique to exponentiation, it's also true of 281474976710656 + 79766443076872509863361.
2) This is a vague notion and you're going to have to deal with the problem of the smallest number that isn't expressible (I believe it might be the same as "feasible numbers", and there's a whole theory here)
3) the whole tenor of his critique suggests something a bit more foundational (predicativity, I guess...?)