On the contrary, this is a problem that goes "complicated-easy-complicated" the more you think about it. You're at stage two, so here's the mindfuck: Why aren't we constantly stuck at local minima? Surely these problems we're throwing at ANNs aren't all convex.