It's infrastructure, so it's all inherently neutral. SSL is used by banks, protesters, and criminals alike. You can't weaken it for one group without weakening it for everyone. It's also global: you can't backdoor an IRC client only for marijuana users in the US, for instance.
So if you get to surveil pedophiles in the US, it means that Saudi Arabia gets to surveil homosexuals. We're on the same infrastructure.
Also, it's important to recognize that illegal behavior is a critical part of Democratic change. If SSL could discriminate based on your intent to break a law so we could arrest them all, people campaigning for marijuana legalization would all be in jail, and the law would not be changingl. So would people in the 60s campaigning for civil rights, and every homosexual in the country. There is always a grey area period of time in which people break a law because they don't believe in it. That period of civil disobedience is how laws end up getting changed. Even (especially) morality laws against things like sexuality, drugs, or alcohol. It's important to a living democracy that the police are not a perfect force.
There might be a way to stop pedophiles and kidnappers, and rapists that we haven't thought of simply because we're not willing to talk about how we could do it.
Highways have police. Where are the digital police? I'm not sure I prefer such a thing, but why don't we even discuss it?