First, you are correct about trespassers. But even if you only consider passengers, planes are still safer per passenger-mile than trains.
Second, commercial planes are very safe. Private planes... not in the same league.