The vast majority of deaths by train involve "trespassers", which is code for "dimwits who bypassed crossing gates and got smashed by the train that couldn't stop". Usually not even the train drivers are injured, much less the passengers.
But airplanes are very safe - perhaps mostly because it's hard for idiots to drive in front of them.
First, you are correct about trespassers. But even if you only consider passengers, planes are still safer per passenger-mile than trains.
Second, commercial planes are very safe. Private planes... not in the same league.
It's code for suicide. The remainder are as you described.