There is no evolution == better/more fit, as long as reproduction cascade goes uninterrupted, genes can evolve any which way and still survive whether they're neutral or a negative.
It's true that sometimes something can get taken along for the ride by luck of the draw. In which case what's really being selected for is some subgroup of genes as opposed to an individual one. In those cases there's some reason that losing the "detrimental" gene would actually be more detrimental, even if indirectly.