Sucks and is unfair, but it's the same principle that stops the government from, for example, jailing people who used to drink if Prohibition were put back in the Constitution.(1)
(1) There's a carve-out in jurisprudence that if something was illegal in the past, and you're jailed for it, and it is later made legal, your continued incarceration can be re-considered. But that's handled on, generally, a case-by-case basis.
No, it's not: ex post facto is a term of art for retroactive criminalization; QI applies only to civil liability. Retrospective enhancements to civil liability are not Constitutionally prohibited.