Sure Infineon can probably get my data, but that's far beyond the scope of my threat model.
As long as the system is open to putting your own keys on there I'm fine with it.
As long as software that uses the TPM cannot detect whether you tampered with the TPM or not, it is principally all right.
But as I wrote down: this is exactly the opposite of what trusted computing was invented for: make the machine trustable (for the companies that have control over the TPM/trusted computing), because the user is distrusted.