A USB is tiny, and you can shrink it's footprint with USB-C. You can also buy USB keys with tamper-proof housings that will blow a fuse if opened to be physically compromised. Coupled with strong post-quantum crypto, that key is relatively secure, even if physically discovered.
That's just the technical bit. You can also split the key in half and transfer the other half somewhere, which creates legal protection. You could also create a housing for the key so it's not easily discoverable.
If all that sounds a bit extra, circle back to that the perpetrator has 4.5 Billion worth of something.
It does sound like a lot of work. I think I'd go with the $5 wrench option.