When it all goes bankrupt, they can pay off the bonds for x¢ in the dollar and own SpaceX.
Perhaps if the gov could organize a little better, they'd make sure SpaceX owed lots of taxes and put themselves in front of the queue for ownership and screw other creditors (especially foreign).
Edit: looks like the US military doesn't spend that much on SpaceX: https://londoneconomics.co.uk/blog/publication/crouching-riv...