See https://github.com/windmill-labs/windmill-sync-example
it should be trivial to push that to a self-hosted setup if you need it.
I have not tried to take my entire workspace from self-hosted to cloud. But I have been exporting and importing individual scripts/flows/apps from self-hosted to cloud.
(no affiliation, just a happy user)