As a citizen of a country where a lot of foreigners move in, it's not great. And we have nowhere the level of quality of life of Japan to fall from.
And yet! Singapore is consistently ranked as one of the happiest countries in the world. This contradicts the common western association of personal liberty correlating with happiness.