You first remove rate limits, then implement and release exponential backoff on frontend, then apply rate limits again (on a small segment of users first, then more). No biggie, you just need to be very careful. And boss needs to chill for that time, which is unlikely to happen.