Well, it doesn't have to be as good as GPT4. I've gotten a ton of use/power out of a local llama model. Even GPT3 was "good enough" for a lot of tasks. I mostly use ChatGPT because its the most convenient.
OpenAI has pretty much the "best" model and first mover advantage. They can lose the latter, and might struggle to keep the former.