It might not require specialized hardware in the future as GPUs get more powerful and we have other techniques such as LoRA for fine-tuning the models. We might see a distributed training [1] effort harnessing thousands of gamer GPUs worldwide, as well. All of this powered by open source software. Also there could be advances in the training software making it vastly more efficient.
1. https://arxiv.org/pdf/2301.11913.pdf