Whatever they've built this year presumably uses all the positive/negative feedback on ChatGPT that they have a year worth of data now...
Another examples is the Be My Eyes data - presumably the vision part of GPT-4 was trained on the archive of data the blind assistance app has, and that could be an exclusive deal with OpenAI.