zlacker

The complaint has specific examples they got from ChatGPT.

There is a precedent: There were some exploit prompts that could be used to get ChatGPT to emit random training set data. It would emit repeated words or gibberish that then spontaneously converged on to snippets of training data.

OpenAI quickly worked to patch those and, presumably, invested energy into preventing it from emitting verbatim training data.

It wasn’t as simple as asking it to emit verbatim articles, IIRC. It was more about it accidentally emitting segments of training data for specific sequences that were semi rare enough.

>>Aurorn+(OP)
1. The data emitted by that buffer-overflow-y prompt is both non-deterministic and actual training only appears a fraction of the time. There no prompt that allowed for reproducible targeting of data sets.

2. OpenAI's "patch" for that was to use their content moderation filter to flag those types of requests. They've done the same thing for copyrighted content requests. It's both annoying because those requests aren't against the ToS but it also shows that nothing has been inherently "fixed". I wouldn't even say it was patched.. they just put a big red sticker over it.

>>Aurorn+(OP)
Hm... Why would people not just paste in sections of the book to the "raw" model in the playground (gpt instead of chatgpt) and just see if it completes the text correctly? Is the concern that chatgpt may have used the book for training data but not the original llm?

>>Aurorn+(OP)
> OpenAI quickly worked to patch those

So it was a problem, but isn't anymore?

>>kevinw+x2
edit: i meant to say "used the book for chat finetuning/rlhf but not the original llm". Also, I saw one example of the regurgitation by openAI of a NYT article, and it was indeed GPT-4, not ChatGPT.