zlacker

I don't think getting training data is that hard still, the biggest platforms that locked down their APIs still use them for their mobile apps and can easily be reverse engineered to find keys or undocumented endpoints (or in the case of reddit, an entirely different internal API with less limits and a lot more info leaks...)

>>Shekel+(OP)
Can you explain the reddit one?

>>bloqs+n
Assuming the Reddit app does not use certificate pinning, you can use your computer to provide internet to your phone and then use an app like Charles Proxy to inspect requests being made from an app. Pretty easy to reverse engineer the API.

If the app does use certificate pinning, then you can use an Android phone and a modified app that removes the logic that enforces certificate pinning. This is more involved but also not impossible.

>>Shekel+(OP)
I think its a lot harder, while you still have lots of lawsuits coming in against AI models

>>4death+71
That does not sound like the proper way to do an openAI 2.0. If Reddit ever hears that's how an AI company scraped them, they'll get sued for fun and profits.

>>philis+z1
You can legally scrape anything that does not require a login in the US. You can also legally train an AI on it for now.

>>philis+z1
The point is that the data is easily accessible. If you wanted to get your hands on the data while simultaneously keeping them clean, contract with a Russian contracting company to give you a data dump. You don't need to know how they got it.

>>philis+z1
you're aware openai trained on a boatload of pirated ebooks?

they "steal" access to data because the LLM launders it on the other end

>>4death+c3
Well, until discovery, wherein your deliberate not knowing will be a pretty big deal.

>>wahnfr+C3
That is frustrating to no end. If I pirate one book I should pay a hefty fine. If a company does it it's unlocking untapped value.

>>az226+m2
Are you referring to the LinkedIn case? There has not been a decision on the legality of scraping in that matter

>>Shekel+(OP)
Easier than that would just be downloading the torrent of all of Reddit through Sept 2023.

https://academictorrents.com/details/89d24ff9d5fbc1efcdaf9d7...

>>monoca+q7
That's fascinating that the total size is so tiny — only 2.4 TB‽

I assume this must be only the text portion, and heavily compressed?

>>wahnfr+C3
What do you base this on?

Llms know the contents of books because they are analyzed, reviewed and spoken about everywhere. Pick some obscure book that doesn't show up on any social media and ask about it's contents. GPT won't have a clue

>>4death+c3
Subcontracting out your crimes isn’t going to fly in court.

>>bloqs+n
The reddit app uses an undocumented graphql based api seperate from the publicly available rest api used by third party apps.

>>philis+z1
It's essentially impossible to prove in court that training data was obtained or used improperly unless you go and tell on yourself. And even then it requires you to actually make someone with a lot of money mad, or to not have enough money yourself. Certainly microsoft would have already caught lots of flak for training their models on every github repo, instead they got a minor paddling from the public eye that went away after not much time had passed.

>>bko+O8
https://qz.com/openai-books-piracy-microsoft-meta-google-cha....

What's your evidence contrary to this? Sounds like your common sense rather than inside knowledge

>>q7xvh9+98
Text really doesn't take up that much space, and in addition it compresses pretty well.

The entire English language Wikipedia is only around 60GB in a format that can be readily searched and randomly accessed (ZIM), for example: https://kiwix.org/

>>lxgr+6d
Does Kiwix actually work? I see people hyping it here but I could never get it to actually, y'know, download the file and display the wikipedia on my phone.

>>4death+71
Yeah! <3 https://github.com/mitmproxy/android-unpinner

>>lmm+2e
It works perfectly for me, both on iOS and macOS.

>>lmm+2e
Just downloaded. Doesn't seem to want to download Wikipedia on my phone, it says "detecting if filesystem supports 4gb files"

>>lmm+2e
Kiwix worked for me. IIRC there may be difficulties opening an archive that was downloaded outside of the mobile app, but archives downloaded in-app were fine.

For the mobile app I used one of the smaller Wikipedia subsets, since I didn't want to take up too much space on my phone. The full offline Wikipedia download is saved to my laptop.

>>monoca+q7
The question is: if you then added all of Usenet before, say, 1992, would the effective intelligence of the trained LLM go up or down?

>>mr_toa+R9
Really? It's done pretty regularly to limit liability.

>>PaulDa+Ih
I can't speak to intelligence, but the result would be ignorant in a meaningful way.

>>mr_toa+R9
If it's done in a country where it's legal, maybe even processed in the same country and all you take out is the weights, I bet it gets a bit muddier.

>>Shekel+(OP)
That would still pose a legal problem.

>>Shekel+1b
It is not impossible. You can call witnesses, refer to emails, source code etc.

>>4death+qi
They make a point out of not directly asking for the crime when they do that. Just increasing pressure on subcontractors that leads to cutting corners including the law.

It is harder to prove to a "should have known" standard compared to say buying stolen speakers from the back of a truck for 20% of the list price.

>>4death+71
Why y’all desperate to steal data to train non intelligent software? Reddit and others should sue for license violations.

>>wahnfr+9b
Did you read the article (this one misstates the case but if you look at the one linked about the lawsuit)? This is a lawsuit. Nothing has been proven. Burden of proof is on you

>>Nasrud+YA
There’s an implicit assumption in your argument that you’re going to directly ask for a crime to be committed. Why are you assuming that? You’ll go to a contractor and say “we want Reddit data.” Anyone with even mild technical competence can figure out how to get it.