zlacker

Your response makes me wonder if poisoning the well is possible by submitting code to Github with multiple languages and coding styles. A single file with a function signature written in Javascript and the body written in Python + Ruby. Enough code would surely break the AI model behind it. Unless Copilot has some sort of ingestion validation which wouldn’t surprise.

>>yeptha+(OP)
In any training with code I've done, we've written a parser that validates against tree sitter grammars to make sure it's at least syntactically valid against some known subset of languages we're training on.

>>yeptha+(OP)
Probably but you would have to submit an absurdly large amount of code to make a dent. Practically unreasonable considering their training corpus is also increasing per lines of public code submitted on github.

So not only would you have to submit a insanely large amount of code but you're also racing against literally millions of users writing legitimate code at any period of time.

>>stu2b5+T1
Why not just use AI to generate the code, and automate submission via APIs?

>>Grimm1+u1
I’m which case shifting strategies toward code that looks correct but isn’t using shared syntax between languages as well as language specific gotchas.

>>stu2b5+T1
> Probably but you would have to submit an absurdly large amount of code to make a dent.

So how about an already poisoned well. How up to date is the average Github project on encryption standards?

>>yeptha+Y3
Yeah but if malicious intent is a concern you can just spin up a sandboxed instance to run the code to check first.

Really the thing is there's not way to ascribe correctness to a piece of code right, like humans fail at this even. The only "correct" code is like rote algorithmic code that has a well defined method of operation. And there's likely a lot more correct examples of that, like way more than you'd ever be able to poison.

You may be able to be misleading though by using names that say one thing but do another, but again you'd be fighting against the tide of correctly named things.

>>yeptha+(OP)
I don't know if this is true, but I would assume that the tokenizers they used for Codex use actual language parsers which would drop invalid files like this and make this attack infeasible.

When I was playing around a couple years ago with the Fastai courses in language modeling I used the Python tokenize module to feed my model, and with excellent parser libraries like Lark[0] out there it wouldn't take that long to build real quality parsers.

Of course I could be totally wrong and they might just be dumping pure text in, shutter.

[0]: https://github.com/lark-parser/lark