zlacker

My main concern is not even mentioned in this article and there are hardly any comments here addressing it: Privacy / allowing 3rd parties to read and potentially train on your proprietary source code.

I've used LLMs to crank out code for tedious things (like generating C-APIs and calling into poorly documented libraries) but I'm not letting them touch my code until I can run it 100% locally offline. Would love to use the agentic stuff but from what I've heard it's still too slow to run on a high end workstation with a single 4080.

Or have things got better lately, and crucially is there good VisualStudio integration for running local agents / LLMs?

>>Fredki+(OP)
The problem is, it's like ads. Even if it's a paid subscription, the incentive to hoover up your code, metadata about your code or everything they can get away with is just too strong. I think it will just happen at some point, unless you run them 100% locally or hosted somewhere but self managed.

>>Fredki+(OP)
Do you store code on GitHub? If so, how is GH's guarantee to not use your code different from Cursor's (with privacy mode enabled)?

>>oblio+q
Yes and even if they were squeaky clean and didn't do anything bad with your code, there's also the possibility that they've been hacked and code is getting siphoned off somewhere.

>>julvo+B
No I definitely don't use GitHub. Everything is entirely in-house.

But even if I did, there's a much more solid foundation of trust there, whereas these AI companies have been very shady with their 'better to ask for forgiveness, than permission' attitudes of late.

>>Fredki+R
Fair enough. In that case small models like devstral [1] are probably your best bet

[1] https://mistral.ai/news/devstral

>>julvo+B
Do you commit your client code to Github?

>>Fredki+R
All the model providers have offerings that promise not to train on your code. Can you trust them not to do it anyway? Maybe not. What's the actual damage if they did? You have to balance the expected productivity loss from forgoing the use of these tools with the risk that comes from sharing your code with them. You may want to reevaluate that somewhat frequently. I think there is a tendency in some circles to be a little bit to precious with their code.

>>Fredki+(OP)
So would you be interested in a simple serverless service that lets you run LLM’s and is focused on privacy and security first instead of trying to run it on an expensive workstation with a single 4080?

>>Fredki+(OP)
If you're big enough, you pay for on-prem frontier level tools. If not, you can do all right with a good GPU and local model, but it's not the same. And yeah there are plenty of plugins like Continue that support local or on-prem hosted models.

>>Fredki+(OP)
> but I'm not letting them touch my code until I can run it 100% locally offline

You can do this already with Ollama, RooCode, and a Docker compatible container engine.

>>Fredki+(OP)
Cursor has "privacy mode" in Pro mode that supposedly prevents training on your code.

>>julvo+B
Cursor has no privacy mode whatsoever. I have been able to exfiltrate just about anything from it.

>>Fredki+(OP)
Honestly is your code that revolutionary? Let's say I steal it, what can I do with it?