zlacker

I feel that was more true 1-2 years ago. These days I find Claude Code write almost as good (or as bad depending on your perspective) Elixir code as JavaScript code and there must be less Elixir code in the training data.

>>victor+(OP)
There's certainly a lot more JS code out there to train on, but the quality of the Elixir code is likely overall much better.

>>victor+(OP)
I personally find it much more painful to generate valid Rust code that compiles and does what I want than e.g. valid python code that runs and does what I want.

>>victor+(OP)
in my daily experience Claude Code writes better Elixir code than JS (React). Surely this has to do with the quality of the training material

>>jszymb+k5
i think it's pretty clear that some of "the things you expect to make an LLM good at a language" (like strong typing) are not actually the case. other things like "don't indirect your code by jumping to something unexpected" might be more important.

>>dnauti+Y7
If anything llms would be poorer in codegen for static languages because they are more verbose - More tokens to generate and use limited context windows parsing code.

The advantage rather for llms in strongly typed languages is that compilers can catch errors early and give the model early automated feedback so you don’t have to.

With weakly typed (and typically interpreted) languages they will need to run the code which maybe quite slow to do so or not realistic.

Simply put agentic coding loops prefer stronger static analysis capabilities.

>>agos+o6
Can’t confirm or deny comparison with JS but I can second that it write decent elixir

The only problem I’ve ever had was on maybe 3 total occasions it’s added a return statement, I assume because of the syntax similarity with ruby

>>manque+b9
not necessarily. if your refactoring loop requires too many cycles you'll fall off the attention context window.

also, some nonstatic languages have a habit of having least surprise in their codebases -- it's often possible to effectively guess the types flowing through at the callsite. zero refactoring feedback necessary is better than even one.

>>pjm331+6p
I’ve found Claude (at least until Opus 4) would routinely fail at writing a bash script. For example it would end an if block with }. Or get completely lost with environment variables and subshells.

But those are exactly the same mistakes most humans make when writing bash scripts, which makes them inherently flaky.

Ask it to write code in a language with types, a “logical” syntax where there are no tricky gotchas, with strict types, and a compiler which enforces those rules, and while LLMs struggle to begin with, they eventually produce code which is nearly clean and bug free. Works much better if there is an existing codebase where they can observe and learn from existing patterns.

On the other hand asking them to write JavaScript and Python, sure they fly, but they confidently implement code full of hidden bugs.

The whole “amount of training data” is completely overblown. I’ve seen code do well even with my own made up DSL. If the rules are logical and you explain the rules to it and show it existing patterns, the can mostly do alright. Conversely there is so much bad JavaScript and Python code in their training data that I struggle to get them to produce code in my style in these languages.