zlacker

I'm inclined to agree with you in principle, but there's much, much less Haskell examples in their training corpus than for JavaScript or Python.

>>eru+(OP)
And yet the models I've used have been great with Rust, which pales in lines of code to JavaScript (or Python or PHP or Perl or C or C++).

>>kstrau+A
I've also had decent experiences with Rust recently. I haven't done enough Haskell programming in the AI era to really say.

But it could be that different programming languages are a bit like different human languages for these models: when they have more than some threshold of training data, they can express their general problem solving skills in any of them? And then it's down to how much the compiler and linters can yell at them.

For Rust, I regularly tell them to make `clippy::pedantic` happy (and tell me explicitly when they think that the best way to do that is via an explicit ignore annotation in the code to disable a certain warning for a specific line).

Pedantic clippy is usually too.. pedantic for humans, but it seems to work reasonably well with the agents. You can also add clippy::cargo which ain't included in clippy::pedantic.

>>eru+(OP)
You are right that there is significantly more Javascript in the training data, but I can say from experience that I'm a little shocked at how well opus 4.5 has been for me writing Haskell. I'm fairly particular and I end up re-writing a lot of code for style reasons but it can often one shot an acceptable solution that is mostly inline with the rest of the code base.

>>eru+u1
> But it could be that different programming languages are a bit like different human languages for these models: when they have more than some threshold of training data, they can express their general problem solving skills in any of them? And then it's down to how much the compiler and linters can yell at them.

I think this is exactly right.

>>eru+(OP)
True for now, but probably not a durable fact. Synthetic data pipelines should be mostly invariant to the programming language, as long as the output is correct. If anything the additional static analysis makes it more amenable to synthetic data generation.

>>energy+89
> Synthetic data pipelines should be mostly invariant to the programming language, as long as the output is correct.

Well, you can adapt your PHP producing pipeline to produce Haskell code that is correct in the sense of solving the problem at hand, but getting it to produce idiomatic code is probably a lot harder.

>>solomo+M1
Exactly my opinion - I think the more you lock down the "search space" by providing strong and opinionated tooling, the more LLMs perform well. I think of it as starting something like a simulated annealing trying to get a correct solution, versus the same simulated annealing run while using heuristics and bounds to narrow the solution space

>>eru+(OP)
For the little Haskell I've done with llms, I can tell you they're not bad at it.

Actually, Haskell was a bit too hard for me on my own for real projects. Now with AI assistants, I think it could be a great pick.

>>eru+(OP)
From what I've heard—and in my own very limited experiments—LLMs are much better at less popular languages than I would have expected. I've had good results with OCaml, and I've talked to people who've had good results with Haskell and even Unison.

I've also seen multiple startups that have had some pretty impressive performance with Lean and Rocq.

My current theory is that as long as the LLM has sufficiently good baseline performance in a language, the kind of scaffolding and tooling you can build around the pure code generation will have an outsize effect, and languages with expressive type systems have a pretty direct advantage there: types can constrain and give immediate feedback to your system, letting you iterate the LLM generation faster and at a higher level than you could otherwise.

I recently saw a paper[1] about using types to directly constrain LLM output. The paper used TypeScript, but it seems like the same approach would work well with other typed languages as well. Approaches like that make generating typed code with LLMs even more promising.

Abstract:

> Language models (LMs) can generate code but cannot guarantee its correctness often producing outputs that violate type safety, program invariants, or other semantic properties. Constrained decoding offers a solution by restricting generation to only produce programs that satisfy user-defined properties. However, existing methods are either limited to syntactic constraints or rely on brittle, ad hoc encodings of semantic properties over token sequences rather than program structure.

> We present ChopChop, the first programmable framework for constraining the output of LMs with respect to semantic properties. ChopChop introduces a principled way to construct constrained decoders based on analyzing the space of programs a prefix represents. It formulates this analysis as a realizability problem which is solved via coinduction, connecting token-level generation with structural reasoning over programs. We demonstrate ChopChop's generality by using it to enforce (1) equivalence to a reference program and (2) type safety. Across a range of models and tasks, ChopChop improves success rates while maintaining practical decoding latency.

[1]: https://arxiv.org/abs/2509.00360

>>eru+s9
I think the trick with Haskell is that you can write types in such a way that the APIs that get generated are idiomatic and designed well. The implementations of individual functions might be messy or awkward, but as long as those functions are relatively small—which is how I tend to write my non-AI-based Haskell code anyhow!—it's not nearly as important.

>>tikhon+so
As a huge proponent of constrained decoding for LLM reliability, I don't quite think it's the right approach for code. This is because in many programming languages, it is legal to use a function before its declaration. Since this is common in existing code, LLMs will also try to write code that way.

>>tikhon+so
You might be right, but I think you must take into account that (I think) you're not super familiar with these languages as well, so you might not notice all the warts a programmer with a lot of experience in these langs would, and overrate the skill of the LLM.

Nowadays, I write C# and TS at work, and it's absolutely crazy how much better the LLM is at TS, with almost all code being decent the first try, but with C# I need to do a lot of massaging.

>>torgin+ay
I write Haskell professionally and I can tell you that Opus 4.5 can do a great writing industrial Haskell code consistent with an existing code base.

I don't think it is capable of writing galaxy brain Haskell libraries, it absolutely missed the forest for the trees, but if you have an existing code base with consistent patterns it can emulate then it can do a surprisingly good job.

Here is an example side project I have done extremely heavily with Claude: https://github.com/solomon-b/kpbj.fm

I built a library (without Claude) that wraps in an opinionated way Servant and a handful of other common libraries used to build Haskell web-apps and then have let Claude use that to build this site. There is absolutely some hairy code and I have done a ton of manual refactors on what Claude produces, but claude has been highly effective for me here.