zlacker

>>new ways to create specifications.

Thats again programming languages. Real issue with LLMs now is it doesn't matter if it can generate code quickly. Some one still has to read, verify and test it.

Perhaps we need a need a terse programming language. Which can be read quickly and verified. You could call that specification.

>>kamaal+(OP)
Yes, essentially a higher level programming language than what we currently have. A programming language that doesn't have strict syntax, and can be expressed with words or code. And like any other programming language, it includes specifications for the tests and expectations of the result.

The programming language can look more like code in parts where the specification needs to be very detailed. I think people can get intuition about where the LLM is unlikely to be successful. It can have low detail for boilerplate or code that is simple to describe.

You should be able to alter and recompile the specification, unlike the wandering prompt which makes changes faster than normal version control practices keep up with.

Perhaps there's a world where reading the specification rather than the compiled code is sufficient in order to keep cognitive load at reasonable levels.

At very least, you can read compiled code until you can establish your own validation set and create statistical expectations about your domain. Principally, these models will always be statistical in nature. So we probably need to start operating more inside that kind of framework if we really want to be professional about it.

>>deepsq+o4
Simply put whatever you write should produce the same output regardless of how many times you execute it. The more verbose you make it, the more pointless it becomes.

More terse the better.

>>kamaal+Je
for the sake of being downvoted: MASM.

>>tosapp+an
An LLM could speak FPGA.

Good luck auditing that.

>>deepsq+o4
We already have exceptionally high level languages, like Inform7 [0]. The concept doesn't work all that well. Terseness is a value. Its why we end up with so many symbol-heavy languages. Yes, there are tradeoffs, but that is the whole of computer science.

We didn't end up with Lean and Rust, for a lack of understanding in how to create strong specifications. Pascal-like languages fell out of favour, despite having higher readability.

[0] https://learnxinyminutes.com/inform7/

>>kamaal+(OP)
This specification argument seems to boil down to: what if we used Haskell to describe systems to LLMs?

Many of our traditional functional languages, ML family in particular, let you write hyper concise expressions (pure math if you’re in to that sort of thing), craft DSLs of unlimited specifiable power (‘makeTpsReportWith “new cover page format”’), and also in natural language (function names like `emptied cart should have zero items`).

I think if we did that and leveraged the type systems of those languages and the systematic improvements we see from ADTs and pattern matching in those languages, combined with a specification first approach like TDD, that we’d have a great starting point to have an LLM generate the rest of the system perfectly.

… yes, that is just writing Haskell/OCaml/F# with extra steps.

… yes, that level of specification is also the point with those languages where your exploratory type-diddling suddenly goes ‘presto’ and you magically have a fully functioning system.

I guess I’m old-fashioned, but sometimes I wonder if compilers are good for what they’re good for.