Sadly I think git hook managers remove the simplicity of the whole design. Understandably since no one reads manuals anymore and projects don’t mind tacking on yet another module/plugin, however easy.
> hk is written in rust, pre-commit is written in python. hk will be much faster.
I have no idea how fast hk or pre-commit is, i have never used them. What matters to speed is the algorithms used and their complexities. If you implement a shitty algorithm with exponential complexity in Rust it's going to be slower than a linear complexity algorithm in python.
[0]: https://github.com/jdx/mise
What a strange sentence. First Hk will be faster than Lefthook because it will be written in Rust[1], but then later on it says that it doesn't actually matter? But either way it will be faster because it has "advanced parallelism"?
Looking at the comparison to Lefthook it's not clear to me why this isn't a PR to Lefthook. It's already a well-established solution which is present in many package managers. Surely the distinction between "checks" and "fixes" would be possible to introduce there as well?
Do we yet another tool instead of working together on improving the existing ones?
[1]: Which also is not a given. Programs which allocate a lot can be faster in Go since the garbage collector only works on the live set of objects whereas Rust have to explicitly deallocate all memory. CLI tooling is actually kinda a sweet-spot of GCs: You don't want to spend time reclaiming memory until you reach a certain threshold of used memory. In scenarios where you use less than the threshold you end up spending zero cycles on deallocation. (And completely leaking all memory is bound to cause problem on bigger commands or smaller machines.)
I think hk does do all those things, but it's a bit obscured by the focus on Git hooks in the docs. But the docs are also still in a super early state, so maybe that will be fleshed out more in the future.
"scripts": {
"postinstall": "git config --local core.hooksPath etc/hooks"
},There are a lot of command line tools that are written in JS and other scripting languages.
Having the tools that are involved in interactive use be written in a compiled languages gives hope that they might be fast enough to not be annoying.
Me personally I do not install nodejs on my machines. So knowing that this tool is not written in JS is relevant for me.
1. Link the scripts from the worktree to the .git/hooks directory - perhaps using a bootstrap script.
Ref: https://codeinthehole.com/tips/tips-for-using-a-git-pre-comm...
2. Declare the directory in the worktree to be the local git hooks directory.
As much as i'm a proponent of Rust, Go is very capable and is generally a great language. Its warts (as i see them) are not going to be apparent in the parallelism needed for a Git hook manager lol.
???
Also, this tool replaces another which was written in Go, which I would put in a similar performance category as Rust. It shouldn't make a difference in this scenario
When I or a team member do gradle build, npm build or cargo build or similar on a fresh checkout the tool should ask to be installed.
In practice though, hooks sometimes run tools that are were not designed like that and instead modify the file (eg: formatters). However, this is usually done on staged files (in case of the pre-commit hook) and the modification is applied to the working copy. This can cause race condition in that one tool may overwrite the changes made by another tool. But since the source is not modified, it won't end up in a deadlock. It will also fail as desired. So the race is not serious. It will resolve itself after a few runs, unless something is done to prevent it in the first place.
I find hk's choice of programming language for hooks pretty exotic TBH.
This all made me look at lefthook, and it’s pretty darn interesting. I’d been missing out!
1. https://git-scm.com/docs/githooks
2. https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
Here is another site dedicated to git-hooks. You'll find some good examples and resources there:
Could the Go solution add parallelism, sure. Did they? Not yet. Does that mean no other improvement in any other language can ever be written? No.
"Rust is faster" as an off the cuff comment that should have been left out seeing it has triggered some folks to hyper focus on that point.
Me and my colleagues have had numerous issues setting up pre-commit because it inherits Python's atrocious infrastructure.
I'm curious how this is going to deal with actually running plugins? Will it take the same approach as pre-commit and add dedicated not-very-good support for a load of different languages?
They did: Lefthook lets you define a "group" where you can specify that every command should be done in parallel: https://lefthook.dev/configuration/group.html. In addition, it's possible to configure the whole hook to run in parallel through another property: https://lefthook.dev/configuration/parallel.html.
I'm assuming that Hk's innovation here is that it's a bit smarter with what it runs in parallel. Maybe it uses the globs to automatically run commands in parallel which targets different files?
> "Rust is faster" as an off the cuff comment that should have been left out seeing it has triggered some folks to hyper focus on that point.
It's not a a hyper focus: This was the first reason (out of only three) that Hk itself presented as a reason to use it over Lefthook. So yes, I agree: It should have been left out if the intention wasn't for people to focus on it. Put it somewhere in a footnote if it's not so relevant.
Just put your hook logic in a script and copy it to the git hooks folder on startup as necessary -- and then, voila, you've avoided the nonsense of some well-intentioned package whose author thinks Rust in git hooks is a selling point rather than a head scratcher.
In general though, I agree that the blanket statement of “X is good because it’s language Y” is absurd, though I stubbornly cling to the opposite case for NodeJS, because I despise the idea of a frontend language running anything but a browser window. I have no objective defense.
In less than 40 lines of nix, most of which is boilerplate, you have fully cross-platform automatically installed and declaratively managed hooks, as part of your repo.
Need to run it in CI too? Well, no problem! Nix runs equally well in CI as it does locally.
Check this flake for example:
https://github.com/Azeirah/remarks/blob/main/flake.nix
See line 62 in the shell script.
This flake:
1. Manages ALL my dev env dependencies. Including even the specific version of bash that the script is running on, also the specific python dependencies down to the bit.
2. It's (posix-compliant) cross-platform, so it runs on MacOS, WSL, Linux, NixOS, ARM, x86 etc. Also docker.
3. It uses no specialized tools other than Nix, which is now a 20 year old Linux project which is quickly gaining even more traction. Nix is a programming language for reproducible, reliable and declarative dependency management (think docker but with a pure and functional programming language, rather than a recipe with installation instructions)
4. It also creates binaries as well as docker images from the binary. The docker image is like 2 lines of extra code.
Highly highly recommended. It's a bit difficult to wrap your head around initially, but holy damn if all dependency management problems don't just magically disappear forever once you learn Nix.
And I mean ALL of them. Whether it's Linux packages, docker containers, development environments, CI, virtual machines, containers, compilers, C headers from a 1970 Bell Labs project. No matter the architecture or OS (except Windows! :D).
Oh, did I mention it has a lockfile? So you can rollback or upgrade piecewise? Whenever you want?
It's like a programmable version of the superset of npm, cargo, pip, apt, brew, composer, gem, make, cmake, docker, git, Jenkins and even Linux itself (if you dare go the way of NixOS)
I'm sure it's something that can be engineered away and optimized in the Nix core though. Because yeah it isn't a hyperspecialized tool for specifically git hooks alone written in Rust. No.
It's not a saw nor a hammer. It's an entire workshop.
Flakes are less of a workshop, but more like a bus with tools or a heavy-duty toolbox.
or is this "Yet Another Rust Rewrite" (YARR)?
- Pre-commit hook setup
- Run locally anytime (`pre-commit run -a`)
- Check in CI (as Nix flake check)
Example repo: https://github.com/srid/haskell-template
The pre-commit configuration: https://github.com/srid/haskell-template/blob/master/nix/mod...
It claims:
1) Rust is fast, so that helps some.
2) They run hooks in parallel, and that helps a lot.
I like knowing what language a tool is written in. If it's written in Python or JavaScript and it isn't something that's absolutely essential I can just immediately move on. It also lets me know if it's something I'd be willing to contribute to. It's odd that the authors mentioning the language is so triggering for you.
I'll put these benchmarks on that page in different scenarios.
Update: https://github.com/jdx/hk/blob/main/docs/public/benchmark.pn...
As I said in the doc I think real-world performance in a large codebase will show that hk is even faster still—though that will depend on the project in question. It's really just a matter of providing the right levers in the right places and having good defaults.
Despite what everyone here says: yeah, just doing CLIs in Rust will be faster than Go and for CLIs like this milliseconds matter.
My biggest gripes are:
1. Pre-commit hooks are designed to modify files in-place, which turns `git commit` into something that can alter your work tree. IMO those tools should never ever modify files (unless explicitly asked by the user) and only output a diff and exit code that signifies if the check failed.
2. It manages the tools with its own environment but never exposes it. I always have to dig around its cache directory to find out which executable it is running if I have to reproduce the problem after something goes wrong.
ln -sf ../../scripts/git-pre-commit-hook .git/hooks/pre-commit
which simply adds a pre-commit symlink to a script in the repo's scripts/ dir. But hooksPath seems better.> Despite what everyone here says: yeah, just doing CLIs in Rust will be faster than Go and for CLIs like this milliseconds matter.
You've shown nothing to suggest that this is true. On my computer "lefthook --help" is exactly as fast as "hk --help". There's also many other differences between these tools. YAML vs Pkl is one difference. Another one is that Lefthook shells out to "git" to determine the list of staged files, while Hk uses libgit2. Which of these are faster? I'm not sure! It might even depend on repository size and/or other details. And as we all agree: In practice the parallelism strategy will matter the most.
> As I said in the doc I think real-world performance in a large codebase will show that hk is even faster still
I'm absolutely sure that you will be able to tweak hk to become the "fastest" pre-commit runner in your designated category. I'm also pretty sure that similar optimizations will apply quite easily to Lefthook. They're after all doing pretty much the same thing.
> then you removed the parallel support
that was not intentional. I fixed that, but it didn't change the results that much:
https://github.com/jdx/hk/commit/dfe1fc1724b8f6c43b184dc98ac...
In any case, I don't know why anyone would take such a new project "seriously". I certainly don't.
I don't want to have to have 3rd party tools (including things like PRs/issue management) that don't record their info in the repo itself.
Should be possible with the content addressable store underneath and new blob types? Or will that break too much of the existing git?
Direct claim from the article.
First, I intend to be able to define simple "dependencies", but IMO that's not the interesting part.
I am going to use rw mutexes on every file with staged changes. If 2 steps are running against disjoint files there is no problem. If 2 steps are running against "check" (not "fix" which edit files by convention), they can also run at the same time. The only scenario where blocking is necessary is when you have 2 "fix" or 1 "check" and 1 "fix" on an intersection of their files.
For that scenario there is going to be a setting that you can enable to run a steps "check" command first (only if this scenario is about to happen, if no files will be in contention it will simply run "fix"), and if "check" fails, then it will switch to "fix".
This is my current idea and in my head I think it would work. There are caveats, notably running "check" and then "fix" might be slower than just waiting and running "fix" once which is why it needs to be optional behavior. You also may not care about things stomping on each other (maybe the linters themselves have their own locking logic).
mise supports an experimental bootstrapping feature: it can download itself and install the tools required for the project.
See https://mise.jdx.dev/cli/generate/bootstrap.html and https://mise.jdx.dev/continuous-integration.html#bootstrappi...
commit->treeish->blob
and refs?
I had submitted a PoC patch for the bash version that greatly sped it up, but there were a couple of tests I was struggling to get to pass, and I ran out of free time.