zlacker

>>jsnell+(OP)
I've come around to the opinion that (POSIX-style) filesystems are no longer a great idea. Instead, I think something more like Git (plumbing, not porcelain) would be preferable as the underlying storage abstraction.

>>carapa+3t
What exactly are you thinking of? Git manages files after all.

If you mean the underlying data structures, that's basically what modern filesystems are. XFS, APFS, BTRFS etc are all copy-on-write file systems that use git-like structures underneath. In the same way that git branches are "instant" because no data is actually copied, so too can these file systems clone files or whole trees instantly without copying any real data. You can easily "branch" a directory tree, rsync the results to a remote machine, etc.

>>mike_h+LI
For one thing, it would be nice if every directory had a hash that covered all its contents (recursively), like Git tree objects. That way, all sorts of tools that need to check which files in a directory tree have changed – including `git diff`, `make`, file sync tools, and indexed search tools – could immediately skip directories with no changes, without needing a separate fsmonitor tool. The cost would be higher overhead recalculating hashes when files are constantly being updated.

It would also be nice to support filesystem transactions, somewhat analogous to Git commits. POSIX file APIs make it difficult to avoid race conditions in the best circumstances, and extremely difficult if there’s a security boundary involved. You can never check something about a path (e.g. “is this a symlink?”) and then rely on that thing being true, because it could have been concurrently modified between the check and whatever you do next. So you have to rely on the limited atomic semantics available - for example, instead of explicitly checking for a symlink, you can use O_NOFOLLOW - but those are not always sufficient, depending on what you’re trying to do. It shouldn’t be this way. I should be able to take a frozen snapshot of the filesystem, inspect it to my heart’s content, make the changes I want, and finally atomically commit or abort.

Regarding copy-on-write clones, can any of those filesystems actually clone arbitrary directories? In APFS’s case, you can make copy-on-write clones of files, and you can make copy-on-write snapshots of entire volumes, but you can’t just take some random directory and make a copy-on-write clone of it (without individually cloning all the files under the directory). I believe the same limitation exists for some or all of the modern Linux filesystems.

>>comex+v71
I'm giving a (slightly updated) version of my talk at the Storage Network Industry Association Storage Developer's Conference (2022) in Freemont, CA next thursday:

https://storagedeveloper.org/events/sdc-2022/agenda/2022-09-...

"Symbolic links Considered Harmful"

Might be relevant to readers :-).