zlacker

An unpopular opinion, but NFS is super handy and useful in reliable private networks with centralised authentication. Sure, it has its downsides that are being worked on with newer versions of the protocol(4+) with addition complexity, but it sure is useful in closely controlled setups like for HPC clusters.

I ran a HPC cluster for an University, and relied upon good old NFSv3 for shared file storage(both home directories, and research datasets). In addition I also built out a big set of softwares compiled in one server and made available to the entire cluster via a read-only NFS mount point. The whole thing works so reliably without any hiccups whatsoever. To over some the limitations of authentication and authorisation with NFS storage, we use a centralised FreeIPA server that allows all machines in the cluster have the same UID/GID mapping everywhere.

As a cream on top, the storage we expose over NFS is ZFS, that integrates nicely with NFS.

Update 1: Yes, data security is a bit of an afterthought with NFS. As anybody in my network with physical access can mount my central storage to another server physically and access data as long as they can recreate UID/GID locally.. but, if I let someone to do this physically, I have bigger problems to deal with first.

>>reacha+(OP)
You can fix the network access problem by using IPsec.

>>reacha+(OP)
NFSv3 over wireguard is a well-kept secret weapon.

All those IP-based ACLs are suddenly useful...

>>teddyh+W
NFS over TLS:

* https://datatracker.ietf.org/doc/html/draft-ietf-nfsv4-rpc-t...

>>reacha+(OP)
> I ran a HPC cluster for an University, and relied upon good old NFSv3 for shared file storage(both home directories, and research datasets).

Used in lots of places if they don't want to go GPFS, Lustre, maybe CephFS nowadays. Dell-EMC Isilon is used in lots of places for NFS (and SMB): I worked at a place that had >10PB in one file system/namespace (each node both serves traffic and has disk/flash, replicated over a back-end).

> […] we use a centralised FreeIPA server that allows all machines in the cluster have the same UID/GID mapping everywhere.

(Open)LDAP is still very handy as well and used in many places. (AD is technically LDAP+Kerberos.)

>>throw0+hq
Latest draft expired more than a year ago?

>>teddyh+Xr
Implementations for Linux and FreeBSD:

* https://patchwork.kernel.org/project/cifs-client/cover/16503...

* https://www.freshports.org/sysutils/nfs-over-tls/

Activity on the NFSv4 mailing list:

* https://mailarchive.ietf.org/arch/browse/nfsv4/

But no recent commits to the draft:

* https://github.com/chucklever/i-d-rpc-tls

¯\_(ツ)_/¯

>>reacha+(OP)
Pretty much the same setup we run at the university I work for, though for the whole department instead of just one cluster. Combination ZFS exporting shares with it's built in server and controlling autofs mounts from FreeIPA makes it a pretty easy to use system.

Out of curiosity, did you ever try Kereberized NFS for extra security? We tested it out a while back (and still use it in some small circumstances) but never got it stable enough for production use.

Side-note: I wouldn't be surprised if LDAP+NFS is still pretty common across universities, either as a holdover from Sun days or just out of practicality.

>>stryan+oQ
We (well, the IT group at a previous job) used kerberized NFS with Ubuntu (16.04 and 18.04 IIRC) and netapp filers, worked fine.

> Side-note: I wouldn't be surprised if LDAP+NFS is still pretty common across universities, either as a holdover from Sun days or just out of practicality.

Yes, absolutely. Most large enterprises, be it universities or big companies, have some kind of centralized directory (nowadays probably Microsoft AD), and machines (servers and end user clients) are then configured to lookup user and group info from there.