I've used almost all the major NAS operating systems/software for DIY NASes so I wanted to write up a few thoughts on each. At the time of this writing, ZFS is the only serious contender for NAS filesystems, so this review only covers operating systems with ZFS support. At this time, HAMMERFS is still only available on Dragonfly BSD which does not have support for jails and only recently gained VM support. This makes Dragonfly BSD not a suitable host for software such as Plex. At this time, BTRFS is still not stable, and is only available on Linux.

I have installed all these operating systems on the same hardware/ZFS pool, so ZFS has been easily passing the vendor lock-in test. I haven't had any data loss or incompatibility importing my NAS pool in to any of these installs. My comments are only relating to the administration and general use as a NAS.

• FreeNAS: Altogether, a pretty nice piece of software. This gives you the "router webpage UI" feel for your NAS and makes it incredibly easy to one-click most common actions. FreeNAS is just a fancy web interface on top of a standard FreeBSD installation, so there is no worry of data lock-in here. The backend software driving the web interface is rather slow, and FreeNAS is picky about you not installing any software in the "main" system. You should make a jail or VM to install any tools (such as 7zip) which can make basic tasks like unzipping downloads on your NAS annoying. The preferred method would be to unzip them locally, then copy the files over. FreeNAS also had a major software rewrite during one of the major FreeBSD transitions which involved them completely dropping jail support unannounced and the majority of their userbase along with it. They have since reverted this change. A really good OS if you don't want to deal with how painful the various network sharing service configurations are.
• FreeBSD: FreeBSD feels refreshing to use, like older Debian installations. It doesn't fight with you and doesn't have many opinions. For me, the system became a death by many cuts situation. You have to configure all your network sharing services. Samba is never fun to configure, Avahi is miserable. Ezjails aren't really as "EZ" as described. Packages aren't built with an ffmpeg that can transcode MP3 so you will likely have to build all of X in ports (a 3-4 hour endeavor, plus learning the system) to play your music. Once you get your jails created, keeping both them and the base system updated is another adventure. You have all the tools available, but if you don't have heavy FreeBSD administration experience, the system becomes a real drag to maintain.
• Illumos: It just ain't Linux. It ain't BSD. It's not bad, but it's just too far away from what you're probably familiar with to get used to using. (If you're a Solaris administrator, ignore this.) The man pages are great, but what the system is lacking is high level "getting started" guide to introduce and give you good defaults for setting up a NAS. There is some amazing technology here, and I would definitely recommend playing with Illumos at some point, but for a NAS, it was just an uphill battle to learn the system and get back to the same functionality I had with FreeNAS. The package system is well integrated with the zones ("jails"), but setting up both zones and VMs involved hand-editing some JSON. If you have some network administration experience, Illumos may feel very comfortable to you. If you have a non-IT background, I don't recommend using it on your NAS (but still try it!)
• NetBSD: It just feels ancient. I come from an OpenBSD background, and it's missing all the quality of life I would expect out of OpenBSD. If you come from a Linux background, I would recommend trying NetBSD in a VM first to get the feel for it. I got annoyed while trying to set up my disks (the disk subsystem is just slightly different from both FreeBSD and OpenBSD) and the system didn't last much longer past that. The documentation is well-written as you would expect from a BSD project, but the man pages lack the "see also" section so it becomes tricky to find the exact tools you're looking for when you're not familiar with an area.
• Linux: SystemD. There are many jokes about NIH syndrome (not invented here), but every time I pick up Linux again, I have to learn at least one new core tool. Much like FreeBSD, you are going to have to configure all the network sharing services yourself, but unlike FreeBSD you are more likely to find some help on the internet for the exact version your distro ships with so it won't be as painful (usually). Linux is also lacking in the platform stability you will find in other systems (such as the BSDs). The distros that can/do provide a more modern set of packages also tend to have an unstable base that wants frequent updates and suffers from occasional breakage during updates. The distros that provide a more stable base are plagued by woefully out of date packages. This is not the type of situation you want for a NAS.
• NixOS: NixOS is a rather opinionated package and configuration management system that hides a Linux underneath. You're not allowed to touch the Linux, just edit your nixos.conf. If you can't nixos.conf it, you are in for a miserable time. In practice, this tends to work out great for a NAS. It breaks the packages away from the base system as you find in the BSD world, and splits your configuration away from both. This provides a much more gentle segregation than the FreeNAS "one jail, one binary" approach to getting CLI tools on your NAS while still making it incredibly easy to throw out and recreate your installation. The nixos.conf feels like all the wisdom of the Archlinux wiki distilled down in to a tiny service block. All in all, this approach seems to work great for configuring servers and makes service management on the NAS pretty enjoyable while still maintaining control of the parts you want. The documentation for NixOS is rather spotty, and the main manual is a several hundred page HTML file that seems designed to choke up all major browsers.

At the time of this writing, I am using NixOS on my NAS and would highly recommend either NixOS or FreeNAS, unless you have substantial experience using another system.