With btrfs still unstable (I did give it a chance) and Ext4 lacking support for on-the-fly compression and deduplication, ZFS may yet be the only choice if you need such features. Of course, without its being in the kernel, using it as the main filesystem for a GNU/Linux machine is tricky. There are various ways to get Debian GNU/Linux to run on a ZFS root filesystem using zfs-fuse, most of which involve the use of debootstrap and a fair bit of initramfs hacking.

If you don't have a ZFS-ready boot medium on hand that you can use debootstrap with, however, this may not be an option. In particular, the official Debian CD set does not include zfs-fuse as a loadable installer module.

On the other hand, if you can live with / being on a more conventional filesystem, ZFSing /home, /usr, and /var, while not for the faint of heart, is perfectly possible. The following instructions are from memory, so proceed with care, and in particular, keep a rescue disc handy. I cannot guarantee the correctness of any of the following instructions. In fact, errors are very likely! You follow them on your own risk.

Step One: Install Debian GNU/Linux

Note: This step assumes that you are installing a fresh copy of Debian GNU/Linux. If your system is already up and running (on a non-ZFS filesystem), you may skip this step.

First, create a root partition large enough to hold the base system. In theory, a couple hundred megabytes should suffice, but reserving a gigabyte or so doesn't hurt unless you're low on disc space. You may choose to reserve a partition for ZFS at this point as well.

Next, install the base system as usual and let the system reboot.

Finally, on the new Debian system, install the “zfs-fuse” package:

$ aptitude install zfs-fuse

Step Two: Create the ZFS pool and volumes

Note: This step assumes that you are not planning to use an existing ZFS pool. If you do, you may skip this step.

Assuming that /dev/sdaZ is your designated ZFS pool partition, issue the following commands (if you want to use raidz or desire the zpool to span multiple physical devices, you need to consult the ZFS manual):

$ zpool create -O compression=on -O dedup=on tank /dev/sdaZ
$ for x in tank/HOME tank/DEBIAN tank/DEBIAN/var tank/DEBIAN/usr
  do
    zfs create -o compression=on -o dedup=on $x
  done

Depending on your use case, you may want to create additional volumes for, say, /usr/src, /usr/share/doc, /usr/local, et cetera. In this case, deduplication and compression can be enabled on a case-by-case basis.

Step Three: Move /home, /usr, and /var to ZFS

Now for the tricky part. Be sure to question each step thoroughly before hitting the enter key to confirm it.

/home

Moving /home is ridiculously simple:

$ cd /
$ mv home home.OLD
$ zfs set mountpoint=/home tank/HOME
$ mv home.OLD/* /home/
$ rmdir home.OLD

/var

For /var, boot into single-user mode, and do the following:

$ mv /var /rawvar
$ mkdir /var
$ cd /var
$ ln -s ../rawvar/{lock,log,run,tmp} .
$ mv -f /rawvar/{backups,cache,lib,local,mail,opt,spool} /tank/DEBIAN/var/
$ cp -a /rawvar/* /tank/DEBIAN/var/
$ zfs set mountpoint=/zfsvar tank/DEBIAN/var
$ mount -o bind /zfsvar /var

The reason for the general complication is that the zfs-fuse tools rely on at least /var/run to communicate with the zfs-fuse daemon. The reason for not mounting the ZFS var volume at /var directly is that for some reason, the ZFS tools refuse to mount volumes in non-empty directories.

/usr

For /usr, remain in single-user mode, and do the following:

$ cd /usr/lib
$ for x in libz.so* libfuse.so* libcrypto.so*
  do
    dpkg-divert --divert /lib/$x /usr/lib/$x
    mv -vi $x /lib/
    ln -vs /lib/$x
  done
$ cd /
$ mv /usr /usr.OLD
$ zfs set mountpoint=/usr tank/DEBIAN/usr
$ zfs set mountpoint=legacy tank/DEBIAN
$ zfs set mountpoint=legacy tank
$ mv /usr.OLD/* /usr/
$ rmdir /usr.OLD

Yes, this is a bit of a hack, and it will break whenever the libraries' version numbers change. Remember to maintain the library diversions whenever the respective packages are updated or you may end up with an unbootable system!

Step Four: Do startup script fixups

As it is, Debian will not boot properly. This is because of two things: One, zfs-fuse refuses to import the filesystem on startup (don't ask me why). Two, /var is not populated with the stuff that ought to be there.

We will solve both of the above by writing a simple init script that will initialize the zpool and bind-mount /zfsvar at /var.

Create a file called /etc/init.d/mulkzfs with the following contents:

### BEGIN INIT INFO
# Provides:          mulkzfs
# Required-Start:    zfs-fuse
# Required-Stop:     zfs-fuse
# X-Start-Before:    console-setup networking
# Default-Start:     S
# Default-Stop:      0 6
# Short-Description: Mulky ZFS post-initialization
# Description:       Post-initializes ZFS mulkily.
### END INIT INFO

case "$1" in
  start)
    echo "Setting mulky ZFS stuff up."
    /sbin/zpool import tank || :
    /bin/mount -o bind /zfsvar /var
    ;;
  stop)
    echo "Shutting mulky ZFS stuff down."
    /bin/umount /var
    ;;
esac

Make the file executable:

$ chmod +x /etc/init.d/mulkzfs

Adapt the zfs-fuse and FUSE init scripts a bit so they don't depend on the networking stuff to be set up before they are called (lest insserv shall complain about cyclic dependencies):

$ for x in zfs-fuse fuse; do
    sed --in-place 's/remote_fs/local_fs/g' /etc/init.d/$x
  done

And finally, run insserv to have all init scripts rearranged as needed:

$ insserv -v mulkzfs
$ insserv -v

Wrapup

The above should leave you with a hopefully booting, ZFS-based system. If you stumble upon any mistakes in the instructions, please let me know in the comments so I can fix the bugs. If, on the other hand, the instructions actually worked for you, that would be useful information as well.

Good luck!