Setting Up Quotas & Reservations in OpenZFS in Linux

ZFS supports quotas and reservations at the filesystem level. Quotas in ZFS set limits on the amount of space that a ZFS filesystem can use. Reservations in ZFS are used to guarantee a certain amount of space is available to the filesystem for use for apps and other objects in ZFS. Both quotas and reservations apply to the dataset the limits are set on and any descendants of that dataset.

Primarily, quotas are set to limit the amount of space that a particular user of the system can consume so that no one user maximizes the Zpool in the dataset. In this article, I have setup two users in the Linux system: datapioneer (myself as the primary user) and dante (a secondary user of the Linux system) upon which OpenZFS on Linux has been installed.

Let's look at the zpool that I currently have setup in Linux for this example. Running both zpool status and zfs list commands in Debian 10 Linux produce the following profile of the ZFS filesystem:

Here, I have a RAIDZ of 3 SCSI drives with a logs mirror and two-drive SSD Zil SLOG. All drives are ONLINE and the zfspool is also ONLINE and healthy. As shown, the zfspool/data dataset is using 12M of data out of a total space of 19.0G. This means that both datapioneer and dante have access to this total 19.0G of zfspool drive space across the drives. I would like to limit the user, dante, to 5G of drive space so he doesn't inadvertently fill up the Zpool with data. In addition, I would like to reserve 5G of the Zpool under the dataset zfspool/apps for applications. Here's how we can accomplish both.

To limit 5G of drive space as a quota for dante under zfspool/data, we can run the following command:

Setting the quota for dante under the zfspool/data/dante dataset to 5G is performed using: zfs set quota=5G zfspool/data/dante. In the figure above, I ran a zfs list to look at the current listing of drive utilization and you can see that dante has the entire 19.0G available to him. After running the zfs set quota command, we can confirm that we have limited dante to 5G by running the zfs get quota zfspool/data/dante command showing a value of 5G for the property: Quota. Rerunning zfs list also shows that only 4.99G, ~ 5G, is currently available to dante. However, note that the user datapioneer still has full use of the zpool wherein the amount of 19.0G of space is showing as available. The user dante will only be able to fill up 5G of space under the zfspool/data/dante dataset and any attempts to add more data to the drives will be denied. Now, if we add a child dataset called ws under zfspool/data/dante, the quota for /zfspool/data/dante/ws inherits the 5G limitation:

Now, let's set a 5G reservation for apps under /zfspool/apps. What this means is that the ZFS filesystem will set aside 5G of drive space for use by apps only and nothing else will be able to use it. This is different from quotas set up in the system since setting quotas allow drive space use up to a certain value whereas establishing a reservation tells the ZFS filesystem not to allow use of the reserved amount by any object in the system other than apps under /zfspool/apps. To set a reservation of 5G for apps dataset in ZFS, we can run the command:

After setting the 5G reservation in ZFS for apps, we see that the amount USED for zfspool in the system has increased from 24.4M to 5.02G (which includes the 5G reservation for apps) with 14.0G AVAIL instead of 19.0G AVAIL prior to setting up the reservation. Also, note that the amount available for apps under /zfspool/apps remains at 19.0G which does not establish a quota of any kind since this amount AVAIL remains unchanged before and after the reservation was set. However, what should be realized here is that the amount of space AVAIL to dante remains at 5G (as established in the Quota we setup earlier) but, the total amount AVAIL for datapioneer has been reduced from 19.0G to 14.0G (the difference being 5G) which reflects the fact that 5G of space has been reserved for apps and is no longer available to any other object in the zfspool system.

Designating "Hot Spares" in the Storage Pool in OpenZFS - Part 6