BTRFS
BTRFS
BTRFS indeed is a better filesystem, featuring extent based file storage, copy-on-write (COW), easy snapshots and filesystem compression, deduplication and defragmentation. It supports in place upgrade from ext2/3/4 to btrfs with an optional rollback. It can even rebalance itself.
What it does not support is encryption, nor swap files.
Create
mkfs.btrfs -f -L testbtrfs /dev/sdd1 /dev/sdd2 /dev/sdd3
mkfs.btrfs -L home /dev/sdd1
[root@oel7 ~]# mkfs.btrfs -L home /dev/sdd1
btrfs-progs v3.19.1
See http://btrfs.wiki.kernel.org for more information.
Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
fs created label home on /dev/sdd1
nodesize 16384 leafsize 16384 sectorsize 4096 size 30.00GiB
Migrate
Btrfs supports in place migration from ext2/3/4 with an option to rollback. The original extfs image stays untouched after the migration, and it can be used to rollback the entire extfs-btrfs upgrade. This is offline upgrade.
The migration is offline, so the filesystem needs to be unmounted first and cleaned by forcing fsck on it.
umount /dev/sdb1
fsck -f /dev/sdb1
Only then the convertion may start and it takes quite a long time.
btrfs-convert /dev/sdb1
mount /dev/sdb1 /btrfs
btrfs subvolume list /btrfs
ID 256 gen 25 top level 5 path ext2_saved
Directly after the conversion the btrfs can be mounted rw, the extra space is consumed by btrfs metadata only. The actual file data stays on extfs and changes are performed by making a differential COW to ext2 image.
The ext2_saved subvolume contains a file named “image” which indeed is
an image of extfs. It can be accessed (read-only) by mounting it through
a loop device. When it is not required, it can be deleted:
btrfs subvolume delete /btrfs/ext2_saved
When we are happy with the new btrfs it can be defragmented and balanced:
btrfs filesystem defrag -r /btrfs
btrfs balance start /btrfs
Conversion process explained in detail here.
Subvolumes and Snapshots
Btrfs subvolume has its hierarchy and relations between other subvolumes. A subvolume in btrfs can be accessed from the parent subvolume (as a subdirectory) or as separately mounted filesystem. A btrfs filesystem consists of one or more subvolumes, the default one’s id is 5 (alias 0).
A btrfs snapshot is also a kind of a subvolume, sharing its data with the parent subvolume and possibly other snapshots.The snaphost can be writable but due to its nature, the subsequent changes on it are visible in the snapshot itself and not in its source snapshot.
[root@oel7 ~]# btrfs subvolume create /home/sub1
Create subvolume '/home/sub1'
[root@oel7 ~]# btrfs subvolume create /home/sub2
Create subvolume '/home/sub2'
[root@oel7 ~]# btrfs subvolume list /home
ID 258 gen 14 top level 5 path sub1
ID 259 gen 15 top level 5 path sub2
External Articles on Subvolumes
- Subvolumes and Snapshots in LWN
Snapper
Snapper is a tool which helps with managing snapshosts. It can compare snapshosts, revert between snapshots and supports automatic snapshots timelines.
Configuration
[root@oel7 ~]# ls /home
jxa oracle sub1 sub2
[root@oel7 ~]# snapper -c home create-config /home
[root@oel7 ~]# snapper -c home list
Type | # | Pre # | Date | User | Cleanup | Description | Userdata
-------+---+-------+------+------+---------+-------------+---------
single | 0 | | | root | | current |
[root@oel7 ~]# snapper list
Unknown config.
[root@oel7 ~]# snapper -c root create-config /home
Creating config failed (subvolume already covered).
[root@oel7 ~]# snapper delete-config home
Command 'delete-config' does not take arguments.
[root@oel7 ~]# snapper -c home delete-config
[root@oel7 ~]# snapper -c root create-config /home
[root@oel7 ~]# snapper list
Type | # | Pre # | Date | User | Cleanup | Description | Userdata
-------+---+-------+------+------+---------+-------------+---------
single | 0 | | | root | | current |
[root@oel7 ~]# snapper list-configs
Config | Subvolume
-------+----------
root | /home
/home config
snapper set-config ALLOW_USERS="oracle"
snapper set-config TIMELINE_LIMIT_HOURLY="24"
snapper set-config TIMELINE_LIMIT_DAILY="14"
snapper set-config TIMELINE_LIMIT_MONTHLY="6"
snapper set-config TIMELINE_LIMIT_YEARLY="0"
snapper get-config
Key | Value
-----------------------+-------
ALLOW_GROUPS |
ALLOW_USERS | oracle
BACKGROUND_COMPARISON | yes
EMPTY_PRE_POST_CLEANUP | yes
EMPTY_PRE_POST_MIN_AGE | 1800
FSTYPE | btrfs
NUMBER_CLEANUP | yes
NUMBER_LIMIT | 50
NUMBER_MIN_AGE | 1800
SUBVOLUME | /home
TIMELINE_CLEANUP | yes
TIMELINE_CREATE | yes
TIMELINE_LIMIT_DAILY | 14
TIMELINE_LIMIT_HOURLY | 24
TIMELINE_LIMIT_MONTHLY | 6
TIMELINE_LIMIT_YEARLY | 0
TIMELINE_MIN_AGE | 1800
cd /home
mv jxa/.cache jxa/.cache-old
btrfs subvolume create /home/jxa/.cache
chmod 700 jxa/.cache
chown jxa:jxa jxa/.cache
mv jxa/.cache-old/* jxa/.cache/
rmdir jxa/.cache-old
# other subvolumes @home
jxa/.virtualbox
jxa/Downloads
jxa/tmp
jxa/VirtualBox VMs
jxa/.cache
root fs config
mv log log-old
btrfs subvolume create /var/log
chmod 775 log
chown root:syslog log
mv log-old/* log
rmdir log-old
mv cache cache-old
btrfs subvolume create /var/cache
chmod 755 cache
mv cache-old/* cache
rmdir cache-old
mv /tmp /tmp-old
btrfs subvolume create /tmp
chmod 1777 /tmp
mv /tmp-old/* /tmp
rmdir /tmp-old
mv /var/tmp /var/tmp-old
btrfs subvolume create /var/tmp
chmod 1777 /var/tmp
mv /var/tmp-old/* /var/tmp
rmdir /var/tmp-old
snapper -c root2 create-config /
root@ubuntu15b:/var# snapper list-configs
Config | Subvolume
-------+----------
root | /home
root2 | /
snapper -c root2 set-config ALLOW_USERS="jxa"
snapper -c root2 set-config TIMELINE_LIMIT_HOURLY="24"
snapper -c root2 set-config TIMELINE_LIMIT_DAILY="7"
snapper -c root2 set-config TIMELINE_LIMIT_MONTHLY="0"
snapper -c root2 set-config TIMELINE_LIMIT_YEARLY="0"
snapper -c root2 get-config
Suse recommends to create following subvolumes:
/tmp
/opt
/srv
/var/crash
/var/spool
/var/log
/var/run
/var/tmp
Tuning
snapper
I like snapper to ignore a list of directories when doing
reports (comparing changes). So I add a filter file
/etc/snapper/filters/mysnap.txt
with following lines:
*/.git
*/.cache
updatedb
Prevent updatedb of indexing .snapshots directories.
The configuration to adjust is located in /etc/updatedb.conf:
PRUNENAMES=".git .bzr .hg .svn .cache .snapshots"
kernel
Use the latest kernel available. Even with LTS Ubuntu (14.04 at the moment) you may use 4.2 kernel.
Maintenance
BTRFS occupies more space when snapper is active. Depending on configuration and on the filesystem activity, it might actually occupy much more than a regular filesystem does. So the rule of thumb is to plan btrfs size at least twice what a regular filesystem would take. If possible, plan it more.
monitoring
Simple df is not enough at BTRFS filesystem because it does not count in
snapshot data. More appropriate command is btrfs filesystem show:
root@Ubuntu16a:~# btrfs filesystem show /home
Label: 'home' uuid: b7dfa550-e798-463d-ad65-78d0002318a8
Total devices 1 FS bytes used 9.59GiB
devid 1 size 30.00GiB used 12.31GiB path /dev/sdb1
The filesystem above shows 9.59GB of filsystem data but it takes 12.31GB at BTRFS (devid line).
Planning wide space margin might not be enough, particularly during often
system updates and upgrades the space pressure might go high and
other-than-fs-increase solution is often required. In such the situation you
may sacrifice snapshots history or remove some of them selectively. The command
you are searching for is snapper delete (remove|rm) number | number1-number2,
for example:
snapper -c root2 delete 63-97
Above removes some incremental changes and usually releases sometimes significant amount of diskspace.
challenges
It happend deleting most of my snapshots was not enough and
btrfs fi show still reported the filsystem being full.
Marc Merlin wrote good article on the issue.
Below are his advices:
Unfortunately it’s not uncommon for a btrfs device to fill up due to the fact that it does not rebalance chunks (3.18+ has started freeing empty chunks, which is a step in the right direction).
btrfs balance start /home &
while sleep 5;
do btrfs fi show /home;
btrfs balance status -v /home;
done
Resources
- BTRFS wiki.kernel.org - primary source of information on BTRFS.
- How to manage BTRFS Storage Pools, Subvolumes and Snapshots - RedHat documentation on LVM CLI.
- Incremental Backup
- Subvolumes and Snapshots - an LWN article.
- Snappper - automatic snapshots.
- Tuning Snapper - ArchLinux Wiki.
- ArchLinux on BTRFS
- BTRFS in Oracle Enterprise Liinux
- BTRFS features, Oracle’s summary.
- Fixing BTRFS full problems
- Moving BTRFS subvolume to another disk