Luci4 Wiki Blog - linux:btrfs

compression

Anonymous (anonymous@undisclosed.example.com) — 2025-06-08T13:56:11+00:00

BTRFS Compression - Notes

It is possible to mount a Btrfs filesystem with different compression settings and change those settings as needed. Here’s how you can do it:

Mounting with Compression: You can initially mount the Btrfs filesystem with a specified compression method. For example, to mount with maximum compression using zlib, you can use:

sudo mount -o compress=zlib /dev/sdX1 /mnt/mybtrfs

Unloading Data: After moving some data, you can unmount the drive:

sudo umount /mnt/mybtrfs

Remounting with Different Compression: You can then remount the Btrfs filesystem with a different compression option. For example, to use zstd with maximum compression, you would do:

sudo mount -o compress=zstd:9 /dev/sdX1 /mnt/mybtrfs

Important Notes

Compression Options: Btrfs supports several compression algorithms, such as zlib, zstd, and lzo. Each has different performance and compression characteristics.

Per-File Compression: Btrfs allows you to set compression options on a per-file basis, but when mounting, the specified option applies to new files created or copied to the mounted filesystem.

Existing Files: Existing files will not be recompressed when you change the mount options. To recompress existing files with a new compression method, you would need to copy them again or use a command to explicitly change their compression.

Example of Changing Compression on Existing Files

To change the compression of existing files, you can use the following command:

sudo btrfs property set /mnt/mybtrfs/yourfile compress zstd:9

Remember that existing files will not automatically change compression until you take explicit action to re-compress them.

Supported Compression Types in Btrfs

No Compression

sudo mount -o compress=none /dev/sdX1 /mnt/mybtrfs

/etc/fstab line for mounting at boot:

/dev/sdX1 /mnt/mybtrfs btrfs defaults,compress=none 0 0

No compression is applied.

Pros: Fast read/write performance; no CPU overhead.
Cons: Disk space.

Zlib Compression

sudo mount -o compress=zlib /dev/sdX1 /mnt/mybtrfs

fstab:

/dev/sdX1 /mnt/mybtrfs btrfs defaults,compress=zlib 0 0

Uses the Zlib algorithm, providing moderate compression ratios.

Pros: Good balance between compression ratio and speed.
Cons: Higher CPU usage compared to no compression.

LZO Compression

sudo mount -o compress=lzo /dev/sdX1 /mnt/mybtrfs

fstab:

/dev/sdX1 /mnt/mybtrfs btrfs defaults,compress=lzo 0 0

LZO is a fast, lightweight compression algorithm.

Pros: Very fast compression and decompression speeds.
Cons: Compression ratios are typically lower than Zlib.

Zstd Compression

sudo mount -o compress=zstd /dev/sdX1 /mnt/mybtrfs

fstab:

/dev/sdX1 /mnt/mybtrfs btrfs defaults,compress=zstd 0 0

Zstd (Zstandard) offers high compression ratios and good speed.

Pros: Can achieve better compression ratios than Zlib and LZO while maintaining decent speed.
Cons: Slightly higher CPU usage compared to LZO.

Zstd with Level Options

sudo mount -o compress=zstd:9 /dev/sdX1 /mnt/mybtrfs

fstab:

/dev/sdX1 /mnt/mybtrfs btrfs defaults,compress=zstd:9 0 0

You can specify compression levels (1-15) with Zstd for fine-tuning.

Pros: Flexibility in balancing compression ratio vs. speed.
Cons: Higher levels (e.g., 9 or above) may increase CPU usage significantly.

Summary

Compression Type	Pros	Cons
No Compression	Fast performance, no CPU overhead	More disk space required
Zlib	Good balance of speed and compression	Higher CPU usage than no compression
LZO	Very fast compression/decompression	Lower compression ratios
Zstd	High compression with decent speed	Slightly higher CPU usage than LZO
Zstd with Levels	Flexible for tuning performance	High levels increase CPU usage

snippet.cpp

by:
▖     ▘▖▖
▌ ▌▌▛▘▌▙▌
▙▖▙▌▙▖▌ ▌
 
written: a while ago. Edited: June 8 2025

error-checks

Anonymous (anonymous@undisclosed.example.com) — 2025-06-08T14:29:13+00:00

BTRFS Error Checks

To perform an error check on a Btrfs filesystem, you can use the btrfs check command, which is designed for checking and repairing Btrfs filesystems. Here’s how to perform an error check:

1. Check the Btrfs Filesystem without Making Changes:

Before performing any modifications, it's always good to run the check in a read-only mode to identify any issues.

sudo btrfs check --readonly /dev/sdX

Replace /dev/sdX with the actual device where your Btrfs filesystem is located (e.g., /dev/sda1).
This command will scan for errors but won’t make any changes.

2. Check and Repair the Btrfs Filesystem (if needed):

If you want to perform repairs, you can use the --repair option. Be cautious with this option, as it can result in data loss.

sudo btrfs check --repair /dev/sdX

3. Checking a Mounted Btrfs Filesystem:

You can also check for errors in a mounted Btrfs filesystem by using the btrfs scrub command. This scrubs the filesystem for data inconsistencies:

sudo btrfs scrub start /mountpoint

Replace /mountpoint with the actual mount point of the Btrfs filesystem (e.g., /mnt).
This process runs in the background, checking the data and metadata blocks.
To monitor the progress of a scrub operation, run:

sudo btrfs scrub status /mountpoint

4. Log Scrub Results:

After a scrub completes, you can view detailed information about any errors found or fixed using the status command mentioned earlier.

5. Repairing Specific Issues:

If you're dealing with specific issues like a corrupt superblock or other particular problems, there are more advanced options, but these should be used with caution and preferably after backing up data.

What Does `btrfs scrub` Do?

The btrfs scrub command in Btrfs is used to verify data integrity by checking for any inconsistencies between data and metadata on disk. Here's a more detailed breakdown of what it does:

Data Integrity Check: The scrub operation systematically reads all data and metadata blocks of the Btrfs filesystem, verifying that the content matches the checksums that were stored when the data was written. This ensures that the data hasn't been corrupted since it was last written.
- Error Detection: If the scrub finds any blocks that don’t match their checksum (i.e., corrupted data), it logs these errors. If redundancy (like a RAID setup) is used, it can attempt to fix the issue by reading the same data from another redundant copy (e.g., from a mirrored disk in RAID 1 or RAID 10) and automatically repairing the corrupted data.
Involves All Devices: On multi-device setups, like Btrfs RAID configurations, the scrub runs across all devices, checking for issues in any of the mirrored or striped copies of the data.

Key Points:

1. Non-intrusive: The scrub operation is safe to run while the filesystem is mounted and in use, as it doesn’t alter the data itself unless an error is detected and corrected (in RAID setups). 2. Runs in Background: It runs in the background, allowing the system to continue operating normally. You can check its progress using the status option. 3. Finds and Fixes Silent Corruption: Btrfs has built-in checksumming for both metadata and data. A scrub operation ensures that any silent data corruption (e.g., bit rot) is detected and, if possible, corrected.

How Scrub Works in RAID:

- RAID 1 (mirroring): If a block’s checksum doesn’t match the stored checksum, Btrfs will compare it with the mirrored block from the other device. If the other copy is correct, it will be used to repair the corrupted block.

- RAID 5/6 (striping with parity): If a block is corrupted, the parity blocks are used to reconstruct the correct data and repair the error.

Commands for Scrubbing

Start Scrub:

sudo btrfs scrub start /mountpoint

This begins the scrub operation on the specified Btrfs filesystem.

Check Scrub Status:

sudo btrfs scrub status /mountpoint

This shows the current progress of the scrub and any errors that have been found or repaired.

Cancel Scrub (if needed): sudo btrfs scrub cancel /mountpoint

This cancels the ongoing scrub operation.

Output Example of Scrub Status:

After completing a scrub, you can see the results with:

sudo btrfs scrub status /mountpoint

Example output:

scrub status for btrfs filesystem /mnt/volume
scrub started at Sun Sep 29 11:27:53 2024 and finished after 02:11:08
total bytes scrubbed: 500.00GiB with 0 errors

In this case, no errors were detected during the scrub of a 500 GiB filesystem.

When to Run `btrfs scrub`?

- Periodically: Regular scrubs are recommended to ensure data integrity. Many users run it monthly or quarterly. - Before Backups: It's a good practice to scrub the filesystem before taking backups to ensure that no corrupted data gets backed up. - After Power Failures or Hardware Issues: If there’s a concern about potential disk corruption (e.g., after a power failure), running a scrub can help detect any issues.

When Scrub Doesn’t Repair:

- If you're not using RAID or some form of data redundancy, scrub can detect corruption but will not be able to repair it, as there’s no redundant data to fall back on.

In short, btrfs scrub is a powerful tool for verifying and maintaining the integrity of data on a Btrfs filesystem, especially in scenarios where data redundancy (like RAID) is involved.

hd-migration

Anonymous (anonymous@undisclosed.example.com) — 2025-06-08T14:28:26+00:00

Complete Guide for Migrating to a New BTRFS Formatted Hard Drive

Using a USB enclosure to transfer data from your old hard drive to a new one is a common and practical approach, especially when you don’t have extra slots in your machine.

Considerations:

Safety: You can ensure that the new drive is fully prepared and tested before replacing the old one.
Data Verification: Transferring data over USB allows you to verify the transfer before making any changes to your system.
Transfer Speed: Depending on the USB version (e.g., USB 2.0 vs. USB 3.0), transfer speeds may vary. USB 3.0 is significantly faster and is recommended if your drives support it.
Power Supply: Ensure that the USB enclosure provides enough power for your hard drive, especially if it’s a larger 3.5-inch drive.
Data Integrity: Always verify the data after transfer, as mentioned earlier, using diff or similar tools to ensure nothing was lost or corrupted.

Format and initialize

Identify the Drive: list block devices to find your new hard drive:

lsblk

Look for the new device (e.g., /dev/sdb).

(Option 1) Partition the SSD using `parted`

Start parted with your SSD:

# Replace `/dev/sdX` with the correct device identifier.
sudo parted /dev/sdX

Create a new partition table:

mklabel gpt

Create a new partition:

mkpart primary btrfs 0% 100%

This command creates a primary partition that occupies the entire drive.

Exit parted:

quit

(Option 2) Partition the SSD using `fdisk`

Open fdisk to Create a New Partition:

Start fdisk:

# Replace sdX with your drive identifier (e.g., /dev/sdb).
sudo fdisk /dev/sdX

Create a New Partition:

Type g to create a new GPT partition table (if you want GPT; skip if you prefer MBR).

Type n to create a new partition.

Choose the partition number (default is fine).

For the first sector, press Enter to accept the default.

For the last sector, press Enter to use the entire disk (or specify a size).

Type w to write changes and exit fdisk.

Format the New Partition as Btrfs:

Format the partition you just created (e.g., /dev/sdX1):

sudo mkfs.btrfs /dev/sdX1

Mount the Partition: create a mount point and mount it:

sudo mkdir /mnt/mybtrfs
sudo mount /dev/sdX1 /mnt/mybtrfs

Verify the Filesystem: check the newly created Btrfs filesystem:

sudo btrfs filesystem show

Check if the partition is mounted:

df -h

You should see your Btrfs partition listed.

Add to /etc/fstab for Automatic Mounting: to mount the partition automatically at boot, edit the /etc/fstab file:

sudo vim /etc/fstab

/dev/sdX1 /mnt/mybtrfs btrfs defaults 0 0

Files Migration

To replace your old hard drive with the new one and copy all data—including hidden files and preserving permissions—you can use the rsync command. Here’s how to do it step by step:

Steps to Replace Old Hard Drive with New One

Mount Both Hard Drives:

Ensure both the old and new hard drives are mounted. You might have something like this:
Old drive (e.g., /dev/sda1 mounted at /mnt/old_drive)
New drive (e.g., /dev/sdb1 mounted at /mnt/mybtrfs)

If not already mounted, create mount points and mount them:

sudo mkdir /mnt/old_drive
sudo mount /dev/sda1 /mnt/old_drive

Copy Data Using rsync:

Use the following rsync command to copy all data from the old drive to the new drive while preserving permissions and copying hidden files:

sudo rsync -aAv /mnt/old_drive/ /mnt/mybtrfs/

Breakdown of options:

-a: Archive mode; preserves permissions, timestamps, and symlinks.
-A: Preserves ACLs (Access Control Lists).
-v: Verbose; shows the progress of the transfer.
Optional -X option, will not allow compression!
-X: Preserves extended attributes.

Verify the Copy: after the copying process completes, you can verify that the data has been copied correctly:

sudo diff -r /mnt/old_drive/ /mnt/mybtrfs/

This will show any differences between the two directories.

Unmount Both Drives: once the data is copied and verified, unmount both drives:

sudo umount /mnt/old_drive
sudo umount /mnt/mybtrfs

Replace the Old Drive: physically replace the old hard drive with the new one in your system.

Data Integrity

While rsync has features that help ensure data integrity, it’s a good idea to perform additional verification after the transfer to ensure everything has copied correctly. Using checksums or file comparison tools will help confirm that your data is intact and uncorrupted.

rsync itself does not guarantee data integrity, but it provides several mechanisms to help ensure that data is transferred correctly and is uncorrupted.

Here’s how you can use rsync effectively and what you can do to enhance data integrity:

Features of `rsync`

File Comparison: by default, rsync checks file sizes and timestamps to determine if files need to be copied. This helps prevent unnecessary copies.

Checksum Verification: you can use the --checksum option, which forces rsync to compare files using checksums instead of relying solely on size and timestamp. This is more resource-intensive but increases the likelihood of identifying changes or corruption:

sudo rsync -aAv --checksum /mnt/old_drive/ /mnt/mybtrfs/

Verbose Output: the -v option gives you a detailed output of the transfer, allowing you to monitor the process.

Post-Transfer Verification

To ensure that data is uncorrupted after the transfer, consider using additional verification methods:

diff Command: After using rsync, you can compare the source and destination directories using:

sudo diff -r /mnt/old_drive/ /mnt/mybtrfs/

This will report any differences between the two directories.

Checksums: you can generate checksums (e.g., using md5sum or sha256sum) for all files in both locations and compare them:

cd /mnt/old_drive && find . -type f -exec sha256sum {} + | sort > /tmp/old_checksums.txt
cd /mnt/mybtrfs && find . -type f -exec sha256sum {} + | sort > /tmp/new_checksums.txt
diff /tmp/old_checksums.txt /tmp/new_checksums.txt

NOTE: several default settings have changed in version 5.15, please make sure this does not affect your deployments:

DUP for metadata (-m dup)
enabled no-holes (-O no-holes)
enabled free-space-tree (-R free-space-tree)

snippet.cpp

by:
▖     ▘▖▖
▌ ▌▌▛▘▌▙▌
▙▖▙▌▙▖▌ ▌
 
written: a while ago. Edited: June 8 2025

trim

Anonymous (anonymous@undisclosed.example.com) — 2025-06-08T14:12:04+00:00

Guide to enable TRIM on a Btrfs filesystem

Enable TRIM on your Btrfs filesystem, either through continuous or periodic TRIM operations.

Check Your SSD

Ensure your SSD is recognized and has Btrfs installed. You can check connected drives using:

lsblk

Mount the Btrfs Filesystem with TRIM Option

You can enable TRIM either during mounting or set it up for periodic trimming.

Option A: Enable TRIM During Mounting

Mount with the discard Option: Replace /dev/sdX1 and /mnt/mybtrfs with your actual device identifier and mount point.

sudo mount -o discard /dev/sdX1 /mnt/mybtrfs

Add to /etc/fstab: To make the change persistent across reboots, add the following line to your /etc/fstab file:

/dev/sdX1 /mnt/mybtrfs btrfs defaults,discard 0 0

Option B: Schedule Periodic TRIM

In this guide I will schedule the trim-script with systemd, you could instead run the script periodically using a simple cronjob, with some tradeoffs, explained in this table:

Feature	`cron`	`systemd` timer
Ease of setup	✔️ Easier	❌ More involved
Logging	❌ Manual (unless set)	✔️ Via `journalctl`
Missed job handling	❌ Not handled	✔️ `Persistent=true`
Service dependencies	❌ None	✔️ Fully supported
Portability	✔️ Highly portable	❌ Linux only (with `systemd`)

Unmount the Filesystem (if needed): If you want to remove the discard option and use periodic TRIM instead:

sudo umount /mnt/mybtrfs

Re-mount without discard: Mount the filesystem normally:

sudo mount /dev/sdX1 /mnt/mybtrfs

Create a TRIM Script: Create a simple script for periodic TRIM. For example, create a file named btrfs-trim.sh:

sudo vim /usr/local/bin/btrfs-trim.sh

Add the following lines:

#!/bin/bash
btrfs filesystem df /mnt/hd1
sudo btrfs device stats /mnt/hd1
sudo fstrim /mnt/hd1

Make the Script Executable:

sudo chmod +x /usr/local/bin/btrfs-trim.sh

Schedule the Script with systemd: Create a systemd service and timer to run the script regularly.

Create the Service File:

sudo vim /etc/systemd/system/btrfs-trim.service

Add the following:

[Unit]
Description=Run Btrfs TRIM

[Service]
Type=oneshot
ExecStart=/usr/local/bin/btrfs-trim.sh

Create the Timer File:

sudo vim /etc/systemd/system/btrfs-trim.timer

Add the following:

[Unit]
Description=Run Btrfs TRIM weekly

[Timer]
OnCalendar=weekly
Persistent=true

[Install]
WantedBy=timers.target

Enable and Start the Timer:

sudo systemctl enable btrfs-trim.timer
sudo systemctl start btrfs-trim.timer

Verify TRIM

You can check if TRIM is functioning by monitoring your SSD’s performance and usage over time. You can also look at the output of the TRIM command you scheduled to see if it executes without errors.

snippet.cpp

by:
▖     ▘▖▖
▌ ▌▌▛▘▌▙▌
▙▖▙▌▙▖▌ ▌
 
written: a while ago. Edited: June 8 2025