They are using consumer drives.
In the last 12 months I've done 100+ pull the power plug tests while compiling i.e. while writing. Every cold boot subsequent had no complaints, normal in every way. No special commands or repairs.
Thing is? This isn't a scientific sample. It's not enough data to understand what are pseudorandom hardware problems. Do you think these tests prove I have good hardware? I don't. I know there are bugs in my SSD firmware. I just haven't been hit by one at the same time as a power plug pull.
Btrfs is tested way more than this though. Both synthetically using xfstests, including dm-log-writes tests to demonstrate proper power fail behavior by the filesystem. Facebook does these tests continuously. So do many other companies. Things can still go wrong in ways no file system can completely project you from.
But also other filesystems don't checksum my data. Why should I have confidence in my data otherwise?
--- Chris Murphy
On Thu, Jun 25, 2020 at 11:57 PM Chris Murphy lists@colorremedies.com wrote:
They are using consumer drives.
In the last 12 months I've done 100+ pull the power plug tests while compiling i.e. while writing. Every cold boot subsequent had no complaints, normal in every way. No special commands or repairs.
Thing is? This isn't a scientific sample. It's not enough data to understand what are pseudorandom hardware problems. Do you think these tests prove I have good hardware? I don't. I know there are bugs in my SSD firmware. I just haven't been hit by one at the same time as a power plug pull.
Btrfs is tested way more than this though. Both synthetically using xfstests, including dm-log-writes tests to demonstrate proper power fail behavior by the filesystem. Facebook does these tests continuously. So do many other companies. Things can still go wrong in ways no file system can completely project you from.
But also other filesystems don't checksum my data. Why should I have confidence in my data otherwise?
The answer to your question "Why should I have confidence in my data" is because of the same reason you are giving for using btrfs. Because years (decades?) of using ext4, I haven't had a single problem with my files (that I've seen), due to the filesystem, even without a checksum. xfs is not up to decades, but it's certainly around a decade, for me.
A different use case I haven't seen mentioned yet is using sd-cards. I use sd-cards a lot in small raspberry-pi type/size machines. I've been able to use some of these for several years at a time. I'm always prepared to re-install these at any time, but others might not. How does btrfs do on sd-cards?
Troy
On Fri, Jun 26, 2020 at 10:16 AM Troy Dawson tdawson@redhat.com wrote:
On Thu, Jun 25, 2020 at 11:57 PM Chris Murphy lists@colorremedies.com wrote:
They are using consumer drives.
In the last 12 months I've done 100+ pull the power plug tests while compiling i.e. while writing. Every cold boot subsequent had no complaints, normal in every way. No special commands or repairs.
Thing is? This isn't a scientific sample. It's not enough data to understand what are pseudorandom hardware problems. Do you think these tests prove I have good hardware? I don't. I know there are bugs in my SSD firmware. I just haven't been hit by one at the same time as a power plug pull.
Btrfs is tested way more than this though. Both synthetically using xfstests, including dm-log-writes tests to demonstrate proper power fail behavior by the filesystem. Facebook does these tests continuously. So do many other companies. Things can still go wrong in ways no file system can completely project you from.
But also other filesystems don't checksum my data. Why should I have confidence in my data otherwise?
The answer to your question "Why should I have confidence in my data" is because of the same reason you are giving for using btrfs. Because years (decades?) of using ext4, I haven't had a single problem with my files (that I've seen), due to the filesystem, even without a checksum. xfs is not up to decades, but it's certainly around a decade, for me.
A different use case I haven't seen mentioned yet is using sd-cards. I use sd-cards a lot in small raspberry-pi type/size machines. I've been able to use some of these for several years at a time. I'm always prepared to re-install these at any time, but others might not. How does btrfs do on sd-cards?
I haven't seriously tested this recently, but last I checked into this, it worked okay. SD cards have a higher degree of quality variance, though. And that *does* cause issues, since those bubble up way more obviously than they do in XFS and Ext4. I think you'd be more likely to notice the poor I/O bandwidth that generally affects all filesystems on SD cards than any Btrfs-specific problems, though.
-- 真実はいつも一つ！/ Always, there's only one truth!
On Thu, Jun 25, 2020 at 11:57 PM Chris Murphy <lists(a)colorremedies.com> wrote:
The answer to your question "Why should I have confidence in my data" is because of the same reason you are giving for using btrfs. Because years (decades?) of using ext4, I haven't had a single problem with my files (that I've seen), due to the filesystem, even without a checksum.
The thing you trust in this case, is not ext4 though. It's not designed to even detect the problem under discussion so you can't trust it to protect you from the problem. What you actually trust is the drive's ECC and there's a ton of data out there indicating you shouldn't.
Not least of which is a very long period of time with file systems that did not even apply checksums to their own metadata. You aren't alone in trusting the hardware vendors, even file system design implicitly trusted them. But due in part to ZFS and btrfs, and their own bug reports, they've learned of these betrayals and now ext4 and xfs also at least checksum their own metadata. If you implicitly trust the vendors, why checksum metadata?
A different use case I haven't seen mentioned yet is using sd-cards. I use sd-cards a lot in small raspberry-pi type/size machines. I've been able to use some of these for several years at a time. I'm always prepared to re-install these at any time, but others might not. How does btrfs do on sd-cards?
Less total writes than other file systems. With compression it reduces write amplification significantly, and increases sdcard lifespan by a lot (which maybe isn't saying much for consumer sdcards but set that aside for now).
I'm using btrfs on a Raspberry Pi Zero, 512 MB RAM, compress=zstd:1. Read and write performance seem about the same, perhaps reads are a touch faster but it's purely anecdotal.
I have seen these cards die in various ways, never one time a read error from the card itself. First indication are checksum mismatch errors from btrfs, i.e. the card is returning garbage for some blocks, soon thereafter the card "bricks" itself into a read-only state. i can still mount the file system and get my data out. And most of the time when it's read-only, it's still accepting write commands without error, it's just not persistent. Write a block, no error. Read that block, old data. It's plainly failing to write yet no error from the card at all.
-- Chris Murphy
On Fri, Jun 26, 2020 at 8:11 AM Chris Murphy chrismurphy@fedoraproject.org wrote:
On Thu, Jun 25, 2020 at 11:57 PM Chris Murphy <lists(a)colorremedies.com> wrote:
The answer to your question "Why should I have confidence in my data" is because of the same reason you are giving for using btrfs. Because years (decades?) of using ext4, I haven't had a single problem with my files (that I've seen), due to the filesystem, even without a checksum.
The thing you trust in this case, is not ext4 though. It's not designed to even detect the problem under discussion so you can't trust it to protect you from the problem. What you actually trust is the drive's ECC and there's a ton of data out there indicating you shouldn't.
Not least of which is a very long period of time with file systems that did not even apply checksums to their own metadata. You aren't alone in trusting the hardware vendors, even file system design implicitly trusted them. But due in part to ZFS and btrfs, and their own bug reports, they've learned of these betrayals and now ext4 and xfs also at least checksum their own metadata. If you implicitly trust the vendors, why checksum metadata?
A different use case I haven't seen mentioned yet is using sd-cards. I use sd-cards a lot in small raspberry-pi type/size machines. I've been able to use some of these for several years at a time. I'm always prepared to re-install these at any time, but others might not. How does btrfs do on sd-cards?
Less total writes than other file systems. With compression it reduces write amplification significantly, and increases sdcard lifespan by a lot (which maybe isn't saying much for consumer sdcards but set that aside for now).
I'm using btrfs on a Raspberry Pi Zero, 512 MB RAM, compress=zstd:1. Read and write performance seem about the same, perhaps reads are a touch faster but it's purely anecdotal.
I have seen these cards die in various ways, never one time a read error from the card itself. First indication are checksum mismatch errors from btrfs, i.e. the card is returning garbage for some blocks, soon thereafter the card "bricks" itself into a read-only state. i can still mount the file system and get my data out. And most of the time when it's read-only, it's still accepting write commands without error, it's just not persistent. Write a block, no error. Read that block, old data. It's plainly failing to write yet no error from the card at all.
The quality of sd-cards is all over the place, I totally agree. Nowadays, if it isn't the brand, and specific type from that brand, that I use, then it isn't worth my time. Even if someone gives me one for free. Especially if someone gives me one for free.
And, the quality of many of the small pi sized machines, isn't all that good. I was going to list examples, but I'll keep it short. Enough to say, I'm always prepared to re-install these at any time.
Thanks for the info about btrfs on sd-cards.
Troy
There are "industrial" sd cards that cost more. Maybe $50 for a 4G. But they are almost impervious. They are expensive but a good value if you want something more reliable and get sick of buying new sd cards (or dealing with warranty returns). ATP, Delkin, etc brands that don't really come up in consumer space. These are almost all SLC. Much lower density. So they aren't as fast either. While you don't need compression as much to mitigate write amplification failures you'll benefit from space savings considering you'll end up with much less flash for the same price as consumer.
-- Chris Murphy
-
Chris Murphy -
Chris Murphy -
Neal Gompa -
Troy Dawson