Should I trust Ted Ts'o ?
Par benoit12 le vendredi, avril 3 2009, 14:52 - Lien permanent
Ted Ts'o says that the good behaviour for saving a file is
One argument that has commonly been made on the various comment streams is that when replacing a file by writing a new file and the renaming “file.new” to “file”, most applications don’t need a guarantee that new contents of the file are committed to stable store at a certain point in time; only that either the new or the old contents of the file will be present on the disk. So the argument is essentially that the sequence:
fd = open(“foo.new”, O_WRONLY); write(fd, buf, bufsize); fsync(fd); close(fd); rename(“foo.new”, “foo”);
and he adds :
Openoffice, being a portable application, that has to work on other operating systems and filesystems (for example, like Solaris's UFS), does do open/write/close/fsync/rename. So you're safe if you're using OpenOffice (and emacs, and vim).
So i looked on my Debian/Sid at my favorite editors:
- Vim 7.2
- Emacs 22
- Gedit 2.24.0
- OpenOffice.org 3.0.1
see how they really do (and if they preserve ACL when saving a file named “foo”).
Only the Paranoid survive
Vim
- copy “foo” to “foo~”, with protective mode and ACL
- write the new content to “foo”
- fsync on “foo”
FAILED
- not what Ted Ts'o told us ! If a crash happens, you get a corrupted “foo” and as the backup “foo~” is not fsync'ed, it may be also corrupted. Complete disaster;
- file doesn't change, so it's the same inode, same mode and same ACL;
- writes the file content 2 times (backup + new).
(PS: also checked vi on solaris, it behaves just like vim, total disaster.)
Emacs
- rename “foo” to “foo~”
- re-create “foo” for writing the new content
- fscync on “foo”
- chmod to restore mode
- remove “foo~”
FAILED
- again, not what Ted Ts'o told us :) If a crash happens, you get a corrupted “foo”, but at least the backup file is safe. Half-disaster;
- file has been re-created, so it's a new inode, ACL are not preserved.
Gedit
- open “.gedit-save-XXXXXX”
- chmod + chown + setxattr on that temp file to restore mode and ACL
- write to that temporary file the new content
- rename “foo” to “foo~”
- rename “.gedit-save-XXXXXX” to “foo”
FAILED
- no fsync at all ! If it wasn't missing, it would do what Ted Ts'o's advises;
- Half-disaster: “foo” may be incomplete, you would have to restore the “foo~” backup file (actually it's the original file).
OpenOffice.org
- backup “foo” to ~/.openoffice.org/3/user/backup/
- overwrites “foo” reading data from a temporary in “/tmp/svmpb.tmp/”
FAILED
- no fsync;
- Complete disaster: both “foo” and its backup may be broken;
- writes twice the data;
- file doesn't change, so it's the same inode, same mode and same ACL.
Suddenly, i don't feel like editing /etc/passwd with these text editors (but i haven't find a replacement yet). If the server was to crash, it might not reboot at all. Brrrr.
TODO:
- look/open bug for emacs about ACL;
- check newer gedit;
- find out why Ted Ts'o thinks vim/emacs/openoffice behave safely.
It looks like Ted Ts'o was wrong :/
Commentaires
you can use gedit 2.26 ;)
"Suddenly, i don't feel like editing /etc/passwd with these text editors (but i haven't find a replacement yet). If the server was to crash, it might not reboot at all. Brrrr."
Aren't the odds of this happening insanely small? Do the servers you work on not have a UPS and encounter Kernel oops on a regular basis?
If so, perhaps you have more to worry about than the saving behavior of your text editor.
Honestly the only time I'm worried about all these fsync/saving issues I see floating around are on my laptop which has some flaky hardware drivers and therefore stands a realistic chance of oopsing between fsyncs on me. Even then I don't think I've ever encountered data loss yet.
And wait a second... aren't you supposed to be using tools like usermod, useradd, etc to alter /etc/passwd?!
I think this nicely highlights how user error is still the greatest threat to system stability. ;-)
schmichael > Don't forget about network filesystems. There are much more likely to fail. /etc/passwd was an example, but that kind of thing could happen with any important text file in /etc. (/etc/init.d/sshd ? ;). And we're only talking about text editor :)
But you're right: carefully written tools such as useradd are safe (Ted T'so's pattern)
Paolo > I knew it :) I'll check this as soon as possible.
Emacs backup behaviour can be modified with a couple of settings.
There's a backup-by-copying variable, setting it to t should avoid the save-to-new-inode-and-lose-ACL problem. It's off by default.
oliv > This emacs option would make it as bad as vim. I don't care about the inode, i care about the ACL. Gedit solved this problem using libacl, i guess vim too.
nice, constructive post, thanks for sharing, truly in spirit of proper GNOME developer, aye?
gah.
You said Vim can "corrupt" the old version of the file because it doesn't fsync() the rename: is this true? Doesn't safely renaming files fall under the jurisdiction of the filesystem? Doesn't journaling in ext3/jfs/etc. take care of this problem?
gpasswd WIN?
open("/etc/group+", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 5
umask(077) = 0777
fchown(5, 0, 0) = 0
fchmod(5, 0644) = 0
fstat(5, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
write(5, "root:x:0: daemon:x:1: bin:x:2: sy"..., 1341) =1341
fsync(5) = 0
close(5) = 0
lstat("/etc/group", {st_mode=S_IFREG|0644,st_size=1335, ...}) = 0
rename("/etc/group+", "/etc/group") = 0
Hello Donny, here journaling does not help (unless you have a battery-backed memory, then you can replay 'lost' I/O on reboot). Before hiting the disk, data stays in memory, and if you crash while writing the file to disk, you're write is incomplete (data in memory vanishes). That's why the way vim/emacs/openoffice behave is not safe: they make a backup of your data and then start to overwrite your file. If your computer crashes during this overwrite, then your file is at best truncated. Of course you can pull the backup, but that's quite annoying ... and if the backup file has not been synced to disk before the crash, you lost the two of them. Ted Ts'o explains how to write robust application by writing a new file, syncing it to the disk and then renaming it (which atomic). This way, on a crash, you either get the old or the new file, anyway a valid file.
Russ > yep, passwd/useradd/etc WIN. They match exactly the pattern Ted Ts'o describes.
If the file had hard links before it should have the same hard links afterward. That's why I don't like the rename() approach for normal editing - but I do use it for the likes of passwd, shadow, sudoers.
"Suddenly, i don't feel like editing /etc/passwd with these text editors (but i haven't find a replacement yet). If the server was to crash, it might not reboot at all. Brrrr."
You should never directly edit /etc/passwd or /etc/group. That's why 'vipw' and 'vigr' exist. They have 'vi' in the name but set the EDITOR env. var. to whatever you want.
"Suddenly, i don't feel like editing /etc/passwd with these text editors (but i haven't find a replacement yet). If the server was to crash, it might not reboot at all. Brrrr."
You cannot edit /etc/passwd safely even with passwd since PAM itself does not call fsync, see https://www.redhat.com/archives/pam...
Again, /etc/passwd was a (silly) example ...
tjs > Interesting link http://git.kernel.org/?p=linux/kern... but that doesn't fix emacs/vim/OOo
Antonomasia: Rename breaking hard links is sometimes desired - eg back in the day people would copy kernel trees as hardlinks which makes diff very fast, then when they edited a file the hardlink would break, so diff still worked.