Philip Prindeville wrote:
I'm a flailing at cluefulness here. Maybe someone can set me straight.
I run "yum" nightly (as as service), but I see a lot of "*.rpmnew" files being left around.
What's most bizarre is that the original RPM files haven't been changed, and often the two files have the same size, contents (and hence MD5 signature), permissions, ownership, etc. Even the same file modification date in most cases.
So why do they get left behind?
# cd /etc/security # ls -ltr chroot* -rw-r--r-- 1 root root 82 Aug 1 05:18 chroot.conf.rpmnew -rw-r--r-- 1 root root 82 Aug 1 05:18 chroot.conf # diff -c chroot.conf.rpmnew chroot.conf # mv chroot.conf.rpmnew chroot.conf #
Thanks,
-Philip
Bingo:
# ls -l /etc/security/chroot.conf* -rw-r--r-- 1 root root 82 Aug 1 05:18 /etc/security/chroot.conf -rw-r--r-- 1 root root 82 Aug 1 05:18 /etc/security/chroot.conf.rpmnew # perl -e 'print join(", ", stat("/etc/security/chroot.conf")), "\n"' 64768, 721510, 33188, 1, 0, 0, 0, 82, 1154431139, 1154431139, 1156023232, 4096, 8 # perl -e 'print join(", ", stat("/etc/security/chroot.conf.rpmnew")), "\n"' 64768, 719917, 33188, 1, 0, 0, 0, 82, 1154431128, 1154431128, 1156023323, 4096, 8 # perl -e 'print scalar localtime((stat("/etc/security/chroot.conf.rpmnew"))[9]), "\n"' Tue Aug 1 05:18:48 2006 # perl -e 'print scalar localtime((stat("/etc/security/chroot.conf"))[9]), "\n"' Tue Aug 1 05:18:59 2006 #
And there you have it.
Why are the packages being generated with a few seconds jitter?
This seems to be generating a lot of .rpmnew files gratuitously.
-Philip
On Sun, Dec 10, 2006 at 03:50:06PM -0700, Philip Prindeville wrote:
Philip Prindeville wrote:
I'm a flailing at cluefulness here. Maybe someone can set me straight.
I run "yum" nightly (as as service), but I see a lot of "*.rpmnew" files being left around.
What's most bizarre is that the original RPM files haven't been changed, and often the two files have the same size, contents (and hence MD5 signature), permissions, ownership, etc. Even the same file modification date in most cases.
So why do they get left behind?
# cd /etc/security # ls -ltr chroot* -rw-r--r-- 1 root root 82 Aug 1 05:18 chroot.conf.rpmnew -rw-r--r-- 1 root root 82 Aug 1 05:18 chroot.conf # diff -c chroot.conf.rpmnew chroot.conf # mv chroot.conf.rpmnew chroot.conf #
Thanks,
-Philip
Bingo:
# ls -l /etc/security/chroot.conf* -rw-r--r-- 1 root root 82 Aug 1 05:18 /etc/security/chroot.conf -rw-r--r-- 1 root root 82 Aug 1 05:18 /etc/security/chroot.conf.rpmnew # perl -e 'print join(", ", stat("/etc/security/chroot.conf")), "\n"' 64768, 721510, 33188, 1, 0, 0, 0, 82, 1154431139, 1154431139, 1156023232, 4096, 8 # perl -e 'print join(", ", stat("/etc/security/chroot.conf.rpmnew")), "\n"' 64768, 719917, 33188, 1, 0, 0, 0, 82, 1154431128, 1154431128, 1156023323, 4096, 8 # perl -e 'print scalar localtime((stat("/etc/security/chroot.conf.rpmnew"))[9]), "\n"' Tue Aug 1 05:18:48 2006 # perl -e 'print scalar localtime((stat("/etc/security/chroot.conf"))[9]), "\n"' Tue Aug 1 05:18:59 2006 #
And there you have it.
Why are the packages being generated with a few seconds jitter?
This seems to be generating a lot of .rpmnew files gratuitously.
Looks a bit like multilib. E.g. config files existing in two packages, an i386 and an x86_64 one.
If it were a single package and rpm upgrades it it registers that the config files have not changed, so the new one can be put in place.
But if you do that for two packages in a row that contain exactly the same config files, then the first update will have changed the config files properly, but the second update will think the user changed the config files manually and will go the rpmnew route.
So the question is: Is that a x86_64 system? If yes, then we need to think about config files in multilib. If not, then forget about the above consiracy theory. ;)
Axel Thimm wrote:
On Sun, Dec 10, 2006 at 03:50:06PM -0700, Philip Prindeville wrote:
Philip Prindeville wrote:
I'm a flailing at cluefulness here. Maybe someone can set me straight.
I run "yum" nightly (as as service), but I see a lot of "*.rpmnew" files being left around.
What's most bizarre is that the original RPM files haven't been changed, and often the two files have the same size, contents (and hence MD5 signature), permissions, ownership, etc. Even the same file modification date in most cases.
So why do they get left behind?
# cd /etc/security # ls -ltr chroot* -rw-r--r-- 1 root root 82 Aug 1 05:18 chroot.conf.rpmnew -rw-r--r-- 1 root root 82 Aug 1 05:18 chroot.conf # diff -c chroot.conf.rpmnew chroot.conf # mv chroot.conf.rpmnew chroot.conf #
Thanks,
-Philip
Bingo:
# ls -l /etc/security/chroot.conf* -rw-r--r-- 1 root root 82 Aug 1 05:18 /etc/security/chroot.conf -rw-r--r-- 1 root root 82 Aug 1 05:18 /etc/security/chroot.conf.rpmnew # perl -e 'print join(", ", stat("/etc/security/chroot.conf")), "\n"' 64768, 721510, 33188, 1, 0, 0, 0, 82, 1154431139, 1154431139, 1156023232, 4096, 8 # perl -e 'print join(", ", stat("/etc/security/chroot.conf.rpmnew")), "\n"' 64768, 719917, 33188, 1, 0, 0, 0, 82, 1154431128, 1154431128, 1156023323, 4096, 8 # perl -e 'print scalar localtime((stat("/etc/security/chroot.conf.rpmnew"))[9]), "\n"' Tue Aug 1 05:18:48 2006 # perl -e 'print scalar localtime((stat("/etc/security/chroot.conf"))[9]), "\n"' Tue Aug 1 05:18:59 2006 #
And there you have it.
Why are the packages being generated with a few seconds jitter?
This seems to be generating a lot of .rpmnew files gratuitously.
Looks a bit like multilib. E.g. config files existing in two packages, an i386 and an x86_64 one.
If it were a single package and rpm upgrades it it registers that the config files have not changed, so the new one can be put in place.
But if you do that for two packages in a row that contain exactly the same config files, then the first update will have changed the config files properly, but the second update will think the user changed the config files manually and will go the rpmnew route.
So the question is: Is that a x86_64 system? If yes, then we need to think about config files in multilib. If not, then forget about the above consiracy theory. ;)
Well, in two cases that I looked at, that turned out to be true.
Not sure if there were any exceptions. Maybe samba-common. I'll pay attention in the future when I see it happen next...
I'm still not clear, though: if the file being installed is part of the sources that's being built (i.e. it's not a generated file), and the makefile that does the install invoked "cp --preserve=timestamps" then both the .i386 and the .x86_64 copies should have an identical timestamp. Right?
-Philip
-Philip
Philip Prindeville wrote:
I'm still not clear, though: if the file being installed is part of the sources that's being built (i.e. it's not a generated file), and the makefile that does the install invoked "cp --preserve=timestamps" then both the .i386 and the .x86_64 copies should have an identical timestamp. Right?
Consider the case where the installed file(s) are build-time generated. These are *very* unlikely to have identical timestamps (having been built separately on different archs/buildhosts).
-- Rex
On Sun, Dec 10, 2006 at 10:17:41PM -0600, Rex Dieter wrote:
Philip Prindeville wrote:
I'm still not clear, though: if the file being installed is part of the sources that's being built (i.e. it's not a generated file), and the makefile that does the install invoked "cp --preserve=timestamps" then both the .i386 and the .x86_64 copies should have an identical timestamp. Right?
Consider the case where the installed file(s) are build-time generated. These are *very* unlikely to have identical timestamps (having been built separately on different archs/buildhosts).
I think that's it. Going back to Philip's example on /etc/security/chroot.conf (he didn't mention which distro, but the one matching his timestamps is FC5): On FC5/x86_64 one gets:
# rpm -qf /etc/security/chroot.conf pam-0.99.5.0-5.fc5 pam-0.99.5.0-5.fc5 # TZ=C ls -ld --full-time /etc/security/chroot.conf -rw-r--r-- 1 root root 82 2006-08-01 11:18:48.000000000 +0000 /etc/security/chroot.conf # rpm -V pam.i386| grep chroot # rpm -V pam.x86_64| grep chroot .......T c /etc/security/chroot.conf
So the i386 package thinks all is fine, while the x86_64 package thinks the config file has been altered manually. So the x86_64 package will not touch that file on upgrading and generate an rpmnew file.
BTW I wonder why multilib allowed i386 to win over x86_64 on my system. Looks like a different issue, but perhaps the above only triggers in such cases.
There are several points to learn here:
a) always use install -p or similar, e.g. preserve the time stamps while packaging. This is a general rule of reason and we see here that it can trigger other bugs
b) avoid packaging config files in multilibed packages. E.g. pam should split off a libpam subpackage and we should only multilib that.
c) investigate what happens on multilib upgrades and config files. Something is obvioulsy broken there.
Rex Dieter wrote:
Philip Prindeville wrote:
I'm still not clear, though: if the file being installed is part of the sources that's being built (i.e. it's not a generated file), and the makefile that does the install invoked "cp --preserve=timestamps" then both the .i386 and the .x86_64 copies should have an identical timestamp. Right?
Consider the case where the installed file(s) are build-time generated. These are *very* unlikely to have identical timestamps (having been built separately on different archs/buildhosts).
-- Rex
I would have thought that to be much less common. Also, sometimes the jitter between the .x86_64 and the .i386 files are only a few seconds (often less than 30).
It shouldn't be too hard to cache the timestamp on one of those files, and if the MD5 checksum and size are identical, then substitute in the previous timestamp... Say always using the .i386 timestamp, for instance.
The downside to that is that if you find a bug that only affects .x86_64 architecture (not that unlikely), then you have to kick out both binary RPM's with new version numbers. It also means that when building, you'd always have to build the "reference" architecture first.
-Philip
On Sun, Dec 10, 2006 at 08:36:34PM -0700, Philip Prindeville wrote:
Axel Thimm wrote:
Looks a bit like multilib. E.g. config files existing in two packages, an i386 and an x86_64 one.
Well, in two cases that I looked at, that turned out to be true.
Not sure if there were any exceptions. Maybe samba-common. I'll pay attention in the future when I see it happen next...
Well, samba-common has the same fate:
# rpm -qf --qf '%{name}-%{version}-%{release}@%{arch}\n' /etc/samba/lmhosts /etc/samba/smb.conf | sort -u samba-common-3.0.23c-2@i386 samba-common-3.0.23c-2@x86_64 # TZ=C ls -l --full-time /etc/samba/lmhosts /etc/samba/smb.conf -rw-r--r-- 1 root root 20 2006-09-02 02:59:06.000000000 +0000 /etc/samba/lmhosts -rw-r--r-- 1 root root 9765 2006-09-02 02:59:06.000000000 +0000 /etc/samba/smb.conf # rpm -V samba-common.x86_64 | grep /etc/samba .......T c /etc/samba/lmhosts .......T c /etc/samba/smb.conf # rpm -V samba-common.i386 | grep /etc/samba
So, again the timestamps of the config files in the two coinstalled packages differ and an upgrade will make one of them think the config file was changed.
packaging@lists.fedoraproject.org
-
Axel Thimm -
Philip Prindeville -
Rex Dieter