Florian, DJ,
This took me quite some time to test in a variety of combinations to ensure that I got everything right. Initially I developed all of these as stand-along lua programs to test for robustness under various conditions. Then I ported them to rpm and used the posix library to some extent (though minimally). At this point I relied on strace logs to extensively verify operation of rpm and the scripts under various conditions.
Summary of the changes:
* There were two copies of glibc_post_upgrade, one in the patch and one in dist-git, and I removed both. There
* I updated glibc.spec to removal all handling of the glibc_post_upgrade binary.
* The 4 "steps" of glibc_post_upgrade were converted directly to lua.
* Step 1: We don't check to see which architecture we're running on, and instead we search *all* multilib directories during upgrade and remove anything that would be bad. This does mean that if for some reason you had /lib64/power6 on x86_64 we would remove libraries from that directory during an upgrade. I consider this a reasonable simplification, the directories are orthogonal and you really shouldn't have them mixed like that in a normal install.
* Step 2: I fix /etc/ld.so.conf with a slightly safer operation of writing a temporary file and then moving it into place. Unfortunately lua doesn't have good temp name support, so I use a static name, and if the 'rename' operation fails we leave the tmp file /etc/glibc_post_upgrade.ld.so.conf in place. Should we call it /etc/ld.so.conf.rpmnew to be more in line with what rpm would do?
* Step 3: We discussed that this step of re-running ldconfig is required since the cache format might change so we recreate the cache early.
* Step 4: This is the same issue as step 3, we might have an out-of-date gconv cache so we recreate it as early as possible.
I've done extensive testing, but there were a lot of manual steps involved in this process of testing, and it's possible I missed something.
What I'm looking for:
- Review of the C code to Lua conversion. - Review of the overall steps and if they make sense.
I'd like to get this into Rawhide ASAP for general testing.
Thanks!
Cheers, Carlos.
8< -- 8< -- 8< From ad67afbb5f2a2c147abc97f7830fdb027fb3405f Mon Sep 17 00:00:00 2001 From: Carlos O'Donell carlos@redhat.com Date: Thu, 16 May 2019 21:43:50 -0400 Subject: [PATCH] Convert glibc_post_upgrade to lua.
--- glibc-post_upgrade.patch | 272 -------------------------------------- glibc.spec | 157 +++++++++++++++++++--- glibc_post_upgrade.c | 274 --------------------------------------- 3 files changed, 139 insertions(+), 564 deletions(-) delete mode 100644 glibc-post_upgrade.patch delete mode 100644 glibc_post_upgrade.c
diff --git a/glibc-post_upgrade.patch b/glibc-post_upgrade.patch deleted file mode 100644 index a64adfc..0000000 --- a/glibc-post_upgrade.patch +++ /dev/null @@ -1,272 +0,0 @@ -Short description: RPM Post-upgrade cleanup program. -Author(s): Fedora glibc team glibc@lists.fedoraproject.org -Origin: PATCH -Upstream status: not-needed - -A helper program is needed to clean up the system configuration -early during RPM package installation, so that other scriptlets -can run successfully. - -diff --git a/elf/Makefile b/elf/Makefile -index 2a432d8beebcd207..368dcae477fff2ae 100644 ---- a/elf/Makefile -+++ b/elf/Makefile -@@ -117,6 +117,14 @@ others-extras = $(ldconfig-modules) - endif - endif - -+# This needs to be statically linked because it is executed at a time -+# when there might be incompatible shared objects on disk, and the -+# purpose of this program is to remove them (among other things). -+others-static += glibc_post_upgrade -+others += glibc_post_upgrade -+glibc_post_upgrade-modules := static-stubs -+CFLAGS-glibc_post_upgrade.c += -DGCONV_MODULES_DIR='"$(gconvdir)"' -+ - # To find xmalloc.c and xstrdup.c - vpath %.c ../locale/programs - -@@ -559,6 +567,8 @@ $(objpfx)sln: $(sln-modules:%=$(objpfx)%.o) - - $(objpfx)ldconfig: $(ldconfig-modules:%=$(objpfx)%.o) - -+$(objpfx)glibc_post_upgrade: $(glibc_post_upgrade-modules:%=$(objpfx)%.o) -+ - SYSCONF-FLAGS := -D'SYSCONFDIR="$(sysconfdir)"' - CFLAGS-ldconfig.c += $(SYSCONF-FLAGS) -D'LIBDIR="$(libdir)"' \ - -D'SLIBDIR="$(slibdir)"' -diff --git a/elf/glibc_post_upgrade.c b/elf/glibc_post_upgrade.c -new file mode 100644 -index 0000000000000000..19b59f70e2308032 ---- /dev/null -+++ b/elf/glibc_post_upgrade.c -@@ -0,0 +1,229 @@ -+#include <sys/types.h> -+#include <sys/wait.h> -+#include <stdio.h> -+#include <errno.h> -+#include <unistd.h> -+#include <sys/time.h> -+#include <dirent.h> -+#include <stddef.h> -+#include <fcntl.h> -+#include <string.h> -+ -+#define LD_SO_CONF "/etc/ld.so.conf" -+#define ICONVCONFIG "/usr/sbin/iconvconfig" -+ -+#define verbose_exec(failcode, path...) \ -+ do \ -+ { \ -+ char *const arr[] = { path, NULL }; \ -+ vexec (failcode, arr); \ -+ } while (0) -+ -+__attribute__((noinline)) static void vexec (int failcode, char *const path[]); -+__attribute__((noinline)) static void says (const char *str); -+__attribute__((noinline)) static void sayn (long num); -+__attribute__((noinline)) static void message (char *const path[]); -+ -+int -+main (void) -+{ -+ char initpath[256]; -+ -+ char buffer[4096]; -+ struct pref { -+ const char *p; -+ int len; -+ } prefix[] = { { "libc-", 5 }, { "libm-", 5 }, -+ { "librt-", 6 }, { "libpthread-", 11 }, -+ { "librtkaio-", 10 }, { "libthread_db-", 13 } }; -+ int i, j, fd; -+ off_t base; -+ ssize_t ret; -+ -+ /* In order to support in-place upgrades, we must immediately remove -+ obsolete platform directories after installing a new glibc -+ version. RPM only deletes files removed by updates near the end -+ of the transaction. If we did not remove the obsolete platform -+ directories here, they would be preferred by the dynamic linker -+ during the execution of subsequent RPM scriptlets, likely -+ resulting in process startup failures. */ -+ const char *remove_dirs[] = -+ { -+#if defined (__i386__) -+ "/lib/i686", -+ "/lib/i686/nosegneg", -+#elif defined (__powerpc64__) && _CALL_ELF != 2 -+ "/lib64/power6", -+#endif -+ }; -+ for (j = 0; j < sizeof (remove_dirs) / sizeof (remove_dirs[0]); ++j) -+ { -+ size_t rmlen = strlen (remove_dirs[j]); -+ fd = open (remove_dirs[j], O_RDONLY); -+ if (fd >= 0 -+ && (ret = getdirentries (fd, buffer, sizeof (buffer), &base)) -+ >= (ssize_t) offsetof (struct dirent, d_name)) -+ { -+ for (base = 0; base + offsetof (struct dirent, d_name) < ret; ) -+ { -+ struct dirent *d = (struct dirent *) (buffer + base); -+ -+ for (i = 0; i < sizeof (prefix) / sizeof (prefix[0]); i++) -+ if (! strncmp (d->d_name, prefix[i].p, prefix[i].len)) -+ { -+ char *p = d->d_name + prefix[i].len; -+ -+ while (*p == '.' || (*p >= '0' && *p <= '9')) p++; -+ if (p[0] == 's' && p[1] == 'o' && p[2] == '\0' -+ && p + 3 - d->d_name -+ < sizeof (initpath) - rmlen - 1) -+ { -+ memcpy (initpath, remove_dirs[j], rmlen); -+ initpath[rmlen] = '/'; -+ strcpy (initpath + rmlen + 1, d->d_name); -+ unlink (initpath); -+ break; -+ } -+ } -+ base += d->d_reclen; -+ } -+ close (fd); -+ } -+ } -+ -+ int ldsocfd = open (LD_SO_CONF, O_RDONLY); -+ struct stat ldsocst; -+ if (ldsocfd >= 0 && fstat (ldsocfd, &ldsocst) >= 0) -+ { -+ char p[ldsocst.st_size + 1]; -+ if (read (ldsocfd, p, ldsocst.st_size) == ldsocst.st_size) -+ { -+ p[ldsocst.st_size] = '\0'; -+ if (strstr (p, "include ld.so.conf.d/*.conf") == NULL) -+ { -+ close (ldsocfd); -+ ldsocfd = open (LD_SO_CONF, O_WRONLY | O_TRUNC); -+ if (ldsocfd >= 0) -+ { -+ size_t slen = strlen ("include ld.so.conf.d/*.conf\n"); -+ if (write (ldsocfd, "include ld.so.conf.d/*.conf\n", slen) -+ != slen -+ || write (ldsocfd, p, ldsocst.st_size) != ldsocst.st_size) -+ _exit (109); -+ } -+ } -+ } -+ if (ldsocfd >= 0) -+ close (ldsocfd); -+ } -+ -+ /* If installing bi-arch glibc, rpm sometimes doesn't unpack all files -+ before running one of the lib's %post scriptlet. /sbin/ldconfig will -+ then be run by the other arch's %post. */ -+ if (! access ("/sbin/ldconfig", X_OK)) -+ verbose_exec (110, -+ (char *) "/sbin/ldconfig", -+ (char *) "/sbin/ldconfig"); -+ -+ if (! utimes (GCONV_MODULES_DIR "/gconv-modules.cache", NULL)) -+ { -+ const char *iconv_cache = GCONV_MODULES_DIR "/gconv-modules.cache"; -+ const char *iconv_dir = GCONV_MODULES_DIR; -+ verbose_exec (113, -+ (char *) ICONVCONFIG, -+ (char *) "/usr/sbin/iconvconfig", -+ (char *) "-o", -+ (char *) iconv_cache, -+ (char *) "--nostdlib", -+ (char *) iconv_dir); -+ } -+ -+ _exit(0); -+} -+ -+void -+vexec (int failcode, char *const path[]) -+{ -+ pid_t pid; -+ int status, save_errno; -+ int devnull = 0; -+ -+ if (failcode < 0) -+ { -+ devnull = 1; -+ failcode = -failcode; -+ } -+ pid = vfork (); -+ if (pid == 0) -+ { -+ int fd; -+ if (devnull && (fd = open ("/dev/null", O_WRONLY)) >= 0) -+ { -+ dup2 (fd, 1); -+ dup2 (fd, 2); -+ close (fd); -+ } -+ execv (path[0], path + 1); -+ save_errno = errno; -+ message (path); -+ says (" exec failed with errno "); -+ sayn (save_errno); -+ says ("\n"); -+ _exit (failcode); -+ } -+ else if (pid < 0) -+ { -+ save_errno = errno; -+ message (path); -+ says (" fork failed with errno "); -+ sayn (save_errno); -+ says ("\n"); -+ _exit (failcode + 1); -+ } -+ if (waitpid (0, &status, 0) != pid || !WIFEXITED (status)) -+ { -+ message (path); -+ says (" child terminated abnormally\n"); -+ _exit (failcode + 2); -+ } -+ if (WEXITSTATUS (status)) -+ { -+ message (path); -+ says (" child exited with exit code "); -+ sayn (WEXITSTATUS (status)); -+ says ("\n"); -+ _exit (WEXITSTATUS (status)); -+ } -+} -+ -+static void -+says (const char *str) -+{ -+ write (1, str, strlen (str)); -+} -+ -+static void -+sayn (long num) -+{ -+ char string[sizeof (long) * 3 + 1]; -+ char *p = string + sizeof (string) - 1; -+ -+ *p = '\0'; -+ if (num == 0) -+ *--p = '0'; -+ else -+ while (num) -+ { -+ *--p = '0' + num % 10; -+ num = num / 10; -+ } -+ -+ says (p); -+} -+ -+static void -+message (char *const path[]) -+{ -+ says ("/usr/sbin/glibc_post_upgrade: While trying to execute "); -+ says (path[0]); -+} diff --git a/glibc.spec b/glibc.spec index df07e14..ce8d082 100644 --- a/glibc.spec +++ b/glibc.spec @@ -87,7 +87,7 @@ Summary: The GNU libc libraries Name: glibc Version: %{glibcversion} -Release: 19%{?dist} +Release: 20%{?dist}
# In general, GPLv2+ is used by programs, LGPLv2+ is used for # libraries. @@ -137,7 +137,6 @@ Source12: ChangeLog.old # - See each individual patch file for origin and upstream status. # - For new patches follow template.patch format. ############################################################################## -Patch1: glibc-post_upgrade.patch Patch2: glibc-fedora-nscd.patch Patch3: glibc-rh697421.patch Patch4: glibc-fedora-linux-tcsetattr.patch @@ -401,8 +400,11 @@ libraries, as well as national language (locale) support. /sbin/ldconfig %end
-# We need to run ldconfig manually because ldconfig cannot handle the -# relative include path in the /etc/ld.so.conf file we gneerate. +# We need to run ldconfig manually because __brp_ldconfig assumes that +# glibc itself is always installed in $RPM_BUILD_ROOT, but with sysroots +# we may be installed into a subdirectory of that path. Therefore we +# unset __brp_ldconfig and run ldconfig by hand with the sysroots path +# passed to -r. %undefine __brp_ldconfig
###################################################################### @@ -1094,10 +1096,6 @@ truncate -s 0 %{glibc_sysroot}/etc/gai.conf truncate -s 0 %{glibc_sysroot}%{_libdir}/gconv/gconv-modules.cache chmod 644 %{glibc_sysroot}%{_libdir}/gconv/gconv-modules.cache
-# Install the upgrade program -install -m 700 build-%{target}/elf/glibc_post_upgrade \ - %{glibc_sysroot}%{_prefix}/sbin/glibc_post_upgrade.%{_target_cpu} - ############################################################################## # Install debug copies of unstripped static libraries # - This step must be last in order to capture any additional static @@ -1120,8 +1118,8 @@ rm -rf %{glibc_sysroot}%{_prefix}/share/zoneinfo # # XXX: Ideally ld.so.conf should have the timestamp of the spec file, but there # doesn't seem to be any macro to give us that. So we do the next best thing, -# which is to at least keep the timestamp consistent. The choice of using -# glibc_post_upgrade.c is arbitrary. +# which is to at least keep the timestamp consistent. The choice of using +# %{SOURCE0} is arbitrary. touch -r %{SOURCE0} %{glibc_sysroot}/etc/ld.so.conf touch -r sunrpc/etc.rpc %{glibc_sysroot}/etc/rpc
@@ -1358,8 +1356,7 @@ chmod 0444 master.filelist # - The partial (lib*_p.a) static libraries, include files. # - The static files, objects, unversioned DSOs, and nscd. # - The bin, locale, some sbin, and share. -# - The use of [^gi] is meant to exclude all files except glibc_post_upgrade, -# and iconvconfig, which we want in the main packages. +# - We want iconvconfig in the main package. # - All the libnss files (we add back the ones we want later). # - All bench test binaries. # - The aux-cache, since it's handled specially in the files section. @@ -1375,7 +1372,7 @@ cat master.filelist \ -e 'nscd' \ -e '%{_prefix}/bin' \ -e '%{_prefix}/lib/locale' \ - -e '%{_prefix}/sbin/[^gi]' \ + -e '%{_prefix}/sbin/iconvconfig' \ -e '%{_prefix}/share' \ -e '/var/db/Makefile' \ -e '/libnss_.*.so[0-9.]*$' \ @@ -1452,10 +1449,10 @@ grep '%{_libdir}/lib.*.a' < master.filelist \ ###############################################################################
# All of the bin and certain sbin files go into the common package except -# glibc_post_upgrade.* and iconvconfig which need to go in glibc. Likewise -# nscd is excluded because it goes in nscd. +# iconvconfig which needs to go in glibc. Likewise nscd is excluded because +# it goes in nscd. grep '%{_prefix}/bin' master.filelist >> common.filelist -grep '%{_prefix}/sbin/[^gi]' master.filelist \ +grep '%{_prefix}/sbin/iconvconfig' master.filelist \ | grep -v 'nscd' >> common.filelist # All of the files under share go into the common package since they should be # multilib-independent. @@ -1749,7 +1746,6 @@ $run_ldso /usr/bin/valgrind --error-exitcode=1 \
%endif # %{run_glibc_tests}
- %pre -p <lua> -- Check that the running kernel is new enough required = '%{enablekernel}' @@ -1758,7 +1754,129 @@ if rpm.vercmp(rel, required) < 0 then error("FATAL: kernel too old", 0) end
-%post -p %{_prefix}/sbin/glibc_post_upgrade.%{_target_cpu} +%post -p <lua> +-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they would be preferred by the dynamic linker +-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures. */ + +-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +local remove_prefixes = { "libc-", + "libm-", + "librt-", + "libpthread-", + "librtkaio-", + "libthread_db-" } + +-- We are going to search these directories. +local remove_dirs = { "/lib/i686", + "/lib/i686/nosegneg", + "/lib64/power6", + "/lib64/power7", + "/lib64/power8" } + +-- Walk all the directories with files we need to remove... +for _, rdir in ipairs (remove_dirs) do + if posix.access (rdir) then + -- If the directory exists we look at all the files... + local remove_files = posix.files (rdir) + for rfile in remove_files do + for _, rprefix in ipairs (remove_prefixes) do + -- Does the literal prefix match? + local prefix = string.find (rfile, rprefix, 1, true) + -- Does it end with '*.so'? + local dso = string.match (rfile, "%.so$") + if (prefix == 1 and (dso ~= nil)) then + -- Removing file... + os.remove (rdir .. '/' .. rfile) + end + end + end + end +end + +-- (2) Update /etc/ld.so.conf +-- Next we update /etc/ld.so.conf to ensure that it starts with +-- a literal "include ld.so.conf.d/*.conf". + +local ldsoconf = "/etc/ld.so.conf" +local ldsoconf_tmp = "/etc/glibc_post_upgrade.ld.so.conf" + +local conf_fd = io.open (ldsoconf, "r") + +if conf_fd ~= nil then + + -- We must have a "include ld.so.conf.d/*.conf" line. + local have_include = false + for line in conf_fd:lines () do + -- This must match, and we don't ignore whitespace. + if string.match (line, "^include ld.so.conf.d/%*%.conf$") ~= nil then + have_include = true + end + end + conf_fd:close () + + if not have_include then + -- Insert "include ld.so.conf.d/*.conf" line at the start of the + -- file. We only support one of these post upgrades running at + -- a time (temporary file name is fixed). + local tmp_fd = io.open (ldsoconf_tmp, "w") + if tmp_fd ~= nil then + tmp_fd:write ("include ld.so.conf.d/*.conf\n") + for line in io.lines (ldsoconf) do + tmp_fd:write (line .. "\n") + end + tmp_fd:close () + local res = os.rename (ldsoconf_tmp, ldsoconf) + if res == nil then + io.stdout:write ("Error: Unable to update configuration file (rename).\n") + end + else + io.stdout:write ("Error: Unable to update configuration file (open).\n") + end + end +end + +-- (3) Rebuild ld.so.cache early. +-- If the format of the cache changes then we need to rebuild +-- the cache early to avoid any problems running binaries with +-- the new glibc. + +if posix.access ("%{_prefix}/sbin/ldconfig", "x") then + local pid = posix.fork () + if pid == 0 then + posix.exec ("%{_prefix}/sbin/ldconfig") + elseif pid > 0 then + posix.wait (pid) + end +end + +-- (4) Update gconv modules cache. +-- If the /usr/lib/gconv/gconv-modules.cache exists, then update it +-- with the latest set of modules that were just installed. +-- We assume that the cache is in %{_libdir}/gconv and called +-- "gconv-modules.cache". + +local iconv_dir = "%{_libdir}/gconv" +local iconv_cache = iconv_dir .. "/gconv-modules.cache" +if posix.utime (iconv_cache) == 0 then + local pid = posix.fork () + if pid == 0 then + posix.exec ("%{_prefix}/sbin/iconvconfig", + "-o", iconv_cache, + "--nostdlib", + iconv_dir) + elseif pid > 0 then + posix.wait (pid) + end +else + io.stdout:write ("Error: Missing " .. iconv_cache .. " file.\n") +end
%posttrans all-langpacks -e -p <lua> -- If at the end of the transaction we are still installed @@ -1899,6 +2017,9 @@ fi %files -f compat-libpthread-nonshared.filelist -n compat-libpthread-nonshared
%changelog +* Thu May 16 2019 Carlos O'Donell carlos@redhat.com - 2.29.9000-20 +- Convert glibc_post_upgrade to lua. + * Wed May 15 2019 Florian Weimer fweimer@redhat.com - 2.29.9000-19 - Auto-sync with upstream branch master, commit 32ff397533715988c19cbf3675dcbd727ec13e18: diff --git a/glibc_post_upgrade.c b/glibc_post_upgrade.c deleted file mode 100644 index 9014857..0000000 --- a/glibc_post_upgrade.c +++ /dev/null @@ -1,274 +0,0 @@ -#include <sys/types.h> -#include <sys/wait.h> -#include <stdio.h> -#include <errno.h> -#include <unistd.h> -#include <sys/time.h> -#include <dirent.h> -#include <stddef.h> -#include <fcntl.h> -#include <string.h> -#include <sys/stat.h> -#include <elf.h> - -#define LD_SO_CONF "/etc/ld.so.conf" -#define ICONVCONFIG "/usr/sbin/iconvconfig" - -#define verbose_exec(failcode, path...) \ - do \ - { \ - char *const arr[] = { path, NULL }; \ - vexec (failcode, arr); \ - } while (0) - -__attribute__((noinline)) void vexec (int failcode, char *const path[]); -__attribute__((noinline)) void says (const char *str); -__attribute__((noinline)) void sayn (long num); -__attribute__((noinline)) void message (char *const path[]); -__attribute__((noinline)) int check_elf (const char *name); - -int -main (void) -{ - struct stat statbuf; - char initpath[256]; - - char buffer[4096]; - struct pref { - char *p; - int len; - } prefix[] = { { "libc-", 5 }, { "libm-", 5 }, - { "librt-", 6 }, { "libpthread-", 11 }, - { "librtkaio-", 10 }, { "libthread_db-", 13 } }; - int i, j, fd; - off_t base; - ssize_t ret; - - /* In order to support in-place upgrades, we must immediately remove - obsolete platform directories after installing a new glibc - version. RPM only deletes files removed by updates near the end - of the transaction. If we did not remove the obsolete platform - directories here, they would be preferred by the dynamic linker - during the execution of subsequent RPM scriptlets, likely - resulting in process startup failures. */ - const char *remove_dirs[] = - { -#if defined (__i386__) - "/lib/i686", - "/lib/i686/nosegneg", -#elif defined (__powerpc64__) && _CALL_ELF != 2 - "/lib64/power6", - "/lib64/power7", - "/lib64/power8", -#endif - }; - for (j = 0; j < sizeof (remove_dirs) / sizeof (remove_dirs[0]); ++j) - { - size_t rmlen = strlen (remove_dirs[j]); - fd = open (remove_dirs[j], O_RDONLY); - if (fd >= 0 - && (ret = getdirentries (fd, buffer, sizeof (buffer), &base)) - >= (ssize_t) offsetof (struct dirent, d_name)) - { - for (base = 0; base + offsetof (struct dirent, d_name) < ret; ) - { - struct dirent *d = (struct dirent *) (buffer + base); - - for (i = 0; i < sizeof (prefix) / sizeof (prefix[0]); i++) - if (! strncmp (d->d_name, prefix[i].p, prefix[i].len)) - { - char *p = d->d_name + prefix[i].len; - - while (*p == '.' || (*p >= '0' && *p <= '9')) p++; - if (p[0] == 's' && p[1] == 'o' && p[2] == '\0' - && p + 3 - d->d_name - < sizeof (initpath) - rmlen - 1) - { - memcpy (initpath, remove_dirs[j], rmlen); - initpath[rmlen] = '/'; - strcpy (initpath + rmlen + 1, d->d_name); - unlink (initpath); - break; - } - } - base += d->d_reclen; - } - close (fd); - } - } - - int ldsocfd = open (LD_SO_CONF, O_RDONLY); - struct stat ldsocst; - if (ldsocfd >= 0 && fstat (ldsocfd, &ldsocst) >= 0) - { - char p[ldsocst.st_size + 1]; - if (read (ldsocfd, p, ldsocst.st_size) == ldsocst.st_size) - { - p[ldsocst.st_size] = '\0'; - if (strstr (p, "include ld.so.conf.d/*.conf") == NULL) - { - close (ldsocfd); - ldsocfd = open (LD_SO_CONF, O_WRONLY | O_TRUNC); - if (ldsocfd >= 0) - { - size_t slen = strlen ("include ld.so.conf.d/*.conf\n"); - if (write (ldsocfd, "include ld.so.conf.d/*.conf\n", slen) - != slen - || write (ldsocfd, p, ldsocst.st_size) != ldsocst.st_size) - _exit (109); - } - } - } - if (ldsocfd >= 0) - close (ldsocfd); - } - - /* If installing bi-arch glibc, rpm sometimes doesn't unpack all files - before running one of the lib's %post scriptlet. /sbin/ldconfig will - then be run by the other arch's %post. */ - if (! access ("/sbin/ldconfig", X_OK)) - verbose_exec (110, "/sbin/ldconfig", "/sbin/ldconfig"); - - if (! utimes (GCONV_MODULES_DIR "/gconv-modules.cache", NULL)) - { - char *iconv_cache = GCONV_MODULES_DIR"/gconv-modules.cache"; - char *iconv_dir = GCONV_MODULES_DIR; - verbose_exec (113, ICONVCONFIG, "/usr/sbin/iconvconfig", - "-o", iconv_cache, - "--nostdlib", iconv_dir); - } - - _exit(0); -} - -void -vexec (int failcode, char *const path[]) -{ - pid_t pid; - int status, save_errno; - int devnull = 0; - - if (failcode < 0) - { - devnull = 1; - failcode = -failcode; - } - pid = vfork (); - if (pid == 0) - { - int fd; - if (devnull && (fd = open ("/dev/null", O_WRONLY)) >= 0) - { - dup2 (fd, 1); - dup2 (fd, 2); - close (fd); - } - execv (path[0], path + 1); - save_errno = errno; - message (path); - says (" exec failed with errno "); - sayn (save_errno); - says ("\n"); - _exit (failcode); - } - else if (pid < 0) - { - save_errno = errno; - message (path); - says (" fork failed with errno "); - sayn (save_errno); - says ("\n"); - _exit (failcode + 1); - } - if (waitpid (0, &status, 0) != pid || !WIFEXITED (status)) - { - message (path); - says (" child terminated abnormally\n"); - _exit (failcode + 2); - } - if (WEXITSTATUS (status)) - { - message (path); - says (" child exited with exit code "); - sayn (WEXITSTATUS (status)); - says ("\n"); - _exit (WEXITSTATUS (status)); - } -} - -void -says (const char *str) -{ - write (1, str, strlen (str)); -} - -void -sayn (long num) -{ - char string[sizeof (long) * 3 + 1]; - char *p = string + sizeof (string) - 1; - - *p = '\0'; - if (num == 0) - *--p = '0'; - else - while (num) - { - *--p = '0' + num % 10; - num = num / 10; - } - - says (p); -} - -void -message (char *const path[]) -{ - says ("/usr/sbin/glibc_post_upgrade: While trying to execute "); - says (path[0]); -} - -int -check_elf (const char *name) -{ - /* Play safe, if we can't open or read, assume it might be - ELF for the current arch. */ - int ret = 1; - int fd = open (name, O_RDONLY); - if (fd >= 0) - { - Elf32_Ehdr ehdr; - if (read (fd, &ehdr, offsetof (Elf32_Ehdr, e_version)) - == offsetof (Elf32_Ehdr, e_version)) - { - ret = 0; - if (ehdr.e_ident[EI_CLASS] - == (sizeof (long) == 8 ? ELFCLASS64 : ELFCLASS32)) - { -#if defined __i386__ - ret = ehdr.e_machine == EM_386; -#elif defined __x86_64__ - ret = ehdr.e_machine == EM_X86_64; -#elif defined __powerpc64__ - ret = ehdr.e_machine == EM_PPC64; -#elif defined __powerpc__ - ret = ehdr.e_machine == EM_PPC; -#elif defined __s390__ || defined __s390x__ - ret = ehdr.e_machine == EM_S390; -#elif defined __x86_64__ - ret = ehdr.e_machine == EM_X86_64; -#elif defined __sparc__ - if (sizeof (long) == 8) - ret = ehdr.e_machine == EM_SPARCV9; - else - ret = (ehdr.e_machine == EM_SPARC - || ehdr.e_machine == EM_SPARC32PLUS); -#else - ret = 1; -#endif - } - } - close (fd); - } - return ret; -}
* Carlos O'Donell:
# XXX: Ideally ld.so.conf should have the timestamp of the spec file, but there # doesn't seem to be any macro to give us that. So we do the next best thing, -# which is to at least keep the timestamp consistent. The choice of using -# glibc_post_upgrade.c is arbitrary. +# which is to at least keep the timestamp consistent. The choice of using +# %{SOURCE0} is arbitrary. touch -r %{SOURCE0} %{glibc_sysroot}/etc/ld.so.conf touch -r sunrpc/etc.rpc %{glibc_sysroot}/etc/rpc
This should use SOURCE0 in the comment, to avoid a lint warning due to a macro in a comment.
# All of the bin and certain sbin files go into the common package except -# glibc_post_upgrade.* and iconvconfig which need to go in glibc. Likewise -# nscd is excluded because it goes in nscd. +# iconvconfig which needs to go in glibc. Likewise nscd is excluded because +# it goes in nscd. grep '%{_prefix}/bin' master.filelist >> common.filelist -grep '%{_prefix}/sbin/[^gi]' master.filelist \ +grep '%{_prefix}/sbin/iconvconfig' master.filelist \ | grep -v 'nscd' >> common.filelist
Shouldn't this be echo instead of grep?
+%post -p <lua> +-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they would be preferred by the dynamic linker +-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures. */
+-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +local remove_prefixes = { "libc-",
"libm-","librt-","libpthread-","librtkaio-","libthread_db-" }
Should this be more restrictive, eventually generating a match pattern like "libc%-[1-9][0-9]*%.[0-9]+%.so"?
Or should we reserve this for a future change? It would allow us to fix downgrades if we nuke additional library copies that have names that do not match the current library.
+-- (2) Update /etc/ld.so.conf +-- Next we update /etc/ld.so.conf to ensure that it starts with +-- a literal "include ld.so.conf.d/*.conf".
+local ldsoconf = "/etc/ld.so.conf" +local ldsoconf_tmp = "/etc/glibc_post_upgrade.ld.so.conf"
+local conf_fd = io.open (ldsoconf, "r")
+if conf_fd ~= nil then
- -- We must have a "include ld.so.conf.d/*.conf" line.
- local have_include = false
- for line in conf_fd:lines () do
- -- This must match, and we don't ignore whitespace.
- if string.match (line, "^include ld.so.conf.d/%*%.conf$") ~= nil then
have_include = true- end
- end
- conf_fd:close ()
I think you could io.lines here as well (because you do not use break).
+-- (3) Rebuild ld.so.cache early. +-- If the format of the cache changes then we need to rebuild +-- the cache early to avoid any problems running binaries with +-- the new glibc.
+if posix.access ("%{_prefix}/sbin/ldconfig", "x") then
Not sure why the access check is needed here?
- local pid = posix.fork ()
- if pid == 0 then
- posix.exec ("%{_prefix}/sbin/ldconfig")
Suggest: assert(posix.exec ("%{_prefix}/sbin/ldconfig")), to terminate execution in the subprocess on error. But this a bit tricky.
- elseif pid > 0 then
- posix.wait (pid)
- end
+end
Maybe add a comment that you use posix.exec to avoid the use of the shell, which may not be installed yet?
You should put this into a function, to reuse it below.
Thanks, Florian
On 5/17/19 10:52 AM, Florian Weimer wrote:
- Carlos O'Donell:
# XXX: Ideally ld.so.conf should have the timestamp of the spec file, but there # doesn't seem to be any macro to give us that. So we do the next best thing, -# which is to at least keep the timestamp consistent. The choice of using -# glibc_post_upgrade.c is arbitrary. +# which is to at least keep the timestamp consistent. The choice of using +# %{SOURCE0} is arbitrary. touch -r %{SOURCE0} %{glibc_sysroot}/etc/ld.so.conf touch -r sunrpc/etc.rpc %{glibc_sysroot}/etc/rpc
This should use SOURCE0 in the comment, to avoid a lint warning due to a macro in a comment.
Fixed.
# All of the bin and certain sbin files go into the common package except -# glibc_post_upgrade.* and iconvconfig which need to go in glibc. Likewise -# nscd is excluded because it goes in nscd. +# iconvconfig which needs to go in glibc. Likewise nscd is excluded because +# it goes in nscd. grep '%{_prefix}/bin' master.filelist >> common.filelist -grep '%{_prefix}/sbin/[^gi]' master.filelist \ +grep '%{_prefix}/sbin/iconvconfig' master.filelist \ | grep -v 'nscd' >> common.filelist
Shouldn't this be echo instead of grep
Worse, this is a bug, it moves only iconvconfig into common.fileslist, and I missed this because I didn't do a file-by-file-by-rpm comparison.
The previous grep is '^gi' which is to say "Anything that doesn't start with g or i" so the we do want grep, but we want:
I've fixed this now.
I did a file-by-file-by-rpm comparison and it was clean (should do this every time I touch these list edit steps).
+%post -p <lua> +-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they would be preferred by the dynamic linker +-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures. */
+-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +local remove_prefixes = { "libc-",
"libm-","librt-","libpthread-","librtkaio-","libthread_db-" }Should this be more restrictive, eventually generating a match pattern like "libc%-[1-9][0-9]*%.[0-9]+%.so"?
The original code was more relaxed than your regexp, since it allowed [.0-9] in any order at all in the characters between the prefix and the ending '.so' suffix.
My new code is even more relaxed, allowing prefix.*.so.
I will adjust it to use a fixed regexp to minimize problems with potential future multilibs from other projects.
Fixed.
Or should we reserve this for a future change? It would allow us to fix downgrades if we nuke additional library copies that have names that do not match the current library.
I will restrict the match as you wrote above, but no further for now.
I think we can revisit non-multilib downgrades later.
+-- (2) Update /etc/ld.so.conf +-- Next we update /etc/ld.so.conf to ensure that it starts with +-- a literal "include ld.so.conf.d/*.conf".
+local ldsoconf = "/etc/ld.so.conf" +local ldsoconf_tmp = "/etc/glibc_post_upgrade.ld.so.conf"
+local conf_fd = io.open (ldsoconf, "r")
+if conf_fd ~= nil then
- -- We must have a "include ld.so.conf.d/*.conf" line.
- local have_include = false
- for line in conf_fd:lines () do
- -- This must match, and we don't ignore whitespace.
- if string.match (line, "^include ld.so.conf.d/%*%.conf$") ~= nil then
have_include = true- end
- end
- conf_fd:close ()
I think you could io.lines here as well (because you do not use break).
Simplified then to use posix.access and io.lines.
Fixed.
+-- (3) Rebuild ld.so.cache early. +-- If the format of the cache changes then we need to rebuild +-- the cache early to avoid any problems running binaries with +-- the new glibc.
+if posix.access ("%{_prefix}/sbin/ldconfig", "x") then
Not sure why the access check is needed here?
The original C code says this:
-+ /* If installing bi-arch glibc, rpm sometimes doesn't unpack all files -+ before running one of the lib's %post scriptlet. /sbin/ldconfig will -+ then be run by the other arch's %post. */
This seems like gibberish. RPM always unpacks all the files. It has to.
I'm removing it, we don't do any similar check for iconvconfig, we expect it's installed and executable or we fail.
Fixed.
- local pid = posix.fork ()
- if pid == 0 then
- posix.exec ("%{_prefix}/sbin/ldconfig")
Suggest: assert(posix.exec ("%{_prefix}/sbin/ldconfig")), to terminate execution in the subprocess on error. But this a bit tricky.
Fixed. I'm using assert for ldconfig and iconvconfig.
- elseif pid > 0 then
- posix.wait (pid)
- end
+end
Maybe add a comment that you use posix.exec to avoid the use of the shell, which may not be installed yet?
Fixed.
You should put this into a function, to reuse it below.
Fixed.
v2 - Fixed SOURCE0 reference in comment. - Fixed iconvconfig inclusion in glibc.filelist. - Fixed iconvconfig removal from common.filelist. - Created post_exec function to run execs with asserts to check for errors. - Switched step 1 to use tighter regexps. - Switched step 1 to use %{_libdir} where possible. - Fixed step 2 to use io.lines and simplify. - Always run ldconfig in step 3 using post_exec - Step 4 uses post_exec also.
That resolves all the immediate issues you and DJ raised.
Comments appreciated on v2.
* Carlos O'Donell:
# All of the bin and certain sbin files go into the common package except -# glibc_post_upgrade.* and iconvconfig which need to go in glibc. Likewise -# nscd is excluded because it goes in nscd. +# iconvconfig which needs to go in glibc. Likewise nscd is excluded because +# it goes in nscd.
Maybe add “because it is used in post scriptlet below”?
+-- We use lua's posix.exec because there may be no shell that we can +-- run during glibc upgrade. +function post_exec (program, ...)
- local pid = posix.fork ()
- if pid == 0 then
- assert (posix.exec (program, ...))
- elseif pid > 0 then
- posix.wait (pid)
- end
+end
Hehe. GNU-style Lua.
+-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they may be preferred by the dynamic linker +-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures.
+-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +-- We employ a tight match on libc-X.Y.so where X > 2 and Y > 2?, and +-- related libraries so we would match "libc-2.20.so" and higher. +remove_regexps = { "libc%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so",
"libm%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","librt%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","libpthread%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","librtkaio%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","libthread_db%%-[1-9][0-9]*%%.[0-9]+%%.so" }
Untested:
local remove_regexps = {"c", "m", "rt", "pthread", "rtkaio", "thread_db", "nss_compat", "nss_db", "nss_dns", "nss_hesiod", "nss_files", "nss_nis"} for i = 1, #remove_regexps do remove_regexps[i] = ("lib" .. remove_regexps[i] .. "%%-[2-9][0-9]*%%.[0-9]+%%.so$") done remove_regexps[#remove_regexps + 1] = "libthread_db%%-1%%.0%%.so"
Note: No [2-9] group in the second version component, as that is not future-proof. And added $ at the end. And list the irregular libthread_db name separately.
- if string.match (line, "^include ld.so.conf.d/%*%.conf$") ~= nil then
Should this use %%*%%?
Rest looks okay to me.
Thanks, Florian
On 5/31/19 12:49 PM, Florian Weimer wrote:
- Carlos O'Donell:
# All of the bin and certain sbin files go into the common package except -# glibc_post_upgrade.* and iconvconfig which need to go in glibc. Likewise -# nscd is excluded because it goes in nscd. +# iconvconfig which needs to go in glibc. Likewise nscd is excluded because +# it goes in nscd.
Maybe add “because it is used in post scriptlet below”?
Fixed.
+-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they may be preferred by the dynamic linker +-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures.
+-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +-- We employ a tight match on libc-X.Y.so where X > 2 and Y > 2?, and +-- related libraries so we would match "libc-2.20.so" and higher. +remove_regexps = { "libc%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so",
"libm%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","librt%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","libpthread%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","librtkaio%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","libthread_db%%-[1-9][0-9]*%%.[0-9]+%%.so" }Untested:
local remove_regexps = {"c", "m", "rt", "pthread", "rtkaio", "thread_db", "nss_compat", "nss_db", "nss_dns", "nss_hesiod", "nss_files", "nss_nis"}
This is in controlled by install_different() in glibc.spec, in that we only install 5 variant libraries, but the removal set must be the superset of all possible previously deployed libraries, and so should be the full set of libraries. I've expanded this to more than the above.
Fixed.
for i = 1, #remove_regexps do remove_regexps[i] = ("lib" .. remove_regexps[i] .. "%%-[2-9][0-9]*%%.[0-9]+%%.so$") done remove_regexps[#remove_regexps + 1] = "libthread_db%%-1%%.0%%.so"
Fixed.
Note: No [2-9] group in the second version component, as that is not future-proof. And added $ at the end. And list the irregular libthread_db name separately.
Good idea.
Fixed.
- if string.match (line, "^include ld.so.conf.d/%*%.conf$") ~= nil then
Should this use %%*%%?
Ha! Yes, works because the degenerate regexp is still valid.
Fixed.
Rest looks okay to me.
v3 - Added comment about why iconvconfig is in glibc rpm. - Added multilib removal coverage for all installed libs. - Includes support for irregular named libs. - Includes support for tighter regexp which is still future proof. - Fixed ld.so.conf string match.
Retested removal of all libs in a multilib upgrade (simulated with fake files). Retested locally for multilib upgrades. Retested locally for ld.so.conf edits. Retested locally for gconv-modules.cache update. Retested locally to verify no installed files changed locations.
OK for rawhide?
* Carlos O'Donell:
+-- We employ a tight match on libc-X.Y.so where X > 2 and Y > 2?, and +-- related libraries so we would match "libc-2.20.so" and higher. +remove_regexps = {} +for i = 1, #install_libs do
- remove_regexps[i] = ("lib" .. install_libs[i]
.. "%%-[2-9][0-9]*%%.[0-9]+%%.so$")+end
Sorry, I think the comment does not match the regexp anymore.
Thanks, Florian
On 5/31/19 5:13 PM, Florian Weimer wrote:
- Carlos O'Donell:
+-- We employ a tight match on libc-X.Y.so where X > 2 and Y > 2?, and +-- related libraries so we would match "libc-2.20.so" and higher. +remove_regexps = {} +for i = 1, #install_libs do
- remove_regexps[i] = ("lib" .. install_libs[i]
.. "%%-[2-9][0-9]*%%.[0-9]+%%.so$")+end
Sorry, I think the comment does not match the regexp anymore.
Ah, yes, I guess you're right.
+-- We employ a tight match on libc-X.Y.so where X > 2 and Y > 0, and +-- related libraries, so we would match "libc-2.0.so" and higher.
Fixed.
"Carlos O'Donell" carlos@redhat.com writes:
+remove_regexps = {} +for i = 1, #install_libs do
- remove_regexps[i] = ("lib" .. install_libs[i]
.. "%%-[2-9][0-9]*%%.[0-9]+%%.so$")+end
Won't match libc-10.1 but let's worry about that later :-)
On 5/31/19 5:32 PM, DJ Delorie wrote:
"Carlos O'Donell" carlos@redhat.com writes:
+remove_regexps = {} +for i = 1, #install_libs do
- remove_regexps[i] = ("lib" .. install_libs[i]
.. "%%-[2-9][0-9]*%%.[0-9]+%%.so$")+end
Won't match libc-10.1 but let's worry about that later :-)
-- We employ a tight match where X.Y is in [2.0,9.9*], so we would -- match "libc-2.0.so" and so on up to "libc-9.9*".
Fixed.
* Carlos O'Donell:
On 5/31/19 5:32 PM, DJ Delorie wrote:
"Carlos O'Donell" carlos@redhat.com writes:
+remove_regexps = {} +for i = 1, #install_libs do
- remove_regexps[i] = ("lib" .. install_libs[i]
.. "%%-[2-9][0-9]*%%.[0-9]+%%.so$")+end
Won't match libc-10.1 but let's worry about that later :-)
-- We employ a tight match where X.Y is in [2.0,9.9*], so we would -- match "libc-2.0.so" and so on up to "libc-9.9*".
Fixed.
Did you change the regexp as well? Dropping the [0-9]*?
Thanks, Florian
On 5/31/19 5:51 PM, Florian Weimer wrote:
- Carlos O'Donell:
On 5/31/19 5:32 PM, DJ Delorie wrote:
"Carlos O'Donell" carlos@redhat.com writes:
+remove_regexps = {} +for i = 1, #install_libs do
- remove_regexps[i] = ("lib" .. install_libs[i]
.. "%%-[2-9][0-9]*%%.[0-9]+%%.so$")+end
Won't match libc-10.1 but let's worry about that later :-)
-- We employ a tight match where X.Y is in [2.0,9.9*], so we would -- match "libc-2.0.so" and so on up to "libc-9.9*".
Fixed.
Did you change the regexp as well? Dropping the [0-9]*?
No, but I see your point, the current expression obviously supports more than that, and so the comment is inaccurate.
It's not useful to have '[0-9]*' following the [2-9] since we'll never get that high in any reasonable long-term timeline.
I'll drop the '[0-9]*' and commit that with the above comment.
Testing looks good.
I've pushed the glibc_post_upgrade conversion to lua. With your recent removal of build-locale-archive, we've killed two statically compiled binaries in one step and made things simpler.
"Carlos O'Donell" carlos@redhat.com writes:
v2
- Fixed SOURCE0 reference in comment.
- Fixed iconvconfig inclusion in glibc.filelist.
- Fixed iconvconfig removal from common.filelist.
- Created post_exec function to run execs with asserts to check for errors.
- Switched step 1 to use tighter regexps.
- Switched step 1 to use %{_libdir} where possible.
- Fixed step 2 to use io.lines and simplify.
- Always run ldconfig in step 3 using post_exec
- Step 4 uses post_exec also.
diff --git a/glibc-post_upgrade.patch b/glibc-post_upgrade.patch deleted file mode 100644 index a64adfc..0000000 --- a/glibc-post_upgrade.patch +++ /dev/null
Deletes unneeded patch. OK.
-Release: 21%{?dist} +Release: 22%{?dist}
Ok
-Patch1: glibc-post_upgrade.patch
Removes unneeded patch. Ok.
-# We need to run ldconfig manually because ldconfig cannot handle the -# relative include path in the /etc/ld.so.conf file we gneerate. +# We need to run ldconfig manually because __brp_ldconfig assumes that +# glibc itself is always installed in $RPM_BUILD_ROOT, but with sysroots +# we may be installed into a subdirectory of that path. Therefore we +# unset __brp_ldconfig and run ldconfig by hand with the sysroots path +# passed to -r.
Ok.
-# Install the upgrade program -install -m 700 build-%{target}/elf/glibc_post_upgrade \
- %{glibc_sysroot}%{_prefix}/sbin/glibc_post_upgrade.%{_target_cpu}
Don't install the program we no longer need. OK.
-# which is to at least keep the timestamp consistent. The choice of using -# glibc_post_upgrade.c is arbitrary. +# which is to at least keep the timestamp consistent. The choice of using +# SOURCE0 is arbitrary.
Ok.
-# - The use of [^gi] is meant to exclude all files except glibc_post_upgrade, -# and iconvconfig, which we want in the main packages. +# - We want iconvconfig in the main package and we do this by using +# a double negation of -v and [^i] so it removes all files in +# sbin *but* iconvconfig.
Ok. Possibly TMI but ok. It's a weird thing it's doing, hard to document...
- -e '%{_prefix}/sbin/[^gi]' \
- -e '%{_prefix}/sbin/[^i]' \
Take out reference to program we no longer have, leave the rest alone. Ok.
-# glibc_post_upgrade.* and iconvconfig which need to go in glibc. Likewise -# nscd is excluded because it goes in nscd. +# iconvconfig which needs to go in glibc. Likewise nscd is excluded because +# it goes in nscd.
Ok.
-grep '%{_prefix}/sbin/[^gi]' master.filelist \ +grep '%{_prefix}/sbin' master.filelist \
- | grep -v '%{_prefix}/sbin/iconvconfig' \
Net result - everything in sbin except iconconfig, both ways. Ok.
-%post -p %{_prefix}/sbin/glibc_post_upgrade.%{_target_cpu} +%post -p <lua> +-- We use lua's posix.exec because there may be no shell that we can +-- run during glibc upgrade. +function post_exec (program, ...)
- local pid = posix.fork ()
- if pid == 0 then
- assert (posix.exec (program, ...))
- elseif pid > 0 then
- posix.wait (pid)
- end
+end
Ok.
+-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they may be preferred by the dynamic linker +-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures.
+-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +-- We employ a tight match on libc-X.Y.so where X > 2 and Y > 2?, and +-- related libraries so we would match "libc-2.20.so" and higher. +remove_regexps = { "libc%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so",
"libm%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","librt%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","libpthread%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","librtkaio%%-[2-9][0-9]*%%.[2-9][0-9]+%%.so","libthread_db%%-[1-9][0-9]*%%.[0-9]+%%.so" }
Noted that libthread_db is exception. Will not match libc-3.10.
+-- We are going to search these directories. +local remove_dirs = { "%{_libdir}/i686",
"%{_libdir}/i686/nosegneg","%{_libdir}/power6","%{_libdir}/power7","%{_libdir}/power8" }
List of obsolete multilibs, ok.
+-- Walk all the directories with files we need to remove... +for _, rdir in ipairs (remove_dirs) do
- if posix.access (rdir) then
- -- If the directory exists we look at all the files...
- local remove_files = posix.files (rdir)
- for rfile in remove_files do
for _, rregexp in ipairs (remove_regexps) do- -- Does it match the regexp?
- local dso = string.match (rfile, rregexp)
if (dso ~= nil) then-- Removing file...os.remove (rdir .. '/' .. rfile)- end
end- end
- end
+end
Ok.
+-- (2) Update /etc/ld.so.conf +-- Next we update /etc/ld.so.conf to ensure that it starts with +-- a literal "include ld.so.conf.d/*.conf".
+local ldsoconf = "/etc/ld.so.conf" +local ldsoconf_tmp = "/etc/glibc_post_upgrade.ld.so.conf"
+if posix.access (ldsoconf) then
- -- We must have a "include ld.so.conf.d/*.conf" line.
- local have_include = false
- for line in io.lines (ldsoconf) do
- -- This must match, and we don't ignore whitespace.
- if string.match (line, "^include ld.so.conf.d/%*%.conf$") ~= nil then
have_include = true- end
- end
- if not have_include then
- -- Insert "include ld.so.conf.d/*.conf" line at the start of the
- -- file. We only support one of these post upgrades running at
- -- a time (temporary file name is fixed).
- local tmp_fd = io.open (ldsoconf_tmp, "w")
- if tmp_fd ~= nil then
tmp_fd:write ("include ld.so.conf.d/*.conf\n")for line in io.lines (ldsoconf) dotmp_fd:write (line .. "\n")endtmp_fd:close ()local res = os.rename (ldsoconf_tmp, ldsoconf)if res == nil thenio.stdout:write ("Error: Unable to update configuration file (rename).\n")end- else
io.stdout:write ("Error: Unable to update configuration file (open).\n")- end
- end
+end
Ok.
+-- (3) Rebuild ld.so.cache early. +-- If the format of the cache changes then we need to rebuild +-- the cache early to avoid any problems running binaries with +-- the new glibc.
+-- Note: We use _prefix because Fedora's UsrMove says so. +post_exec ("%{_prefix}/sbin/ldconfig")
Ok.
+-- (4) Update gconv modules cache. +-- If the /usr/lib/gconv/gconv-modules.cache exists, then update it +-- with the latest set of modules that were just installed. +-- We assume that the cache is in _libdir/gconv and called +-- "gconv-modules.cache".
+local iconv_dir = "%{_libdir}/gconv" +local iconv_cache = iconv_dir .. "/gconv-modules.cache" +if (posix.utime (iconv_cache) == 0) then
- post_exec ("%{_prefix}/sbin/iconvconfig",
"-o", iconv_cache,"--nostdlib",iconv_dir)+else
- io.stdout:write ("Error: Missing " .. iconv_cache .. " file.\n")
+end
Ok.
diff --git a/glibc_post_upgrade.c b/glibc_post_upgrade.c deleted file mode 100644 index 9014857..0000000 --- a/glibc_post_upgrade.c +++ /dev/null
Deleting unneeded file, ok.
"Carlos O'Donell" carlos@redhat.com writes:
diff --git a/glibc-post_upgrade.patch b/glibc-post_upgrade.patch deleted file mode 100644
This deletes one of the .c files and it's Makefile rules, ok.
diff --git a/glibc_post_upgrade.c b/glibc_post_upgrade.c deleted file mode 100644
This deletes the other. Ok.
-Patch1: glibc-post_upgrade.patch
Ok.
-# We need to run ldconfig manually because ldconfig cannot handle the -# relative include path in the /etc/ld.so.conf file we gneerate. +# We need to run ldconfig manually because __brp_ldconfig assumes that +# glibc itself is always installed in $RPM_BUILD_ROOT, but with sysroots +# we may be installed into a subdirectory of that path. Therefore we +# unset __brp_ldconfig and run ldconfig by hand with the sysroots path +# passed to -r.
Ok.
-# Install the upgrade program -install -m 700 build-%{target}/elf/glibc_post_upgrade \
- %{glibc_sysroot}%{_prefix}/sbin/glibc_post_upgrade.%{_target_cpu}
Ok.
-# which is to at least keep the timestamp consistent. The choice of using -# glibc_post_upgrade.c is arbitrary. +# which is to at least keep the timestamp consistent. The choice of using +# %{SOURCE0} is arbitrary. touch -r %{SOURCE0} %{glibc_sysroot}/etc/ld.so.conf touch -r sunrpc/etc.rpc %{glibc_sysroot}/etc/rpc
Source0 was the .tar.gz file anyway, so this just corrects the comment. OK, caveat Florian's comment about lint.
-# - The use of [^gi] is meant to exclude all files except glibc_post_upgrade, -# and iconvconfig, which we want in the main packages. +# - We want iconvconfig in the main package.
- -e '%{_prefix}/sbin/[^gi]' \
- -e '%{_prefix}/sbin/iconvconfig' \
These two are EXCLUDING files from glibc.filelist. The comment and code are both consistent in that we go from "glibc_post_upgrade and iconfconfig" to just "iconvconfig". Ok.
# All of the bin and certain sbin files go into the common package except -# glibc_post_upgrade.* and iconvconfig which need to go in glibc. Likewise -# nscd is excluded because it goes in nscd. +# iconvconfig which needs to go in glibc. Likewise nscd is excluded because +# it goes in nscd.
Likewise.
grep '%{_prefix}/bin' master.filelist >> common.filelist -grep '%{_prefix}/sbin/[^gi]' master.filelist \ +grep '%{_prefix}/sbin/iconvconfig' master.filelist \ | grep -v 'nscd' >> common.filelist
The old source EXCLUDES iconvconfig, the new source REQUIRES it. Use [^i] instead.
Er, OK. Or spurious whitespace changes. Your choice :-)
-%post -p %{_prefix}/sbin/glibc_post_upgrade.%{_target_cpu} +%post -p <lua> +-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they would be preferred by the dynamic linker
"may be"
+-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures. */
Ok.
+-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +local remove_prefixes = { "libc-",
"libm-","librt-","libpthread-","librtkaio-","libthread_db-" }+-- We are going to search these directories. +local remove_dirs = { "/lib/i686",
"/lib/i686/nosegneg","/lib64/power6","/lib64/power7","/lib64/power8" }+-- Walk all the directories with files we need to remove... +for _, rdir in ipairs (remove_dirs) do
- if posix.access (rdir) then
We're not accounting for $prefix but the original didn't either. So this is a "if exists" check. Ok.
- -- If the directory exists we look at all the files...
- local remove_files = posix.files (rdir)
- for rfile in remove_files do
for _, rprefix in ipairs (remove_prefixes) do- -- Does the literal prefix match?
- local prefix = string.find (rfile, rprefix, 1, true)
- -- Does it end with '*.so'?
- local dso = string.match (rfile, "%.so$")
if (prefix == 1 and (dso ~= nil)) then
Ok.
-- Removing file...os.remove (rdir .. '/' .. rfile)- end
end- end
- end
+end
Ok.
+-- (2) Update /etc/ld.so.conf +-- Next we update /etc/ld.so.conf to ensure that it starts with +-- a literal "include ld.so.conf.d/*.conf".
+local ldsoconf = "/etc/ld.so.conf" +local ldsoconf_tmp = "/etc/glibc_post_upgrade.ld.so.conf"
+local conf_fd = io.open (ldsoconf, "r")
+if conf_fd ~= nil then
- -- We must have a "include ld.so.conf.d/*.conf" line.
- local have_include = false
- for line in conf_fd:lines () do
- -- This must match, and we don't ignore whitespace.
- if string.match (line, "^include ld.so.conf.d/%*%.conf$") ~= nil then
have_include = true- end
- end
- conf_fd:close ()
Ok.
- if not have_include then
- -- Insert "include ld.so.conf.d/*.conf" line at the start of the
- -- file. We only support one of these post upgrades running at
- -- a time (temporary file name is fixed).
- local tmp_fd = io.open (ldsoconf_tmp, "w")
- if tmp_fd ~= nil then
tmp_fd:write ("include ld.so.conf.d/*.conf\n")for line in io.lines (ldsoconf) dotmp_fd:write (line .. "\n")endtmp_fd:close ()
Ok.
local res = os.rename (ldsoconf_tmp, ldsoconf)if res == nil thenio.stdout:write ("Error: Unable to update configuration file (rename).\n")end- else
io.stdout:write ("Error: Unable to update configuration file (open).\n")- end
- end
+end
Ok.
+-- (3) Rebuild ld.so.cache early. +-- If the format of the cache changes then we need to rebuild +-- the cache early to avoid any problems running binaries with +-- the new glibc.
+if posix.access ("%{_prefix}/sbin/ldconfig", "x") then
Inconsistent use of ${prefix} ? The old script used /sbin/ldconfig, and we're checking /etc not ${prefix}/etc, etc. If this is intentional, needs explaining in the comment.
- local pid = posix.fork ()
- if pid == 0 then
- posix.exec ("%{_prefix}/sbin/ldconfig")
- elseif pid > 0 then
- posix.wait (pid)
- end
No error handling.
+-- (4) Update gconv modules cache. +-- If the /usr/lib/gconv/gconv-modules.cache exists, then update it +-- with the latest set of modules that were just installed. +-- We assume that the cache is in %{_libdir}/gconv and called +-- "gconv-modules.cache".
+local iconv_dir = "%{_libdir}/gconv" +local iconv_cache = iconv_dir .. "/gconv-modules.cache"
Ok.
+if posix.utime (iconv_cache) == 0 then
- local pid = posix.fork ()
- if pid == 0 then
- posix.exec ("%{_prefix}/sbin/iconvconfig",
"-o", iconv_cache,"--nostdlib",iconv_dir)- elseif pid > 0 then
- posix.wait (pid)
- end
Again, no error handling.
+else
- io.stdout:write ("Error: Missing " .. iconv_cache .. " file.\n")
+end
Ok.
+* Thu May 16 2019 Carlos O'Donell carlos@redhat.com - 2.29.9000-20
Remember to tweak the timestamp as you go ;-)
On 5/17/19 5:56 PM, DJ Delorie wrote:
grep '%{_prefix}/bin' master.filelist >> common.filelist -grep '%{_prefix}/sbin/[^gi]' master.filelist \ +grep '%{_prefix}/sbin/iconvconfig' master.filelist \ | grep -v 'nscd' >> common.filelist
The old source EXCLUDES iconvconfig, the new source REQUIRES it. Use [^i] instead.
Fixed. Using grep -v and iconvconfig, since that's more explicit.
Er, OK. Or spurious whitespace changes. Your choice :-)
Fixed. Removed spurious change.
-%post -p %{_prefix}/sbin/glibc_post_upgrade.%{_target_cpu} +%post -p <lua> +-- (1) Remove multilib libraries from previous installs. +-- In order to support in-place upgrades, we must immediately remove +-- obsolete platform directories after installing a new glibc +-- version. RPM only deletes files removed by updates near the end +-- of the transaction. If we did not remove the obsolete platform +-- directories here, they would be preferred by the dynamic linker
"may be"
+-- during the execution of subsequent RPM scriptlets, likely +-- resulting in process startup failures. */
Ok.
Fixed. Removed trailing C comment close.
+-- We are going to remove these libraries. Generally speaking we remove +-- all core libraries in the multilib directory. +local remove_prefixes = { "libc-",
"libm-","librt-","libpthread-","librtkaio-","libthread_db-" }+-- We are going to search these directories. +local remove_dirs = { "/lib/i686",
"/lib/i686/nosegneg","/lib64/power6","/lib64/power7","/lib64/power8" }+-- Walk all the directories with files we need to remove... +for _, rdir in ipairs (remove_dirs) do
- if posix.access (rdir) then
We're not accounting for $prefix but the original didn't either. So this is a "if exists" check. Ok.
We should.
Fixed. Uses %{_libdir}.
+-- (3) Rebuild ld.so.cache early. +-- If the format of the cache changes then we need to rebuild +-- the cache early to avoid any problems running binaries with +-- the new glibc.
+if posix.access ("%{_prefix}/sbin/ldconfig", "x") then
Inconsistent use of ${prefix} ? The old script used /sbin/ldconfig, and we're checking /etc not ${prefix}/etc, etc. If this is intentional, needs explaining in the comment.
Yes, in Fedora we install into %{_prefix} and so the use is intentional, on your system you'll see /sbin is a symlink to /usr/sbin, and this is part of the MoveToUsr transition (which glibc never fully completed).
The old script did use /sbin/ldconfig and it was correct, but may not be correct at some point in the future.
The new lua script uses %{_prefix} and so is more correct (though not quite perfect with sysroots, but I'll fix that later).
Lastly, we don't apply %{_prefix} to /etc installed files, this is controlled by sysconfdir.
Fixed. Comment added.
- local pid = posix.fork ()
- if pid == 0 then
- posix.exec ("%{_prefix}/sbin/ldconfig")
- elseif pid > 0 then
- posix.wait (pid)
- end
No error handling.
Fixed. Using assert.
+-- (4) Update gconv modules cache. +-- If the /usr/lib/gconv/gconv-modules.cache exists, then update it +-- with the latest set of modules that were just installed. +-- We assume that the cache is in %{_libdir}/gconv and called +-- "gconv-modules.cache".
+local iconv_dir = "%{_libdir}/gconv" +local iconv_cache = iconv_dir .. "/gconv-modules.cache"
Ok.
+if posix.utime (iconv_cache) == 0 then
- local pid = posix.fork ()
- if pid == 0 then
- posix.exec ("%{_prefix}/sbin/iconvconfig",
"-o", iconv_cache,"--nostdlib",iconv_dir)- elseif pid > 0 then
- posix.wait (pid)
- end
Again, no error handling.
Fixed. Using assert.
+else
- io.stdout:write ("Error: Missing " .. iconv_cache .. " file.\n")
+end
Ok.
+* Thu May 16 2019 Carlos O'Donell carlos@redhat.com - 2.29.9000-20
Remember to tweak the timestamp as you go ;-)
Fixed.
-
Carlos O'Donell -
DJ Delorie -
Florian Weimer