I would like to see a quick review for this.
I think have to do this, which is unfortunate. It increases build time by about ten minutes on x86-64 (with lots of CPU cores).
Proposed patch:
diff --git a/glibc.spec b/glibc.spec index 73ecd93..54c96b2 100644 --- a/glibc.spec +++ b/glibc.spec @@ -908,7 +908,9 @@ make -j1 install_root=%{glibc_sysroot} install -C build-%{target} # locales. %ifnarch %{auxarches} pushd build-%{target} -make %{?_smp_mflags} -O install_root=%{glibc_sysroot} \ +# Do not use a parallel make here because the hardlink optimization in +# localedef is not fully reproducible when running concurrently. +make install_root=%{glibc_sysroot} \ install-locales -C ../localedata objdir=`pwd` popd %endif
I think this could also explain the x86_64/i686 multilib differences for the locale archive that we saw downstream occasionally.
Thanks, Florian
On 11/21/18 1:12 PM, Florian Weimer wrote:
I would like to see a quick review for this.
I think have to do this, which is unfortunate. It increases build time by about ten minutes on x86-64 (with lots of CPU cores).
Proposed patch:
diff --git a/glibc.spec b/glibc.spec index 73ecd93..54c96b2 100644 --- a/glibc.spec +++ b/glibc.spec @@ -908,7 +908,9 @@ make -j1 install_root=%{glibc_sysroot} install -C build-%{target} # locales. %ifnarch %{auxarches} pushd build-%{target} -make %{?_smp_mflags} -O install_root=%{glibc_sysroot} \ +# Do not use a parallel make here because the hardlink optimization in +# localedef is not fully reproducible when running concurrently. +make install_root=%{glibc_sysroot} \ install-locales -C ../localedata objdir=`pwd` popd %endif
I think this could also explain the x86_64/i686 multilib differences for the locale archive that we saw downstream occasionally.
This is fine with me.
My *preference* would be to do a parallel install, followed by hardlink de-dup pass that we write and execute as part of the build. I think those two things can use multiple threads and save on the time.
* Carlos O'Donell:
On 11/21/18 1:12 PM, Florian Weimer wrote:
I would like to see a quick review for this.
I think have to do this, which is unfortunate. It increases build time by about ten minutes on x86-64 (with lots of CPU cores).
Proposed patch:
diff --git a/glibc.spec b/glibc.spec index 73ecd93..54c96b2 100644 --- a/glibc.spec +++ b/glibc.spec @@ -908,7 +908,9 @@ make -j1 install_root=%{glibc_sysroot} install -C build-%{target} # locales. %ifnarch %{auxarches} pushd build-%{target} -make %{?_smp_mflags} -O install_root=%{glibc_sysroot} \ +# Do not use a parallel make here because the hardlink optimization in +# localedef is not fully reproducible when running concurrently. +make install_root=%{glibc_sysroot} \ install-locales -C ../localedata objdir=`pwd` popd %endif
I think this could also explain the x86_64/i686 multilib differences for the locale archive that we saw downstream occasionally.
This is fine with me.
My *preference* would be to do a parallel install, followed by hardlink de-dup pass that we write and execute as part of the build. I think those two things can use multiple threads and save on the time.
Right, but that needs some sort of --no-hard-link option for localedef, I think.
Thanks, Florian
On 11/26/18 8:54 AM, Florian Weimer wrote:
- Carlos O'Donell:
On 11/21/18 1:12 PM, Florian Weimer wrote:
I would like to see a quick review for this.
I think have to do this, which is unfortunate. It increases build time by about ten minutes on x86-64 (with lots of CPU cores).
Proposed patch:
diff --git a/glibc.spec b/glibc.spec index 73ecd93..54c96b2 100644 --- a/glibc.spec +++ b/glibc.spec @@ -908,7 +908,9 @@ make -j1 install_root=%{glibc_sysroot} install -C build-%{target} # locales. %ifnarch %{auxarches} pushd build-%{target} -make %{?_smp_mflags} -O install_root=%{glibc_sysroot} \ +# Do not use a parallel make here because the hardlink optimization in +# localedef is not fully reproducible when running concurrently. +make install_root=%{glibc_sysroot} \ install-locales -C ../localedata objdir=`pwd` popd %endif
I think this could also explain the x86_64/i686 multilib differences for the locale archive that we saw downstream occasionally.
This is fine with me.
My *preference* would be to do a parallel install, followed by hardlink de-dup pass that we write and execute as part of the build. I think those two things can use multiple threads and save on the time.
Right, but that needs some sort of --no-hard-link option for localedef, I think.
For the record I proposed a patch for that upstream: https://www.sourceware.org/ml/libc-alpha/2018-11/msg00681.html
Once we have this option we can then use --no-hard-links to avoid any hard link usage, and then have a deterministic build result that we can post-process with hardlink?