I may have gotten this idea from someone else, so my apologies if it looks like I'm presenting it as my own. Just speak up and you'll get the glory and adulation of the tens that are reading this list.
As you may know, I am working on converting us from an initrd+install.img to one giant initrd containing everything from both. My baseline of just shoving everything into one image was 172 MB compressed. This is pretty big - it increases memory requirements and causes a variety of weird lags in bootup. For just one data point, a VM here refused to boot the boot.iso with anything less than 1 GB of memory.
After some strategic removals as outlined in a previous mail, I reduced this image to 131 MB compressed, 386 MB uncompressed. This is much better and is quite a bit smaller than what we have today. It reduced my boot.iso memory requirements to 560 MB. However, it's still not close to what I want. My goal is a 100 MB compressed initrd.
So it's time to think a little more drastically. Here it is: What if we kicked all translations out of the initrd?
This further reduces the initrd to 114 MB compressed, 329 MB uncompressed. Check out all those memory savings, plus think about how much faster downloads will go if doing pxeboot. As an English speaker, I can't see how anyone would disapprove of this plan.
No? Not my best idea ever? Well, I'm not advocating removing translation support from anaconda. I just want to move it. What if instead of building in every single translation to one giant file that everyone has to download, we moved each language into its own specific support image file?
This file would basically be another cpio archive containing /usr/share/locale/<lang>, perhaps plus some other files as they turn up. These would be stored in a known location given by .treeinfo, perhaps in a directory within images/. Then in loader when you select your language, the proper support image is fetched, mounted, and away we go. For people using boot.iso or the DVD, there would be no downloading - this would all just be mounted silently in the background. For pxebooting, it would mean downloading a separate file, but it would only be the one smaller, specific one so it should save time and bandwidth.
It may sound like this is violating my principle that we should move everything needed for installation into one single image to be downloaded, but I don't think so. I think once I get all the horrible method-specific poking, transferring, and mounting code ripped out, adding support for transferring and mounting one single image should be pretty straightforward.
Thoughts?
- Chris
John Reiser (jreiser@bitwagon.com) said:
So it's time to think a little more drastically. Here it is: What if we kicked all translations out of the initrd?
+1
Another +1 if you kick out the fonts, too!
You'll probably get more savings from that than from translations. However, this is much more complex, as instead of just excluding a directory, you'll have to parse at least one of comps or the fontconfig files, to see which fonts go with which language.
Bill
You'll probably get more savings from that than from translations. However, this is much more complex, as instead of just excluding a directory, you'll have to parse at least one of comps or the fontconfig files, to see which fonts go with which language.
/usr/share/fonts is 31 MB uncompressed, vs. /usr/share/locale at 55 MB uncompressed. I don't know offhand how those compare compressed.
But you're right - it's quite a bit more complex. The good news is we'd be doing it at image construction time which means a simpler language available to do this complex stuff. But let's not work on this unless we have to.
- Chris
So it's time to think a little more drastically. Here it is: What if we kicked all translations out of the initrd?
+1
Another +1 if you kick out the fonts, too!
I haven't quite decided how to handle that yet, since a lot of language share the same fonts and if I moved them into the language support files, it'd mean a fair amount of duplication. That could mean making split media even more difficult to creat.
But yes, this is probably the next thing to look into. Fonts are huge.
- Chris
So it's time to think a little more drastically. Here it is: What if we kicked all translations out of the initrd?
+1
Another +1 if you kick out the fonts, too!
Ah, this language splitting thing was your idea (http://osdir.com/ml/anaconda-devel-list/2010-08/msg00047.html). Thanks for coming up with this. I think we can run with it.
- Chris
just to make sure, are there in initrd any hardware drivers which are not required for booting?
Perhaps, but that's hard to figure out.
/modules is 15 MB uncompressed and /firmware is 8 MB uncompressed. So while there's some savings possible here, figuring out what can go and what can stay is tricky. I am trying to stick to the obvious and easy stuff for now. If we have to dig in a little deeper and make harder decisions later, if required.
- Chris
As discussed in #anaconda on freenode:
11:39 < rvykydal> ok, trying hard to nitpick - to get the language image, english-only UI will be minimal to none, right? no network configuration / target selection needed? 11:40 < clumens> yeah, that could be a problem 11:41 < clumens> we could maybe still include the anaconda.mo and nm-applet.mo in the initrd, though. 11:42 < clumens> or loader could be its own domain and include only that 11:43 < rvykydal> clumens, that makes sense
- Chris
Moving them out was easy:
diff --git a/scripts/mk-images b/scripts/mk-images index da9b508..cda2e1c 100755 --- a/scripts/mk-images +++ b/scripts/mk-images @@ -365,6 +365,21 @@ makeproductfile() { fi }
+makelangs() { + root=$1 + destdir=$2 + + for d in $root/usr/share/locale/*; do + if [ ! -d $d ]; then + continue + fi + + lang=$(basename $d) + ( cd $root ; find usr/share/locale/$lang | cpio -c -o | gzip -9 > $destdir/$lang.img ) + rm -rf $d + done +} + makeinitrd() { INITRDMODULES="" KEEP="" @@ -468,6 +483,11 @@ elif [ "$BUILDARCH" = "sparc" -o "$BUILDARCH" = "sparcv9" -o "$BUILDARCH" = "spa arches="sparc64" fi
+# Split the languages out into their own support images before building the +# initrd. +mkdir -p $TOPDESTPATH/images/langs +makelangs $IMGPATH $TOPDESTPATH/images/langs + foundakernel="" for KERNELARCH in $arches; do for kernelvers in $kerneltags; do
clumens@exeter:~/install/20100826/x86_64/os/images$ ls boot.iso efiboot.img efidisk.img langs/ pxeboot/ clumens@exeter:~/install/20100826/x86_64/os/images$ ls -l pxeboot/ total 114892 -rw-r--r--. 2 root root 113547228 Aug 26 14:06 initrd.img -rwxr-xr-x. 2 root root 3966416 Aug 26 14:06 vmlinuz* clumens@exeter:~/install/20100826/x86_64/os/images$ ls langs/ af.img bn_IN.img de.img fi.img hr.img it.img mk.img ne.img pt.img sk.img sv.img uk.img ar.img ca.img el.img fr.img hu.img ja.img ml.img nl.img pt_BR.img sl.img ta.img vi.img as.img cs.img es.img gu.img id.img kn.img mr.img or.img ro.img sl_SI.img te.img zh_CN.img bg.img cy.img et.img he.img ilo.img ko.img ms.img pa.img ru.img sr.img tg.img zh_TW.img bn.img da.img fa.img hi.img is.img mai.img nb.img pl.img si.img sr@latin.img tr.img zu.img
The boot.iso has the same layout, so no problem there. Bringing them back in will be a little harder, but I think pretty straightforward once all the method stuff is gone.
Incidentally, boot.iso will now boot in a VM with as little as 500 MB of memory. That's a good improvement.
I haven't heard any significant problems on this plan yet but maybe I should give it a day or two. It might be nice to get a rel-eng opinion on adding all these extra files to the langs/ directory. Obviously, this is F-15 timeframe.
- Chris
anaconda-devel@lists.stg.fedoraproject.org