Anaconda-devel July 2007

anaconda-devel@lists.stg.fedoraproject.org

29 participants
25 discussions

Start a nNew thread

PATCH:Fix [only allows one param]
by Hiroshi Hara 14 Aug '07

14 Aug '07

Hi, I made the patch which fixes [only allows one param] Would you review the attached patch, please? thank you

2 1

PATCH:CIDR prefix on netconfig_dialog.py
by HARA Hiroshi 14 Aug '07

14 Aug '07

Hi, Did you forget to remove these lines? It seems that if these lines is not removed, CIDR prefix can not be recognized . would you check attached patch, please?

2 4

nfs/livecd install hack
by Jerry Vonau 07 Aug '07

07 Aug '07

Hi All: I've been playing around with both anaconda and the live-cd, and I had an idea to kind of merge the two. I took advantage up the RHupdates override on a nfs install, to have anaconda use a squashfs.img that is on the nfs server in /LiveOS/,(this could use a bit of work...) creating the same device name as the the live-cd install is looking for. Once that is done, anaconda uses a patched livecd.py to install to the harddrive. This is more of a proof of concept them prime code.. ;) First I altered fedora.py in installclass, flags.py, bootloader.py, livecd.py, and added a new one called livefs.py, adding them in /RHupdates/ to incorporate my changes with out really having to play with the loader. dmc: I like where you are going with the reboot-less install, looking forward to your progress. Jerry --- /home/cvsroot/anaconda/installclasses/fedora.py 2007-02-06 10:14:43.000000000 -0600 +++ installclasses/fedora.py 2007-06-19 17:44:46.000000000 -0500 @@ -45,22 +45,12 @@ anaconda.dispatch.skipStep("partition") def getMethod(self, methodstr): + if os.access("/mnt/source/RHupdates/livefs.py", os.R_OK): + import livefs + import livecd + return livecd.LiveCDImageMethod + if methodstr.startswith("livecd://"): import livecd return livecd.LiveCDImageMethod + return BaseInstallClass.getMethod(self, methodstr) def getBackend(self, methodstr): + if os.access("/mnt/source/RHupdates/livefs.py", os.R_OK): + import livecd + return livecd.LiveCDCopyBackend + if methodstr.startswith("livecd://"): import livecd return livecd.LiveCDCopyBackend --- /home/cvsroot/anaconda/flags.py 2006-10-18 16:33:24.000000000 -0500 +++ flags.py 2007-06-17 20:11:23.000000000 -0500 @@ -66,6 +66,7 @@ self.__dict__['flags']['cmdline'] = self.createCmdlineDict() self.__dict__['flags']['useIPv4'] = True self.__dict__['flags']['useIPv6'] = True + self.__dict__['flags']['livefs'] = 0 # for non-physical consoles like some ppc and sgi altix, # we need to preserve the console device and not try to # do things like bogl on them. this preserves what that --- /home/cvsroot/anaconda/bootloader.py 2007-01-16 13:58:13.000000000 -0600 +++ bootloader.py 2007-07-02 15:43:20.000000000 -0500 @@ -37,7 +37,7 @@ return # FIXME: this is a hack... - if flags.livecd: + if flags.livecd or flags.livefs: return if anaconda.id.ksdata: --- /home/cvsroot/anaconda/livecd.py 2007-07-02 14:05:21.000000000 -0500 +++ livecd.py 2007-07-11 20:49:18.000000000 -0500 @@ -86,9 +75,12 @@ class LiveCDImageMethod(installmethod.InstallMethod): def __init__(self, method, rootpath, intf): """@param method livecd://live-block-device""" - installmethod.InstallMethod.__init__(self, method, rootpath, intf) + installmethod.InstallMethod.__init__(self, method, rootpath, intf) self.osimg = method[8:] + if flags.livefs: + log.info("livefs mode - changing os.img") + self.osimg = "/dev/live-osimg" if not stat.S_ISBLK(os.stat(self.osimg)[stat.ST_MODE]): intf.messageWindow(_("Unable to find image"), _("The given location isn't a valid %s " @@ -316,9 +295,72 @@ f.close() # rebuild the initrd(s) - vers = self.kernelVersionList() - for (n, arch, tag) in vers: - packages.recreateInitrd(n, anaconda.rootPath) + if flags.livefs: + log.info("livefs mode - need bootloader") + log.info("livefs mode - need to find kernel") + # guaranteed to only have one kernel installed at this point + + kver = subprocess.Popen(["/usr/bin/ls", "%s/lib/modules/"%anaconda.rootPath], stdout=subprocess.PIPE).communicate()[0].strip() + print "Kernel version is %s"%kver + log.info("Kernel version is %s" %(kver)) + + # create new initramfs + os.system("/usr/sbin/chroot %s /sbin/mkinitrd -v -f /boot/initrd-%s.img %s"%(anaconda.rootPath, kver, kver)) + log.info("livefs mode - rebuild initrd") + # Figure out grub device name + ##HACK + diskDeviceFile="/dev/sda" +# grubDevNumber=int(rootDeviceFile[diskDeviceFile.__len__():]) - 1 +# grubDev="(hd0,%s)"%grubDevNumber + grubDev="(hd0,0)" + osName="Fedora" + + log.info("livefs mode - need to write grub.conf") + # create grub.conf (see above how grubDev is computed) + grub = open(anaconda.rootPath + "/boot/grub/grub.conf", "w") + grub.write("default=0\n") + grub.write("timeout=5\n") + grub.write("splashimage=%s/grub/splash.xpm.gz\n"%grubDev) + grub.write("hiddenmenu\n") + grub.write("title %s (%s)\n"%(osName, kver)) + grub.write(" root %s\n"%grubDev) + #grub.write(" kernel /boot/vmlinuz-%s ro quiet root=UUID=%s\n"%(kver, uuid)) + grub.write(" kernel /vmlinuz-%s ro root=LABEL=%s\n"%(kver, "/")) + grub.write(" initrd /initrd-%s.img\n"%kver) + grub.close() + + os.system("rm -f %s/etc/grub.conf"%anaconda.rootPath) + os.system("ln -s ../boot/grub/grub.conf %s/etc/grub.conf"%anaconda.rootPath) + os.system("cat %s/etc/grub.conf"%anaconda.rootPath) + + + # copy necessary grub files + log.info("livefs mode - need to write MBR") + os.system("cp %s/usr/share/grub/i386-redhat/*stage* %s/boot/grub/"%(anaconda.rootPath, anaconda.rootPath)) + # install grub +# if useMBR: +# grubMbrParam = "(hd0)" +# else: +# grubMbrParam = grubDev +### FIXME hardcode +# f = os.popen("/usr/sbin/chroot %s /sbin/grub --verbose --batch --device-map=/dev/null", "w"%(anaconda.rootPath)) + f = os.popen("/usr/sbin/chroot /mnt/sysimage /sbin/grub --verbose --batch --device-map=/dev/null", "w") + f.write("device (hd0) %s\n"%diskDeviceFile) + f.write("root %s\n"%grubDev) +# f.write("setup --stage2=/boot/grub/stage2 --prefix=/boot/grub (hd0)\n") +# f.write("setup --stage2=/grub/stage2 --prefix=/grub (hd0)\n") + f.write("setup (hd0)\n") + f.write("quit\n") + f.flush() + f.close() + log.info("livefs mode - DONE") + + else: + + # rebuild the initrd(s) + vers = self.kernelVersionList() + for (n, arch, tag) in vers: + packages.recreateInitrd(n, anaconda.rootPath) def writeConfiguration(self): pass livefs.py: #Jerry's anaconda init import os,sys import isys import string import iutil from flags import flags # Lets get some loops isys.makeDevInode("loop1", "/tmp/loop1") isys.makeDevInode("loop2", "/tmp/loop2") isys.makeDevInode("loop3", "/tmp/loop3") isys.makeDevInode("loop4", "/tmp/loop4") # set livefs flag flags.livefs = "1" # Look for squash #imgpath = None #imglist = [ "/mnt/source/LiveOS/squashfs.img", "/mnt/source/squashfs.img" ] #for p in imglist: # if (os.access(p, os.X_OK)): # imgpath = p #if os.access("/mnt/source/LiveOS/squashfs.img", os.R_OK) # imgpath = "/mnt/source/LiveOS/squashfs.img" #if os.access("/mnt/source/squashfs.img", os.R_OK): # imgpath = "/mnt/source/squashfs.img" # found squash HACK hard code.... Got to work on those skills..... # isys.losetup("/tmp/loop3", "%s", readOnly = 1 %imgpath) isys.losetup("/tmp/loop3", "/mnt/source/LiveOS/squashfs.img", readOnly = 1) os.mkdir("/squashfs") isys.mount("/tmp/loop3", "/squashfs", fstype = 'squashfs', readOnly = 1) isys.losetup("/tmp/loop4", "/squashfs/os.img", readOnly = 1) os.system("ln -s /tmp/loop4 /dev/live-osimg") break # Look for ext3 #imgpath = None #imglist = [ "/mnt/source/LiveOS/ext3fs.img", "/mnt/source/ext3fs.img" ] #for p in imglist: # if (os.access(p, os.X_OK)): # imgpath = p # break #if os.access("/mnt/source/LiveOS/ext3fs.img", os.R_OK) # imgpath = "/mnt/source/LiveOS/ext3fs.img" #if os.access("/mnt/source/ext3fs.img", os.R_OK): # imgpath = "/mnt/source/ext3fs.img" # found ext3 for sysroot # isys.losetup("/tmp/loop4", imgpath , readOnly = 1) # isys.losetup("/tmp/loop4", "/mnt/source/ext3fs.img", readOnly = 1) # os.system("ln -s /tmp/loop4 /dev/live-osimg")

4 9

[PATCH] generate module-info on the fly
by Bill Nottingham 06 Aug '07

06 Aug '07

The attached patch generates the module-info file on the fly from the module lists in the kernel, so we don't need to keep updating a static file any more. Requires the kernel spec changes at: https://www.redhat.com/archives/fedora-kernel-list/2007-July/msg00109.html Caveats: 1) module descriptions are read from the modules themselves. Ergo, they may be horribly ugly and inconsistent 2) it's possible this will miss a module or two in its initial incarnation loader2/Makefile | 1 - scripts/mk-images | 8 +++++--- utils/Makefile | 1 + utils/genmodinfo | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 60 insertions(+), 4 deletions(-) Bill

5 11

[BOGUS PATCH] warn liveinst users about mandatory rootfs format
by Douglas McClendon 06 Aug '07

06 Aug '07

I mentioned on fedora-devel that I might try to put together (today) a patch to fix 250301. I said this because I know jkatz is out for the week, and it seems like this would be a _really_ nice thing to fix for f8t1. However, even though I have been posting anaconda patches of late, I am hardly a seasoned python&anaconda hacker, able to instantaneously fix things like this. Basically I can feel myself falling into that cliche of only having the motivation to work on the features that I'm having fun with, and not the bugs that are more important to the wider community. So in case that happens. Here is what I would consider the bare minimal patch that ought to be applied for F8T1. From what I can gather so far, the next better patch would be to replicate queryNoFormatPreExisting as something like liveInstMustFormatRoot, and invoke it appropriately in the 4 places where the query one is invoked. The next better beyond that would be to implement a file level copy installation mechanism so that it actually is possible to leave '/' unformatted. Or find a way to disallow the option of not formatting in the UI. And quite probably Jeremy or other more seasoned anaconda people might know of better solutions than those. -dmc

2 2

Patch to add [stage2] section to .treeinfo
by Will Woods 31 Jul '07

31 Jul '07

This patch, as the subject says, will add a [stage2] section to .treeinfo. This will help other tools find the stage2 images, for things like upgrades and such. Comments welcomed. -w

2 1

Re: LiveCD wiping root partition?
by Douglas McClendon 31 Jul '07

31 Jul '07

Douglas McClendon wrote: > Michel Salim wrote: >> I just did a clean install of Fedora 7 from the live CD onto my laptop, >> which previously had a Fedora install upgraded from one of the F7 test >> releases, partitioned as suggested by anaconda (LVM, one swap partition, >> everything else under '/') >> >> When reinstalling, I kept the partition layout and specifically told >> Anaconda *not* to reformat / (having booted in rescue mode beforehand, >> and >> removing everything but /home). Anaconda gave a warning that the leftover >> files might interfere with the installed system, which gave the >> impression >> that those files won't actually be removed during installation. > > I don't remember the specific warning, but if it is not clearly > indicating that unselecting the format option on '/' is not allowed, > then that is a bug that should be fixed either with better user > messages, or alternate installation mechanisms. WOW. I was wrong. There is NO such message. That is a horrible bug. I may try to check to see if it's still in the latest anaconda, and provide some sort of simple patch later today. Also, to answer your question more thoroughly than my first reply- Yes, after the dd, if there is a seperate /usr or other partitions, files are then copied from / to there. This is all very related to my turboLiveInst patch which I recently posted to livecd-list and anaconda-devel. I'm surprised I don't remember hearing about this bug before. I had personally run into the same warning you saw, but that is just a general warning that has nothing to do with the livecd installer case specifically, and the livecd installer will stupidly let you just march along with the / fs not scheduled for formatting, even though it is going to anyway. -dmc > > I didn't have anything to do with what's in F7, though I did know enough > about it to catch the fact that the 'formatting / filesystem' during the > process was meaningless and time consuming, so that will be gone from > from F8. > >> >> As it turns out, however, the old contents are completely gone (I'm >> trying >> out different recovery tools now to see if I could rescue some of the >> data). >> It's as if the live CD simply used dd to transfer the install image to >> the >> hard drive (in which case, how does it actually handle different >> partition >> layouts, e.g. /usr, /var, /home on separate partitions -- does it just >> move >> the files afterwards?) >> >> As it stands it seems that the Live CD is a very dangerous tool, at >> least as >> long as >> 1) the default behaviour of Anaconda is to put everything under / >> 2) it does not carry more warnings about what it will do to the / >> partition >> during installation >> >> Could someone let us know how the live CD actually performs its work? > > > It takes a 4.0G ext3 fsimage from the cd, effectively dd's it to the > chosen root volume, then does a resize2fs. > > > It >> would aid tremendously in the data recovery part. Would 'dd'-ing the >> entire >> partition to an external drive, and mounting it on a Windows computer >> (most >> recovery tools are unfortunately for that platform) preserve all the data >> required? I'm assuming that the new installation overwrote the same >> parts of >> the disk that was used to hold the OS in the previous install. > > As long as you haven't done much, data that was sitting in the first > 4.0G of the filesystem will be gone, but data in the rest should be > largely untouched. I have no idea how to go about trying to recover it > (haven't used any windows tools), except asking for expensive > professional date recovery help. Or if all I cared about was a text > file, using 'strings' and 'grep' on chunks of the disk data. > > -dmc >

1 0

Revised: [PATCH] turboLiveInst - improves livecd/usb installer speed by 15-20+%
by Douglas McClendon 27 Jul '07

27 Jul '07

Attached is a revised version of my turboLiveInst patch to livecd-tools and anaconda. This version is more polished. I.e. bugs have been fixed, complexity removed, and therefore should be easier to review. I performed some anecdotal performance tests, on a sony vaio vgn-n250e. I used a 30G destination volume for all tests, and when using usbstick media, it was media that reported 8.5MB/s from hdparm -t. I did have selinux disabled, and did not use the prelink option. I'd love to hear performance numbers from differing test rigs. The performance results I got were- install from cdrom without turboLiveInst: copy: 250s postinst: 86s install from cdrom with turboLiveInst: copy: 299s postinst: 84s install from usb without turboLiveInst: copy: 226s postinst: 72s install from usb with turboLiveInst: copy: 175s postinst: 72s Conclusions: On this testrig, installing from cdrom, turboLiveInst yielded a 20% speedup in copy, which resulted in an end to end install speedup of 15%. Installing from usb, turboLiveInst yielded a 29% speedup in copy, which resulted in an end to end install speedup of 20%. I did test copy-to-ram mode, and the resulting benefits were laughably huge. But this is only because this laptop has 1G of ram, and very strange behaviour occurs this near the threshold of having too little ram to use this feature. Though this is still an argument in favor of turboLiveInst, in that somehow it found itself on the better side of the threshold. I would expect that with 2G of ram, the benefit would be on the order of 35-50% speedup, as the main thing masking the benefit in the cdrom case, is the slow access to install media, which hides the benefit of cutting the needed disk writes nearly in half. The secondary benefit of turboLiveInst is that it removes the artificial limitation that the target rootfs must be greater than 4.0G, instead of the 2.1G actual uncompressed size of the contents of the LiveCD. Jeremy has pushed back against this patch because of complexity. Hopefully this round of polishing will make the patch much easier to read and understand. In addition, since the cleanupDeleted patch which this one depends on has already been merged, that should also make this a bit more palatable. Jeremy also brought up the idea of doing a file level copy installation, rather than the current block level mechanism. This _is_ a good idea, in my opinion, in that it will more intelligently support situations such as a separate /usr filesystem. In addition there is no way that turboLiveInst or the existing block level mechanism can be made to support xfs or other non-ext3 destination filesystems chosen by the user (should those options return). But- I would argue that file level installations may suffer badly due to cdrom seeking. I would also argue that there is no reason why turboLiveInst, could not be the first choice for installation technique, with fallbacks to file level copies for the seperate /usr and xfs type scenarios. Ultimately I just hope that turboLiveInst gets serious consideration for F8, via performance comparison with whatever other options may exist. Finally, here are some notes on the architecture, which may help you to understand the code- As I mentioned in the original patch, the basic idea is to simply, at livecd-creator build time, use a device mapper snapshot to generate a delta file between the 4.0G filesystem housing 2.1G of data, and a 2.1G filesystem holding the same data. Then at anaconda/liveinst time, that delta file is used to recreate a virtual image of the 2.1G filesystem, so that it can be copied to the installation target, rather than the 4.0G filesystem which includes 1.9G of zeros that needn't be written to disk. I ended up using /dev/loop118 in the initramfs init (mayflower generated) to expose the delta file. /dev/loop118 was mknod'd already by mayflower, but not actually used for anything. I used it to expose the delta file (osmin.tgz), because /dev/loop121 was being used to expose the 4.0G os.img, and it seemed simplest to use an identical mechanism to expose that data. Because by the time anaconda runs, the original cdrom and squashfs filesystems have been lazy unmounted, a simple cp at that time was not an option(?). I chose to extract the delta file (osmin, a 16MB sparse file containing 1.2M of data, compressed to 25kb on cdrom) into /dev/shm (i.e. ram) so that reads from it would not try to go to the cdrom. This may not have been necessary. I chose to calculate the size of the filesystem at livecd-creator time, and include it with osmin as osmin.size(together forming osmin.tgz). This is less complex than what I did in the first pass, which was to use dumpe2fs to calculate it at ananconda time. As always, questions, comments, criticisms, and especially testers are more than welcome. peace... -dmc

1 0

anaconda vs /etc/mtab -- fixed recently?
by Jeffrey Law 26 Jul '07

26 Jul '07

In late 2006 I stumbled upon a bug in which anaconda would throw an exception if a %post script changed /etc/mtab to be a link to /proc/mounts. I had assumed this was due to anaconda's attempts to update /etc/mtab as it unmounted filesystems before it exited. I even went as far as to strace anaconda to verify it indeed was mucking around with /etc/mtab after the %post script ran. However, I went to reproduce (and then fix) this problem today, but couldn't reproduce it. Does anyone know for sure if this was fixed? Or should I dig deeper to see if there's another element necessary to trigger the failure? (the original kickstart that triggered this failure had a lot of other stuff in its %post that perhaps could play a role in triggering the failure). Jeff

2 1

[PATCH] remove plip support
by Bill Nottingham 26 Jul '07

26 Jul '07

I strongly suspect that this does not work, as we haven't even included the modules in forever. Bill

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Anaconda-devel July 2007