livecd April 2011

livecd@lists.fedoraproject.org

14 participants
16 discussions

norhgb doesn't work
by Lars Bjørndal 03 May '11

03 May '11

Hello, list In my kickstart file, I use 'bootloader --append=nomodeset norhgb'. Even though, I get the graphical boot when booting the live media. Last, I tried it, building a Fedora 15 livecd. What can I do to get the old text output during boot? Regards, Lars

6 8

livecd-iso-to-disk : cannot create Legacy BIOS bootable partition with gptmbr.bin from syslinux
by Gregory Fowler 28 Apr '11

28 Apr '11

Hi I think that the disk utility "parted" is broken with regard to making hard drive partitions bootable. At least when syslinux gptmbr.bin file is used to format it. This is causing the livecd-iso-to-disk utility to incorrectly create a USB stick that is capable of UEFI boot but not Legacy BIOS boot. Question: *Can we get parted to write the bootable flag on the gptmbr.bin partition in the same way that sgdisk accomplishes this? *Or can we use the better tool in this situation (sgdisk) and patch the livecd-iso-to-disk script to enable the use of sgdisk when making USB sticks with the gptmbr.bin file from syslinux? For this test you need ... UEFI 2.1 compliant system Legacy BIOS bootable system Livecd ISO image the was correctly mastered as a UEFI and Legacy Bootable system. livecd-tools-16.3 parted gdisk syslinux (with gptmbr.bin) Run bash -x /usr/bin/livecd-iso-to-disk --reset-mbr --format --efi image.iso /dev/sdb and save the output. ###SAMPLE Output shows sgdisk does not see partition as bootable after parted does its work. ### Then after sgdisk does its work, it shows the partition as bootable. ... ... Press Enter to continue or ctrl-c to abort + read + umount /dev/sdb /dev/sdb1 + : + /sbin/parted --script /dev/sdb mklabel gpt ++ LC_ALL=C ++ /sbin/parted --script -m /dev/sdb 'unit b print' ++ grep '^/dev/sdb:' + partinfo='/dev/sdb:8019509248B:scsi:512:512:gpt:hp v165w;' ++ echo /dev/sdb:8019509248B:scsi:512:512:gpt:hp 'v165w;' ++ cut -d : -f 2 ++ sed -e 's/B$//' + size=8019509248 + /sbin/parted --script /dev/sdb unit b mkpart '"EFI System Partition"' fat32 1048576 8018460672 set 1 boot on + '[' -x /usr/sbin/sgdisk ']' + SGDISK= ++ /usr/sbin/sgdisk /dev/sdb --attributes=1:show + SGDISK= + echo 'SGDISK reports /dev/sdb attributes: ' SGDISK reports /dev/sdb attributes: + echo + egrep -i 'bios boot' + echo 'parted failed to make the partition Legacy BIOS bootable; USING SGDISK ' parted failed to make the partition Legacy BIOS bootable; USING SGDISK + /usr/sbin/sgdisk /dev/sdb --attributes=1:set:2 The operation has completed successfully. ++ /usr/sbin/sgdisk /dev/sdb --attributes=1:show + SGDISK='1:2:1 (legacy BIOS bootable)' + echo 'SGDISK reports /dev/sdb attributes: 1:2:1 (legacy BIOS bootable)' SGDISK reports /dev/sdb attributes: 1:2:1 (legacy BIOS bootable) ... ... + echo 'Target device is now set up with a Live image!' Target device is now set up with a Live image! + exitclean + RETVAL=0 + '[' -d /media/srctmp.NpEBDs ']' + '[' -d /media/tgttmp.ui8qin ']' + exit 0 Notice that parted is supposed to create a "bootable Flag" for the partition on the gptmbr.bin formatted USB stick. If we reboot from the stick in legacy mode, we get the syslinux error "Missing Operating System" see http://syslinux.zytor.com/wiki/index.php/Common_Problems#Missing_Operating_… http://syslinux.zytor.com/wiki/index.php/Common_Problems#Missing_OS_.28gptm… If we patch the script like below and use it to re-create the USB stick ( make it use the gdisk package's "sgdisk" util to make the partition bootable), then we can reboot from the USB stick in Legacy Mode and we have a proper Legacy/UEFI bootable USB stick that was formatted with syslinux gptmbr.bin. ##Patch to prove this out --- livecd-iso-to-disk 2011-04-28 09:52:40.000000000 -0400 +++ livecd-iso-to-disk_oracle 2011-04-28 09:52:41.000000000 -0400 @@ -422,6 +422,20 @@ partinfo=$(LC_ALL=C /sbin/parted --script -m $device "unit b print" |grep ^$device:) size=$(echo $partinfo |cut -d : -f 2 |sed -e 's/B$//') /sbin/parted --script $device unit b mkpart '"EFI System Partition"' fat32 1048576 $(($size - 1048576)) set 1 boot on + #parted may not properly set the gptmbr partition as bootable. + #Use sgdisk to check and fix + if [ -x /usr/sbin/sgdisk ] ; then + SGDISK="" + SGDISK=$(/usr/sbin/sgdisk $device --attributes=1:show) + echo "SGDISK reports $device attributes: $SGDISK" + if ! echo $SGDISK | egrep -i 'bios boot' ; then + echo "parted failed to make the partition Legacy BIOS bootable; USING SGDISK " + /usr/sbin/sgdisk $device --attributes=1:set:2 + SGDISK=$(/usr/sbin/sgdisk $device --attributes=1:show) + echo "SGDISK reports $device attributes: $SGDISK" + fi + fi + # Sometimes automount can be _really_ annoying. echo "Waiting for devices to settle..." /sbin/udevadm settle Thanks Gregory Fowler

2 2

minimum centos livecd and available ttys
by hai wu 28 Apr '11

28 Apr '11

Using Centos minimum livecd ks file at https://projects.centos.org/trac/livecd/browser/trunk/CentOS5/centos-livecd…, and there is only one tty available (tty0). What else would be needed to have multiple ttys when livecd boots up?

2 2

Extract original ISO from initrd.img file
by hai wu 28 Apr '11

28 Apr '11

Is there a tool to auto-extract livecd/ISO file from initrd.img (which was generated from livecd-iso-to-pxeboot command)?

3 3

Fedora 15 live iso build fails
by Valent Turkovic 17 Apr '11

17 Apr '11

Hi, I'm not sure if this is a know issue or not, I tried to build Fedora 15 live iso but it fails because of unmet dependency: Error creating Live CD : Failed to build transaction : udisks-1.0.1-8.fc15.i686 requires ntfsprogs Has anybody else tried creating Fedora 15 live iso and got it to work or you also had same issue? Here is a full log: # livecd-creator -t /home/temp/ --cache=/home/yum-cache/ -f Fedora-15 -c /home/fedora-live-desktop.ks /usr/lib/python2.7/site-packages/pykickstart/commands/partition.py:251: UserWarning: A partition with the mountpoint / has already been defined. warnings.warn(_("A partition with the mountpoint %s has already been defined.") % pd.mountpoint) mke2fs 1.41.12 (17-May-2010) Filesystem label=_Fedora-15 OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 262144 inodes, 1048576 blocks 10485 blocks (1.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1073741824 32 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. tune2fs 1.41.12 (17-May-2010) Setting maximal mount count to -1 Setting interval between checks to 0 seconds Retrieving http://ftp.heanet.ie/pub/fedora/linux/development/15/i386/os/repodata/repom… ...OK Retrieving http://ftp.heanet.ie/pub/fedora/linux/development/15/i386/os/repodata/bebf4… ...OK Retrieving http://ftp.heanet.ie/pub/fedora/linux/development/15/i386/os/repodata/95b23… ...OK Error creating Live CD : Failed to build transaction : udisks-1.0.1-8.fc15.i686 requires ntfsprogs Cheers, Valent. -- follow me - www.twitter.com/valentt & http://kernelreloaded.blog385.com linux, anime, spirituality, wireless, scuba, linuxmce smart home, zwave ICQ: 2125241, Skype: valent.turkovic, MSN: valent.turkovic(a)hotmail.com

2 1

[PATCH 2/5] Beta version of editliveos.py (edit-livecd)
by Frederick Grose 15 Apr '11

15 Apr '11

The full script is available at https://bugzilla.redhat.com/show_bug.cgi?id=448030#c41 [Patch 2/5] Author: Frederick Grose <fgrose(a)sugarlabs.org> Date: Thu Apr 14 09:48:41 2011 -0400 Support refreshing of attached LiveOS devices. Provide a new feature to refresh an attached LiveOS source image with a renewed rootfs and freshed overlay and, optionally, skip building a new .iso file. Updated base_on() to support selinux contexts, provide a liveos_mount() function, new functions to check disc space for building and refreshing (with an extra space option to test or adjust staging space). Provide an option to bypass the break to shell or script to permit uninterrupted processing. Include the kernel architecture of the source in the .iso image name, support F15-style rd.luks=0 in the boot configuration adjustment, and provide updated usage statements. diff --git a/tools/edit-livecd b/tools/edit-livecd index 659cfae..e156895 100755 --- a/tools/edit-livecd +++ b/tools/edit-livecd @@ -69,8 +69,24 @@ class LiveImageEditor(LiveImageCreator): self.clone = False """Signals when to copy a running LiveOS image as base.""" - self._include = None - """A string of file or directory paths to include in __copy_img_root.""" + self.refresh_only = False + """Signals that a filesystem refresh only is requested. No iso9660 + installation image will be produced. If the source has never been + configured, this value is set to None.""" + + self._overlay = None + """Signals the existence of an filesystem overlay file in base_on.""" + + self._include = [] + """A string of file or directory paths in the outer device filesystem + to include in __copy_src_root().""" + + self._exclude = [] + """A string of file or directory paths in the outer device filesystem + to exclude from __copy_src_root().""" + + self._releasefile = [] + """A string of file or directory paths to include in _brand().""" self._builder = os.getlogin() """The name of the Remix builder for _branding. @@ -82,10 +98,10 @@ class LiveImageEditor(LiveImageCreator): current compression or lack thereof. Compression type options vary with the version of the kernel and SquashFS used.""" - self.skip_compression = False + self.skip_compression = None """Controls whether to use squashfs to compress the image.""" - self.skip_minimize = False + self.skip_minimize = None """Controls whether an image minimizing snapshot should be created.""" self._isofstype = "iso9660" @@ -110,6 +126,15 @@ class LiveImageEditor(LiveImageCreator): self._LiveImageCreatorBase__isodir = None """directory where the iso is staged""" + self.squashmnt = None + """Mount object for the source LiveOS SquashFS.""" + + self.liveosmnt = None + """Mount object for the source LiveOS image.""" + + self.extra_loops = [] + """List of extra loop devices created for block device sources.""" + # properties def __get_image(self): if self._LoopImageCreator__imagedir is None: @@ -166,6 +191,7 @@ class LiveImageEditor(LiveImageCreator): finally: os.unlink(path) + def mount(self, base_on, cachedir = None): """mount existing file system. @@ -176,7 +202,9 @@ class LiveImageEditor(LiveImageCreator): We also need to get some info about the image before we can mount it. base_on -- the <LIVEIMG.src> a LiveOS.iso file or an attached LiveOS - device, such as, /dev/live for a currently running image. + device, such as, /dev/live for a currently running image, + or <base_on>/mnt/live for a live-mounted (overlay-mounted) + LiveOS device. cachedir -- a directory in which to store a Yum cache; Not used in edit-liveos. @@ -196,13 +224,56 @@ class LiveImageEditor(LiveImageCreator): makedirs(self._LoopImageCreator__imagedir) makedirs(self._ImageCreator_outdir) - if self.clone: + self._LiveImageCreatorBase__isodir = os.path.join( + self._ImageCreator__builddir, 'iso') + + if self._src_type == 'blk': + srcmnt = DiskMount(RawDisk(None, base_on), self._mkdtemp('blk-')) + elif self._src_type == 'iso': + srcmnt = DiskMount(LoopbackDisk(base_on, None), + self._mkdtemp('iso-')) + elif self._src_type == 'live': + srcmnt = Mount(os.path.join('/mnt', 'live')) + + self.srcmnt = srcmnt + + try: + srcmnt.mount() + except MountError, e: + raise CreatorError("Failed to mount '%s' : %s" % (base_on, e)) + else: + srcmntdir = srcmnt.mountdir + liveosdir = os.path.join(srcmntdir, 'LiveOS') + if not os.path.exists(liveosdir): + raise CreatorError("No LiveOS directory on %s" % base_on) + + files = os.listdir(liveosdir) + for f in files: + if f.find('overlay-') == 0: + self._overlay = f + break + if os.path.exists(os.path.join(liveosdir, 'ext3fs.img')): + self.squashmnt = False + + if self._space_to_proceed(srcmntdir, base_on): + self.__copy_src_root(srcmntdir, base_on) + else: + raise CreatorError('''Insufficient space to stage a new build. + Exiting...''') + finally: + srcmnt.cleanup() + + if self.clone and self._overlay: # Need to clone base_on into ext3fs.img at this point self._LoopImageCreator__fslabel = self.name self._base_on(base_on) else: + self.liveosmnt.cleanup() + if self.squashmnt.mounted: + self.squashmnt.cleanup() LiveImageCreator._base_on(self, base_on) - self._LoopImageCreator__fstype = get_fsvalue(self._image, 'TYPE') + self._LoopImageCreator__fstype = get_fsvalue('value', 'TYPE', None, + self._image) self._get_fslabel() self.fslabel = self._LoopImageCreator__fslabel @@ -210,140 +281,377 @@ class LiveImageEditor(LiveImageCreator): self._LoopImageCreator__instloop = ExtDiskMount( ExistingSparseLoopbackDisk(self._image, - self._LoopImageCreator__image_size), + self._LoopImageCreator__image_size), self._ImageCreator_instroot, self._fstype, self._LoopImageCreator__blocksize, self.fslabel, - self.tmpdir) + tmpdir=self.tmpdir) try: self._LoopImageCreator__instloop.mount() except MountError, e: raise CreatorError("Failed to loopback mount '%s' : %s" % (self._image, e)) - cachesrc = cachedir or (self._ImageCreator__builddir + "/yum-cache") - makedirs(cachesrc) + if self._script or self._shell: + if self._src_type == 'iso': + cachesrc = cachedir or (self._ImageCreator__builddir + "/yum-cache") + makedirs(cachesrc) - for (f, dest) in [("/sys", None), ("/proc", None), - ("/dev/pts", None), ("/dev/shm", None), - (cachesrc, "/var/cache/yum")]: - self._ImageCreator__bindmounts.append(BindChrootMount(f, self._instroot, dest)) + for (f, dest) in [("/sys", None), ("/proc", None), + ("/dev/pts", None), ("/dev/shm", None), + (cachesrc, "/var/cache/yum")]: + self._ImageCreator__bindmounts.append(BindChrootMount( + f, self._instroot, dest)) - self._do_bindmounts() + self._do_bindmounts() - os.symlink("../proc/mounts", self._instroot + "/etc/mtab") + os.symlink("/proc/self/mounts", self._instroot + "/etc/mtab") - self.__copy_img_root(base_on) - self._brand(self._builder) - def _base_on(self, base_on): - """Clone the running LiveOS image as the basis for the new image.""" + def liveos_mount(self, srcmntdir, base_on, ops=None): + """Mount the LiveOS image in the iso, or installed device.""" - self.__fstype = 'ext4' - self.__image_size = 4096L * 1024 * 1024 - self.__blocksize = 4096 + if self.liveosmnt and self.liveosmnt.mounted: + return self.liveosmnt + + liveosdir = os.path.join(srcmntdir, 'LiveOS') + squash_img = os.path.join(liveosdir, 'squashfs.img') + rootfs_img = os.path.join(liveosdir, 'ext3fs.img') + + if self._src_type == 'live': + return + + self.extra_loops.extend([add_loop(8)]) + self.extra_loops.extend([add_loop(9)]) + self.extra_loops.extend([add_loop(10)]) + if self._src_type == 'blk' and self._overlay: + if not os.path.exists(rootfs_img): + self.liveosmnt = LiveImageMount(RawDisk(None, base_on), + self._mkdtemp('live-'), self._overlay, + self._ImageCreator__builddir) + elif os.path.exists(squash_img): + self.squashmnt = DiskMount(LoopbackDisk(squash_img, None), + self._mkdtemp('squash-'), ops=ops) + rootfs_img = os.path.join(self.squashmnt.mountdir, + 'LiveOS', 'ext3fs.img') + try: + self.squashmnt.mount() + except MountError, e: + raise CreatorError("Failed to mount '%s' : %s" % + (squash_img, e)) + + if os.path.exists(rootfs_img): + self.liveosmnt = DiskMount(LoopbackDisk(rootfs_img, None), + self._mkdtemp('ext3-'), ops=ops) + elif not self.liveosmnt: + raise CreatorError("No LiveOS image found on '%s', Exiting..." % + base_on) - self.__instloop = ExtDiskMount(SparseLoopbackDisk(self._image, - self.__image_size), - self._instroot, - self.__fstype, - self.__blocksize, - self.fslabel, - self.tmpdir) try: - self.__instloop.mount() + self.liveosmnt.mount() except MountError, e: - raise CreatorError("Failed to loopback mount '%s' : %s" % - (self._image, e)) + raise CreatorError("Failed to mount '%s' : %s" % + (rootfs_img, e)) + else: + if self.squashmnt is None: + self.squashmnt = self.liveosmnt.squashmnt - subprocess.call(['rsync', '-ptgorlHASx', '--specials', '--progress', - '--include', '/*/', - '--exclude', '/etc/mtab', - '--exclude', '/etc/blkid/*', - '--exclude', '/dev/*', - '--exclude', '/proc/*', - '--exclude', '/home/*', - '--exclude', '/media/*', - '--exclude', '/mnt/live', - '--exclude', '/sys/*', - '--exclude', '/tmp/*', - '--exclude', '/.liveimg*', - '--exclude', '/.autofsck', - '/', self._instroot]) - subprocess.call(['sync']) + return self.liveosmnt - self._ImageCreator__create_minimal_dev() - self.__instloop.cleanup() + def _space_to_proceed(self, srcmntdir, base_on): + """Provide estimated size requirements and availability of staging + resources for the build. + """ - def __copy_img_root(self, base_on): - """helper function to copy root content of the base LiveIMG to - ISOdir""" + # Determine compression and minimization states + liveosdir = os.path.join(srcmntdir, 'LiveOS') + squash_img = os.path.join(liveosdir, 'squashfs.img') + img = squash_img + if not os.path.exists(squash_img): + img = os.path.join(liveosdir, 'osmin.img') + if not os.path.exists(img): + if self.skip_minimize is None: + self.skip_minimize = True + if self.skip_compression is None: + # default to compressed in all cases + self.skip_compression = False + if self.skip_compression is None: + self.skip_compression = False + # 'self.compress_type = None' will force reading it from + # base_on. + if self.compress_type is None: + self.compress_type = squashfs_compression_type(img) + if self.compress_type == 'undetermined': + # 'gzip' for compatibility with older versions. + self.compress_type = 'gzip' + + du_args = ['/usr/bin/du', '-csxb', '--files0-from=-'] ignore_list = ['ext3fs.img', 'squashfs.img', 'osmin.img', 'home.img', 'overlay-*'] if self.clone: ignore_list.remove('home.img') - includes = 'boot, /EFI, /syslinux, /LiveOS' - if self._include: - includes += ", " + self._include - - imgmnt = DiskMount(RawDisk(0, base_on), self._mkdtemp()) + if os.path.exists(os.path.join(liveosdir, 'home.img')): + du_args.extend(['--exclude=/home/*']) + for fn in ignore_list: + du_args.extend(['--exclude=%s' % fn]) + + # remove duplicate files to reduce iso size + efi_vmlinuz = os.path.join(srcmntdir, 'EFI', 'boot', 'vmlinuz0') + efi_initrd = os.path.join(srcmntdir, 'EFI', 'boot', 'initrd0.img') + if os.path.exists(efi_vmlinuz): + self._exclude.extend(['%s' % efi_vmlinuz]) + self._exclude.extend(['%s' % efi_initrd]) + + if self._src_type == 'live': + stdin = '/' + elif self._src_type in ['blk', 'iso']: + self.liveosmnt = self.liveos_mount(srcmntdir, base_on, 'ro') + stdin = self.liveosmnt.mountdir + rootfs_size = long(rcall(du_args, stdin, stderr=False).split()[-2]) + # staged copy + required = rootfs_size + + if self.skip_compression: + # Space on iso image + required += rootfs_size else: - imgmnt = DiskMount(LoopbackDisk(base_on, 0), self._mkdtemp()) + # assuming a 2:1 compression ratio, at most + required += rootfs_size / 2L + + if not self.skip_minimize: + required += 8192L + + tmp_available = long(disc_free_info(self.tmpdir, '-TB1')[4]) + e2img_size = 1024L * 1024L * 512L + if tmp_available > e2img_size: + e2img = tempfile.NamedTemporaryFile(dir=self.tmpdir) + if self._src_type == 'live': + device = '/dev/mapper/live-rw' + elif self._src_type in ['blk', 'iso']: + device = disc_free_info(self.liveosmnt.mountdir)[0] + rcall(['/sbin/e2image', '-r', device, e2img.name]) + e2img_size = tmp_available - long(disc_free_info( + self.tmpdir, '-TB1')[4]) + if os.stat(self.tmpdir).st_dev == os.stat(self.output).st_dev: + available = tmp_available + required += e2img_size + else: + available = long(disc_free_info(self.output, '-TB1')[4]) - self._LiveImageCreatorBase__isodir = self._ImageCreator__builddir + "/iso" + for fn in self._exclude: + du_args.extend(['--exclude=%s' % os.path.join(srcmntdir, + fn.lstrip(os.sep))]) - try: - imgmnt.mount() - except MountError, e: - raise CreatorError("Failed to mount '%s' : %s" % (base_on, e)) + if self.clone: + includes = ['boot', 'EFI', 'isolinux', 'syslinux', 'LiveOS'] + files0from = '' + for fn in includes: + self._include.extend(['%s' % fn]) + for i, fn in enumerate(self._include): + src = os.path.join(srcmntdir, fn.lstrip(os.sep)) + files0from += '%s\0' % src + self._include[i] = src else: + files0from = srcmntdir + + tobe_copied = long( + rcall(du_args, files0from, stderr=False).split()[-2]) + # 1 staged copy and another in iso9660 file. + required += tobe_copied * 2L + # amount for uncertain overhead + required += self._extra_space + + print '\n{:20,} bytes set to be copied'.format(tobe_copied), + print 'from the source filesystem {:s}.'.format(base_on) + print '{:20,} bytes are in the LiveOS filesystem.\n'.format( + rootfs_size) + print '{:20,} bytes are estimated to be required'.format(required), + print 'for staging this build.\n' + print '{:20,} bytes are available'.format(available), + print 'on {:s} for staging the build.'.format(self.output) + if available <= required: + needed = (required - available) / (1024 * 1024) + print >> sys.stderr, "\n" + "There is insufficient space to build a remix on %s.\n" + "%s MiB of additional space is needed." % (self.output, needed) + liveosmnt.cleanup() + return False + return True + + + def __copy_src_root(self, srcmntdir, base_on): + """helper function to copy root content of the base LiveOS source to + ISOdir + + """ + ignore_list = ['ext3fs.img', 'squashfs.img', 'osmin.img', 'home.img', + 'overlay-*'] + + if self.clone: + ignore_list.remove('home.img') + liveosdir = os.path.join(srcmntdir, 'LiveOS') + + for fn in self._exclude: + ignore_list.extend(['%s' % fn]) + + isodir = self._LiveImageCreatorBase__isodir + + if self.clone: # include specified files or directories - if self.clone: - baseimg = os.path.join(imgmnt.mountdir, 'LiveOS', - 'squashfs.img') - # 'self.compress_type = None' will force reading it from - # base_on. - if self.compress_type is None: - self.compress_type = squashfs_compression_type(baseimg) - if self.compress_type == 'undetermined': - # 'gzip' for compatibility with older versions. - self.compress_type = 'gzip' - - dst = self._LiveImageCreatorBase__isodir - print includes - for fd in includes.split(', /'): - src = os.path.join(imgmnt.mountdir, fd) - if os.path.isfile(src): - shutil.copy2(src, os.path.join(dst, fd)) - elif os.path.isdir(src): - shutil.copytree(src, os.path.join(dst, fd), + tgt_list = [] + for fp in self._include: + if os.path.exists(fp): + tgt_list.extend([os.path.basename(fp)]) + if os.path.isfile(fp): + shutil.copy2(fp, os.path.join(isodir, tgt_list[-1])) + elif os.path.isdir(fp): + shutil.copytree(fp, os.path.join(isodir, tgt_list[-1]), symlinks=True, ignore=shutil.ignore_patterns( *ignore_list)) + print """ + \rDevice folders and files to be included in the new image: + \r %s""" % tgt_list + else: + #copy over everything but squashfs.img or ext3fs.img + print """ + \rDevice folders & files to be excluded from the source root copy: + \r %s""" % ignore_list + shutil.copytree(srcmntdir, isodir, + ignore=shutil.ignore_patterns(*ignore_list)) + + bootfolder = os.path.join(isodir, 'isolinux') + installedpath = os.path.join(isodir, 'syslinux') + if os.path.exists(installedpath): + os.rename(installedpath, bootfolder) + + # build symlinks to reduce iso size + efi_vmlinuz = os.path.join(isodir, 'EFI', 'boot', 'vmlinuz0') + isolinux_vmlinuz = os.path.join(bootfolder, 'vmlinuz0') + efi_initrd = os.path.join(isodir, 'EFI', 'boot', 'initrd0.img') + isolinux_initrd = os.path.join(bootfolder, 'initrd0.img') + + if os.path.exists(os.path.join(srcmntdir, 'EFI', 'boot', 'vmlinuz0')): + os.unlink(efi_vmlinuz) + os.unlink(efi_initrd) + os.symlink(isolinux_vmlinuz, efi_vmlinuz) + os.symlink(isolinux_initrd, efi_initrd) + + subprocess.call(['sync']) + + + def _base_on(self, base_on): + """Clone the running or /dev/mapper LiveOS image as the basis for the + new image. + + """ + + if self._src_type == 'blk': + base_on = self.liveosmnt.mountdir + os.sep + if self.squashmnt: + image = self.liveosmnt.imgloop else: - #copy over everything but squashfs.img or ext3fs.img - shutil.copytree(imgmnt.mountdir, - self._LiveImageCreatorBase__isodir, - ignore=shutil.ignore_patterns(*ignore_list)) - subprocess.call(['sync']) + image = self.liveosmnt.disk + try: + self.srcmnt.mount() + except MountError, e: + raise CreatorError("Failed to mount '%s' : %s" % + (self.srcmnt.device, e)) + self.__image_size = os.stat(image.lofile).st_size + device = image.device + elif self._src_type == 'live': + base_on = '/' + device = '/dev/mapper/live-rw' + liveosdir = os.path.join('/mnt', 'live', 'LiveOS') + rootfs_img = os.path.join(liveosdir, 'ext3fs.img') + squashmnt = None + if not os.path.exists(rootfs_img): + squashloop = get_loop('/sysroot/LiveOS/squashfs.img') + squashmnt = DiskMount(Disk(None, squashloop), + self._mkdtemp('squash-')) + try: + squashmnt.mount() + except MountError, e: + raise CreatorError("Failed to mount '%s' : %s" % + (squashloop, e)) + rootfs_img = os.path.join(squashmnt.mountdir, + 'LiveOS', 'ext3fs.img') + self.__image_size = os.stat(rootfs_img).st_size + if squashmnt: + squashmnt.unmount() + self.__fstype = get_fsvalue('value', 'TYPE', None, device) + self.__blocksize = os.stat(device).st_blksize + if self.squashmnt is not None: + self.srcmnt.unmount() + + self.__instloop = ExtDiskMount(SparseLoopbackDisk(self._image, + self.__image_size), + self._instroot, + self.__fstype, + self.__blocksize, + self.fslabel, + tmpdir=self.tmpdir) + try: + self.__instloop.mount() + except MountError, e: + raise CreatorError("Failed to loop mount '%s' : %s" % + (self._image, e)) + else: + livecfg = 'in' + dbusid = 'ex' + if self.refresh_only: + dbusid = 'in' + if not os.path.exists(os.path.join(base_on, + '.liveimg-configured')): + self.refresh_only = None + + home = 'in' + if os.path.exists(os.path.join(self._LiveImageCreatorBase__isodir, + 'LiveOS', 'home.img')): + home = 'ex' + print "\nCopying the source image..." + subprocess.call(['rsync', '-ptgorlHASxXW', '--specials', '--stats', + '--partial-dir=.rsync-partial', + '--exclude', '- /selinux/', + '--include', '/*/', + '--exclude', '/dev/*', + '--exclude', '/etc/blkid/*', + '--exclude', '/etc/mtab', + '--%sclude' % home, '/home/*', + '--exclude', '/proc/*', + '--exclude', '/mnt/live', + '--exclude', '/sys/*', + '--exclude', '/tmp/*', + '--%sclude' % dbusid, '/var/lib/dbus/machine-id', + '--%sclude' % livecfg, '/.liveimg*', + '--exclude', '/.autofsck', + '%s' % base_on, self._instroot]) + makedirs(os.path.join(self._instroot, 'selinux')) + subprocess.call(['/bin/sync']) + + if self._src_type == 'blk': + self.liveosmnt.cleanup() + finally: - imgmnt.cleanup() + self.__instloop.cleanup() - def _brand (self, _builder): + def _brand(self, _builder): """Adjust the image branding to show its variation from original - source by builder and build date.""" + source by builder and build date. + + """ self.fslabel = self.name dt = time.strftime('%d-%b-%Y') - lst = ['isolinux/isolinux.cfg', 'syslinux/syslinux.cfg', - 'syslinux/extlinux.conf'] + lst = ['isolinux/isolinux.cfg', 'isolinux/syslinux.cfg', + 'isolinux/extlinux.conf'] for f in lst: fpath = os.path.join(self._LiveImageCreatorBase__isodir, f) if os.path.exists(fpath): @@ -365,13 +673,17 @@ class LiveImageEditor(LiveImageCreator): ntext = dt.translate(None, '-') + '-' + _builder + '-Remix-' + release # Update fedora-release message with Remix details. - releasefiles = '/etc/fedora-release, /etc/generic-release' + releasefiles = ['fedora-release', 'generic-release', 'issue'] + for i, rf in enumerate(releasefiles): + releasefiles[i] = os.path.join(self._instroot, 'etc', rf) if self._releasefile: - releasefiles += ', ' + self._releasefile - for fn in releasefiles.split(', '): - if os.path.exists(fn): + for fn in self._releasefile: + releasefiles.extend([os.path.join(self._instroot, + fn.lstrip(os.sep))]) + for rfpath in releasefiles: + if os.path.exists(rfpath): try: - with open(self._instroot + fn, 'r') as f: + with open(rfpath, 'r') as f: text = ntext + '\n' + f.read() open(f.name, 'w').write(text) except IOError, e: @@ -379,16 +691,16 @@ class LiveImageEditor(LiveImageCreator): (f.name, e)) self._releasefile = ntext - self.name += '-' + os.uname()[4] + '-' + time.strftime('%Y%m%d.%H%M') + kernel = os.path.join(self._LiveImageCreatorBase__isodir, 'isolinux', + 'vmlinuz0') + arch = get_file_info(kernel)[7].split('.')[-1] + self.name += '-' + arch + '-' + time.strftime('%Y%m%d.%H%M') def _configure_bootloader(self, isodir): """Restore the boot configuration files for an iso image boot.""" bootfolder = os.path.join(isodir, 'isolinux') - oldpath = os.path.join(isodir, 'syslinux') - if os.path.exists(oldpath): - os.rename(oldpath, bootfolder) cfgf = os.path.join(bootfolder, 'isolinux.cfg') for f in ['syslinux.cfg', 'extlinux.conf']: @@ -400,7 +712,9 @@ class LiveImageEditor(LiveImageCreator): '-e', 's/Welcome to .*/Welcome to ' + self._releasefile + '!/', '-e', 's/root=[^ ]*/root=live:CDLABEL=' + self.name + '/', '-e', 's/rootfstype=[^ ]* [^ ]*/rootfstype=auto ro/', - '-e', 's/liveimg .* quiet/liveimg quiet/', cfgf] + '-e', 's/liveimg .* rd_NO_LUKS/liveimg quiet rhgb rd_NO_LUKS/', + '-e', 's/liveimg .* rd.luks=0/liveimg quiet rhgb rd.luks=0/', + cfgf] dev_null = os.open("/dev/null", os.O_WRONLY) try: @@ -408,29 +722,209 @@ class LiveImageEditor(LiveImageCreator): stdout = subprocess.PIPE, stderr = dev_null).communicate()[0] return 0 - except IOError, e: raise CreatorError("Failed to configure bootloader file: %s" % e) return 1 finally: os.close(dev_null) + + def no_new_iso(self, iso): + """Hijacked the _LiveImageCreatorBase__create_iso base method to + prevent creating a new iso when the refresh_only option is selected. + + """ + + self.refresh(iso) + + def refresh(self, iso): + """Hijacked the _LiveImageCreatorBase__implant_md5sum base method to + insert code to refresh a running or livemounted image's ext3fs.img and + overlay files (just before the staged files are deleted). + + """ + + if not self.refresh_only: + # Implant an isomd5sum. + if os.path.exists("/usr/bin/implantisomd5"): + implantisomd5 = "/usr/bin/implantisomd5" + elif os.path.exists("/usr/lib/anaconda-runtime/implantisomd5"): + implantisomd5 = "/usr/lib/anaconda-runtime/implantisomd5" + else: + logging.warn("isomd5sum not installed; not setting up mediacheck") + return + subprocess.call([implantisomd5, iso]) + + if self._src_type == 'live': + print """ + \r Notice: Refreshing from running images is not yet supported. + """ + return + elif self._src_type == 'iso': + return + + print "\nRefreshing the source device with the new image..." + isodir = self._LiveImageCreatorBase__isodir + + try: + self.srcmnt.mount() + except MountError, e: + raise CreatorError("Failed to mount '%s' : '%s'" % + (self.srcmnt.mountdir, e)) + else: + if self._space_for_refresh(self.srcmnt, isodir): + liveosdir = os.path.join(self.srcmnt.mountdir, 'LiveOS') + if self._overlay: + self._overlay = os.path.join(liveosdir, self._overlay) + dd_args = ['/bin/dd', 'if=/dev/zero', + 'of=%s' % self._overlay, 'bs=64k', 'count=1', + 'conv=notrunc'] + + staged_LiveOS = os.path.join(isodir, 'LiveOS') + + def _proclists(fname, subt=None): + if fname: + fpath = os.path.join(liveosdir, fname) + if os.path.exists(fpath): + dellist.extend([fpath]) + if subt: + filelist.extend([os.path.join(staged_LiveOS, subt)]) + target = os.path.join(liveosdir, subt) + tgtlist.extend([target]) + dellist.extend([target]) + + filelist, tgtlist, dellist = ([], [], []) + if self.skip_minimize: + _proclists('osmin.img') + else: + _proclists(None, 'osmin.img') + if self.skip_compression: + _proclists('squashfs.img', 'ext3fs.img') + elif self.squashmnt: + _proclists('ext3fs.img', 'squashfs.img') + else: + filelist.extend([os.path.join(os.path.dirname(self._image), + 'LiveOS', 'ext3fs.img')]) + target = os.path.join(liveosdir, 'ext3fs.img') + tgtlist.extend([target]) + dellist.extend([target]) + + for f in ['syslinux.cfg', 'extlinux.conf']: + cfgf = os.path.join(self.srcmnt.mountdir, 'syslinux', f) + if os.path.exists(cfgf): + cfgf_sed = ['/bin/sed', '-i', '-e', + 's/Welcome to .*/Welcome to ' + self._releasefile + '!/', + cfgf] + + environ = os.environ.copy() + cmd_cp = '/bin/cp' + cmd_rm = '/bin/rm' + + # Copy new ext3fs.img or squashfs.img and osmin.img, and reset + # the overlay. + for f in dellist: + rcall([cmd_rm, f], env=environ) + for i, f in enumerate(filelist): + rcall([cmd_cp, '--remove-destination', f, tgtlist[i]], + env=environ) + if self._overlay: + print "Resetting the overlay file..." + rcall(dd_args, env=environ) + rcall(cfgf_sed, env=environ) + + # flag refresh image as configured. (This has already been done + # for the refresh_only case.) + if self.refresh_only is not None: + self.liveosmnt.ops = 'rw' + try: + self.liveosmnt.mount() + except MountError, e: + raise CreatorError("Failed to mount '%s' : %s" % + (self.liveosmnt.mountdir, e)) + else: + shutil.copy('/.liveimg-configured', + self.liveosmnt.mountdir) + shutil.copy('/.liveimg-late-configured', + self.liveosmnt.mountdir) + finally: + self.liveosmnt.cleanup() + + finally: + self.srcmnt.unmount() + + + def _space_for_refresh(self, srcmnt, isodir): + """Check for sufficient space on the installation device for a refresh. + """ + + du_args = ['/usr/bin/du', '-csxb', '--files0-from=-'] + + include = ['squashfs.img', 'osmin.img', 'ext3fs.img', + 'livecd-iso-to-disk'] + include2 = include[:] + files0from = '' + for i, fn in enumerate(include): + include[i] = os.path.join(isodir, 'LiveOS', fn) + files0from += '%s\0' % include[i] + if not self.squashmnt: + files0from += '%s\0' % os.path.join(os.path.dirname(self._image), + 'LiveOS', 'ext3fs.img') + tobe_copied = long( + rcall(du_args, files0from, stderr=False).split()[-2]) + files0from = '' + for i, fn in enumerate(include2): + include2[i] = os.path.join(srcmnt.mountdir, 'LiveOS', fn) + files0from += '%s\0' % include2[i] + tobe_deleted = long( + rcall(du_args, files0from, stderr=False).split()[-2]) + available = long(disc_free_info(srcmnt.mountdir, + '-TB1')[4]) + tobe_deleted + files0from = os.path.join(srcmnt.mountdir, 'LiveOS', self._overlay) + overlay_size = long( + rcall(du_args, files0from, stderr=False).split()[-2]) + files0from = os.path.join(srcmnt.mountdir, 'LiveOS', 'home.img') + home_size = long(rcall(du_args, files0from, stderr=False).split()[-2]) + reduced_overlay_size = available + overlay_size + print '\n{:20,} bytes to be copied'.format(tobe_copied) + print '{:20,} bytes to be deleted'.format(tobe_deleted) + print '{:20,} bytes in home.img'.format(home_size) + print '{:20,} bytes in overlay'.format(overlay_size) + print '{:20,} bytes available'.format(available) + if available <= 0: + print '{:20,} bytes in potential overlay'.format( + reduced_overlay_size) + return False + return True + + + def _unmount_instroot(self): + """Undo anything performed in mount().""" + + if self.liveosmnt: + self.liveosmnt.cleanup() + if self.squashmnt: + self.squashmnt.cleanup() + def parse_options(args): - parser = optparse.OptionParser(usage = """ - %prog [-n=<name>] - [-o=<output>] - [-s=<script.sh>] - [-t=<tmpdir>] - [-e=<excludes>] - [-f=<exclude-file>] - [-i=<includes>] - [-r=<releasefile>] - [-b=<builder>] + parser = optparse.OptionParser(usage = ''' + %prog [-n, --name <name>] + [-o, --output <output directory>] + [-s, --script <script.sh>] + [-N, --noshell] + [-t, --tmpdir <tmpdir>] + [-e, --exclude <exclude, s>] + [-i, --include <include, s>] + [-r, --releasefile <releasefile, s>] + [-b, --builder <builder>] [--clone] - [-c=<compress_type>] + [--refresh_only] + [--no-refresh] + [-c, --compress-type <compression type>] + [--compress] [--skip-compression] [--skip-minimize] - <LIVEIMG.src>""") + [--extra-space] + <LIVEIMG.src>''') parser.add_option("-n", "--name", type="string", dest="name", help="name of new LiveOS (don't include .iso, it will " @@ -443,46 +937,74 @@ def parse_options(args): help="specify script to run chrooted in the LiveOS " "fsimage") + parser.add_option("-N", "--noshell", action="store_false", + dest="shell", default=True, + help="Specify no break to shell after edit.") + parser.add_option("-t", "--tmpdir", type="string", dest="tmpdir", default="/var/tmp", help="Temporary directory to use (default: /var/tmp)") parser.add_option("-e", "--exclude", type="string", dest="exclude", - help="Specify directory or file patterns to be excluded " - "from the rsync copy of the filesystem.") - - parser.add_option("-f", "--exclude-file", type="string", - dest="exclude_file", - help="Specify a file containing file patterns to be " - "excluded from the rsync copy of the filesystem.") + help='''A string of file or directory paths in the outer device + filesystem to exclude from __copy_src_root(). Include multiple files + as "file1, file2, ..." ''') parser.add_option("-i", "--include", type="string", dest="include", - help="Specify directory or file patterns to be included " - "in copy_img_root.") + help='''A string of file or directory paths in the outer device + filesystem to include in __copy_src_root(). Include multiple files + as "file1, file2, ..." ''') parser.add_option("-r", "--releasefile", type="string", dest="releasefile", - help="Specify release file/s for branding.") + help='''Specify release file/s for branding. Include + multiple files as "file1, file2, ..." ''') parser.add_option("-b", "--builder", type="string", dest="builder", default=os.getlogin(), help="Specify the builder of a Remix.") - parser.add_option("", "--clone", action="store_true", dest="clone", - help="Specify that source image is LiveOS block device.") + parser.add_option("", "--clone", action="store_true", + dest="clone", default=False, + help="""Specify copying of the LiveOS root filesystem. + This option will also refresh the source filesystem, + unless the --no-refresh option is also invoked.""") + + parser.add_option("", "--refresh-only", action="store_true", + dest="refresh_only", default=False, + help='''Specify replacing the squashfs.img or ext3fs.img + of the source LiveOS installation instance with such + files from the new build, and resetting any overlay. + No new iso installation file will be produced.''') + + parser.add_option("", "--no-refresh", action="store_true", + dest="no_refresh", default=False, + help='''Specify no refresh of the source filesystems.''') - parser.add_option("-c", "--compress_type", type="string", + parser.add_option("-c", "--compress-type", type="string", dest="compress_type", help="Specify the compression type for SquashFS. Will " "override the current compression or lack thereof.") + parser.add_option("", "--compress", action="store_true", + dest="compress", default=None, + help="Specify compression of the filesystem image, " + "ext3fs.img. Used when overriding the base_on " + "compression status.") + parser.add_option("", "--skip-compression", action="store_true", - dest="skip_compression", default=False, - help="Specify no compression of filesystem, ext3fs.img") + dest="skip_compression", default=None, + help="Specify no compression of filesystem image, " + "ext3fs.img") parser.add_option("", "--skip-minimize", action="store_true", - dest="skip_minimize", default=False, + dest="skip_minimize", default=None, help="Specify no osmin.img minimal snapshot.") + parser.add_option("", "--extra-space", type="string", + dest="extra_space", default='2', + help='''Specify extra space to reserve for unexpected + staging area needs.''') + setup_logging(parser) (options, args) = parser.parse_args() @@ -491,36 +1013,105 @@ def parse_options(args): parser.print_usage() sys.exit(1) - print args[0] + print "\nSource image at %s" % args[0] return (args[0], options) -def get_fsvalue(filesystem, tag): + +def get_fsvalue(format=None, tag=None, token=None, filesystem=None): + """Return filesystem information based on a blkid tag, token, or device.""" + dev_null = os.open('/dev/null', os.O_WRONLY) - args = ['/sbin/blkid', '-s', tag, '-o', 'value', filesystem] + args = ['/sbin/blkid', '-c', '/dev/null'] + if format: + args.extend(['-o', format]) + if tag: + args.extend(['-s', tag]) + if token: + args.extend(['-l', '-t', token]) + if filesystem: + args.extend([filesystem]) try: fs_type = subprocess.Popen(args, stdout=subprocess.PIPE, - stderr=dev_null).communicate()[0] + stderr=dev_null).communicate()[0].rstrip() except IOError, e: - raise CreatorError("Failed to determine fs %s: %s" % value, e ) + raise CreatorError("Failed to determine fs %s: %s" % value, e) finally: os.close(dev_null) + return fs_type - return fs_type.rstrip() -def rebuild_iso_symlinks(isodir): - # remove duplicate files and rebuild symlinks to reduce iso size - efi_vmlinuz = "%s/EFI/boot/vmlinuz0" % isodir - isolinux_vmlinuz = "%s/isolinux/vmlinuz0" % isodir - efi_initrd = "%s/EFI/boot/initrd0.img" % isodir - isolinux_initrd = "%s/isolinux/initrd0.img" % isodir +def rcall(args, stdin=None, stderr=True, env=None): + out, err, p, environ = None, None, None, None + if env: + environ = os.environ.copy() + environ.update(env) + try: + p = subprocess.Popen(args, stdin=subprocess.PIPE, env=environ, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + out, err = p.communicate(stdin) + except OSError, e: + raise CreatorError(u"Failed executing:\n'%s'\n'%s'" % (args, e)) + except: + raise CreatorError(u"""Failed to execute:\n'%s' + \renviron: '%s'\nstdout: '%s'\nstderr: '%s'""" % + (args, environ, out, err)) + else: + if p.returncode != 0 and stderr: + raise CreatorError(u"""Error in call:\n'%s'\nenviron: '%s' + \rstdout: '%s'\nstderr: '%s'\nreturncode: '%s'""" % + (args, environ, out, err, p.returncode)) + return out + + +def disc_free_info(path, options=None): + """Return the disc free information for a file or device path.""" + + info = [None] * 7 + if os.path.exists(path): + st_mode = os.stat(path).st_mode + if stat.S_ISBLK( + st_mode) or os.path.ismount(path) or stat.S_ISDIR(st_mode): + df_args = ['/bin/df'] + if options: + df_args.extend([options]) + df_args.extend([path]) + devinfo = rcall(df_args).splitlines() + info = devinfo[1].split(None) + if len(info) == 1: + info.extend(devinfo[2].split(None)) + return info + + +def get_loop(path): + """Return the loop device for a given mount point path.""" + + loopdevice = None + for loop in rcall(['/sbin/losetup', '-a']).splitlines(): + info = loop.split() + if path == info[2].strip('()'): + loopdevice = info[0].rstrip(':') + break + return loopdevice - if os.path.exists(efi_vmlinuz): - os.remove(efi_vmlinuz) - os.remove(efi_initrd) - os.symlink(isolinux_vmlinuz,efi_vmlinuz) - os.symlink(isolinux_initrd,efi_initrd) + +def add_loop(n=8): + """Add a loop device.""" + + while os.path.exists('/dev/loop%s' % n): + n += 1 + os.mknod('/dev/loop%s' % n, 0660 | stat.S_IFBLK, os.makedev(7, n)) + return '/dev/loop%s' % n + + +def get_file_info(path): + """Retrieve file information.""" + + info = [None] * 16 + if os.path.exists(path): + info = rcall(['/usr/bin/file', '-b', path]).split() + return info def main(): # LiveOS set to <LIVEIMG.src> @@ -530,51 +1121,91 @@ def main(): print >> sys.stderr, "You must run edit-liveos as root" return 1 - if stat.S_ISBLK(os.stat(LiveOS).st_mode): - name = get_fsvalue(LiveOS, 'LABEL') + '.edited' - elif options.name and options.name != os.path.basename(LiveOS): + try: + long(options.extra_space) + except ValueError, e: + raise CreatorError('''Notice, --extra-space: %s is not a number. + \rPlease correct this.''' % options.extra_space, e) + + name = os.path.basename(LiveOS) + ".edited" + src_type = 'iso' + st_mode = os.stat(LiveOS).st_mode + if stat.S_ISBLK(st_mode): + src_type = 'blk' + name = get_fsvalue('value', 'LABEL', None, LiveOS) + '.edited' + if LiveOS == '/dev/live': + src_type = 'live' + if options.name and options.name != os.path.basename(LiveOS): name = options.name - else: - name = os.path.basename(LiveOS) + ".edited" if options.output: output = options.output else: - output = os.path.dirname(LiveOS) - if output == '/dev': + output = os.path.dirname(os.path.normpath(LiveOS)) + if output in ['/dev']: output = options.tmpdir editor = LiveImageEditor(name) - editor._exclude = options.exclude - editor._exclude_file = options.exclude_file - editor._include = options.include + editor._src_type = src_type + if options.exclude: + editor._exclude = options.exclude.split(', ') + if options.include: + editor._include = options.include.split(', ') + editor._script = options.script + editor._shell = options.shell editor.clone = options.clone + if options.refresh_only: + editor.refresh_only = options.refresh_only + editor.clone = True editor.tmpdir = options.tmpdir + editor.output = output editor._builder = options.builder - editor._releasefile = options.releasefile + if options.releasefile: + editor._releasefile = options.releasefile.split(', ') editor.compress_type = options.compress_type editor.skip_compression = options.skip_compression + if options.compress: + editor.skip_compression = False editor.skip_minimize = options.skip_minimize + editor._extra_space = long(options.extra_space) * 1024L * 1024L try: editor.mount(LiveOS, cachedir = None) + editor._brand(editor._builder) editor._configure_bootloader(editor._LiveImageCreatorBase__isodir) if options.script: print "Running edit script '%s'" % options.script editor._run_script(options.script) - else: + elif editor._shell: print "Launching shell. Exit to continue." print "----------------------------------" editor.launch_shell() - rebuild_iso_symlinks(editor._LiveImageCreatorBase__isodir) editor.unmount() + if options.refresh_only: + self.refresh_only = True + editor._LiveImageCreatorBase__create_iso = editor.no_new_iso + if not editor.skip_compression: + print '''\nThe new image will now be resquashed. + Please wait...''' + elif options.clone and not options.no_refresh: + editor._LiveImageCreatorBase__implant_md5sum = editor.refresh + print '\nThe new image will now be packaged. Please wait...' editor.package(output) + call(['/bin/sync']) logging.info("%s.iso saved to %s" % (editor.name, output)) except CreatorError, e: - logging.error(u"Error editing LiveOS : %s" % e) + logging.error(u"Error editing LiveOS : '%s'" % e) + print "\nError editing LiveOS: '%s'" % e return 1 finally: + name = editor.name + for loop in editor.extra_loops: + if loop: + call(['/bin/rm', '-f', loop]) editor.cleanup() + print 'LiveOS edit has completed.' + if not editor.refresh_only: + print ' See %s in %s' % (name, output) return 0

1 1

[PATCH 5/5] Document --copy-home --include options
by Frederick Grose 14 Apr '11

14 Apr '11

[Patch 5/5] Author: Frederick Grose <fgrose(a)sugarlabs.org> Date: Thu Apr 14 11:12:38 2011 -0400 Document --copy-home and --include options. diff --git a/docs/livecd-iso-to-disk.pod b/docs/livecd-iso-to-disk.pod index 7ef0c14..5550fc6 100644 --- a/docs/livecd-iso-to-disk.pod +++ b/docs/livecd-iso-to-disk.pod @@ -6,7 +6,7 @@ livecd-iso-to-disk - installs bootable Live images onto USB/SD storage devices. =head1 SYNOPSIS -B<livecd-iso-to-disk> [--help] [--noverify] [--format] [--reset-mbr] [--efi] [--skipcopy] [--force] [--xo] [--xo-no-home] [--timeout <time>] [--totaltimeout <time>] [--extra-kernel-args <args>] [--multi] [--livedir <dir>] [--compress] [--skipcompress] [--swap-size-mb <size>] [--overlay-size-mb <size>] [--home-size-mb <size>] [--delete-home] [--crypted-home] [--unencrypted-home] <source> <target device> +B<livecd-iso-to-disk> [--help] [--noverify] [--format] [--reset-mbr] [--efi] [--skipcopy] [--force] [--xo] [--xo-no-home] [--timeout <time>] [--totaltimeout <time>] [--extra-kernel-args <args>] [--multi] [--livedir <dir>] [--compress] [--skipcompress] [--swap-size-mb <size>] [--overlay-size-mb <size>] [--home-size-mb <size>] [--delete-home] [--crypted-home] [--unencrypted-home] [--copy-home] [--include <file s>] <source> <target device> Simplest @@ -135,6 +135,20 @@ Specifies the default option to encrypt a new persistent home directory if --hom Prevents the default option to encrypt a new persistent home directory. +=item --copy-home + +This option allows one to copy a persistent home folder from one LiveOS image to the target image. Changes already made in the source home directory will be propagated to the new image. + +=over 4 + +WARNING: User-sensitive information, such as password cookies and user and application data, will be copied to the new image! Scrub this information before using this option. + +=back + +item --include <path s> + +This option allows one to copy one or more file or directory paths from the outer source filesystem to the same path on the target. Multiple paths should be specified in one string, i.e., --include "path1 path2 ..." + =back =head1 CONTRIBUTORS

1 0

[PATCH 4/5] livecd-iso-to-disk --copy-home --include options
by Frederick Grose 14 Apr '11

14 Apr '11

[Patch 4/5] Author: Frederick Grose <fgrose(a)sugarlabs.org> Date: Thu Apr 14 10:53:09 2011 -0400 Support installation of customized builds with --copy-home and --include options. Custom isos may package home.img and extra files or directories. Fix logic on unmounting and removing SRCMNT & TGTMNT. Include uncompressed ext3fs.img in detectsrctype() logic. Use loop mount for SRCMNT on running Live images. Add error messages and disc space estimation adjustments. Tolerate user changes to 'quiet rhgb' in configuration file. Support F15-style rd.luks=0 in configuration file adjustment. diff --git a/tools/livecd-iso-to-disk.sh b/tools/livecd-iso-to-disk.sh index 5261aab..de48ec4 100755 --- a/tools/livecd-iso-to-disk.sh +++ b/tools/livecd-iso-to-disk.sh @@ -33,6 +33,7 @@ shortusage() { [--compress] [--skipcompress] [--swap-size-mb <size>] [--overlay-size-mb <size>] [--home-size-mb <size>] [--delete-home] [--crypted-home] [--unencrypted-home] + [--copy-home] [--include <file s>] <source> <target device> (Enter livecd-iso-to-disk --help on the command line for more information.)" @@ -242,6 +243,20 @@ usage() { --unencrypted-home Prevents the default option to encrypt a new persistent home directory. + --copy-home + This option allows one to copy a persistent home folder from one LiveOS + image to the target image. Changes already made in the source home + directory will be propagated to the new image. + WARNING: User-sensitive information, such as password cookies and + user and application data, will be copied to the new image! Scrub + this information before using this option. + + --include <path s> + This option allows one to copy one or more file or directory paths from + the outer source filesystem to the same path on the target. + Multiple paths should be specified in one string, i.e., + --include "path1 path2 ..." + CONTRIBUTORS livecd-iso-to-disk: David Zeuthen, Jeremy Katz, Douglas McClendon, @@ -272,8 +287,14 @@ usage() { cleanup() { sleep 2 - [ -d "$SRCMNT" ] && umount $SRCMNT && rmdir $SRCMNT - [ -d "$TGTMNT" ] && umount $TGTMNT && rmdir $TGTMNT + if [[ -d $SRCMNT ]]; then + umount $SRCMNT + rmdir $SRCMNT + fi + if [[ -d $TGTMNT ]]; then + umount $TGTMNT + rmdir $TGTMNT + fi } exitclean() { @@ -594,7 +615,8 @@ if [ $(id -u) != 0 ]; then fi detectsrctype() { - if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then + if [[ -e $SRCMNT/LiveOS/squashfs.img || + -e $SRCMNT/LiveOS/ext3fs.img ]]; then srctype=live return fi @@ -652,6 +674,8 @@ shopt -s extglob cryptedhome=1 keephome=1 +copyhome= +copyhomesize=0 homesizemb=0 swapsizemb=0 overlaysizemb=0 @@ -659,6 +683,7 @@ srctype= imgtype= LIVEOS=LiveOS HOMEFILE="home.img" +includes= if [[ "$*" =~ "--help" ]]; then usage @@ -747,6 +772,14 @@ while [ $# -gt 2 ]; do --delete-home) keephome="" ;; + --copy-home) + copyhome=1 + cryptedhome="" + ;; + --include) + includes=($2) + shift + ;; *) echo "invalid arg -- $1" shortusage @@ -795,6 +828,7 @@ fi # do some basic sanity checks. checkMounted $TGTDEV +! [[ /dev/live == $1 || /dev/sr0 == $1 ]] && checkMounted $SRC # Format the device if [ -n "$format" -a -z "$skipcopy" ]; then @@ -850,10 +884,13 @@ fi # FIXME: would be better if we had better mountpoints SRCMNT=$(mktemp -d /media/srctmp.XXXXXX) -if [ -b $SRC ]; then - mount -o ro "$SRC" $SRCMNT || exitclean -elif [ -f $SRC ]; then +if [[ -f $SRC || /dev/live == $1 || /dev/sr0 == $1 ]]; then mount -o loop,ro "$SRC" $SRCMNT || exitclean + imgtype='iso' + [[ /dev/live == $1 ]] && imgtype='live' +elif [[ -b $SRC ]]; then + mount -o ro "$SRC" $SRCMNT || exitclean + imgtype='blk' else echo "$SRC is not a file or block device." exitclean @@ -865,11 +902,53 @@ trap exitclean SIGINT SIGTERM detectsrctype -if [ -f "$TGTMNT/$LIVEOS/$HOMEFILE" -a -n "$keephome" -a "$homesizemb" -gt 0 ]; then - echo "ERROR: Requested keeping existing /home and specified a size for /home" - echo "Please either don't specify a size or specify --delete-home" +if [[ -d $SRCMNT/LiveOS ]]; then + SRCLIVEOSDIR=$SRCMNT/LiveOS +else + SRCLIVEOSDIR=$SRCMNT +fi +if [[ ! -f $SRCMNT/$LIVEOS/$HOMEFILE && -n $copyhome ]]; then + echo -e "ERROR: There appears to be no persistent /home.img on the source + Please check your inputs.\n" + exitclean +fi +if [[ -f $SRCMNT/$LIVEOS/$HOMEFILE && -n $copyhome && -n $cryptedhome ]]; then + echo -e "WARNING: You asked for an --encrypted-home and to copy the source's + /home.img to your target device. The --encrypted-home option is only + available for newly-created /home directories. (If the /home.img on + the source is encrypted, that feature will carry over.)\n + Press Enter to continue, or Ctrl-c to abort.\n" + read +fi +if [[ -f $TGTMNT/$LIVEOS/$HOMEFILE && -n $keephome && $homesizemb -gt 0 ]]; then + echo -e "ERROR: The target has an existing /home and you requested that a + new /home.img be created. + To remove an existing /home on the target, you must explicitly specify + --delete-home. + Please adjust your home options.\n" + exitclean +fi +if [[ -f $TGTMNT/$LIVEOS/$HOMEFILE && -n $keephome && -n $copyhome && \ + -s $SRCMNT/$LIVEOS/$HOMEFILE ]]; then + echo -e "ERROR: The target has an existing /home and you requested that one + from the source be copied to the target device. + To remove an existing /home on the target, you must explicitly specify + --delete-home. + Please adjust your home options.\n" exitclean fi +if [[ -s $SRCMNT/$LIVEOS/$HOMEFILE && -n $copyhome && $homesizemb -gt 0 ]]; then + echo -e "ERROR: You requested a new home AND a copy of one from the source. + Please request only one of these options.\n" + exitclean +fi +if [[ ! -s $SRCLIVEOSDIR/$HOMEFILE && -n $copyhome && $overlaysizemb -gt 0 ]]; then + echo -e "NOTICE: There appears to be no persistent /home.img on this image. + Would you LIKE to continue with just the persistent overlay?\n + Press Enter to continue, or Ctrl-c to abort.\n" + read + copyhome="" +fi if [ -n "$efi" ]; then if [ -d $SRCMNT/EFI/BOOT ]; then @@ -894,11 +973,11 @@ else fi if [[ live == $srctype ]]; then - targets="$TGTMNT/$SYSLINUXPATH" - [[ -n $efi ]] && targets+=" $TGTMNT$EFI_BOOT" - [[ -n $xo ]] && targets+=" $TGTMNT/boot/olpc.fth" - duTable=($(du -c -B 1M $targets 2> /dev/null || :)) - tbd=$((tbd + ${duTable[*]: -2:1})) + targets="$TGTMNT/$SYSLINUXPATH" + [[ -n $efi ]] && targets+=" $TGTMNT$EFI_BOOT" + [[ -n $xo ]] && targets+=" $TGTMNT/boot/olpc.fth" + duTable=($(du -c -B 1M $targets 2> /dev/null || :)) + tbd=$((tbd + ${duTable[*]: -2:1})) fi if [[ -n $skipcompress ]] && [[ -s $SRCMNT/LiveOS/squashfs.img ]]; then @@ -931,22 +1010,52 @@ if [[ live == $srctype ]]; then [[ -n $xo ]] && sources+=" $SRCMNT/boot/olpc.fth" duTable=($(du -c -B 1M "$thisScriptpath" $sources 2> /dev/null || :)) livesize=$((livesize + ${duTable[*]: -2:1})) + [[ -s $SRCLIVEOSDIR/$HOMEFILE && -n $copyhome ]] && \ + copyhomesize=($(du -s -B 1M $SRCLIVEOSDIR/$HOMEFILE)) + if [[ 'iso' == $imgtype ]]; then + [[ -e $SRCMNT/boot ]] && findargs=$SRCMNT/boot + findargs+=" $SRCMNT"' -xdev -mindepth 1 -maxdepth 1 + ( -type d -o -type f ) + ! ( -name isolinux -o -name LiveOS -o -name EFI -o -name olpc.fth ) + -print0' + fi + if [[ -n $includes ]]; then + includes=(${includes[@]/#/$SRCMNT}) + for i in ${!includes[*]}; do + if [[ ! -e ${includes[i]} ]]; then + echo -e "\n${includes[i]#$SRCMNT} does not exist. + Please check your input. Exiting..."; exitclean + fi + done + fi + includes_size=0 + if [[ -n $includes || -n $findargs ]]; then + duTable=($(find ${includes[*]} $findargs | \ + du -cs -B 1M --files0-from=-)) + includes_size=${duTable[*]: -2:1} + fi fi freespace=($(df -B 1M --total $TGTDEV)) freespace=${freespace[*]: -2:1} if [[ live == $srctype ]]; then - tba=$((overlaysizemb + homesizemb + livesize + swapsizemb)) + tba=$((overlaysizemb + homesizemb + copyhomesize + includes_size + \ + livesize + swapsizemb)) if ((tba > freespace + tbd)); then needed=$((tba - freespace - tbd)) - printf "\n The live image + overlay, home, & swap space, if requested, - \r will NOT fit in the space available on the target device.\n - \r + Size of live image: %10s MiB\n" $livesize + printf "\n The live image + overlay, home, includes, & swap space, + \r if requested, will NOT fit in the space available on the + \r target device.\n + \r + Size of live image: %10s MiB\n" $livesize (($overlaysizemb > 0)) && \ printf " + Overlay size: %16s\n" $overlaysizemb (($homesizemb > 0)) && \ printf " + Home directory size: %9s\n" $homesizemb + (($copyhomesize > 0)) && \ + printf " + Copy home directory size: %4s\n" $copyhomesize + (($includes_size > 0)) && \ + printf " + Included files size: %9s\n" $includes_size (($swapsizemb > 0)) && \ printf " + Swap overlay size: %11s\n" $swapsizemb printf " = Total requested space: %6s MiB\n" $tba @@ -1030,6 +1139,18 @@ if [ "$srctype" = "live" -a -z "$skipcopy" ]; then if [ -f $SRCMNT/LiveOS/osmin.img ]; then copyFile $SRCMNT/LiveOS/osmin.img $TGTMNT/$LIVEOS/osmin.img || exitclean fi + if [[ -s $SRCLIVEOSDIR/$HOMEFILE && -n $copyhome ]]; then + copyFile $SRCLIVEOSDIR/$HOMEFILE $TGTMNT/$LIVEOS/$HOMEFILE || exitclean + fi + cp_mode='-ax' + [[ 'vfat' == $TGTFS || 'msdos' == $TGTFS ]] && cp_mode='-dRx' + if [[ 'iso' == $imgtype ]]; then + find $findargs -exec cp $cp_mode -t $TGTMNT {} + 1> /dev/null || + exitclean + fi + if [[ -n $includes ]]; then + cp $cp_mode -t $TGTMNT ${includes[*]} || exitclean + fi sync fi @@ -1112,7 +1233,8 @@ if [[ live == $srctype ]]; then if [[ -d $SRCMNT/syslinux/ ]]; then echo "Preparing boot config file." sed -i -e "s/root=live:[^ ]*/root=live:CDLABEL=name/"\ - -e "s/liveimg .* quiet/liveimg quiet/"\ + -e "s/liveimg .* rd_NO_LUKS/liveimg quiet rhgb rd_NO_LUKS/"\ + -e "s/liveimg .* rd.luks=0/liveimg quiet rhgb rd.luks=0/"\ $BOOTCONFIG $BOOTCONFIG_EFI sed -i -e "s/^timeout.*$/timeout\ 100/"\ -e "/^totaltimeout.*$/d" $BOOTCONFIG

1 0

[PATCH 3/5] Liveimage-mount within running LiveOS instance
by Frederick Grose 14 Apr '11

14 Apr '11

[Patch 3/5] Author: Frederick Grose <fgrose(a)sugarlabs.org> Date: Thu Apr 14 10:37:13 2011 -0400 Support usage within a booted LiveOS instance. Provide functions to add loop devices and remove then on exit, a LiveImageMountError(Exception) class for better error handling, and random device mapper names. diff --git a/tools/liveimage-mount b/tools/liveimage-mount index 80d67d1..1ee5b16 100755 --- a/tools/liveimage-mount +++ b/tools/liveimage-mount @@ -23,10 +23,13 @@ import os import sys import stat +import time import getopt +import random import tempfile import subprocess +extra_loops = [] def usage(ecode): print """Usage: @@ -40,6 +43,18 @@ def usage(ecode): and [ARGS] = [arg1[ arg2[ ...]]]\n""" sys.exit(ecode) +class LiveImageMountError(Exception): + """An exception base class for all liveimage-mount errors.""" + def __init__(self, msg): + Exception.__init__(self, msg) + def __str__(self): + try: + return str(self.message) + except UnicodeEncodeError: + return repr(self.message) + + def __unicode__(self): + return unicode(self.message) def call(*popenargs, **kwargs): ''' @@ -62,43 +77,57 @@ def call(*popenargs, **kwargs): return rc -def rcall(args, env=None): +def rcall(args, stdin=None, stderr=True, env=None): + out, err, p, environ = None, None, None, None if env: environ = os.environ.copy() environ.update(env) - else: - environ = None try: - p = subprocess.Popen(args, stdout=subprocess.PIPE, - stderr=subprocess.PIPE, env=environ) - out, err = p.communicate() + p = subprocess.Popen(args, stdin=subprocess.PIPE, env=environ, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + out, err = p.communicate(stdin) except OSError, e: - raise CreatorError(u"Failed to execute:\n'%s'\n'%s'" % (args, e)) + raise LiveImageMountError(u"Failed executing:\n'%s'\n'%s'" % (args, e)) except: - raise CreatorError(u"""Failed to execute:\n'%s' - \renviron: '%s'\nstdout: '%s'\nstderr: '%s'\nreturncode: '%s'""" % - (args, environ, out, err, p.returncode)) + raise LiveImageMountError(u"""Failed to execute:\n'%s' + \renviron: '%s'\nstdout: '%s'\nstderr: '%s'""" % + (args, environ, out, err)) else: - if p.returncode != 0: - raise CreatorError(u"""Error in call:\n'%s'\nenviron: '%s' - \rstdout: '%s'\nstderr: '%s'\nreturncode: '%s'""" % - (args, environ, out, err, p.returncode)) + if p.returncode != 0 and stderr: + raise LiveImageMountError(u'''Error in call:\n'%s'\nenviron: '%s' + \rstdout: %s\nstderr: %sreturncode: %s''' % + (args, environ, out, err, p.returncode)) return out -def get_device_mountpoint(path): - """Return the device and mountpoint for a file or device path.""" +def get_loop(path): + """Return the loop device for a given mount point path.""" + + loopdevice = None + for loop in rcall(['/sbin/losetup', '-a']).splitlines(): + info = loop.split() + if path == info[2].strip('()'): + loopdevice = info[0].rstrip(':') + break + return loopdevice + + +def add_loop(n=8): + """Add a loop device.""" + + while os.path.exists('/dev/loop%s' % n): + n += 1 + os.mknod('/dev/loop%s' % n, 0660 | stat.S_IFBLK, os.makedev(7, n)) + extra_loops.extend(['/dev/loop%s' % n]) + return '/dev/loop%s' % n - info = list() - info[0:5] = [None] * 6 - if os.path.exists(path): - st_mode = os.stat(path).st_mode - if stat.S_ISBLK(st_mode) or os.path.ismount(path): - devinfo = rcall(['/bin/df', path]).splitlines() - info = devinfo[1].split(None) - if len(info) == 1: - info.extend(devinfo[2].split(None)) - return [info[0], info[5]] + +def loop_setup(path): + """Make and associate a loop device with an image file or device.""" + + loop = add_loop() + call(['/sbin/losetup', loop, path]) + return loop def main(): @@ -145,18 +174,16 @@ def main(): img_type = 'blk' imgloop = None overlayloop = None + liveosmnt = tempfile.mkdtemp(prefix='livemnt-device-') else: img_type = 'iso' - - if img_type is 'blk': - liveosmnt = tempfile.mkdtemp(prefix='livemnt-device-') - if img_type is 'iso': liveosmnt = tempfile.mkdtemp(prefix='livemnt-iso-') liveosdir = os.path.join(liveosmnt, 'LiveOS') homemnt = None dm_cow = None mntlive = None + rmmntdir = None command = args[2:] verbose = not command @@ -164,13 +191,12 @@ def main(): squashmnt = tempfile.mkdtemp(prefix='livemnt-squash-') squashloop = None imgloop = None - losetup_args = ['/sbin/losetup', '-f', '--show'] try: - if img_type is 'blk': + if img_type == 'blk': call(['/bin/mount', liveos, liveosmnt]) - elif img_type is 'iso': - liveosloop = rcall(losetup_args + [liveos]).rstrip() + elif img_type == 'iso': + liveosloop = loop_setup(liveos) call(['/bin/mount', '-o', 'ro', liveosloop, liveosmnt]) squash_img = os.path.join(liveosdir, 'squashfs.img') @@ -182,12 +208,12 @@ def main(): ecode = 1 return else: - squashloop = rcall(losetup_args + [squash_img]).rstrip() + squashloop = loop_setup(squash_img) call(['/bin/mount', '-o', 'ro', squashloop, squashmnt]) ext3_img = os.path.join(squashmnt, 'LiveOS', 'ext3fs.img') - if img_type is 'blk': - imgloop = rcall(losetup_args + [ext3_img]).rstrip() + if img_type == 'blk': + imgloop = loop_setup(ext3_img) imgsize = rcall(['/sbin/blockdev', '--getsz', imgloop]).rstrip() files = os.listdir(liveosdir) overlay = None @@ -195,18 +221,19 @@ def main(): if f.find('overlay-') == 0: overlay = f break - overlayloop = rcall(['/sbin/losetup', '-f']).rstrip() if overlay: - call(['/sbin/losetup', overlayloop, os.path.join(liveosdir, - overlay)]) - dm_cow = 'live-rw' + overlayloop = loop_setup(os.path.join(liveosdir, overlay)) + dm_cow = "live-rw-%d-%d" % (os.getpid(), + random.randint(0, 2**16)) + else: overlay = tempfile.NamedTemporaryFile(dir='/dev/shm') print "\npreparing temporary overlay..." call(['/bin/dd', 'if=/dev/null', 'of=%s' % overlay.name, 'bs=1024', 'count=1', 'seek=%s' % (512 * 1024)]) - call(['/sbin/losetup', overlayloop, overlay.name]) - dm_cow = 'live-ro' + overlayloop = loop_setup(overlay.name) + dm_cow = "live-ro-%d-%d" % (os.getpid(), + random.randint(0, 2**16)) call(['/sbin/dmsetup', '--noudevrules', '--noudevsync', 'create', dm_cow, '--table=0 %s snapshot %s %s p 8' % (imgsize, imgloop, overlayloop)]) @@ -214,14 +241,15 @@ def main(): home_path = os.path.join(liveosdir, 'home.img') if os.path.exists(home_path): homemnt = os.path.join(destmnt, 'home') - homeloop = rcall(losetup_args + [home_path]).rstrip() + homeloop = loop_setup(home_path) call(['/bin/mount', homeloop, homemnt]) mntlive = os.path.join(destmnt, 'mnt', 'live') if not os.path.exists(mntlive): os.makedirs(mntlive) + rmmntdir = True call(['/bin/mount', '--bind', liveosmnt, mntlive]) - elif img_type is 'iso': - imgloop = rcall(losetup_args + [ext3_img]).rstrip() + elif img_type == 'iso': + imgloop = loop_setup(ext3_img) call(['/bin/mount', '-o', 'ro', imgloop, destmnt]) if mount_hacks: @@ -232,7 +260,7 @@ def main(): args = [] args.extend(command) live = '' - if img_type is 'blk': + if img_type == 'blk': live = 'live-' print """Starting process with this command line: \r%s\n %s is %smounted.""" % (' '.join(command), liveos, live) @@ -247,7 +275,7 @@ def main(): print "Starting subshell in chroot, press Ctrl-D to exit..." ecode = subprocess.call(['chroot', destmnt], stdin=sys.stdin, stdout=sys.stdout, stderr=sys.stderr) else: - if dm_cow == 'live-ro': + if img_type == 'blk' and dm_cow[:7] == 'live-ro': status = ' with NO LiveOS persistence,' else: status = '' @@ -268,11 +296,13 @@ def main(): if img_type is 'blk': if mntlive: call(['/bin/umount', mntlive]) - os.rmdir(mntlive) + if rmmntdir: + time.sleep(2) + os.rmdir(mntlive) if os.path.ismount(destmnt): call(['/bin/umount', destmnt]) if img_type is 'blk': - m_cow: + if dm_cow: call(['/sbin/dmsetup', '--noudevrules', '--noudevsync', 'remove', dm_cow]) if overlayloop: @@ -288,6 +318,10 @@ def main(): os.rmdir(squashmnt) if not os.path.ismount(liveosmnt): os.rmdir(liveosmnt) + time.sleep(2) + for loop in extra_loops: + if loop: + call(['/bin/rm', '-f', loop]) if verbose: print "Cleanup complete"

1 0

[PATCH 1/5] Support Live image mounting in fs.py
by Frederick Grose 14 Apr '11

14 Apr '11

[Patch 1/5] Author: Frederick Grose <fgrose(a)sugarlabs.org> Date: Thu Apr 14 09:29:14 2011 -0400 Support Live image mounting. Provide DiskMount with mount options, DeviceMapperSnapshot with a device attribute, and a new LiveImageMount class. diff --git a/imgcreate/fs.py b/imgcreate/fs.py index d5307a2..0871182 100644 --- a/imgcreate/fs.py +++ b/imgcreate/fs.py @@ -324,7 +324,6 @@ class LoopbackDisk(Disk): self.device = None - class SparseLoopbackDisk(LoopbackDisk): """A Disk backed by a sparse file via the loop module.""" def __init__(self, lofile, size): @@ -377,11 +376,12 @@ class Mount: class DiskMount(Mount): """A Mount object that handles mounting of a Disk.""" - def __init__(self, disk, mountdir, fstype = None, rmmountdir = True): + def __init__(self, disk, mountdir, fstype=None, rmmountdir=True, ops=None): Mount.__init__(self, mountdir) self.disk = disk self.fstype = fstype + self.ops = ops self.rmmountdir = rmmountdir self.mounted = False @@ -435,6 +435,8 @@ class DiskMount(Mount): args = [ "/bin/mount", self.disk.device, self.mountdir ] if self.fstype: args.extend(["-t", self.fstype]) + if self.ops: + args.extend(['-o', self.ops]) rc = call(args) if rc != 0: @@ -530,6 +532,7 @@ class ExtDiskMount(DiskMount): self.__resize_filesystem(size) return minsize + class DeviceMapperSnapshot(object): def __init__(self, imgloop, cowloop): self.imgloop = imgloop @@ -541,6 +544,7 @@ class DeviceMapperSnapshot(object): def get_path(self): if self.__name is None: return None + return self.device return os.path.join("/dev/mapper", self.__name) path = property(get_path) @@ -569,6 +573,7 @@ class DeviceMapperSnapshot(object): string.join(args, " ")) self.__created = True + self.device = os.path.join('/dev/mapper', self.__name) def remove(self, ignore_errors = False): if not self.__created: @@ -610,6 +615,89 @@ class DeviceMapperSnapshot(object): except ValueError: raise SnapshotError("Failed to parse dmsetup status: " + out) + +class LiveImageMount(object): + """A class for mounting a LiveOS image installed with an active overlay.""" + + def __init__(self, disk, mountdir, overlay, tmpdir='/tmp'): + self.disk = disk + self.mountdir = mountdir + self.overlay = overlay + self.tmpdir = tmpdir + self.__created = False + self.squashmnt = None + self.homemnt = None + self.mntlive = None + + def __create(self): + if self.__created: + return + self.liveosdevmnt = DiskMount(self.disk, + os.path.join(self.tmpdir, 'device')) + self.liveosdevmnt.mount() + liveosdir = os.path.join(self.liveosdevmnt.mountdir, 'LiveOS') + sqfs_img = os.path.join(liveosdir, 'squashfs.img') + if os.path.exists(sqfs_img): + self.squashloop = LoopbackDisk(sqfs_img, None) + self.squashmnt = DiskMount(self.squashloop, + os.path.join(self.tmpdir, 'squash')) + self.squashmnt.mount() + rootfs_img = os.path.join(self.squashmnt.mountdir, + 'LiveOS', 'ext3fs.img') + else: + rootfs_img = os.path.join(liveosdir, 'ext3fs.img') + if not os.path.exists(rootfs_img): + raise SnapshotError("Failed to find a LiveOS root image.") + self.imgloop = LoopbackDisk(rootfs_img, None) + self.overlay = os.path.join(liveosdir, self.overlay) + self.cowloop = LoopbackDisk(self.overlay, None) + home_img = os.path.join(liveosdir, 'home.img') + self.dm_liveimage = DeviceMapperSnapshot(self.imgloop, self.cowloop) + self.dm_livemount = DiskMount(self.dm_liveimage, self.mountdir) + if os.path.exists(home_img): + homedir = os.path.join(self.mountdir, 'home') + self.homemnt = LoopbackMount(home_img, homedir) + self.__created = True + + def mount(self): + try: + self.__create() + if not self.liveosdevmnt.mounted: + self.liveosdevmnt.mount() + if not self.squashmnt.mounted: + self.squashmnt.mount() + self.dm_livemount.mount() + if self.homemnt: + self.homemnt.mount() + mntlivedir = os.path.join(self.mountdir, 'mnt', 'live') + if not os.path.exists(mntlivedir): + os.makedirs(mntlivedir) + self.mntlive = BindChrootMount(self.liveosdevmnt.mountdir, + '', mntlivedir) + self.mntlive.mount() + except MountError, e: + raise SnapshotError("Failed to mount %s : %s" % (self.disk, e)) + self.cleanup() + + def unmount(self): + if self.mntlive: + self.mntlive.unmount() + if self.homemnt: + self.homemnt.unmount() + self.dm_livemount.unmount() + self.liveosdevmnt.unmount() + + def cleanup(self): + self.unmount() + if self.homemnt: + self.homemnt.cleanup() + self.dm_liveimage.remove() + if self.squashmnt: + self.squashmnt.cleanup() + self.liveosdevmnt.cleanup() + self.__created = False + + def create_image_minimizer(path, image, compress_type, target_size = None, tmpdir = "/tmp"): """

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

livecd April 2011