Anaconda-devel April 2016

anaconda-devel@lists.stg.fedoraproject.org

13 participants
10 discussions

Notifying post-install tools about screens seen by the user
by Martin Kolman 06 May '16

06 May '16

The idea is that we should not show configuration screens the users has already visited during the installation in post-installation tools, such as Initial Setup and Gnome Initial Setup. An example of such a screen is the timezone spoke - if the user already visited the timezone spoke during the installation it is unlikely to be useful showing it again in IS or GIS. So there should be a way for Anaconda to tell the post-installation tools what screens have been visited by a user during the installation, so that the tools can hide or skip the given screen. While Anaconda already produces a kickstart file after successful installation, the resulting commands in the file don't really map to individual screens and there is no information about what commands have been generated automatically, based on user action or which were part of an initial kickstart file used for the installation. Also parsing a kickstart file in a robust way really requires Pykickstart, which might not be straightforward to use (at least without wrappers) from post -install utilities not written in Python, such as Gnome Initial Setup. Therefore a separate simple configuration file forwarding the needed information about visited screens to post-installation tools is needed. Manual installation description file proposal * use the "INI file like" config file format - can be easily parsed from Python using the configparser module - should be also easy to parse from C and other programming languages due to widespread use of INI files * stored in /etc/anaconda/manual_installation.ini - other name/path suggestions welcome! :) * header sections corresponding to spokes * sections for now only contain a visited=1 key/value pair * could be easily extended in the future to include more information about what user did in the spoke if needed * example config file snippet: [DatetimeSpoke] visited=1 * it seems sensible to write sections only for spokes that the user visited, not all spokes * all sub-classes of the Spoke class should be recorder if they are visited, including addons * Initial Setup will also need to record spoke visits as there might be another setup tool chained behind it (and GIS indeed runs after IS on RHEL7) * should we also report hubs the user has visited ? at least for now the number of hubs and their order are fixed Looking forward to your feedback! :)

10 28

Dual stack installation with anaconda
by A.J. Werkman 27 Apr '16

27 Apr '16

Is anaconda supposed to support an installation that configures the NIC IPv4/IPv6 dual stack? I don't seem to be able to find the right parameters to do this. I filed a bug about this issue some time ago. https://bugzilla.redhat.com/show_bug.cgi?id=971269 Regards, Koos Werkman.

2 1

Re: /boot/efi size, 260MiB minimum for FAT32 ESP) -- WAS: /boot size
by Chris Murphy 24 Apr '16

24 Apr '16

Anaconda's current ESP default produces a 200MiB FAT16 volume using 4KiB clusters. The Windows 8 and 10 (enterprise) installers use this without complaint. It's Advanced Format 4kn compatible. I see neither an advantage or disadvantage to the change in terms of compatibility. If blivet folks want to make non-removable drive ESPs 260MiB so that they can be formatted FAT32 with a 4KiB cluster by mkdosfs, that's reasonable. Asking the user about any of this is not reasonable. All of your suggestions have confused me, so there's no way I'd agree to subjecting all anaconda users who have an empty drive and user manual storage configuration to this. I've been wanting to get rid of bootloader UI from custom storage config for years, not make it worse by adding more unnecessary UI options for the user to have to sift through to get a Fedora installation. About the MSR specifically: a. Microsoft says the MSR must go immediately before the basic data partition, and your proposal doesn't do that nor can it. [1] b. The Microsoft (Windows 10 Enterprise) installer creates an MSR immediately before the basic data partition, when installed into unallocated space. [2] On Fri, Apr 22, 2016 at 1:31 PM, Bryan Smith <b.j.smith(a)ieee.org> wrote: >> It's convincing that the ESP needs to use 4KiB clusters to support 4kn >> drives. But I'm unconvinced any UEFI spec mandates that the ESP needs >> to be FAT32. > > So ... why not give the user the option of FAT32 without confusing them? Because giving them the option confuses them. >> It's suggested that this ought to be true but the >> language I've seen so far doesn't say it must be true. > > And I'm trying to get you to understand that this is a very, very > valid option for people who _wipe_ their disks and create a _new_ GPT > disk label. Something we can do in Anaconda. I just did it in Anaconda and followed it up with a Windows 10 installation and I'm not experiencing the problem you suggest needs a solution. So at the moment I think you're confused. > P.S. Please, please understand that I'm just trying to avoid a lot of > issues that many people run into, with a simple checkbox when someone > installs Linux on a new or wiped disk (not an existing one). A lot of issues that people run into? This is the first time I've ever heard of this issue and when I tried to reproduce the problem, I couldn't. [1] "It is particularly important that the MSR be created before other primary data partitions." https://msdn.microsoft.com/en-us/library/windows/hardware/dn640535%28v=vs.8… [2] https://paste.fedoraproject.org/359079/ -- Chris Murphy

2 4

/boot size
by Chris Lumens 24 Apr '16

24 Apr '16

For RHEL7, I've got a bug about needing to make /boot larger. For once, it's not a private bug so I can even show you: https://bugzilla.redhat.com/show_bug.cgi?id=1270883 The fix is to make /boot larger so more kernels can fit, say, 1GB. It's not a big change but it raises a bigger point. If I make this change just for rhel7-branch, it will be a regression when we get to RHEL8. To prevent that from happening, I also need to make this change on master. But that will affect all new installs of Fedora. Does anyone think that is going to be a problem? Does it need to be discussed more widely first? - Chris

4 5

/boot/efi size, 260MiB minimum for FAT32 ESP) -- WAS: /boot size
by Bryan Smith 22 Apr '16

22 Apr '16

Ross Smith wrote: > I don't think it will be an issue. I've already been bitten by this with a > ~1300 efi booted desktops and would welcome the updated > configuration. Speaking of which ... Q: Shouldn't we be using 260MiB FAT32 as a minimum EFI System Partition (ESP) size going forward, especially as 4KiB logical blocks become regular? o Just some history (detail) ... - For NT6.0 (Vista), Microsoft recommended (and its installers created) a 200MiB FAT32 ESP - This size has seemingly "stuck" with most distros, even though ... - For NT6.1+ (7+/2008+), Microsoft recommended (and its installers created) a 100MiB FAT32 ESP However ... - If the logical block size is 4KiB, the smallest FAT32 file system is 260MiB Although ... - The ESP can be FAT16 (just like good'ole ARC) instead of FAT32 - Every ESP I've used, including Windows 7 and 8 x64, seems to install fine on a FAT32 ESP Hence why I'm bringing it up. Just curious if there has been much discussion on this. I did some google searches on the archives and couldn't find anything. -- bjs -- Bryan J Smith - http://www.linkedin.com/in/bjsmith E-mail: b.j.smith at ieee.org or me at bjsmith.me

3 9

Anaconda multi-selection article published
by Jiří Konečný 18 Apr '16

18 Apr '16

It's mainly for an informative purpose. If somebody wants to read it, here is the link. http://blog-jkonecny.rhcloud.com/2016/04/18/multi-selection-in-anaconda / Your feedback is appreciated. Jirka

1 0

Potential deprecation in RHEL 7.3 ADDON API
by Jiří Konečný 18 Apr '16

18 Apr '16

I have RFE which required to add payload class to addons so I created this PR: https://github.com/rhinstaller/anaconda/pull/571 This PR will cause that all existing addons to get deprecation warning message. This includes our OSCAP and KDUMP addons. The change is really small. You only need to change two lines of code. I'm going to create PR for our addons when this PR receive ACK. This change happens on master and it will be also in RHEL 7.3 . I want to give you notification about this change before it happens by this mail. If I missed something or you don't want this change please write comments to PR or reply to this mail. Thank you, Jirka K.

5 9

Running kickstart-tests in openQA: should we?
by Adam Williamson 15 Apr '16

15 Apr '16

Hey folks! My silly weekend project is now at the point where I figure we should either decide what it's good for, or stop wasting time on it. So I wrote a thing which runs the anacaonda 'kickstart-tests' in openQA. You can find it on the kickstart-tests branch of openqa_fedora: https://bitbucket.org/rajcze/openqa_fedora/branch/kickstart-tests ## What does it actually do? Very roughly, it: 1. takes the kickstart-tests .ks.in and .sh files 2. produces .ks files (which you are expected to upload to some web server) 3. produces one openQA 'test suite' (recipe) for each kickstart-test 4. merges the 'test suites' into some existing file (which you then load to an openQA instance) ## What doesn't it do? 1. Produce the driver disk image, or install.img (you can build the driver disk and place it in openQA's 'hdds' location; install.img test is not handled) 2. Produce the test repos (you are expected to do that, make them available, and feed the locations to the converter) 3. Work with a few tests (all probably fixable, but at this point I don't want to invest more time without a few decisions) 4. Give you an openQA instance or actually run the tests (you're expected to bring your own instance and trigger the tests) ## What does it look like? Kinda like this: https://openqa.happyassassin.net/tests/overview?version=24&groupid=1&build=… Quick notes there: the driverdisk test hard fails because I haven't bothered to generate the image and send it to my openQA server yet. The ntp and raid-1 test fails are actually genuine fails - the tests are testing things that have changed in anaconda since F24 Alpha, so the fails are perfectly correct. 'container' is now on the 'tests skipped because I know they don't work' list, but it shows up in the UI because I ran and cancelled it once before that. You can click the green dots to see the detailed report for each test. ## What does it 'replace'? It takes the place of run_kickstart_tests.sh , run_one_ks.sh and kstest-runner , pretty much, by using openQA to do all the work of spinning up VMs and running installs and gathering results. ## How do I use it? Roughly: git clone https://bitbucket.org/rajcze/openqa_fedora.git cd openqa_fedora git checkout kickstart-tests git clone https://github.com/rhinstaller/kickstart-tests ./kstest-converter kickstarts kickstart-tests http://www/ks http://www/http (NFS_REPO_URL) ks/ scp ks/*.ks www:/var/www/html/ks/ PYTHONPATH=kickstart-tests/lib kickstart-tests/scripts/make-addon-pkgs.py scp -pr http www:/var/www/html/ ./perl2json templates templates.json ./kstest-converter templates kickstart-tests/ (KS_URL) templates.json merged.json scp merged.json openqa:/tmp ssh openqa cd /var/lib/openqa/share/tests git clone https://bitbucket.org/rajcze/openqa_fedora.git cd openqa_fedora git checkout kickstart-tests cd /var/lib/openqa/share/factory/iso wget https://dl.fedoraproject.org/pub/alt/stage/24_Alpha-1.7/Server/x86_64/iso/F… /usr/share/openqa/script/load_templates /tmp/merged.json --clean /usr/share/openqa/script/client isos post ISO=Fedora-Server-netinst-x86_64-24_Alpha-1.7.iso DISTRI=fedora VERSION=24 FLAVOR=kstests ARCH=x86_64 BUILD=Fedora-24-20160322.4-KSTESTS I've assumed you've got a dumb web server called 'www' with a document root of /var/www/html and an openqa server called 'openqa'. in the kstest-converter kickstarts command, the first arg is a URL where the generated kickstart files will be accessible; the second is a URL where the HTTP test repo will be available; the third is a URL where the NFS test repo will be available (though in fact that's one of the tests I have disabled ATM so you can just type whatever you like, it won't be used). You can optionally pass --http to override the default HTTP base repo URL (which for kstest-converter is the official mirrorlist URL), and --ftp to override the default FTP base repo URL (which is that Texas mirror, same as kickstart-tests' default). The openQA commands are standard ones for loading the templates file and kicking off tests on a given ISO; in a real deployment that stuff would be done automatically. For now I've made the kickstart tests their own 'flavor', but that's easily changeable. I've also been using a BUILD value with -KSTESTS on the end just so it's easy to identify the tests in the openQA UI. ## What are the possible benefits of this over the 'official' runner? The most obvious benefit, I guess, is simply that doing this or something like it would get anaconda out of the business of maintaining a test runner written in shell script. Other than that... Using openQA has some other potential benefits. You get a video of the test run, which is nice. We have a fairly sophisticated openQA scheduler code base available which could potentially be used to trigger the tests in all sorts of situations, though I don't know how this compares to the use of Jenkins with the official runner. openQA provides various capabilities for getting information out of both passed and failed tests; for now I don't have this hooked up as well as I could (uploading anaconda logs on failure doesn't work because the way Fedora's openQA tests are currently set up, the test running during a kickstart install is not considered an 'anaconda' test). But it's pretty easy to fix that. The way I implemented the openQA integration it actually *boots the installed system* to check the test result - it doesn't just grab the RESULT file out of the disk image like the official runner. So this approach actually tests that the installed system boots, which is, you know, probably a good thing. (It also avoids some of the hoop-jumping some of the tests have to do to get at the RESULT file, because in the booted system, it's always just /root/RESULT except that one test where it's /home/RESULT). openQA also gives you a pretty good experience for storing and querying the results, though again I don't know how that compares to the Jenkins setup. I don't know how good the scalability story is with the 'official' runner; I can see that it runs multiple tests at a time and can even spread them across machines with parallel, but I dunno how robust or scalable that is. openQA is getting relatively good at scalability; it's tied to a single controller / web UI box, but you can keep scaling up the actual test runners by just adding more and more 'worker host' boxes. Taskotron may be even better at this, though. openQA has fedmsg integration, which is nice. ## How well do the tests plug in to openQA? I'd say 'reasonably well'. I had the basic logic up and running in a couple of hours, it's all been detail work since then. I take quite a few short cuts in parsing the .sh files to create the test suites, there's quite a few sloppy assumptions in there, but that all could be firmed up. There are a few interesting points of difference. The most obvious is the way the official runner can set up kinda transient assets for each test: spinning up an HTTP server to serve the test repo just for the length of the test execution, for instance. openQA's design doesn't *really* provide an easy way to do that. It'd be possible but maybe require upstream patches to openQA and it might not be the best idea. So in my design, the idea is that you just make the test repositories *permanently* available somehow. If we want to run the tests on the official Fedora deployment of openQA, it's easy to do that and similar prep bits in the ansible plays. Similarly, where the official runner produces the 'final' kickstart files per-test, in the prepare() step, this approach produces 'final' kickstarts one time and makes them permanently available. Again the idea is that this step can be ansiblized, so to update the tests we'd just run the ansible plays. ## What could we possibly do with this? There's a few options. 1) We could keep the jenkins+official runner setup that anaconda's using for doing testing at the level of the anaconda project, and just use this to run the kickstart tests at the Fedora QA distribution testing level. So that whenever we (QA) run the openQA tests for whatever reason, the kickstart tests get run, but keep it as a QA thing. This is kinda the easiest to do short term (especially the easiest for anaconda folks as you pretty much have to do nothing), but medium and long term, it might be a bit more error prone than the other approaches. I suspect the way it would work out, anaconda team would really only care about keeping the tests running in the 'official' runner, and we (QA) would have to spend quite a bit of time keeping the converter in line with the official runner's behaviour. We'd also probably have to get a bit more hardware from somewhere for this case, because suddenly adding another ~50 tests to the set of tests run in the current QA instance will make its test runs quite a lot longer; if we get another box or two we can add more workers to compensate for this. 2) We could try to have QA act as a 'service provider' for anaconda to run the tests in openQA: basically in this scenario we'd give you the necessary credentials to trigger openQA tests in an openQA instance that's owned/maintained by QA. There would be a lot of details to figure out in this case. 3) We could set anaconda up with its own openQA instance for running the tests, and we'd just share tooling stuff like the ansible plays and the openQA scheduling code (however much turns out to be in common). This would mean someone in anaconda team being basically familiar with care and feeding of an openQA instance, which isn't that hard. You'd also need the hardware to run it on. In scenarios 2) and 3), I guess we'd burn down the current kickstart- tests runner, and we could start tweaking the way the tests are implemented (in terms of how prep and asset creation are specified and actually carried out) to fit in more nicely with openQA's design. In scenario 1), kickstart-tests would be pretty much unmodified and we'd have to keep working on the converter to keep its behaviour in line with the official stuff. We *could* invent some kind of intermediate format for specifying test requirements which would be interpreted differently by the openQA and 'native' runners, but I suspect that would sound like a lot of engineering. ## What other choices are there besides using openQA? 1) Forget the whole damn thing and just keep on with our separate approaches. I'm sending this email now because I'm still at the point where I'd be fine with this decision; I've only burned 3 days on this work and I learned a lot about the kickstart-tests while doing it. 2) Keep the separate testing approaches, but possibly look at harmonizing test image generation and test triggering and maybe result storage somehow. 2) Look at running the kickstart-tests in Taskotron (possibly along with Beaker?) instead of openQA. tflink is quite interested in this, I think, and I *may* give it a shot. No promises. 3) Wait for the possibly-inevitable introduction of some sort of 'Central CI'-ish Jenkins thing into Fedora and then see if maybe we want to use that for running these tests. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net

2 6

Anaconda on el6 install - problem loading custom .repo files?
by Dominique Martinet 11 Apr '16

11 Apr '16

Hi, I'm trying to make yum-plugin-priorities work on redhat 6 pxe installs. We have some extra repositories here and we kind of need priorities to get e.g. the right kernel (lustre servers need patching), as well as a couple of tools that might be older than the lastest update but will contain functional patches we need and the install path can break if we install "wrong" packages. Anyway, I was able to get the plugins work straight away on el7 (that tmux is awesome) - just had to drop the plugin, pluginconf, and repo files at the right places in the initrd or at kickstart %pre. On el6 I was able to find /tmp/anaconda-yum.conf with proper pluginpath, pluginconfpath and reposdir as well and it looks like from yum.log that it does get the plugin correctly. What is not working is that I need to set the priorities on the repos themselves, so I wrote the .repo files "manually" and all I get is a lookup for .treeinfo in each baseurl without actually getting the repomd. If I have some repo listed twice it properly warns "Repository foo is listed more than once in the configuration" etc so it does look like it's parsing my .repo files, but it's never getting to their actual setups? If I also setup yum itself (/usr/share/yum-cli and yum) and use the said conf file everything works alright from initrd, so my conf probably doesn't look all that bad. Looking at the code getting all the .treeinfos seems to be used to check the version matches the one in /.buildstamp from the initrd, so it doesn't matter if the file actually exists in the repos or not. I don't really see why it'd be able to try to get these .treeinfo but never actually try to use the repos themselves though; I really just don't get any other request on the fileserver access logs. Any idea? What is kickstart 'repo' line doing different? (Well, it looks like it's not actually creating any file on disk so the repo setup itself would be all in memory? How could I add custom parameters to that if not by repo files?) Thanks, -- Dominique Martinet

1 0

Proposal for testing anaconda updates
by Adam Williamson 01 Apr '16

01 Apr '16

Hey folks! Thought the list might be interested in this, so I'm sending it along: https://fedorahosted.org/rel-eng/ticket/6383 it's an idea I had for solving the problem of testing anaconda updates, which we currently just don't have a good way to do. I would like to see a system which would a) let us test them automatically with openQA, and b) provide a compose for people to run manual tests with. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Anaconda-devel April 2016