infrastructure April 2018

infrastructure@lists.fedoraproject.org

43 participants
69 discussions

FBR: Fix the metadata handling for Rawhide messages
by Sayan Chowdhury 20 Apr '18

20 Apr '18

Hi, Here is a hotfix for fedimg. Right now Rawhide messages are not getting processed. This hotfix will fix the issue. PR for the same[1] [1] https://github.com/fedora-infra/fedimg/pull/103 +1s From b52a1dd1eda25bfbade026a1ebde93839fa3379b Mon Sep 17 00:00:00 2001 From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> Date: Thu, 19 Apr 2018 22:05:27 +0530 Subject: [PATCH 1/2] fedimg: Add the intial files for PR#103 Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> --- files/hotfix/fedimg/consumers.py | 160 +++++++++++++++++++++++++++++++++++++++ roles/fedimg/tasks/main.yml | 8 ++ 2 files changed, 168 insertions(+) create mode 100644 files/hotfix/fedimg/consumers.py diff --git a/files/hotfix/fedimg/consumers.py b/files/hotfix/fedimg/consumers.py new file mode 100644 index 0000000..ce4d662 --- /dev/null +++ b/files/hotfix/fedimg/consumers.py @@ -0,0 +1,160 @@ +# -*- coding: utf-8 -*- +# This file is part of fedimg. +# Copyright (C) 2014-2017 Red Hat, Inc. +# +# fedimg is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# fedimg is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public +# License along with fedimg; if not, see http://www.gnu.org/licenses, +# or write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Authors: David Gay <dgay(a)redhat.com> +# Sayan Chowdhury <sayanchowdhury(a)fedoraproject.org> +""" +This is the `fedmsg consumer`_ that subscribes to the topic emitted after the +completion of the nightly and production compose. The consumer on receving the +message uploads the image using the API of the cloud providers. +""" + +import logging +import multiprocessing.pool + +import fedmsg.consumers +import fedmsg.encoding +import fedfind.release +import fedfind.exceptions + +import fedimg.uploader + +from fedimg.config import PROCESS_COUNT, STATUS_FILTER +from fedimg.utils import get_rawxz_urls, get_value_from_dict + +_log = logging.getLogger(__name__) + + +class FedimgConsumer(fedmsg.consumers.FedmsgConsumer): + """ + A `fedmsg consumer`_ that listens to the pungi compose topics and kicks + of the process to upload the images to various cloud providers. + + Attributes: + topic (str): The topics this consumer is subscribed to. Set to + ``org.fedoraproject.prod.pungi.compose.status.change``. + config_key (str): The key to set to ``True`` in the fedmsg config to + enable this consumer. The key is ``fedimgconsumer.prod.enabled``. + """ + topic = ['org.fedoraproject.prod.pungi.compose.status.change'] + config_key = "fedimgconsumer.prod.enabled" + + def __init__(self, *args, **kwargs): + _log.info("FedimgConsumer initializing") + super(FedimgConsumer, self).__init__(*args, **kwargs) + + # Threadpool for upload jobs + _log.info("Creating thread pool of %s process", PROCESS_COUNT) + self.upload_pool = multiprocessing.pool.ThreadPool( + processes=PROCESS_COUNT + ) + _log.info("FedimgConsumer initialized") + + def consume(self, msg): + """ + This is called when we receive a message matching our topics. + + Args: + msg (dict): The raw message from fedmsg. + """ + _log.info('Received %r %r', msg['topic'], msg['body']['msg_id']) + + msg_info = msg['body']['msg'] + if msg_info['status'] not in STATUS_FILTER: + _log.debug('%s is not valid status' % msg_info['status']) + return + + location = msg_info['location'] + compose_id = msg_info['compose_id'] + try: + compose_metadata = fedfind.release.get_release(cid=compose_id).metadata + except fedfind.exceptions.UnsupportedComposeError: + LOG.debug("%r is unsupported compose" % compose_id) + return + + + # Till F27, both cloud-base and atomic images were available + # under variant CloudImages. With F28 and onward releases, + # cloud-base image compose moved to cloud variant and atomic images + # moved under atomic variant. + prev_rel = ['26', '27'] + if msg_info['release_version'] in prev_rel: + images_meta = get_value_from_dict( + compose_metadata, 'images', 'payload', 'images', 'CloudImages', + 'x86_64') + else: + images_meta = get_value_from_dict( + compose_metadata, 'images', 'payload', 'images', + 'Cloud', 'x86_64') + images_meta.extend(get_value_from_dict( + compose_metadata, 'images', 'payload', + 'images', 'AtomicHost', 'x86_64')) + + images_meta = get_value_from_dict( + compose_metadata, + 'images', + 'payload', + 'images', + 'CloudImages', + 'x86_64' + ) + + if images_meta is None: + _log.debug('No compatible image found to process') + return + + upload_urls = get_rawxz_urls(location, images_meta) + if len(upload_urls) > 0: + _log.info("Start processing compose id: %s", compose_id) + fedimg.uploader.upload( + pool=self.upload_pool, + urls=upload_urls, + compose_id=compose_id, + push_notifications=True + ) + + +class FedimgStagingConsumer(FedimgConsumer): + """ + A `fedmsg consumer`_ that listens to the staging pungi compose topics and + kicks of the process to upload the images to various cloud providers. + + Attributes: + topic (str): The topics this consumer is subscribed to. Set to + ``org.fedoraproject.stg.pungi.compose.status.change``. + config_key (str): The key to set to ``True`` in the fedmsg config to + enable this consumer. The key is ``fedimgconsumer.stg.enabled``. + """ + topic = ['org.fedoraproject.stg.pungi.compose.status.change'] + config_key = "fedimgconsumer.stg.enabled" + + +class FedimgDevConsumer(FedimgConsumer): + """ + A `fedmsg consumer`_ that listens to the dev pungi compose topics and + kicks of the process to upload the images to various cloud providers. + + Attributes: + topic (str): The topics this consumer is subscribed to. Set to + ``org.fedoraproject.dev.pungi.compose.status.change``. + config_key (str): The key to set to ``True`` in the fedmsg config to + enable this consumer. The key is ``fedimgconsumer.dev.enabled``. + """ + topic = ['org.fedoraproject.dev.pungi.compose.status.change'] + config_key = "fedimgconsumer.dev.enabled" diff --git a/roles/fedimg/tasks/main.yml b/roles/fedimg/tasks/main.yml index b0f18bf..2e35608 100644 --- a/roles/fedimg/tasks/main.yml +++ b/roles/fedimg/tasks/main.yml @@ -130,3 +130,11 @@ tags: - fedimg - hotfix + +- name: hotfix - copy the consumers.py over to the site-packages + copy: src="{{ files }}/hotfix/fedimg/consumers.py" dest=/usr/lib/python2.7/site-packages/fedimg/consumers.py + notify: + - restart fedmsg-hub + tags: + - fedimg + - hotfix -- 2.9.4 From 1bf3db45c8154d0a6a0f29836ec14ce24fa404c1 Mon Sep 17 00:00:00 2001 From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> Date: Thu, 19 Apr 2018 22:07:12 +0530 Subject: [PATCH 2/2] fedimg: Add the patch for the PR#103, fix processing Rawhide messages Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> --- files/hotfix/fedimg/consumers.py | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/files/hotfix/fedimg/consumers.py b/files/hotfix/fedimg/consumers.py index ce4d662..8195321 100644 --- a/files/hotfix/fedimg/consumers.py +++ b/files/hotfix/fedimg/consumers.py @@ -85,7 +85,7 @@ class FedimgConsumer(fedmsg.consumers.FedmsgConsumer): try: compose_metadata = fedfind.release.get_release(cid=compose_id).metadata except fedfind.exceptions.UnsupportedComposeError: - LOG.debug("%r is unsupported compose" % compose_id) + _log.debug("%r is unsupported compose" % compose_id) return @@ -106,15 +106,6 @@ class FedimgConsumer(fedmsg.consumers.FedmsgConsumer): compose_metadata, 'images', 'payload', 'images', 'AtomicHost', 'x86_64')) - images_meta = get_value_from_dict( - compose_metadata, - 'images', - 'payload', - 'images', - 'CloudImages', - 'x86_64' - ) - if images_meta is None: _log.debug('No compatible image found to process') return -- 2.9.4 -- Sayan Chowdhury <https://sayanchowdhury.dgplug.org/> Senior Software Engineer, Fedora Engineering - Emerging Platform GPG Fingerprint : 0F16 E841 E517 225C 7D13 AB3C B023 9931 9CD0 5C8B Proud to work at The Open Organization!

5 4

FBR MBS: Fix traceback when reusing components from module build with different buildrequires.
by Jan Kaluža 20 Apr '18

20 Apr '18

Hi, this is FBR to fix MBS traceback which prevents builds of modules in situation when new build-require is added to a module Foo. Currently, MBS tries to check what was the commit hash ("ref") of such buildrequired module in previous build of Foo, but it fails with KeyError, because the previous Foo module build did not build-required this newly added build-required module. Full traceback is here: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/consumer.py", line 240, in process_message further_work = handler(conf, session, msg) or [] File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/handlers/modules.py", line 292, in wait if attempt_to_reuse_all_components(builder, session, build): File "/usr/lib/python2.7/site-packages/module_build_service/utils/reuse.py", line 140, in attempt_to_reuse_all_components previous_module_build = _get_reusable_module(session, module) File "/usr/lib/python2.7/site-packages/module_build_service/utils/reuse.py", line 123, in _get_reusable_module ref2 = old_xmd['mbs']['buildrequires'][br_module_name].get('ref') KeyError: 'platform' Fixed patch [1] addresses this by catching the KeyError in this code block. It has been tested on staging and I verified it fixes this issue. [1] https://src.fedoraproject.org/rpms/module-build-service/c/9d3a4923631efca2f… Regards, Jan Kaluza

3 2

FBR bodhi-pungi: Add AtomicHost variant
by Dusty Mabe 19 Apr '18

19 Apr '18

Hopefully one last change to get the AH artifacts generating in bodhi/pungi.

3 4

[Fedocal] Reminder meeting : Fedora Tagger - future development meeting
by jflory7＠fedoraproject.org 19 Apr '18

19 Apr '18

Dear all, You are kindly invited to the meeting: Fedora Tagger - future development meeting on 2018-04-20 from 14:00:00 to 15:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: This meeting is for stakeholders and interested developers in the future of the Fedora Tagger application. This meeting will focus on figuring out milestones to update the Tagger application and how to support it in Fedora's infrastructure going forward. Anyone is welcome to participate. Please see the [fedora-infra/fedora-tagger#200](https://github.com/fedora-infra/fedora-tagg… for more information. Source: https://apps.fedoraproject.org/calendar/meeting/9217/

1 0

FBR fixes to bodhi-pungi configs for atomichost artifacts
by Dusty Mabe 19 Apr '18

19 Apr '18

This is an attempt to fix https://pagure.io/dusty/failed-composes/issue/176 I'd like to get patrick and one other person to review these changes.

4 7

Freeze Break Request: re-enable fedmsg hook on src
by Kevin Fenzi 19 Apr '18

19 Apr '18

Greetings. Just before freeze in ansible commit dc86b22f9054bdcefd51e46db90433c03d54410c pingou disabled the fedmsg hook because there was a permissions problem reading the fedmsg key. In ansible commit 9cbf120b54a79ecb07931f3a3a1b059525d4d766 patrick fixed this perm issue via a facl. So, I would like to re-enable the hook on normal projects and remove it from forks. (I think we decided forks should only get the pagure hook, not the main fedmsg one). So, I'd like to apply: > diff --git a/roles/git/hooks/files/post-receive-chained b/roles/git/hooks/files/post-receive-chained > index 99b5a2a..5c74da4 100755 > --- a/roles/git/hooks/files/post-receive-chained > +++ b/roles/git/hooks/files/post-receive-chained > @@ -8,7 +8,7 @@ pee \ > /usr/share/git-core/post-receive-alternativearch \ > /usr/lib/python2.7/site-packages/pagure/hooks/files/default_hook.py \ > /usr/lib/python2.7/site-packages/pagure/hooks/files/pagure_hook.py \ > - /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \ > +# /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \ > "/usr/bin/grok-manifest -m /srv/git/grokmirror/manifest.js.gz -t /srv/git/repositories/ -n `pwd`" > > # We used to send emails directly from the git hook here, but now we send to > diff --git a/roles/git/hooks/files/post-receive-chained-forks b/roles/git/hooks/files/post-receive-chained-forks > index 5e0056d..67565e6 100755 > --- a/roles/git/hooks/files/post-receive-chained-forks > +++ b/roles/git/hooks/files/post-receive-chained-forks > @@ -6,7 +6,7 @@ > pee \ > /usr/lib/python2.7/site-packages/pagure/hooks/files/default_hook.py \ > /usr/lib/python2.7/site-packages/pagure/hooks/files/pagure_hook.py \ > - /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \ > +# /usr/lib/python2.7/site-packages/pagure/hooks/files/fedmsg_hook.py \ > "/usr/bin/grok-manifest -m /srv/git/grokmirror/manifest.js.gz -t /srv/git/repositories/ -n `pwd`" > > # We used to send emails directly from the git hook here, but now we send to Run the ansible playbook against pkgs, And then run the script to make sure all projects are set right. +1s? kevin

4 7

Fedora Tagger - Future development meeting
by Elorm Buertey 19 Apr '18

19 Apr '18

Hi everyone, You are kindly invited to the Fedora tagger meeting Starting from Fri, April 20, 2018 - 14:00 UTC and ending on Fri, April 20, 2018 - 15:00:00 UTC At fedora-meeting-1(a)irc.freenode.net This meeting is for stakeholders and interested developers in the future of the Fedora Tagger application. This meeting will focus on figuring out milestones to update the Tagger application and how to support it in Fedora's infrastructure going forward. Anyone is welcome to participate. Please see the link below for more information: https://github.com/fedora-infra/fedora-tagger/issues/200 Regards, Elorm Buertey

1 0

FBR: Check if AMI process is complete before processing
by Sayan Chowdhury 18 Apr '18

18 Apr '18

Hi, I want to push a hotfix for fedimg. This patch would check if the AMI process is complete before proceeding forward. Without this patch the AMIs are failed to copied to different region after it is uploaded to the base region. PR for the same[1] [1] https://github.com/fedora-infra/fedimg/pull/100 +1s From 191af6ec22d5977a432862aa08254f131cd6e9a1 Mon Sep 17 00:00:00 2001 From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> Date: Wed, 18 Apr 2018 23:56:45 +0530 Subject: [PATCH 1/2] fedimg: Add the initial files for the hotfix, fix copied amis Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> --- files/hotfix/fedimg/ec2imgpublisher.py | 300 +++++++++++++++++++++++++++++++++ roles/fedimg/tasks/main.yml | 7 + 2 files changed, 307 insertions(+) create mode 100644 files/hotfix/fedimg/ec2imgpublisher.py diff --git a/files/hotfix/fedimg/ec2imgpublisher.py b/files/hotfix/fedimg/ec2imgpublisher.py new file mode 100644 index 0000000..6e62e04 --- /dev/null +++ b/files/hotfix/fedimg/ec2imgpublisher.py @@ -0,0 +1,300 @@ +# This file is part of fedimg. +# Copyright (C) 2014-2017 Red Hat, Inc. +# +# fedimg is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# fedimg is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public +# License along with fedimg; if not, see http://www.gnu.org/licenses, +# or write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Authors: Sayan Chowdhury <sayanchowdhury(a)fedoraproject.org> +# + +import logging +_log = logging.getLogger(__name__) + +import re + +from time import sleep + +import fedimg.messenger + +from fedimg.utils import external_run_command, get_item_from_regex +from fedimg.utils import get_image_name_from_ami_name +from fedimg.utils import get_image_name_from_ami_name_for_fedmsg +from fedimg.services.ec2.ec2base import EC2Base + + +class EC2ImagePublisher(EC2Base): + """ Comment goes here """ + + def __init__(self, **kwargs): + defaults = { + 'access_key': None, + 'compose_id': None, + 'image_id': None, + 'image_name': 'Fedora-AMI', + 'image_url': None, + 'image_description': 'Fedora AMI Description', + 'service': 'EC2', + 'region': None, + 'secret_key': None, + 'visibility': 'all', + 'push_notifications': False, + } + + for (prop, default) in defaults.iteritems(): + setattr(self, prop, kwargs.get(prop, default)) + + def _retry_till_image_is_public(self, image): + """ Comment goes here """ + + driver = self._connect() + + is_image_public = False + while True: + try: + is_image_public = driver.ex_modify_image_attribute( + image, + {'LaunchPermission.Add.1.Group': 'all'}) + except Exception as e: + if 'InvalidAMIID.Unavailable' in str(e): + # The copy isn't completed yet, so wait for 20 seconds + # more. + sleep(20) + continue + break + + return is_image_public + + def _retry_till_snapshot_is_public(self, snapshot): + + driver = self._connect() + + while True: + is_snapshot_public = driver.ex_modify_snapshot_attribute( + snapshot, + {'CreateVolumePermission.Add.1.Group': 'all'}) + + if is_snapshot_public: + break + + return is_snapshot_public + + + def _retry_till_snapshot_is_available(self, image): + + driver = self._connect() + while True: + image = driver.get_image(image.id) + snapshot_id = image.extra['block_device_mapping'][0]['ebs']['snapshot_id'] + + if snapshot_id: + break + + return snapshot_id + + def _generate_dummy_snapshot_object(self, snapshot_id): + + driver = self._connect() + + snapshot_obj = type('', (), {})() + snapshot_obj.id = snapshot_id + snapshot = driver.list_snapshots(snapshot=snapshot_obj) + + return snapshot + + def _retry_till_blk_mapping_is_available(self, image): + + while True: + image = self._connect().get_image(image_id=image.id) + blk_mapping = image.extra['block_device_mapping'] + + if blk_mapping: + return blk_mapping + + def get_snapshot_from_image(self, image): + """ Comment goes here """ + if isinstance(image, str): + image_id = image + image = self._connect().get_image(image_id) + + blk_mapping = image.extra['block_device_mapping'] + if not blk_mapping: + blk_mapping = self._retry_till_blk_mapping_is_available(image) + + snapshot_id = blk_mapping[0]['ebs']['snapshot_id'] + if snapshot_id is None: + snapshot_id = self._retry_till_snapshot_is_available(image) + + snapshot = self._generate_dummy_snapshot_object(snapshot_id)[0] + + return snapshot + + def get_volume_type_from_image(self, image): + if isinstance(image, str): + image_id = image + image = self._connect().get_image(image_id) + + blk_mapping = image.extra['block_device_mapping'] + if not blk_mapping: + blk_mapping = self._retry_till_blk_mapping_is_available(image) + + return blk_mapping[0]['ebs']['volume_type'] + + def get_virt_type_from_image(self, image): + return 'hvm' + + def publish_images(self, region_image_mapping=None): + """ Comment goes here """ + + published_images = [] + if region_image_mapping is None: + return published_images + + for region, image_id in region_image_mapping: + self.set_region(region) + + _log.info('Publish image (%s) in %s started' % (image_id, region)) + image = self._connect().get_image(image_id=image_id) + is_image_public = self._retry_till_image_is_public(image) + _log.info('Publish image (%s) in %s completed' % (image_id, region)) + + _log.info('Publish snaphsot for image (%s) in %s started' % (image_id, region)) + snapshot = self.get_snapshot_from_image(image) + _log.info('Fetched snapshot for image (%s): %s' % (image_id, snapshot.id)) + is_snapshot_public = self._retry_till_snapshot_is_public(snapshot) + _log.info('Publish snaphsot for image (%s) in %s completed' % (image_id, region)) + + volume_type = self.get_volume_type_from_image(image) + virt_type = self.get_virt_type_from_image(image) + + if self.push_notifications: + fedimg.messenger.notify( + topic='image.publish', + msg=dict( + image_name=get_image_name_from_ami_name_for_fedmsg(image.name), + image_url=self.image_url, + destination=self.region, + service=self.service, + compose=self.compose_id, + extra=dict( + id=image.id, + virt_type=virt_type, + vol_type=volume_type + ) + ) + ) + + fedimg.messenger.notify( + topic='image.upload', + msg=dict( + image_name=get_image_name_from_ami_name_for_fedmsg(image.name), + image_url=self.image_url, + destination=self.region, + service=self.service, + status='completed', + compose=self.compose_id, + extra=dict( + id=image.id, + virt_type=virt_type, + vol_type=volume_type + ) + ) + ) + + published_images.append({ + 'image_id': image.id, + 'is_image_public': is_image_public, + 'snapshot_id': snapshot.id, + 'is_snapshot_public': is_snapshot_public, + 'regions': self.region + }) + + return published_images + + def copy_images_to_regions(self, image_id=None, base_region=None, regions=None): + """ Comment goes here """ + + if (image_id is None) or (regions is None) or (base_region is None): + return + + counter = 0 + copied_images = [] + + self.set_region(base_region) + image = self._connect().get_image(image_id=image_id) + if not image: + return [] + + for region in regions: + _log.info('Copy %s to %s started' % (image_id, region)) + self.set_region(region) + self.image_name = get_image_name_from_ami_name(image.name, region) + + while True: + if counter > 0: + self.image_name = re.sub( + '\d(?!\d)', + lambda x: str(int(x.group(0))+1), + self.image_name + ) + try: + copied_image = self._connect().copy_image( + source_region=base_region, + image=image, + name=self.image_name, + description=self.image_description) + + virt_type = image.extra['virtualization_type'] + volume_type = image.extra['block_device_mapping'][0]['ebs']['volume_type'] + + if self.push_notifications: + fedimg.messenger.notify( + topic='image.copy', + msg=dict( + image_name=get_image_name_from_ami_name_for_fedmsg(copied_image.name), + destination=self.region, + service=self.service, + compose_id=self.compose_id, + extra=dict( + id=copied_image.id, + virt_type=virt_type, + vol_type=volume_type, + source_image_id=image.id + ) + ) + ) + + _log.info('Copy %s to %s is completed.' % (image_id, region)) + copied_images.append({ + 'region': region, + 'copied_image_id': copied_image.id + }) + break + + except Exception as e: + _log.info('Could not register ' + 'with name: %r' % self.image_name) + if 'InvalidAMIName.Duplicate' in str(e): + counter = counter + 1 + else: + _log.info('Failed') + break + + return copied_images + + def deprecate_images(self, image_ids=None, snapshot_perm='all'): + raise NotImplementedError + + def delete_images(self, image_ids=None, snapshot_perm='all'): + raise NotImplementedError diff --git a/roles/fedimg/tasks/main.yml b/roles/fedimg/tasks/main.yml index c21cb8f..b0f18bf 100644 --- a/roles/fedimg/tasks/main.yml +++ b/roles/fedimg/tasks/main.yml @@ -123,3 +123,10 @@ - fedimg - hotfix +- name: hotfix - copy the ec2imgpublisher.py over to the site-packages + copy: src="{{ files }}/hotfix/fedimg/ec2imgpublisher.py" dest=/usr/lib/python2.7/site-packages/fedimg/services/ec2/ec2imgpublisher.py + notify: + - restart fedmsg-hub + tags: + - fedimg + - hotfix -- 2.9.4 From d35fa21e8a6c0fe1e07ed9b74b0f524bba3c6c84 Mon Sep 17 00:00:00 2001 From: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> Date: Wed, 18 Apr 2018 23:57:46 +0530 Subject: [PATCH 2/2] fedimg: Checkin the patch for PR#100. Check if the AMI is complete before proceeding Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012(a)gmail.com> --- files/hotfix/fedimg/ec2imgpublisher.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/files/hotfix/fedimg/ec2imgpublisher.py b/files/hotfix/fedimg/ec2imgpublisher.py index 6e62e04..b1eab48 100644 --- a/files/hotfix/fedimg/ec2imgpublisher.py +++ b/files/hotfix/fedimg/ec2imgpublisher.py @@ -90,6 +90,22 @@ class EC2ImagePublisher(EC2Base): return is_snapshot_public + def _retry_till_image_is_available(self, image_id): + driver = self._connect() + + while True: + try: + image = driver.get_image(image_id) + image_name = image.name + if image_name is None: + continue + return image + except Exception as e: + if 'InvalidAMIID.Unavailable' in str(e): + # The copy isn't completed yet, so wait for 20 seconds + # more. + sleep(20) + continue def _retry_till_snapshot_is_available(self, image): @@ -255,6 +271,8 @@ class EC2ImagePublisher(EC2Base): name=self.image_name, description=self.image_description) + copied_image = self._retry_till_image_is_available(copied_image.id) + virt_type = image.extra['virtualization_type'] volume_type = image.extra['block_device_mapping'][0]['ebs']['volume_type'] -- 2.9.4 -- Sayan Chowdhury <https://sayanchowdhury.dgplug.org/> Senior Software Engineer, Fedora Engineering - Emerging Platform GPG Fingerprint : 0F16 E841 E517 225C 7D13 AB3C B023 9931 9CD0 5C8B Proud to work at The Open Organization!

4 3

Re: How To Contact Folks Interested In Tagger
by Justin W. Flory 17 Apr '18

17 Apr '18

On 04/14/2018 10:15 PM, Elorm Buertey wrote: > Hi. I have created the WhenIsGood survey but I'm not sure of the > best way to send it to the others. I couldn't find their > emails on their profiles. Would posting the link in the Github > issue be okay? > Hi Elorm, Sharing the WhenIsGood link on the GitHub issue for the Fedora Tagger development meeting works; I would also consider sharing it on the Fedora Infrastructure mailing list since there are other people there who may be interested. You can subscribe here: https://lists.fedoraproject.org/admin/lists/infrastructure.lists.fedoraproj… I've included this response to the mailing list already, so you can subscribe and then share the link on this thread for the Fedora Tagger development meeting time. Hope this helps! -- Cheers, Justin W. Flory jflory7(a)gmail.com

2 1

Fwd: mirror failures
by Miroslav Suchý 16 Apr '18

16 Apr '18

Can someone comment on this? I am not really sure if there are some issues on mirrors (not likely) or if this is some issue with Fedora Cloud when network is overloaded (more likely). Miroslav -------- Přeposlaná zpráva -------- Předmět: mirror failures Datum: Mon, 9 Apr 2018 09:38:04 +0200 Od: Michal Novotny <clime(a)redhat.com> Adresa pro odpověď: Community Projects <copr-devel(a)lists.fedorahosted.org> Komu: Cool Other Package Repositories <copr-devel(a)lists.fedorahosted.org> Hello, as of recent month or two I quite often encounter mirror sync failures on COPR builders during the build setup. Could you, please, confirm that this is really the case and link the build logs here if possible? I will then setup some extended monitoring if this proves to be true. Thank you clime _______________________________________________ copr-devel mailing list -- copr-devel(a)lists.fedorahosted.org To unsubscribe send an email to copr-devel-leave(a)lists.fedorahosted.org

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure April 2018