aeolus-devel May 2012

aeolus-devel@lists.fedorahosted.org

34 participants
134 discussions

[PATCH conductor] BZ 801527: wrong error message when imagefactory is down
by Maros Zatko 06 Sep '12

06 Sep '12

From: Maros Zatko <mzatko(a)redhat.com> https://bugzilla.redhat.com/show_bug.cgi?id=801527 API: * new API exception ServiceUnavailable * rescue for ECONNREFUSED when imagefactory is down --- src/app/controllers/api/images_controller.rb | 14 +++++++++----- src/lib/exceptions.rb | 1 + 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/src/app/controllers/api/images_controller.rb b/src/app/controllers/api/images_controller.rb index 465a20b..2bcbeee 100644 --- a/src/app/controllers/api/images_controller.rb +++ b/src/app/controllers/api/images_controller.rb @@ -89,11 +89,15 @@ module Api :template => @tpl.uuid, :environment => @pool_family.name }) - @image = Aeolus::Image::Factory::Image.new(:id => iwhd_image.id) - @image.targets = req[:params][:targets] - @image.template = req[:params][:template] - @image.save! - respond_with(@image) + begin + @image = Aeolus::Image::Factory::Image.new(:id => iwhd_image.id) + @image.targets = req[:params][:targets] + @image.template = req[:params][:template] + @image.save! + respond_with(@image) + rescue Errno::ECONNREFUSED + raise(Aeolus::Conductor::API::ServiceUnavailable.new(503, 'Imagefactory is dead, Jim!')) + end elsif req[:type] == :import account = ProviderAccount.find_by_label(req[:params][:provider_account_name]) raise(Aeolus::Conductor::API::ProviderAccountNotFound.new(404, t("api.error_messages.provider_account_not_found", diff --git a/src/lib/exceptions.rb b/src/lib/exceptions.rb index d56ddb3..9b1edab 100644 --- a/src/lib/exceptions.rb +++ b/src/lib/exceptions.rb @@ -38,6 +38,7 @@ module Aeolus class ProviderImageDeleteFailure < Error; end class ProviderImageNotFound < Error; end class ProviderImageStatusNotFound < Error; end + class ServiceUnavailable < Error; end class TargetImageDeleteFailure < Error; end class TargetImageNotFound < Error; end class TargetImageStatusNotFound < Error; end -- 1.7.7.6

2 2

RFC: Background Processing
by Richard Su 14 Jun '12

14 Jun '12

Hi, This expands on some of the notes Jan provided in other RFCs. delayed_jobs and resque appears to be the most commonly deployed solution. I listed what I thought should be the requirements for a background processing solution. For each requirement I then added some details on how well delayed_jobs and resque could satisfy it. Resque contains most of the features we need. It requires Redis, which is a open source project sponsored by VMware. Redis is available in Fedora. But I don't see Redis available in RHEL and getting it in for RHEL is the big question mark. https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Background_Proce… --- Background Processing # Summary The two most common solutions are delayed_jobs and resque. There is a good write up on github comparing other background processing solutions and why they eventually steered towards delayed_jobs and then resque, https://github.com/blog/542-introducing-resque. The primary differences between delayed_jobs and resque are: At the moment, delayed_jobs doesn't have support for recurring jobs. Resque does support recuring jobs through the resque-scheduler extension/gem. resque provides a sinatra app to monitor the queue. delayed_job doesn't provide monitoring tools out of the box, but we can potential build something on top of rails or simply look at the contents of the database table. resque requires multiple components and potentially could be more difficult to support. It requries a second gem called resque-scheduler. It also uses Redis as its backend and it is currently not available with RHEL. This may be the deal breaker. # Requirements 1. Bucket jobs into different queues. A long running job to check instance status for 1000 instances should not hold up other jobs. The solution should also support multiple workers which would minimize impact of longer running jobs. But using different queues will offer finer grain control. * delayed_job: supports multiple queues through named queues starting with version 3.0. Can start up multiple workers for all queues or for specific queues. * resque: supports multiple queues and workers. 2. Jobs should persist in some way. If a crash occurs, we should be able to restart the system and continue with processing incomplete jobs in the queue. * delayed_job: Jobs persists as objects stored in activerecord entries. * resque: Jobs persists as json objects in redis entries. Using json objects instead of actual objects which may have advanced to a different version makes updating the application potentially easier. 3. Recurring jobs. * delayed_jobs: Not available, in development. * resque: Through resque-scheduler extension. * whenever: A potential alternative to do cron style scheduling [6]. 4. Alerts. Failures should be presented to the user in some way (email, conductor UI) so that appropriate actions can be taken. * delayed_jobs: Support code hooks for different stages in the process. Hooks can be added for error, failure, success.. By default workers will retry a job 25 times. We should use a lower number. No sense in retrying that number of times and holding up the queue if there is a hard failure somewhere in the system. By default it also deletes failed jobs, but can be configured to leave them in the queue with a flag to indicate failure. * resque: Failed jobs can go through additional processing using different failure backends. redis, syslog, custom, etc.. 5. A mechanism to requeue a failed job once the underlying issue has been resolved. If an instance start job fails and there is a network failure to a provider. Once the network is back online, we should have an ability to requeue those jobs. Not sure if this should be automated or if this should be a button somewhere where a user can manually requeue all or select failed jobs. * custom 6. Monitor job status. We should have some way to see what is in the queue. * delayed_jobs: Can only view queue through activerecord database entries. There is no UI so it is more difficult to see what is going on. * resque: Provides a sinatra app to monitor queues, jobs, and workers. 7. Should not enqueue duplicate jobs. * custom 8. Ability to remove jobs from the queues and to place a pause on the queues or jobs. * custom 9. Supportable in Fedora and RHEL * delayed_jobs: We used it in the past. Will need to carry the gem. * resque: Will need to carry the gem. In addition it requires Redis as the backend. Redis is available in Fedora but not in RHEL. Redis is a open source project sponsored by VMware [4]. # Use Cases 1. Dbomatic replacement for instance and realm checking and RHEV instance start. Each RHEV instance that is created will also lead to a job that is enqueued to start that instance. Create a new job to perform instance status check. Create a status check job for each provider account. Allow status check job to be disabled/enabled per provider account. Create a new job to sync realms for all providers. This can be broken up to a job per provider if needed. Create two queues. One for managing instance lifecycle. And a second queue for all other jobs. Start with two workers per queue. Make the number of workers configurable so that it may be adjusted when needed. 2. ldap syncing 3. Generic instance start and stop # Reference [1] https://github.com/collectiveidea/delayed_job/wiki/Named-Queues-Proposal [2] https://github.com/blog/542-introducing-resque Discusses github's use of different background job solutions [3] https://github.com/bvandenbos/resque-scheduler [4] http://redis.io/ [5] http://blog.railsupgrade.com/2011/08/replace-delayedjob-with-resque.html [6] https://github.com/javan/whenever

4 8

[PATCH conductor] Fix some untranslated strings
by Imre Farkas 12 Jun '12

12 Jun '12

From: Imre Farkas <ifarkas(a)redhat.com> --- .../_deployment_card_show.html.mustache | 2 +- src/app/views/images/_status.html.mustache | 8 ++++---- .../views/pools/_pretty_list_header.html.mustache | 3 ++- src/config/locales/en.yml | 1 + 4 files changed, 8 insertions(+), 6 deletions(-) diff --git a/src/app/views/deployments/_deployment_card_show.html.mustache b/src/app/views/deployments/_deployment_card_show.html.mustache index 0f36ee1..4a8cadd 100644 --- a/src/app/views/deployments/_deployment_card_show.html.mustache +++ b/src/app/views/deployments/_deployment_card_show.html.mustache @@ -2,7 +2,7 @@ <h3 class="name"> <%= link_to('{{name}}', '{{path}}') %> </h3> - <span title="{{translated_state}}" class="status {{state}}">alma</span> + <span title="{{translated_state}}" class="status {{state}}">{{state}}</span> <dl class="statistics"> <ul> <li class="uptime"> diff --git a/src/app/views/images/_status.html.mustache b/src/app/views/images/_status.html.mustache index b0688c4..069b14d 100644 --- a/src/app/views/images/_status.html.mustache +++ b/src/app/views/images/_status.html.mustache @@ -32,16 +32,16 @@ <table class="light_table"> <thead> <tr><th> - <strong>Account</strong> + <strong><%= t('images.show.account') %></strong> </th> <th> - Provider + <%= t('images.show.provider') %> </th> <th> - Image UUID + <%= t('images.show.image_uuid') %> </th> <th> - Image URI + <%= t('images.show.image_uri') %> </th> <th class="image_controls"></th> </tr></thead> diff --git a/src/app/views/pools/_pretty_list_header.html.mustache b/src/app/views/pools/_pretty_list_header.html.mustache index 140cd67..38a6008 100644 --- a/src/app/views/pools/_pretty_list_header.html.mustache +++ b/src/app/views/pools/_pretty_list_header.html.mustache @@ -19,6 +19,7 @@ </li> </ul> </dl> + <a class="control" href="#"> - <span>Expand/Collapse</span> + <span><%= t('pools.index.pool_toggle') %></span> </a> diff --git a/src/config/locales/en.yml b/src/config/locales/en.yml index e8c91a3..59c29a0 100644 --- a/src/config/locales/en.yml +++ b/src/config/locales/en.yml @@ -494,6 +494,7 @@ en: not_all_deployments_displayed_alert_html: "Not all deployments are displayed! Go to %{name} Pool details for more! »" global_uptime: Global uptime destroy: Destroy + pool_toggle: Expand/Collapse images: catalog: Catalog catalog_entry: Deployable -- 1.7.6.5

2 1

[PATCH conductor] BZ786535: display failures for instances (rev. 4)
by Imre Farkas 12 Jun '12

12 Jun '12

From: Imre Farkas <ifarkas(a)redhat.com> https://bugzilla.redhat.com/show_bug.cgi?id=786535 Rebased and autoupdate moved to mustache --- src/app/controllers/deployments_controller.rb | 2 +- src/app/helpers/mustache_helper.rb | 1 + src/app/models/deployment.rb | 2 +- src/app/util/taskomatic.rb | 8 +++++ src/app/views/deployments/_alerts_show.html.haml | 24 +++++--------- .../deployments/_deployment_alerts.html.mustache | 34 ++++++++++++++++++++ src/public/javascripts/backbone/views.js | 33 +++++++++++++++++++ src/spec/models/deployment_spec.rb | 2 + 8 files changed, 88 insertions(+), 18 deletions(-) create mode 100644 src/app/views/deployments/_deployment_alerts.html.mustache diff --git a/src/app/controllers/deployments_controller.rb b/src/app/controllers/deployments_controller.rb index 38be20a..3acab91 100644 --- a/src/app/controllers/deployments_controller.rb +++ b/src/app/controllers/deployments_controller.rb @@ -167,7 +167,7 @@ class DeploymentsController < ApplicationController require_privilege(Privilege::VIEW, @deployment) init_new_deployment_attrs save_breadcrumb(deployment_path(@deployment, :viewstate => viewstate_id), @deployment.name) - @failed_instances = @deployment.failed_instances + @failed_instances = @deployment.failed_instances.list(sort_column(Instance), sort_direction) if filter_view? @view = 'instances/list' params[:instances_preset_filter] = "" unless params[:instances_preset_filter] diff --git a/src/app/helpers/mustache_helper.rb b/src/app/helpers/mustache_helper.rb index 78a663e..fabe8e4 100644 --- a/src/app/helpers/mustache_helper.rb +++ b/src/app/helpers/mustache_helper.rb @@ -51,6 +51,7 @@ module MustacheHelper :translated_state => t("instances.states.#{instance.state}"), :public_addresses => instance.public_addresses.present? ? instance.public_addresses : I18n.t('deployments.pretty_view_show.no_ip_address'), :instance_key_present => instance.instance_key.present?, + :last_error => instance.last_error, :stop_enabled => available_actions.include?(InstanceTask::ACTION_STOP), :reboot_enabled => available_actions.include?(InstanceTask::ACTION_REBOOT), :owner => instance.owner.present? ? instance.owner.name : nil, diff --git a/src/app/models/deployment.rb b/src/app/models/deployment.rb index 78af946..77d4c70 100644 --- a/src/app/models/deployment.rb +++ b/src/app/models/deployment.rb @@ -472,7 +472,7 @@ class Deployment < ActiveRecord::Base end def failed_instances - instances.select {|instance| instance.failed?} + instances.failed end def update_state(changed_instance) diff --git a/src/app/util/taskomatic.rb b/src/app/util/taskomatic.rb index a7181dc..463f521 100644 --- a/src/app/util/taskomatic.rb +++ b/src/app/util/taskomatic.rb @@ -73,6 +73,14 @@ module Taskomatic rescue Exception => ex task.state = Task::STATE_FAILED task.message = ex.message + task.instance.update_attributes(:last_error => ex.message) + + # For RHEV-M, since we need to start up the instance after the vm has been created + # we also have to handle create_failed state events separately + if task.instance.state == Instance::STATE_STOPPED && task.action == InstanceTask::ACTION_START && + task.instance.provider_account.provider.provider_type.deltacloud_driver == 'rhevm' + task.instance.update_attributes(:state => Instance::STATE_CREATE_FAILED) + end ensure task.save! if Task.exists?(task.id) end diff --git a/src/app/views/deployments/_alerts_show.html.haml b/src/app/views/deployments/_alerts_show.html.haml index 6742e45..4d2c9de 100644 --- a/src/app/views/deployments/_alerts_show.html.haml +++ b/src/app/views/deployments/_alerts_show.html.haml @@ -1,16 +1,8 @@ -- if @failed_instances.count > 0 - %section.content-section.alerts.collapse_entity - %header - .section-controls - = link_to t("providers.edit.toggle_alerts"), "#", :class => 'collapse alerts' - %h2.alerts - =t 'alerts_label' - %span.label.badge.alert.count= @failed_instances.count - .content.collapsible - %dl.alerts - %ul - - @failed_instances.each do |inst| - %li.alert - %dt.subject.critical= inst.name - %dd.type=t 'alerts.instance_failure' - %dd.desc= "#{inst.last_error.blank? ? inst.state : inst.last_error}" +#deployment-alerts + = render :partial => 'deployment_alerts', + :mustache => { :failed_instance_exists => @failed_instances.count > 0, + :failed_instances_count => @failed_instances.count, + :failed_instances => @failed_instances.map{ |instance| instance_for_mustache(instance) } } + +%script#deploymentAlertTemplate{ :type => 'text/html' } + = render :partial => 'deployment_alerts' diff --git a/src/app/views/deployments/_deployment_alerts.html.mustache b/src/app/views/deployments/_deployment_alerts.html.mustache new file mode 100644 index 0000000..0be2690 --- /dev/null +++ b/src/app/views/deployments/_deployment_alerts.html.mustache @@ -0,0 +1,34 @@ +{{#failed_instance_exists}} + <section class="content-section alerts collapse_entity"> + <header> + <div class="section-controls"> + <%= link_to(t('providers.edit.toggle_alerts'), '#', :class => 'collapse alerts') %> + </div> + <h2 class="alerts"> + <%= t('alerts_label') %> + <span class="label badge alert count">{{failed_instances_count}}</span> + </h2> + </header> + <div class="content collapsible" style="display: block;"> + <dl class="alerts"> + <ul> + {{#failed_instances}} + <li class="alert"> + <dt class="subject critical">{{name}}</dt> + <dd class="type"><%= t('alerts.instance_failure') %></dd> + <dd class="desc"> + {{#last_error}} + {{last_error}} + {{/last_error}} + + {{^last_error}} + {{translated_state}} + {{/last_error}} + </dd> + </li> + {{/failed_instances}} + </ul> + </dl> + </div> + </section> +{{/failed_instance_exists}} diff --git a/src/public/javascripts/backbone/views.js b/src/public/javascripts/backbone/views.js index e3cf278..60b9de2 100644 --- a/src/public/javascripts/backbone/views.js +++ b/src/public/javascripts/backbone/views.js @@ -278,6 +278,39 @@ Conductor.Views.DeploymentsShow = Backbone.View.extend({ instanceCardsHtml += Mustache.to_html($template.html(), instance); }); $instances.html(instanceCardsHtml); + + var $alerts = this.$('#deployment-alerts'); + var alertsVisible = $alerts.find('.collapsible').length == 0 || $alerts.find('.collapsible').is(":visible") + var failedInstances = _.compact(_.map(this.collection.models, function(model) { + if(model.get('failed')) { + return model.toJSON(); + } + else { + return null; + } + })); + + // Render alerts + $alerts.empty(); + if(failedInstances.length == 0) return; + + var $deploymentAlertTemplate = $('#deploymentAlertTemplate'); + $alerts.html(Mustache.to_html($deploymentAlertTemplate.html(), { + 'failed_instance_exists' : failedInstances.length > 0, + 'failed_instances_count' : failedInstances.length, + 'failed_instances' : failedInstances + })); + + // Add callback to Toggle link + $alerts.find('a.collapse').click(function(e) { + e.preventDefault(); + $(this).parents('.collapse_entity').find('.collapsible').slideToggle(80); + }); + + // Restore toggle state on alerts table + if(!alertsVisible) { + $alerts.find('.collapsible').hide(); + } } }); diff --git a/src/spec/models/deployment_spec.rb b/src/spec/models/deployment_spec.rb index 7cf64f2..cc5cced 100644 --- a/src/spec/models/deployment_spec.rb +++ b/src/spec/models/deployment_spec.rb @@ -337,6 +337,8 @@ describe Deployment do @deployment.reload @deployment.stop_instances_and_destroy! + inst1.reload + inst2.reload # this emulates Condor stopping the actual instances # and dbomatic reflecting the changes back to Conductor -- 1.7.6.5

2 1

[PATCH conductor] BZ808393: Fix invalid name error message
by Imre Farkas 12 Jun '12

12 Jun '12

From: Imre Farkas <ifarkas(a)redhat.com> https://bugzilla.redhat.com/show_bug.cgi?id=808393 --- src/app/models/pool.rb | 2 +- src/app/models/pool_family.rb | 2 +- src/app/models/provider.rb | 2 +- src/config/locales/activerecord/en.yml | 14 ++++++++++++++ src/config/locales/activerecord/fr.yml | 14 ++++++++++++++ src/config/locales/activerecord/ja.yml | 14 ++++++++++++++ src/config/locales/en.yml | 1 - src/config/locales/fr.yml | 1 - src/config/locales/ja.yml | 1 - 9 files changed, 45 insertions(+), 6 deletions(-) diff --git a/src/app/models/pool.rb b/src/app/models/pool.rb index 7b6cf89..177de06 100644 --- a/src/app/models/pool.rb +++ b/src/app/models/pool.rb @@ -55,7 +55,7 @@ class Pool < ActiveRecord::Base validates_uniqueness_of :exported_as, :if => :exported_as validates_length_of :name, :maximum => 255 - validates_format_of :name, :with => /^[\w -]*$/n, :message => I18n.t('pools.valid_format_of_name') + validates_format_of :name, :with => /^[\w -]*$/n has_many :permissions, :as => :permission_object, :dependent => :destroy, :include => [:role], diff --git a/src/app/models/pool_family.rb b/src/app/models/pool_family.rb index 8f5ea24..a12068f 100644 --- a/src/app/models/pool_family.rb +++ b/src/app/models/pool_family.rb @@ -45,7 +45,7 @@ class PoolFamily < ActiveRecord::Base has_many :permissions, :as => :permission_object, :dependent => :destroy validates_length_of :name, :maximum => 255 - validates_format_of :name, :with => /^[\w -]*$/n, :message => "must only contain: numbers, letters, spaces, '_' and '-'" + validates_format_of :name, :with => /^[\w -]*$/n validates_presence_of :name validates_uniqueness_of :name diff --git a/src/app/models/provider.rb b/src/app/models/provider.rb index 2033a04..bb4b8a6 100644 --- a/src/app/models/provider.rb +++ b/src/app/models/provider.rb @@ -49,7 +49,7 @@ class Provider < ActiveRecord::Base validates_presence_of :provider_type_id validates_presence_of :url - validates_format_of :name, :with => /^[\w -]*$/n, :message => "must only contain: numbers, letters, spaces, '_' and '-'" + validates_format_of :name, :with => /^[\w -]*$/n validates_length_of :name, :maximum => 255 validate :validate_provider diff --git a/src/config/locales/activerecord/en.yml b/src/config/locales/activerecord/en.yml index 3ac086f..6da4618 100644 --- a/src/config/locales/activerecord/en.yml +++ b/src/config/locales/activerecord/en.yml @@ -343,3 +343,17 @@ en: state: "State" created_at: "Created At" updated_at: "Updated At" + errors: + models: + pool: + attributes: + name: + invalid: "must only contain: numbers, letters, spaces, '_' and '-'" + pool_family: + attributes: + name: + invalid: "must only contain: numbers, letters, spaces, '_' and '-'" + provider: + attributes: + name: + invalid: "must only contain: numbers, letters, spaces, '_' and '-'" diff --git a/src/config/locales/activerecord/fr.yml b/src/config/locales/activerecord/fr.yml index ad6905d..86b4459 100644 --- a/src/config/locales/activerecord/fr.yml +++ b/src/config/locales/activerecord/fr.yml @@ -343,3 +343,17 @@ fr: task: Tâche user: Utilisateur view_state: "Afficher l'état" + errors: + models: + pool: + attributes: + name: + invalid: "doit uniquement contenir : nombres, lettres, espaces, '_' et '-'" + pool_family: + attributes: + name: + invalid: "doit uniquement contenir : nombres, lettres, espaces, '_' et '-'" + provider: + attributes: + name: + invalid: "doit uniquement contenir : nombres, lettres, espaces, '_' et '-'" diff --git a/src/config/locales/activerecord/ja.yml b/src/config/locales/activerecord/ja.yml index 13878c3..9cfef92 100644 --- a/src/config/locales/activerecord/ja.yml +++ b/src/config/locales/activerecord/ja.yml @@ -343,3 +343,17 @@ ja: task: タスク user: ユーザー view_state: 表示状態 + errors: + models: + pool: + attributes: + name: + invalid: 数字、文字、空白、「_」、「-」以外は含ませないようにしてください。 + pool_family: + attributes: + name: + invalid: 数字、文字、空白、「_」、「-」以外は含ませないようにしてください。 + provider: + attributes: + name: + invalid: 数字、文字、空白、「_」、「-」以外は含ませないようにしてください。 diff --git a/src/config/locales/en.yml b/src/config/locales/en.yml index d7a2b75..861f30d 100644 --- a/src/config/locales/en.yml +++ b/src/config/locales/en.yml @@ -461,7 +461,6 @@ en: iwhd_unavailable: Catalog Images are unavailable. It appears the image warehouse is not reachable warning: creation_failed: Pool creation failed. - valid_format_of_name: "must only contain: numbers, letters, spaces, '_' and '-'" list: more_actions: More actions form: diff --git a/src/config/locales/fr.yml b/src/config/locales/fr.yml index 3f787c5..1dafc83 100644 --- a/src/config/locales/fr.yml +++ b/src/config/locales/fr.yml @@ -870,7 +870,6 @@ fr: with_stopped_instances: "Avec des instances arrêtées" properties: properties: "Propriétés de %{pool}" - valid_format_of_name: "doit uniquement contenir : nombres, lettres, espaces, '_' et '-'" privileges: Privilèges properties: Propriétés provider_accounts: diff --git a/src/config/locales/ja.yml b/src/config/locales/ja.yml index d834b4e..efd088c 100644 --- a/src/config/locales/ja.yml +++ b/src/config/locales/ja.yml @@ -870,7 +870,6 @@ ja: with_stopped_instances: 停止インスタンスあり properties: properties: "%{pool} のプロパティ" - valid_format_of_name: 数字、文字、空白、「_」、「-」以外は含ませないようにしてください。 privileges: 特権 properties: プロパティ provider_accounts: -- 1.7.6.5

2 2

RFC: Feature Planning - Design the model changes for support of stateful instances
by jzigmund＠redhat.com 08 Jun '12

08 Jun '12

Hi, I wrote up current status of instance states, and described a proposal how we could implement this feature. Rather than paste in the entire text, I'm just linking to the wiki writeup. https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Design_the_model… Any thoughts or improvement ideas are welcomed. -- Jozef

3 3

[PULL configure] Puppet 2.7 support
by John Eckersberg 05 Jun '12

05 Jun '12

I'm sending this as a pull request[1] instead of patches, as this is a large changeset and the probability of something getting lost or mangled is high. Fedora 17 is now using Puppet 2.7. This corrects some compatibility issues with configure so it will work with Puppet 2.7. These changes are also backwards compatible with Puppet 2.6, so this will work for distros that do not use 2.7 yet (e.g. Fedora 16). Note that you will need the other configure patchset that I just sent to the list ("Various test and rake fixes for F17 compatibility") if you expect to be able to run `rake rpms` after applying this in order to test. To apply this to a branch, just run: git pull git://github.com/jeckersb/aeolus-configure.git puppet-2.7 [1] See http://git-scm.com/docs/git-request-pull --- The following changes since commit 258d5c4e88950d48c5a4bef17e3b084c1beabb1d: RM3172 Allow standalone imagefactory/iwhd installation (2012-05-09 10:56:19 -0500) are available in the git repository at: git://github.com/jeckersb/aeolus-configure.git puppet-2.7 John Eckersberg (2): Fix module structure for puppet 2.7 autoloader Fix broken string interpolation recipes/aeolus/manifests/conductor.pp | 191 +------------------- recipes/aeolus/manifests/conductor/disabled.pp | 39 ++++ recipes/aeolus/manifests/conductor/hwp.pp | 24 +++ recipes/aeolus/manifests/conductor/login.pp | 19 ++ recipes/aeolus/manifests/conductor/logout.pp | 12 ++ recipes/aeolus/manifests/conductor/provider.pp | 18 ++ .../aeolus/manifests/conductor/provider/account.pp | 43 +++++ recipes/aeolus/manifests/conductor/site_admin.pp | 17 ++ recipes/aeolus/manifests/create_bucket.pp | 7 + recipes/aeolus/manifests/deltacloud.pp | 71 -------- recipes/aeolus/manifests/deltacloud/core.pp | 28 +++ recipes/aeolus/manifests/deltacloud/disabled.pp | 18 ++ recipes/aeolus/manifests/deltacloud/ec2.pp | 7 + recipes/aeolus/manifests/image-factory.pp | 35 ---- recipes/aeolus/manifests/image-factory/disabled.pp | 13 ++ recipes/aeolus/manifests/image.pp | 22 +++ recipes/aeolus/manifests/init.pp | 13 -- recipes/aeolus/manifests/iwhd.pp | 32 ---- recipes/aeolus/manifests/iwhd/disabled.pp | 22 +++ recipes/aeolus/manifests/profiles/rhevm.pp | 9 - .../aeolus/manifests/profiles/rhevm/disabled.pp | 3 + recipes/aeolus/manifests/rails.pp | 58 ------ recipes/aeolus/manifests/rails/create/db.pp | 8 + recipes/aeolus/manifests/rails/drop/db.pp | 8 + recipes/aeolus/manifests/rails/migrate/db.pp | 7 + recipes/aeolus/manifests/rails/seed/db.pp | 15 ++ recipes/aeolus/manifests/rhevm/validate.pp | 6 + recipes/aeolus/manifests/selinux.pp | 27 --- recipes/aeolus/manifests/selinux/mode.pp | 10 + recipes/apache/manifests/init.pp | 13 -- recipes/apache/manifests/site.pp | 13 ++ recipes/ntp/manifests/client.pp | 33 ++++ recipes/ntp/manifests/init.pp | 35 ---- recipes/ntp/manifests/server.pp | 2 + recipes/openssl/manifests/certificate.pp | 12 ++ recipes/openssl/manifests/init.pp | 33 ---- recipes/openssl/manifests/key.pp | 19 ++ recipes/postgres/manifests/client.pp | 2 + recipes/postgres/manifests/init.pp | 51 ------ recipes/postgres/manifests/server.pp | 35 ++++ recipes/postgres/manifests/user.pp | 14 ++ 41 files changed, 482 insertions(+), 562 deletions(-) create mode 100644 recipes/aeolus/manifests/conductor/disabled.pp create mode 100644 recipes/aeolus/manifests/conductor/hwp.pp create mode 100644 recipes/aeolus/manifests/conductor/login.pp create mode 100644 recipes/aeolus/manifests/conductor/logout.pp create mode 100644 recipes/aeolus/manifests/conductor/provider.pp create mode 100644 recipes/aeolus/manifests/conductor/provider/account.pp create mode 100644 recipes/aeolus/manifests/conductor/site_admin.pp create mode 100644 recipes/aeolus/manifests/create_bucket.pp delete mode 100644 recipes/aeolus/manifests/deltacloud.pp create mode 100644 recipes/aeolus/manifests/deltacloud/core.pp create mode 100644 recipes/aeolus/manifests/deltacloud/disabled.pp create mode 100644 recipes/aeolus/manifests/deltacloud/ec2.pp create mode 100644 recipes/aeolus/manifests/image-factory/disabled.pp create mode 100644 recipes/aeolus/manifests/image.pp create mode 100644 recipes/aeolus/manifests/iwhd/disabled.pp create mode 100644 recipes/aeolus/manifests/profiles/rhevm/disabled.pp delete mode 100644 recipes/aeolus/manifests/rails.pp create mode 100644 recipes/aeolus/manifests/rails/create/db.pp create mode 100644 recipes/aeolus/manifests/rails/drop/db.pp create mode 100644 recipes/aeolus/manifests/rails/migrate/db.pp create mode 100644 recipes/aeolus/manifests/rails/seed/db.pp create mode 100644 recipes/aeolus/manifests/rhevm/validate.pp delete mode 100644 recipes/aeolus/manifests/selinux.pp create mode 100644 recipes/aeolus/manifests/selinux/mode.pp create mode 100644 recipes/apache/manifests/site.pp create mode 100644 recipes/ntp/manifests/client.pp create mode 100644 recipes/ntp/manifests/server.pp create mode 100644 recipes/openssl/manifests/certificate.pp create mode 100644 recipes/openssl/manifests/key.pp create mode 100644 recipes/postgres/manifests/client.pp create mode 100644 recipes/postgres/manifests/server.pp create mode 100644 recipes/postgres/manifests/user.pp

3 6

Sanitize use of details_tab param
by Samridh 05 Jun '12

05 Jun '12

The details_tab param in provider_accounts_controller::show method has been sanitized curently with only the 'properties' tab value. But it has been done such that it is consistent with the sanitization of the param in other controllers of the conductor. This will ensure wasy changes as and when needed in future.

2 2

[PATCH configure] Various test and rake fixes for F17 compatibility
by John Eckersberg 01 Jun '12

01 Jun '12

This series corrects a number of issues that break rake invocation under F17. Note that the tests were completely broken prior to this; they're still completely broken after this, but they at least will run. We'll need to address that in the future.

2 5

[PATCH conductor 1/2] rufus scheduler integration - RM 3273
by Richard Su 01 Jun '12

01 Jun '12

https://www.aeolusproject.org/redmine/issues/3273 REST api to turn the rufus scheduler on (:new) and off (:destroy) and to check its status (:index). A singleton class JobScheduler encapsulates the rufus scheduler so that only a single scheduler instance runs at any time. Only users who have the ROLE view and modify permissions may control the scheduler. --- aeolus-conductor.spec.in | 2 + src/app/controllers/job_scheduler_controller.rb | 45 +++++++++ src/app/views/job_scheduler/destroy.xml.haml | 2 + src/app/views/job_scheduler/index.xml.haml | 2 + src/app/views/job_scheduler/new.xml.haml | 2 + src/config/routes.rb | 2 + src/lib/job_scheduler.rb | 94 ++++++++++++++++++++ .../controllers/job_scheduler_controller_spec.rb | 84 +++++++++++++++++ 8 files changed, 233 insertions(+), 0 deletions(-) create mode 100644 src/app/controllers/job_scheduler_controller.rb create mode 100644 src/app/views/job_scheduler/destroy.xml.haml create mode 100644 src/app/views/job_scheduler/index.xml.haml create mode 100644 src/app/views/job_scheduler/new.xml.haml create mode 100644 src/lib/job_scheduler.rb create mode 100644 src/spec/controllers/job_scheduler_controller_spec.rb diff --git a/aeolus-conductor.spec.in b/aeolus-conductor.spec.in index a40d517..6cca32f 100644 --- a/aeolus-conductor.spec.in +++ b/aeolus-conductor.spec.in @@ -47,6 +47,7 @@ Requires: rubygem(oauth) Requires: rubygem(rake) Requires: rubygem(delayed_job) >= 2.1.4 Requires: rubygem(paranoia) +Requires: rubygem(rufus-scheduler) Requires: postgresql Requires: postgresql-server Requires: system-logos @@ -189,6 +190,7 @@ haml="app/views/hardware_profiles app/views/realm_mappings \ app/views/deployables app/views/catalogs \ app/views/provider_types app/views/api/images \ app/views/config_servers app/views/provider_realms \ + app/views/job_scheduler \ app/views/logs \ app/views/api/hooks \ app/views/api \ diff --git a/src/app/controllers/job_scheduler_controller.rb b/src/app/controllers/job_scheduler_controller.rb new file mode 100644 index 0000000..b1a2701 --- /dev/null +++ b/src/app/controllers/job_scheduler_controller.rb @@ -0,0 +1,45 @@ +# +# Copyright 2012 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'job_scheduler' + +class JobSchedulerController < ApplicationController + before_filter :require_user_api + respond_to :xml + layout :false + + # GET /jobscheduler + # GET /jobscheduler.xml + def index + require_privilege(Privilege::PERM_VIEW) + respond_with(JobScheduler.activated?) + end + + # GET /jobscheduler/new + # GET /jobscheduler/new.xml + def new + require_privilege(Privilege::PERM_SET) + respond_with(JobScheduler.start) + end + + # DELETE /jobscheduler/1 + # DELETE /jobscheduler/1.xml + def destroy + require_privilege(Privilege::PERM_SET) + respond_with(JobScheduler.stop) + end + +end diff --git a/src/app/views/job_scheduler/destroy.xml.haml b/src/app/views/job_scheduler/destroy.xml.haml new file mode 100644 index 0000000..18e796d --- /dev/null +++ b/src/app/views/job_scheduler/destroy.xml.haml @@ -0,0 +1,2 @@ +!!! XML +%activated=JobScheduler.activated? \ No newline at end of file diff --git a/src/app/views/job_scheduler/index.xml.haml b/src/app/views/job_scheduler/index.xml.haml new file mode 100644 index 0000000..b5c75d6 --- /dev/null +++ b/src/app/views/job_scheduler/index.xml.haml @@ -0,0 +1,2 @@ +!!! XML +%activated=JobScheduler.activated? diff --git a/src/app/views/job_scheduler/new.xml.haml b/src/app/views/job_scheduler/new.xml.haml new file mode 100644 index 0000000..18e796d --- /dev/null +++ b/src/app/views/job_scheduler/new.xml.haml @@ -0,0 +1,2 @@ +!!! XML +%activated=JobScheduler.activated? \ No newline at end of file diff --git a/src/config/routes.rb b/src/config/routes.rb index 265f752..518b013 100644 --- a/src/config/routes.rb +++ b/src/config/routes.rb @@ -238,6 +238,8 @@ Conductor::Application.routes.draw do resources :provider_images end + resources :job_scheduler, :only => [:index, :new, :destroy] + get 'api', :controller => 'api/entrypoint', :action => 'index' namespace :api do resources :images do diff --git a/src/lib/job_scheduler.rb b/src/lib/job_scheduler.rb new file mode 100644 index 0000000..1050ecd --- /dev/null +++ b/src/lib/job_scheduler.rb @@ -0,0 +1,94 @@ +# +# Copyright 2012 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'curb' +require 'rufus-scheduler' + +class JobScheduler + + @@scheduler = nil + + def self.initialize + self.start + end + + def self.start + if @@scheduler.nil? + @@scheduler = Rufus::Scheduler::PlainScheduler.start_new + schedule_tasks(@@scheduler) + end + end + + def self.stop + if @@scheduler + @@scheduler.stop + @@scheduler = nil + end + end + + def self.activated? + if @@scheduler + true + else + false + end + end + + def self.status_via_http(login, password, hostname) + curl = self.curl_request(login, password) + curl.url = job_scheduler_url(hostname, '') + curl.http_get + curl.body_str.to_s + end + + def self.start_via_http(login, password, hostname) + curl = self.curl_request(login, password) + curl.url = job_scheduler_url(hostname, 'new') + curl.http_get + curl.body_str.to_s + end + + def self.stop_via_http(login, password, hostname) + curl = self.curl_request(login, password) + curl.url = job_scheduler_url(hostname, '1') + curl.http_delete + curl.body_str.to_s + end + + private + + def self.schedule_tasks(scheduler) + scheduler.every("1m") do + puts "job scheduler is running" + end + end + + def self.job_scheduler_url(hostname, suffix) + # TODO: is there a better way to generate this url? + url = "https://#{hostname}/conductor/#{JobSchedulerController.controller_path}/#{suffix}" + end + + def self.curl_request(login, password) + curl = Curl::Easy.new + curl.username = login + curl.password = password + curl.headers['HTTP_ACCEPT'] = 'application/xml' + curl.headers['CONTENT_TYPE'] = 'application/xml' + curl.http_auth_types = :basic + return curl + end + +end diff --git a/src/spec/controllers/job_scheduler_controller_spec.rb b/src/spec/controllers/job_scheduler_controller_spec.rb new file mode 100644 index 0000000..75726ec --- /dev/null +++ b/src/spec/controllers/job_scheduler_controller_spec.rb @@ -0,0 +1,84 @@ +# +# Copyright 2012 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' + +describe JobSchedulerController do + render_views + + context "when authenticated as admin" do + + before(:each) do + @admin_permission = FactoryGirl.create :admin_permission + @admin = @admin_permission.user + mock_warden(@admin) + end + + describe "scheduler controls" do + + it "scheduler status from 'GET index'" do + send_and_accept_xml + get :index + response.should be_success + response.body.strip.should == "<activated>false</activated>" + end + + it "turn on scheduler by 'GET new'" do + send_and_accept_xml + get :new + response.should be_success + response.body.strip.should == "<activated>true</activated>" + end + + it "turn off scheduler by 'POST destroy'" do + send_and_accept_xml + post :destroy, :id => 1 + response.should be_success + response.body.strip.should == "<activated>false</activated>" + end + + it "turn scheduler on and then off" do + send_and_accept_xml + get :index + response.should be_success + response.body.strip.should == "<activated>false</activated>" + get :new + response.should be_success + response.body.strip.should == "<activated>true</activated>" + post :destroy, :id => 1 + response.should be_success + response.body.strip.should == "<activated>false</activated>" + end + end + end + + context "when not authenticated as admin" do + + before(:each) do + @non_admin_permission = FactoryGirl.create :pool_user_permission + @user = @non_admin_permission.user + mock_warden(@user) + end + + it "requests should be denied" do + send_and_accept_xml + get :index + response.should_not be_success + end + + end + +end -- 1.7.7.6

2 2

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

aeolus-devel May 2012