rhq-users January 2014

rhq-users@lists.stg.fedorahosted.org

13 participants
16 discussions

monitoring open http sessions on a jboss server
by Attila Heidrich 18 Jan '18

18 Jan '18

Hi, is there a way to monitor the open sessions of a jboss server? I see I have some metrics for the http connector, and the requests per second value is really useful, but we need some more detailed statistics. It would be the best to have some browser to select the parameters... ;) for webapp dveelopers it would be such a great benefit! Afaik, there're some internal values in the jboss server, should I extend the JBoss plugin to get a value, which isn't got now? Attila

4 11

Load factor
by Elmar Knipp 11 Mar '14

11 Mar '14

Hi all, Is there a metric which is similar to the unix load factor, which you see e.g. with "uptime" or "top"? I see "user load" and "system load", but not a combined metric. I want to be alerted when a system is sweating ;-) Thanks, Elmar

3 2

false alerts on process resources
by Attila Heidrich 30 Jan '14

30 Jan '14

Hi! RHQ 4.9 Sometimes we have got "status down" alerts of process resources. The processes haven't got any problems meanwhile, and we don't have the false positives for each and every process on the same platform. The processes are mainly "vboxheadless something"-like processes - at lease we only have problems with these. The alerts are cleared very soon, but sthey are till really annoying. Anyone having the same problem, or preferably a solution? Regards, Attila

3 4

jboss resource: no alert clearing
by Attila Heidrich 30 Jan '14

30 Jan '14

Please take a look at the screenshots here: https://www.dropbox.com/sh/heyowe3rc145ivd/_tlWMpMG3F The resource is available, as it is shown in the timeline, but still there are live alerts. any idea, what is wrong? This is exactly the same as two days before, even the TOD is the same. I have found something at the given time in the Cassandra log: INFO [ScheduledTasks:1] 2014-01-30 04:10:19,381 MessagingService.java (line 860) 1 READ messages dropped in last 5000ms INFO [ScheduledTasks:1] 2014-01-30 04:10:19,381 StatusLogger.java (line 53) Pool Name Active Pending Completed Blocked All Time Blocked INFO [ScheduledTasks:1] 2014-01-30 04:10:19,382 StatusLogger.java (line 68) ReadStage 0 0 23386785 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,382 StatusLogger.java (line 68) RequestResponseStage 0 0 43148496 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,383 StatusLogger.java (line 68) ReadRepairStage 0 0 1722951 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,383 StatusLogger.java (line 68) MutationStage 0 0 76756647 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,383 StatusLogger.java (line 68) ReplicateOnWriteStage 0 0 0 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,383 StatusLogger.java (line 68) GossipStage 0 0 11364876 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,384 StatusLogger.java (line 68) AntiEntropyStage 0 0 69120 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,384 StatusLogger.java (line 68) MigrationStage 0 0 0 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,384 StatusLogger.java (line 68) MemtablePostFlusher 0 0 84294 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,384 StatusLogger.java (line 68) FlushWriter 0 0 1277 0 3 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,385 StatusLogger.java (line 68) MiscStage 0 0 201 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,385 StatusLogger.java (line 68) commitlog_archiver 0 0 0 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,385 StatusLogger.java (line 68) InternalResponseStage 0 0 0 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,386 StatusLogger.java (line 68) AntiEntropySessions 0 0 2048 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,386 StatusLogger.java (line 68) HintedHandoff 0 0 5 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,386 StatusLogger.java (line 78) CompactionManager 0 0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,386 StatusLogger.java (line 90) MessagingService n/a 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,387 StatusLogger.java (line 100) Cache Type Size Capacity KeysToSave Provider INFO [ScheduledTasks:1] 2014-01-30 04:10:19,387 StatusLogger.java (line 101) KeyCache 38797256 38797312 all INFO [ScheduledTasks:1] 2014-01-30 04:10:19,387 StatusLogger.java (line 107) RowCache 0 0 all org.apache.cassandra.cache.SerializingCacheProvider INFO [ScheduledTasks:1] 2014-01-30 04:10:19,387 StatusLogger.java (line 114) ColumnFamily Memtable ops,data INFO [ScheduledTasks:1] 2014-01-30 04:10:19,387 StatusLogger.java (line 117) system.local 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,387 StatusLogger.java (line 117) system.peers 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,388 StatusLogger.java (line 117) system.batchlog 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,388 StatusLogger.java (line 117) system.NodeIdInfo 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,388 StatusLogger.java (line 117) system.LocationInfo 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,388 StatusLogger.java (line 117) system.Schema 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,388 StatusLogger.java (line 117) system.Migrations 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,388 StatusLogger.java (line 117) system.schema_keyspaces 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,389 StatusLogger.java (line 117) system.schema_columns 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,389 StatusLogger.java (line 117) system.schema_columnfamilies 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,389 StatusLogger.java (line 117) system.IndexInfo 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,389 StatusLogger.java (line 117) system.range_xfers 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,389 StatusLogger.java (line 117) system.peer_events 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,389 StatusLogger.java (line 117) system.hints 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,390 StatusLogger.java (line 117) system.HintsColumnFamily 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,390 StatusLogger.java (line 117) rhq.twenty_four_hour_metrics 139188,2097152 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,390 StatusLogger.java (line 117) rhq.raw_metrics 870881,14680064 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,390 StatusLogger.java (line 117) rhq.six_hour_metrics 487836,13631488 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,390 StatusLogger.java (line 117) rhq.metrics_index 660267,4194304 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,390 StatusLogger.java (line 117) rhq.schema_version 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,390 StatusLogger.java (line 117) rhq.one_hour_metrics 163941,5242880 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,391 StatusLogger.java (line 117) system_auth.users 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,391 StatusLogger.java (line 117) system_auth.credentials 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,391 StatusLogger.java (line 117) system_auth.permissions 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,391 StatusLogger.java (line 117) system_traces.sessions 0,0 INFO [ScheduledTasks:1] 2014-01-30 04:10:19,391 StatusLogger.java (line 117) system_traces.events 0,0 Similar happens sometimes, but doesn't look to be stronglyrelated to my problem, since there is no such info from 28-01. /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-02 04:09:21,020 MessagingService.java (line 860) 3 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-06 04:04:16,964 MessagingService.java (line 860) 1 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-07 04:11:18,930 MessagingService.java (line 860) 2 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-14 04:11:49,388 MessagingService.java (line 860) 2 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-14 04:15:19,999 MessagingService.java (line 860) 1 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-21 04:07:32,958 MessagingService.java (line 860) 1 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-23 04:06:56,011 MessagingService.java (line 860) 3 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-23 04:10:53,667 MessagingService.java (line 860) 2 READ messages dropped in last 5000ms /opt/rhq-server-4.9.0/logs/rhq-storage.log: INFO [ScheduledTasks:1] 2014-01-30 04:10:19,381 MessagingService.java (line 860) 1 READ messages dropped in last 5000ms Any idea? Last time I had to restart the agent. Then I have received the clearing message two times. Regards, Attila

2 1

no Alert clearing mail
by Attila Heidrich 28 Jan '14

28 Jan '14

Hi! RHQ 4.9... Sometimes the "clearing alert" doesn't happen, but the alert status is cleared. I have an uncleared alert, while the resource status is OK. In fact, the timeline shows the jboss server unknown, while the status is OK, and the agent log shows the periodic updates even for the recent minutes. Cannot find any anomaly. Other resources on the same Platform seems to work well. Where to look for some traces of "WTF happened"? :) Attila

1 1

RHQ CLI - Obfuscating user/password data
by Andy Overton 27 Jan '14

27 Jan '14

Hi, Does anyone know if it's possible to use obfuscated username/password values to connect to RHQ via the CLI? Seems like an obvious thing to do but from the docs it looks like you can either pass in the username/pass in plain text or get it to prompt you for the password but no option to use obfuscated values. So, I can do: rhq-cli.sh -u rhqadmin -p rhqadmin rhq-cli.sh -u rhqadmin -P but I need to do rhq-cli.sh -u H*%@SDAFSDF -p H*%@SDAFSDF or similar. The problem is this needs to run as a cron job so I can't prompt for the password but don't want it stored as plain text. Thanks, Andy

3 3

RHQ state of the union January 2014
by Heiko W.Rupp 23 Jan '14

23 Jan '14

[ This is the first of a series of postings describing where we stand with RHQ and where we may want to go to. ] As many of you already know, RHQ is the upstream of JBoss Operations Network (JBoss ON) and so RHQ is following the requirements of JBoss ON. RHQ has released 4.9 in September 2013 and is due for another release. That next release will include some performance enhancments on inventory syncing and agent footprint as well as a lot of fixes since RHQ 4.9. This next release is supposed to happen within the next month. The RHQ team recently had a face to face meeting in Brno at the same time that the WildFly team was there as well. There are a few photos available ([1] and [2]) but we missed to create an all-team photo. Some of the outcomes of that meeting are: • RHQ source code will definitively be moved from FedoraHosted to GitHub in the near future. • RHQ will in the future use the JIRA instance at jboss.org instead of Bugzilla (timing unknown at this point). Those two actions will make it easier to contribute in the future when you already have an account at jboss.org From a technical standpoint, we will tackle the following in the near to mid term - please comment on the respective wiki pages: • Audit Subsystem [3] • "Canned DynaGroup expressions" [4] • Add support for https connections of the as7-plugin • Add support Bundles and Drift for the Domain mode of as7/WildFly • Provide a mechanism for out-of-band signaling of state and events from the agent to the server [7] (the current proposal on the wiki is a first draft to illustrate the idea and needs much more work). • Implement signaling of as7/WildFly "dirty" state over the mechanism in the previous item Of course we were also talking about the longer term and here mostly about two items: • UI: The current dependency on the SmartGWT widget set becomes more and more problematic as SmartGWT is not able to keep up with all the browser changes. In addition, the end of life of GWT DevMode is also reducing our ability to further improve the UI. We have thus decided to investigate other options like using Angular.JS for further UI work. We will create a separate UI project on the RHQ GitHub account [5] and will create a UI that can be dropped into RHQ to get more insight how such a Angular.JS driven UI could work for RHQ. This Angular UI will talk via the existing REST-interface to the RHQ server. • Agent: Our current agent is a powerful workhorse, but has its shortcomings. One of those is its runtime footprint that is currently addressed. But even afterwards we want to try to reduce its impact, so that it is easier to run the agent on managed systems. The other part is that the agent currently is only extensible in Java, while operations people are often much more versed in languages like Python. Also the possibility to ad-hoc define some metrics to be evaluated is rather limited (especially when it comes to arbitrary MBean attributes). This means that we will evaluate alternatives to the current agent and also to evaluate how the current agent can be enhanced to support polyglot and smaller runtime environments. In addition to the above we also were talking about enhancing the Alerting subsystem[8] and to introcude a 2-phase-discovery to improve on some of the shortcomings of the current resource discovery [6]. Last but not least we want to thank all contributors for their suggestions, bug reports, enhancement requests and code submissions. Heiko on behalf of the RHQ team Links: [1] https://plus.google.com/114249341487134308671/posts/5nC9PXCDHT8 [2] https://www.facebook.com/media/set/?set=a.250786711757107.1073741829.250589… [3] https://docs.jboss.org/author/display/RHQ/Audit+Subsystem [4] https://docs.jboss.org/author/display/RHQ/Canned+DynaGroup+expressions [5] https://github.com/rhq-project/angular-ui [6] https://docs.jboss.org/author/display/RHQ/2-phase+discovery [7] https://docs.jboss.org/author/display/RHQ/Agent-Server-Backchannel [8] https://docs.jboss.org/author/display/RHQ/Wintermute This one was posted to https://community.jboss.org/en/rhq/blog/2014/01/23/rhq-state-of-the-union-a…

1 0

dependency of resources
by Attila Heidrich 23 Jan '14

23 Jan '14

Hi! I use 4.9... is it possible to define some kind of dependency among the resources? I get all the alerts (processes, network interfaces... one by one) when the server stops. I guess there should be only a single one. Attila

2 1

agent lost for a few minutes
by Attila Heidrich 16 Jan '14

16 Jan '14

Hi! Running on a virtualized Linux, one agent has lost for several minutes. No measurements are present in the servers database, while theres no any sign of problem in the virtualization log, or even in the agent's own log. There is en exception, but the very same happens many time. According the log, it sent the reports regularly for the server: 2014-01-14 10:17:41,788 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementSenderRunner)- Measurement collection for [14] metrics took 94ms - sending report to Server... 2014-01-14 10:18:09,712 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Tue Jan 14 10:18:09 CET 2014 2014-01-14 10:18:09,733 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Tue Jan 14 10:18:09 CET 2014 : Scan [startTime=1389691089712, endTime=1389691089732, runtime=20, isFull=false, isForced=false, numResources=179, numGetAvailabilityCalls=7, numScheduledRandomly=0, numPushedByInterval=6, numAvailabilityChanges=0, numDeferToParent=0] 2014-01-14 10:18:11,788 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementSenderRunner)- Measurement collection for [14] metrics took 67ms - sending report to Server... 2014-01-14 10:18:11,812 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- Starting configuration update check... 2014-01-14 10:18:11,814 INFO [ResourceContainer.invoker.daemon-56351] (org.rhq.plugins.jmx.MBeanResourceComponent)- Configuration management is currently disabled for the ResourceType[id=0, name=java.util.logging, plugin=RHQAgent, category=Service] Resource with key [java.util.logging:type=Logging] - it can be enabled in the Resource's Inventory > Connection tab. 2014-01-14 10:18:12,793 WARN [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- An error occurred while checking for an updated Resource configuration for Resource[id=11763, uuid=74154a3e-07fc-4b0e-88be-f14e95059b8a, type={OpenSSH}SSHD, key=sshd, name=OpenSSH sshd, parent=amweb-se, version=1.0]. net.augeas.AugeasException: get failed at net.augeas.Augeas.processLastCall(Unknown Source) at net.augeas.Augeas.get(Unknown Source) at org.rhq.plugins.sshd.OpenSSHDComponent.getConfig(OpenSSHDComponent.java:189) at org.rhq.plugins.sshd.OpenSSHDComponent.loadResourceConfiguration(OpenSSHDComponent.java:168) at org.rhq.plugins.sshd.OpenSSHDComponent.loadResourceConfiguration(OpenSSHDComponent.java:119) at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:654) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) 2014-01-14 10:18:12,793 WARN [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- An error occurred while checking for an updated Resource configuration for Resource[id=11765, uuid=d9704c88-052e-47e2-8b9e-3495a57bb5c5, type={Apache}Apache HTTP Server, key=/etc/apache2||/etc/apache2/apache2.conf, name=127.0.1.1:80, parent=amweb-se, version=2.2.22]. java.lang.IllegalStateException: Configuration and child resource creation/deletion support for Apache is optional. If you switched it on by enabling Augeas support in the connection settings of the Apache server resource and still get this message, it means that either your Apache version is not supported (only Apache 2.x is supported) or Augeas is not available on your platform. at org.rhq.plugins.apache.ApacheServerComponent.loadResourceConfiguration(ApacheServerComponent.java:369) at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:654) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) 2014-01-14 10:18:12,794 WARN [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- An error occurred while checking for an updated Resource configuration for Resource[id=11799, uuid=63150549-2454-4218-aaad-ddcffc071718, type={Apache}Apache Virtual Host, key=MainServer, name=Main, parent=127.0.1.1:80]. java.lang.IllegalStateException: Configuration and child resource creation/deletion support for Apache is optional. If you switched it on by enabling Augeas support in the connection settings of the Apache server resource and still get this message, it means that either your Apache version is not supported (only Apache 2.x is supported) or Augeas is not available on your platform. at org.rhq.plugins.apache.ApacheVirtualHostServiceComponent.loadResourceConfiguration(ApacheVirtualHostServiceComponent.java:165) at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:654) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) 2014-01-14 10:18:12,795 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- Configuration update check completed in 982ms 2014-01-14 10:18:39,733 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Tue Jan 14 10:18:39 CET 2014 2014-01-14 10:18:39,735 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Tue Jan 14 10:18:39 CET 2014 : Scan [startTime=1389691119733, endTime=1389691119735, runtime=2, isFull=false, isForced=false, numResources=179, numGetAvailabilityCalls=1, numScheduledRandomly=0, numPushedByInterval=0, numAvailabilityChanges=0, numDeferToParent=0] 2014-01-14 10:18:41,789 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementSenderRunner)- Measurement collection for [9] metrics took 73ms - sending report to Server... The Exception above occurs in every hour in the same minute. And this is the servers log: 10:20:08,530 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [amweb] since [Tue Jan 14 10:14:17 UTC 2014]. Will be backfilled since we suspect it is down Any idea how to prevent such events in the future?

2 2

reset pw of rhqadmin
by Elmar Knipp 14 Jan '14

14 Jan '14

Hi, It seems that I forgot my password for the rhqadmin user. Is there a way to retrieve or reset it? I have searched in the docu / forum but could not find a solution. Thanks, Elmar

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

rhq-users January 2014