Plugin Of The Month: Checking UPS alarms

NOTE: This post covers a Nagios/Icinga/Centreon Core compatible plugin addressed to monitor the active alarms. If you are interested in the background information that supports the plugin, see the post Monitoring UPS Devices: UPS-MIB.

Background

Monitoring the UPSs health is critical when trying to keep alive your ITC infrastructure. One of the most supported ways of monitoring UPSs, instead of using proprietary solutions, is SNMP via the MIB UPS (RFC 1628), widely supported by most of the devices in the market.

Specifically, MIB UPS (RFC 1628) supports an active alarms table in the managed device (OID upsAlarmTable, 1.3.6.1.2.1.33.1.6.2). Each table input stores:

• upsAlarmId: An unique active alarm identifier
• upsAlarmDescr: A reference to an alarm description object
• upsAlarmTime: The value of sysUpTime when the alarm condition was detected

MIB includes a list of alarms objects called Well-Known-Alarms:

ID	OID	Description
1	upsAlarmBatteryBad (1.3.6.1.2.1.33.1.6.3.1)	One or more batteries have been determined to require replacement.
2	upsAlarmOnBattery (1.3.6.1.2.1.33.1.6.3.2)	The UPS is drawing power from the batteries.
3	upsAlarmLowBattery (1.3.6.1.2.1.33.1.6.3.3)	The remaining battery run-time is less than or equal to upsConfigLowBattTime.
4	upsAlarmDepletedBattery (1.3.6.1.2.1.33.1.6.3.4)	The UPS will be unable to sustain the present load when and if the utility power is lost.
5	upsAlarmTempBad (1.3.6.1.2.1.33.1.6.3.5)	A temperature is out of tolerance.
6	upsAlarmInputBad (1.3.6.1.2.1.33.1.6.3.6)	An input condition is out of tolerance.
7	upsAlarmOutputBad (1.3.6.1.2.1.33.1.6.3.7)	An output condition (other than OutputOverload) is out of tolerance.
8	upsAlarmOutputOverload (1.3.6.1.2.1.33.1.6.3.8)	The output load exceeds the UPS output capacity.
9	upsAlarmOnBypass (1.3.6.1.2.1.33.1.6.3.9)	The Bypass is presently engaged on the UPS.
10	upsAlarmBypassBad (1.3.6.1.2.1.33.1.6.3.10)	The Bypass is out of tolerance.
11	upsAlarmOutputOffAsRequested (1.3.6.1.2.1.33.1.6.3.11)	The UPS has shutdown as requested, i.e., the output is off.
12	upsAlarmUpsOffAsRequested (1.3.6.1.2.1.33.1.6.3.12)	The entire UPS has shutdown as commanded.
13	upsAlarmChargerFailed (1.3.6.1.2.1.33.1.6.3.13)	An uncorrected problem has been detected within the UPS charger subsystem.
14	upsAlarmUpsOutputOff (1.3.6.1.2.1.33.1.6.3.14)	The output of the UPS is in the off state.
15	upsAlarmUpsSystemOff (1.3.6.1.2.1.33.1.6.3.15)	The UPS system is in the off state.
16	upsAlarmFanFailure (1.3.6.1.2.1.33.1.6.3.16)	The failure of one or more fans in the UPS has been detected.
17	upsAlarmFuseFailure (1.3.6.1.2.1.33.1.6.3.17)	The failure of one or more fuses has been detected.
18	upsAlarmGeneralFault (1.3.6.1.2.1.33.1.6.3.18)	A general fault in the UPS has been detected.
19	upsAlarmDiagnosticTestFailed (1.3.6.1.2.1.33.1.6.3.19)	The result of the last diagnostic test indicates a failure.
20	upsAlarmCommunicationsLost (1.3.6.1.2.1.33.1.6.3.20)	A problem has been encountered in the communications between the agent and the UPS.
21	upsAlarmAwaitingPower (1.3.6.1.2.1.33.1.6.3.21)	The UPS output is off and the UPS is awaiting the return of input power.
22	upsAlarmShutdownPending (1.3.6.1.2.1.33.1.6.3.22)	A upsShutdownAfterDelay countdown is underway.
23	upsAlarmShutdownImminent (1.3.6.1.2.1.33.1.6.3.23)	The UPS will turn off power to the load in less than 5 seconds; this may be either a timed shutdown or a low battery shutdown.
24	upsAlarmTestInProgress (1.3.6.1.2.1.33.1.6.3.24)	A test is in progress, as initiated and indicated by the Test Group. Tests initiated via other implementation-specific mechanisms can indicate the presence of the testing in the alarm table, if desired, via a OBJECT-IDENTITY macro in the MIB document specific to that implementation and are outside the scope of this OBJECT-IDENTITY

Plugin description

check_ups_alarms is a Nagios/Icinga/Centreon Core compatible plugin for checking the active alarms in a UPS MIB compliant device.

The alarms to be checked (see table above) can be defined as a list of IDs in the warning and critical plugin arguments. If one of these alarms becomes active, the plugin will return the status associated to the list where the active alarm is defined.

check_ups_alarms  is based on fetching session data using SNMP v1/2c, so it's necessary that the device being checkek supported this protocol and served info managed by the UPS MIB (RFC 1628).

You can get detailed help and usage examples by running the script with the  --help option.

Usage examples

check_UPS_alarms -H 192.168.0.1

Checks the active alarms on a host with address 192.168.0.1 using SNMP protocol version 1 and 'public' as community. Plugin returns always OK.

check_UPS_alarms -H 192.168.0.1 -w 1..4,11 -c 5..10

Similar to the previous example but returning WARNING if alarm(s) with ids 1 to 4 and 11 are active and CRITICAL if alarms with ids 5 to 10 are active

Download

You can download the latest version of the plugin from GitHub.

The development of this plugin, that now is freely released, implies hours of reading technical documentation, programming and testing. I will be more than glad if you support this effort by clicking in some of the interesting advertisements that you can find on this website.

Last but not least, if you find some bug don't hesitate in contacting me for fixing it quickly. Feedback comments are welcome too!

Tweet

Monitoring UPS devices

From a layered design point of view, the monitoring base layer -or layer 0- includes all devices whose mission is creating a solid physic and environmental platform where upper devices and services could run.

UPS devices are a key part of this layer: from affording not only a continuous supply but also a quality signal to showing the consumption per line, they are a MUST BE when designing from datacenter to global monitoring strategies.

DISCLAIMER: This is mainly a theoretical article. If you're looking for practical info about how to get the most of your device read "Monitoring UPS Devices: UPS-MIB"

Why monitoring a UPS?

Many times, when getting data to start a monitoring project, I find that users claim "Really? Can it be monitored?", then I answer "it MUST be monitored".

Perhaps because UPS devices are in the border between computer and power supply worlds, in the limits where a vast, misterious land populated by volts, ampers and watts starts, the IT crowd deliberately or unconsciously doesn't pay them a bit of its attention.

However, all the IT hard -and thus soft- structure depends on them as suppliers of continuous, clean* power supply, just consider the costs in manpower and the amount of offline service clients of an unattended power cutoff in your CPD due to a unmonitorized out of service UPS.

All UPS are based on batteries as power storage devices and battery life is closely environmental temperature dependent, usually 21 to 25 Celsius degrees (depending on battery specs). In order to maximize their life you have two options: turning you CPD into a freezer spending lots of money in conditioning air maintenance, or monitoring their temperature and fit the environmental temperature to its optimal value... bet it, the last one wins.

Finally, monitoring the UPS output line values will allow you, without the need of using PDUs, knowing when you are reaching the UPS nominal limits or helping you to dimension the right device (and thus saving money) if you need to upgrade the current. And last but not least nowadays, monitoring the UPS input line value will allow you getting the overall** power supply consumption, and thus, the CO2 emission levels of your CPD.

...Well, and how do I get that info?

For a usual IT technician, first obvious option should be adopting a TUIN approach (Toothpicks Under Its Nails). Saddly UPS don't have nails.

Now seriously, you can rely on a propietary monitoring solution like those that some times manufacturers give or much more times they sell. If you have good friends and/or money to get it you will get a nice, colorful, program that only will be useful to monitor one kind of UPS. So if you have a diversified UPS park you will have to use more than one, and you will have to change it if you change the brand when upgrading it.

On the other hand, you can increase the value of your monitoring platform (read Nagios, Zenoss, OpenNMS or any other that supports SNMP traps and/or requests) adding UPS monitoring capabilities. You only need a UPS with SNMP capabilites: many of them have it out of the box, other can get it by mounting a network interface adapter.

When dealing with UPS SNMP MIBs you have, again, two options: basing your monitoring on private, manufacturer MIBs, or adopting an standard position and getting the needed info from the RFC 1628 MIB (formely UPS-MIB). First one is sweet and easy because the manufacturer might had chewed the meal to you, however you'll have to create a service set by manufacturer. Second one will require, in some cases, a higher scripting effort but it will work with all UPS supporting RFC 1628 MIB (that should be -if not all- the most).

As most of you -like me- like the extreme pain, let's go for the second one: "Monitoring UPS devices: UPS-MIB"

Summarizing

Using a few words I've tried to show the pros of monitoring our UPS park: more security, costs saving and more info in our hands to base further decisions. From the many ways that could be selected to monitor an UPS, I've argued why selecting a SNMP based approach and why using the info stored in RFC 1628 MIB.

Interesting links

• The different types of UPS systems (Neil Rasmussen, APC) 

Related posts

• Monitoring UPS devices: UPS-MIB

(*).- Depending on the device characteristics, not all UPS improve the power signal quality. To know more about it check the fantastic Neil Rasmussen's white paper in references.
(**).- Not counting air conditioning consumptions and stating that all CPD devices are attached to UPS network.

Tweet