Open sourcing oozie-graphite

by Thomas Memenga on 2013-07-17

Open sourcing oozie-graphite

We’ve open sourced some very useful glue for Oozie monitoring with graphite on github.

We’ve open sourced some very useful glue for Oozie monitoring with graphite on github.com/syscrest/oozie-graphite and binaries available at https://github.com/syscrest/oozie-graphite/releases

As of now you can find two (independent) modules that push very valuable data into your graphite instance:

GraphiteInstrumentationService

GraphiteInstrumentationService pushes all available internal Oozie instrumentation metrics to your graphite instance. Check out the wiki …

Ever wondered how busy your internal Oozie queue is? Or how you can build your own Oozie dashboard with charts like this? oozie_instrumentation_callablequeue

graphite-mr-counter action

graphite-mr-counter is a custom action that you can add to your workflows to push Hadoop counters into graphite. By utilizing the nominal time from your coordinator actions,you get precise graphs from your bundle / coordinators executed by Oozie … and even updates on reruns! Check out the wiki ….

So let’s say you have a coordinator executing a workflow every six hours and you want some nice charts from stuff like number of input records, bytes read from HDFS or even your own custom counters from these executions? Charts that get updated on reruns ?

That’s exactly what graphite-mr-counter was made for :-) .

graph01_HFDS_BYTES_WRITTEN

More blog entries with practical examples for oozie-graphite will follow soon … stay tuned.