Which monitoring and reporting open source software can be used to allow custom queries of the dataset?

I'm working on a project where it is assumed that a few thousand distributed custom devices need to be monitored. So there are pretty simple questions which shall be answered like: Whats the load/memory/disk space of device X now? What was the disk usage lately? And what was the load at the first time the device was online, maybe years back in time? And also more complicated questions like: Which devices did and how long run a custom task? This one would be used for accounting purpose.

One option to solve this is to build a custom monitoring and reporting tool. But this won't be a simple one day hack, so i wanted to ask if anyone knows some open source software which is able to

  • add new metrices in a very flexible way like ganglia's gmetric tool
  • keep the data forever in some kind of storage to query for simple data points
  • use a json api to query datapoints and to be able to generate complex query's via some kind of api
  • build graphs like everyone already knows from ganglia or such

I've tried ganglia already, but gmetad seems to dump the collected data as soon as the device is offline. But the flexiblity of gmetric is one key feature since in dont know what might be relevant to monitor in the future.

Replay

I suggest taking a look at GRR Rapid Response:

  • Clients are Python based so cross platform
  • Detailed monitoring of client CPU, memory, IO usage and self-imposed limits.
  • Powerful search and download capabilities for files and the Windows registry.
  • AngularJS Web UI and RESTful JSON API.
  • Fully scriptable IPython console access. which means you can have any graphs that you need
  • You can gather almost any data you need via Flows & Hunts you can even have hunts for specific Artifacts.
  • Datastore can be configured as MySQL Advanced or HTTPDatastore + SQLite.
  • The FAQ reports some installations with 32k clients
  • Can be cloud deployed on GCE or EC2
  • Analysis offline with BigQuery, via the iPython console or via export to CSV and then process with Pandas or otherwise.
Category: monitoring Time: 2016-07-31 Views: 1
Tags: monitoring

Related post

iOS development

Android development

Python development

JAVA development

Development language

PHP development

Ruby development

search

Front-end development

Database

development tools

Open Platform

Javascript development

.NET development

cloud computing

server

Copyright (C) avrocks.com, All Rights Reserved.

processed in 0.244 (s). 12 q(s)