Dashboard Pipeline Setup¶

• Dashboard Pipeline Setup
  • Deployment Decisions
  • Installation of Required Software
  • Backend Configuration
    • InfluxDB
  • Startup
  • Usage
  • Notes

Deployment Decisions¶

There are currently two supported backends for storing timeseries data which can be visualized by the grafana dashboard:

1. InfluxDB: More modern timeseries database with well-structured query language
2. Graphite: Well-known monitoring database and web-frontend. Less structured query language and harder to deploy

While InfluxDB provides the much more modern experience and is easier to administer, graphite currently provides better performance with less CPU impact and better control of automatic retention. Depending on your needs, decide on one of the solutions.

Installation of Required Software¶

1. Install rsb, rst
2. Install rsb-performance-monitor
3. Install rsb-performance-monitor-graphite-adapter (git in this project)
4. Install a timeseries database:
   • InfluxDB: version 0.8! as DEB archive or citk
   • Graphite: See https://graphite.readthedocs.org/en/latest/install.html or citk
5. Install grafana version 2 (just extract the pre-built archive somewhere, or citk)

Backend Configuration¶

For both backend solutions you need to decide on a retention policy, that means how long data will be kept in the database. This decides on the storage requirements of your database files and the possibility to scroll back in time in the dashboard.

InfluxDB¶

1. Configure InfluxDB as desired using the example config files and the instructions found here: https://influxdb.com/docs/v0.8/introduction/installation.html. A config file which uses just a local computer and /tmp for data is attached. Please note that in this file the assets line in the admin section needs to be fixed so that it point to the installation of the required files for the admin web interface.
2. Using the web interface (http://host:8083 per default), create a database for the monitoring data. The retention for the proposed default shard space should be set to the desired time you plan to keep data available in the database (E.g. 7d). Duration should probably be set to 1d.

Startup¶

1. Start InfluxDB, e.g. with
   

   $prefix/opt/influxdb/influxdb -config=$prefix/etc/vdemo_scripts/data/influxdb-config-tmp.toml

2. Ensure that a database exists inside the running DB. In this case, we will call it data
   

   curl -X POST 'http://localhost:8086/db?u=root&p=root' -d '{"name": "data"}'

3. Start some rsb-performace-monitor instances, assuming that monitoring results will be published on /monitoring and /host
4. Start rsb-performance-monitor-graphite-adapter with the InfluxDB backend
   

   $prefix/bin/rsbperfmon-graphite -d -b influxdb -a

   
   Assumes that InfluxDB runs on localhost with port 8086 and root:root as credentials. If not, provide the required arguments. -d pre-calculates derivatives of some values for better visualization purposes later on, -a aggregates all subprocesses into the results of a single monitored process since most of the time only the cumulative results of a functional component are interesting.
5. Start a webserver to server the static grafana files, e.g.:
   

    cd $prefix/opt/grafana && python2 -m SimpleHTTPServer 8020

6. Optionally, start an elasticsearch server
   

   $prefix/opt/elasticsearch/bin/elasticsearch -Des.path.data=$HOME/.elasticsearch

   
   Chose an appropriate data path

Usage¶

1. Open the grafana webpage at http://localhost:8020
2. Configure the dashboard as needed
3. Save the dashboard, either to elasticsearch or to your browser

Notes¶

• Use meaningful credentials instead of root:root