Publishing Metrics
Introduction
Publishing metrics from OverOps is a feature that enables the streaming of OverOps data to third-party applications, where it can be used for analytical purposes such as data visualization and anomaly detection. The export of dynamic telemetry data enables a variety of third-party tools, such as Splunk, Datadog, InfluxDB, Grafana, Graphite, and many more, providing control over application data from OverOps.
OverOps supports sending metrics via StatsD, which is an open-source implementation protocol to capture, aggregate and send metrics to modern DevOps tool, and enables visualization of any combination of events sent from OverOps.
Enabling Publishing Metrics from OverOps
When StatsD is enabled, collection and aggregation of metrics is fully automated, providing access to all data displayed in OverOps, for each exception in any analysis or visualization tool, including the data from predefined Dashboard Views, Automated Root Causes, and a number of active JVMs.
Prerequisites
Only services that support the StatsD protocol are able to read metrics from OverOps. Common services in the market that support StatsD include Telegraf, Splunk, and Datadog. Consult your third-party application documentation to verify that your service can receive telemetry data via StatsD.
The export configuration is dynamic, which means that every change made to the OverOps Dashboard is reflected in the data sent to StatsD, providing a steady data flow.
- In Views, when StatsD is configured, the metrics of every new View created is automatically sent to the defined backends through StatsD.
- In the Events Stream, every event publishes its own statistics, enabling the recreating and customizing of the OverOps dashboard in the tool of your choice. In addition, every event can publish a log link to the Root Cause Analysis, which can be directly accessed from the third-party graphic or alarm tool.
- The JVM count sends a query on how many JVMs are currently being monitored.
To publish metrics:
Publish Metrics settings can be updated by account owners and admins only
- Open OverOps Settings Page by clicking on Settings in the top right corner
- Select Publish Metrics In the Side Menu.
- In the Publish Metrics page, turn on StatsD.
The Publish Metrics are now active. Defaults can be changed if necessary. OverOps sends all the metrics recorded in the Views and Events panels to the folder defined in the Output Metric Format section.
Publish Metrics Settings
- Enter the hostname or IP address and port of the StatsD server. For example:
telegraf-statsd-server.company.com:8127
127.0.0.1:8125 (The default StatsD port is 8125).
- Choose the Output metric format
You can change the default folder path of the different metrics (optional).
A default path is provided for each metrics type (Views, Events, Entry Points, etc..).
For more details on metric format, refer to the next section in this article.
6. When finished, click Save.
Once the Publish Metrics are set, metrics are sent from the Controller to the configured services.
Metric Name Formats
This section describes the formats of the metric name to use when publishing to StatsD.
The default patterns listed below per metric type:
Default pattern for Views metrics:
overops.${serviceid}.${application}.${server}.${deployment}.views.${viewname}
Default pattern for Events metrics:
overops.${serviceid}.${application}.${server}.${deployment}.events.${class}.${method}.${event}.${firstseen}
Default pattern for Entry Points metrics:
overops.${serviceid}.${application}.${server}.${deployment}.entrypoints.${entrypointclass}.${entrypointmetrics}
Default pattern for Custom metrics:
{code}overops.${serviceid}.${application}.${server}.${deployment}.metrics.${metric}{code}
Default pattern for System:
overops.${serviceid}.${application}.${server}.system.${systemmetrics}
Default pattern for Diagnostics:
overops.${serviceid}.${application}.${server}.jvmcount
The OverOps-specific variables that can be used by adding them to metric list are described below.
Diagnostics Format
In Diagnostics format, only {serviceid}, {application}, {applicationpid} and {server} may be used.
| Metric | Used for | Description |
|---|---|---|
| ${server} | Views Events Entry Points Diagnostics | The name of the machine which is publishing the StatsD (e.g. "prod04") |
| ${serviceid} | Views Events Entry Points System Diagnostics | The unique numeric ID of the installation key on the machine (e.g. "S1234") |
| ${applicationpid} | Views Events Entry Points Diagnostics | The process ID of the JVM to which the StatsD is related (e.g. "41884") |
| ${application} | Views Events Entry Points System Diagnostics | The name of the JVM to which the StatsD is related (e.g. "worker", "my-webapp") |
| ${deployment} | Views Events Entry Points | The name of the release to which the StatsD is related (e.g. "ver-2017/12/07") |
| ${viewname} | Views | The name of the View that is being published (e.g. "invocations", "times") |
| ${class} | Events | The name of the class to which the StatsD is related (not fully-qualified) |
| ${method} | Events | The name of the method to which the StatsD is related |
| ${systemmetrics} | System | The state of the JVM at the moment of the event, see: JVM View |
| ${eventid} | Events | The unique identifier of the method to which the StatsD relates. |
| ${eventtype} | Events | The type of the event to which the StatsD relates (caught_exception, uncaught_exception, swallowed_exception, log_warning, log_error). |
| ${eventname} | Events | The name of the event to which the StatsD relates. |
| ${eventlink} | Events | The path to the ARC screen of the event in OverOps. Add prefix according to the used 3rd party application (e.g: http://[:8080/]tkp.to/) and copy to browser. |
| ${introducedby} | Events | The name of the first deployment in which the event was introduced (Optional). |
| ${labels} | Events | The names of the labels to which the event is related (Optional). |
| ${entrypointclass} | Entry points Events | The name of the entry point's class to which the StatsD is related (not fully-qualified) |
| ${entrypointmethod} | Events | The name of the entry point's method to which the StatsD is related. |
| ${entrypointmetrics} | Entry Points | Various entry point metrics, such as entry_point_average_runtime, entry_point_total_runtime. |
| ${firstseen} | Events | The time stamp of the first time the event was seen |
| ${infra} | Events | The name if the infrastructure to which the event is routed. |
Configure 3rd-Party Services to Receive StatsD Data from OverOps
Once Publish Metrics integration is enabled, you will need to configure a third-party service to receive data from OverOps.
Updated almost 5 years ago