Show events on a dashboard

This document describes how you can display events on the charts on a dashboard. An event is an activity, such as a deployment or a crash, that affects the operation of a system. Showing events can help you correlate data from different sources when you're troubleshooting an issue.

How events are identified

Dashboards query for events by analyzing the log entries that pass through the Cloud Logging sinks in your Google Cloud project. These log entries include those generated by your project and those routed to your project.

To get notified when one of these events occurs, you can create a log-based alerting policy. In the alerting policy, use the queries provided on the Event types page.

How events are displayed

The charts on your dashboard are annotated as events occur. Events identified by analyzing log entries are shown with a single time reference.

The following screenshot illustrates a chart that is displaying one event that is was identified by analyzing log entries:

Chart displaying event annotations.

As shown in the previous screenshot, the tooltip for an event displays information like the following:

  • Event name.
  • The Google Cloud service or the resource that caused the event.
  • Date and start time of the event.
  • A View button, which when selected opens a page that displays more information about the event. For example, it might open the Logs Explorer, so that you can view log entries.

Dashboard filters, which limit the data displayed by a dashboard, typically filter both time-series data and events.

For custom dashboards, you can also configure whether events are displayed by using the Cloud Monitoring API.

Before you begin

  • To get the permissions that you need to view charts and log entries, and to create alerting policies, ask your administrator to grant you the following IAM roles on your project:

    • View charts, and to view and create alerting policies: Monitoring Editor (roles/monitoring.editor).
    • View all log entries except data access log entries that are created in your Google Cloud project: Logs Viewer (roles/logging.viewer).
    • View data access log entries that are created in your Google Cloud project: Private Logs Viewer (roles/logging.privateLogViewer).

    For more information about granting roles, see Manage access.

    You might also be able to get the required permissions through custom roles or other predefined roles.

Show events

For a list of event types, troubleshooting references, and queries that can be use with the Logs Explorer or with log-based alerting policies, see the Event types page.

Console

To show events in the current session, do the following:

  1. In the Google Cloud console, go to the  Dashboards page:

    Go to Dashboards

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. Select the dashboard that you want to view.
  3. In the dashboard's toolbar, ensure the Events toggle is on.

    If the Events toggle isn't displayed, then the dashboard doesn't support the display of events.

  4. For each event type that you want to display, in the dashboard toolbar, click Add event, and then select the event type from the menu.

    For example, you might configure your dashboard to display "GKE Workload Deployment" events.

    After you add an event type, Cloud Monitoring queries your log data, which takes a few moments, and then adds the annotations to the charts on the dashboard:

    • Info: An informational message.
    • Warning: Indicates a failure, such as a reboot or a crash.
    • Alert: Indicates that an incident was opened.
  5. To view a summary of an event or to access links to log entries, use your pointer to activate the tooltip.

  6. Optional: Your configuration applies only to the current session. To save your configuration, do the following:

    1. In the dashboard toolbar, click Settings, select JSON, and then select JSON editor.
    2. In the JSON editor, add an annotations object. This object contains an array of EventAnnotation objects that specify the event types to display.

      The following shows an example annotations object for a cluster update event. The filter field in this example restricts the events to a specific cluster:

      "annotations": {
        "defaultResourceNames": [],
        "eventAnnotations": [
          {
            "displayName": "",
            "enabled": true,
            "eventType": "GKE_CLUSTER_UPDATE",
            "filter": "resource.labels.cluster_name=my-cluster",
            "resourceNames": []
          }
        ]
      }
      

      For more information about the annotations object, select the API tab of this table.

    3. Click Apply changes and click Save.

API

To configure a custom dashboard to display events, include an annotations field in the Dashboard object. The annotations field contains an instance of a DashboardAnnotations object, which contains two fields:

  • The defaultResourceName field lists the dashboard-level defaults for the Google Cloud projects to search for events. If you don't specify this field, then your selected project is searched.

  • An array of EventAnnotation objects. Each object contains the following:

    • A displayName field, which lets you define the label on the event-specific toggle. When this field isn't specified or when the value of this field is an empty string, the default name for the event is displayed.
    • An enabled field, which controls the value of the event's toggle. When true, the toggle is in the on position and events are shown on the dashboard. When false, the toggle is in the off position.
    • An eventType field, which contains a value from the EventType enumeration.
    • A filter field, which lets you specify a query to append to the event-specific query. When the value of this field is an empty string, then the event-specific query is used.
    • A resourceNames field, which lets you define a list of Google Cloud projects to search for events. When this field is an empty array, your selected project is searched.

The following shows an example annotations object for a cluster update event. The filter field in this example restricts the events to a specific cluster:

"annotations": {
  "defaultResourceNames": [],
  "eventAnnotations": [
    {
      "displayName": "",
      "enabled": true,
      "eventType": "GKE_CLUSTER_UPDATE",
      "filter": "resource.labels.cluster_name=my-cluster",
      "resourceNames": []
    }
  ]
}

For detailed information about how to use the Cloud Monitoring API to create dashboards, see Create and manage dashboards by API and Example: enable events on dashboards.

View event details

When you are viewing a chart that is displaying event annotations, you can use links provided in the tooltip to access additional information:

  • To access details about a specific event, click View.

    The paged opened by the View button is determined by the event type. For example, for an uptime check event, this button opens the dashboard for the uptime check. For a different event type, the Logs Explorer might open.

  • To view the log entries that have timestamps that match an annotation, use your pointer to activate the tooltip for the annotation, go to the summary line of the tooltip, and then click View logs.

Create alerting policies

To get notified when an event appears in your logs, configure a log-based alert. Log-based alerting policies query your log entries as they are received. When the query returns a non-empty result, an incident is created and notifications are sent.

Console

To create a logs-based alert, do the following:

  1. If you haven't configured a notification channel, then do the following:

    1. In the Google Cloud console, go to the  Alerting page:

      Go to Alerting

      If you use the search bar to find this page, then select the result whose subheading is Monitoring.

    2. Click Edit notification channel.
    3. Go to the type of notification channel that you want to configure, click Add new, and then complete the dialog.
  2. In the navigation panel of the Google Cloud console, select Logging, and then select Logs Explorer:

    Go to Logs Explorer

  3. In the toolbar, select Create Alert.

  4. Enter a name for the alerting policy and then click Next.

  5. In the Define log entries to alert on section, enter a query or update the query that was copied from the Query pane when you selected Create Alert.

    For each event type, the Event types page lists a query that is suitable for use in a log-based alerting policy.

  6. Complete the dialog, and then click Save.

API

To create a logs-based alerting policy by using the Cloud Monitoring API, do the following:

  1. Identify the query that is to be included in the condition of the alerting policy. For each event type, the Event types page lists a query that is suitable for use in a log-based alerting policy.

  2. Create the JSON file that contains the representation of the alerting policy. For more information about this step and an example, see Design the alerting policy.

  3. Invoke the alertPolicies.create API method. For example, you might issue the gcloud alpha monitoring policies create command.

Hide events

Console

To hide all events, in the dashboard toolbar, ensure the Events toggle is off.

To hide one type of event, set the toggle for that event type to the off position.

API

To hide one type of event on a custom dashboard, find the EventAnnotation object for the event type, and then set the enabled field to false.

For detailed information about how to use the Cloud Monitoring API, see Create and manage dashboards by API.

What's next