In the last chapter, we looked at the Discover page and how we could make some quick searches across indexed documents. In this chapter, we will look at the Visualize and Dashboard features in Kibana. We will see how we can leverage the power of Kibana, built over Elasticsearch indexes, to build various types of charts and graphs, and awesome dashboards covering various analytics, which can be easily embedded or shared with others.
The Visualize page helps create visualizations in the form of graphs and charts. These visualizations can be saved and viewed individually or can be used in multiple dashboards, which act as a collection of visualizations.
All visualizations in Kibana are based on the aggregation feature of Elasticsearch. Kibana also supports multilevel aggregations to come up with various useful data analytics. Let's take a look at what a Visualize page looks like:
Kibana Visualize page
The Visualize page has two parts—either you can create a new visualization or open an existing one from your saved list.
To create a new visualization, select Visualize from the top menu bar, which opens a new Visualize page, and then click on the New Visualization button on tool bar.
Creating a new visualization is a three step process on the Visualize page:
- Select a visualization type.
- Select a data source (from a new search or an existing saved search).
- Configure the aggregations (metrics and buckets) that are to be used for the visualization on the Edit page.
Kibana supports the following visualizations:
- Area chart
- Data table
- Line chart
- Markdown widget
- Metric
- Pie chart
- Tile map
- Vertical bar chart
Before we start building visualizations of various types, let's understand a bit about Elasticsearch aggregations, which forms the backbone of the visualizations in Kibana.
The metrics and buckets concepts come from the aggregation functionality of Elasticsearch, and they play a vital role when designing a visualization for your dataset in Kibana.
Buckets help distribute documents among multiple buckets containing a subset of indexed documents. Buckets are very similar to the GROUP BY functionality in SQL. They help group documents based on specified criteria, and metrics can be applied on these documents.
Buckets usually represent the X-axis in Kibana charts and it is possible to add sub-buckets to a bucket.
The following buckets are available for the X-axis in Kibana:
- Date Histogram
- Histogram
- Range
- Date Range
- IPV4 Range
- Terms
- Filters
- Significant Terms
Let's take a look at a few important visualizations here.
Date Histogram requires a field name of type date and interval for the configuration. It groups documents as per the specified field and interval specified. For example, if you specify the field bucket as @timestamp and Interval as weekly, documents will be grouped based on weekly data, and then you can apply some metrics, such as Count, Average, and so on, on top of the grouped documents.
Histogram is similar to Date Histogram, except that it requires the field of type numbers and a numeric interval to be specified. It will bucket documents for the particular interval specified in the chosen field. This is like a range aggregation with equal intervals.
Range is like Histogram, but it allows you to configure different ranges as per the requirements, manually. For example, for a field count, you can choose the bucketing range to be 0-1000, 1000-5000, 5000-15000, and so on.
Terms help group documents by the value of any field, which is very similar to the GROUP BY statement in SQL. The Terms aggregation also lets you choose whether you want Top N or Bottom N, or you can specify the order based on metrics too. For example, you can choose to group by a product type and get the top five spends in that product type.
Buckets in visualizations
Metrics represents computations performed on values of fields in each bucket, for example, computing the count, average, minimum, or maximum of a field in the document. Metrics usually represent the Y-axis in Area chart, Vertical bar chart, and Line chart. The types of metrics available in Kibana are:
- Count
- Average
- Sum
- Unique Count
- Min
- Max
- Percentile
- Percentile Ranks
Let's take a look at a few of them.
The Count metric aggregation is very important, and its main purpose is to calculate the count of the number of fields in each bucket in a bucket aggregation.
For example, to count the number of visitors for each of the product categories, you can specify the product category field as bucket aggregation and count metric aggregation.
Buckets and metrics aggregations have Advanced options, which can take JSON input as scripted fields, as described in Chapter 6, Finding Insights with Kibana. The following script is an example:
{ "script" : "doc['volume'].value * 100"}
JSON Input Advanced option
Overall, a New Visualization page looks like this, with the toolbar at the top, metrics and buckets configuration on the left and the preview pane on right-hand side:
Kibana New Visualization page
The toolbar at the top has the options to create a new visualization, save a visualization, open a saved visualization, share a visualization, and refresh it.
Kibana Visualization toolbar
When creating a visualization, Kibana provides two options as a search source:
- From a saved search
- From a new search
Kibana search source selection
From a saved search uses searches that you saved in the Discover page.
From a new search is used to create a new visualization based on a new search.
Now, let's take a look at various visualization types and how they can be used.
Area chart is especially useful to create stacked timelines or distribute data.
Area chart uses metrics as Y-axis and buckets for X-axis. We can also define sub-aggregations in buckets, which give you the functionality of Split Charts (multiple charts based on different aggregations) or Split Area (Area chart split based on different aggregations).
Kibana Area chart
Data table is used to present aggregated data in a tabular format and helps identify Top N kinds of aggregations.
For example, to get the top five clients by the number of hits, the following data table visualization can be used:
Kibana Data table
Line charts are used for high density time series, and are often helpful when comparing one series with another:
Kibana Line chart
Markdown widget is used to display information or instructions on Dashboard and can be used for any requirements for text on Dashboard.
Kibana Markdown widget
Metric is used to show a one number kind of analysis for your field. It can be used to compute the total number of hits or the sum or average of a field.
For example, the following metric can be used to show the average response time of the application over a period of time:
Kibana Metric
Pie charts are often used to show parts of a whole or a percentage relationship. It represents the distribution of data over multiple slices in a pie chart.
A slice of the pie chart is determined by metrics aggregations, which can have the values Count, Sum, or Unique Count. Bucket aggregation defines the type of data that has to be represented in one chart.
For example, the following pie chart can be used to show the distribution of the different response codes of an application:
Kibana Pie chart
Tile maps are used to locate geographic locations based on geo coordinates. It works on the Geohash bucket aggregation, which groups multiple coordinates into one bucket.
Kibana Tile map
Vertical bar chart is a chart that can be used for a variety of purposes and works well with time- and non-time-based fields. It can be used as single bar or stacked as well.
Y-axis is metrics and X-axis is buckets aggregation.
For example, the following Vertical bar chart can be used to show a count of HTTP response codes:
Kibana Vertical bar chart