Index Usage

In order to facilitate the planning and evaluation of your data architecture, InsightEdge provides real-time measurements of the frequency of use and usage patterns of data indexes.

Index Efficiency

InsightEdge allows the definition of indexes of various types (for example, compound or nested) for each object type within a space. These indexes are stored in-memory to enable fast access to data stored in memory or on disk.

One of the key challenges of administrators and developers is to find the best balance between the benefits of having many indexes, and the memory footprint required by the indexes. As a guide, an efficient index means an index that is heavily used by applications or queries, and consumes a minimal amount of storage space. Indexes that are used infrequently can be analyzed for their efficiency.

In order to analyze index efficiency, the user can review two types of information within the Ops Manager:

  • Index size - the memory footprint of the index
  • The relative percentage of index usage, out of the total number of index hits via reads and queries of applications

This information is stored as part of InsightEdge metrics, and can be reviewed using Grafana or Ops Manager.

Viewing Index Usage Statistics

Within Ops Manager there are two indications to help analyze index efficiency:

  • Index size - click on the Indexes tab within the table to review the indexes of each object type. After executing object analysis, you can see an estimate of the size of each index.
  • Relative index usage - the percent each index is used, compared to the overall queries of the specific object type.

The combination of the these two indications provides information of index size compared with its actual usage.

Additional indications of trends and total number of hits are available within Grafana:

  • The Index Hits Trend Chart shows the total number of hits in a given time period. It is possible to monitor the last 24 or 48 hours in order to identify indexes with no or minimal usage.
  • The Total Number of Hits per Index Chart shows a summary or total view of all index hits from the beginning of the deployment.

The combination of both charts enables users to review the overall usage of an index and the actual usage from the last few days. This enables analysis of indexes that where never in use, or perhaps used during specific time frames. An example is an index that was in use a month ago, but due to changes in application code, was not used recently.

In addition to the five-minute index metrics available in Ops Manager, Grafana provides various dashboards that display information for different defined time periods.

Access Index Usage Statistics through Grafana

See Using Grafana for Monitoring and Analytics for details on how to access graphs.

Index Efficiency Charts on the Grafana Space Dashboard

The Space dashboard contains multiple charts or graphs with information such as the number of entries etc. In addition to these charts, two new charts are available – Index Hits Trend and Total Number of Hits per Index.

The Index Hits Trend Chart

The Index Hits Trend chart shows the total number of hits per index by defined time period. It is possible to monitor the last 24 or 48 hours in order to identify indexes with minimal or no usage.

The Total Number of Hits per Index Chart

The Total Number of Hits per Index chart shows a summary or total view of all index hits from the beginning of deployment.

Using the Information in These Charts

The combination of both charts enables users to review the overall usage of an index and the actual usage from the last few days. This enables analyses of indexes that are rarely or never used, or that are used in specific time frames – for example, an index that was in use a month ago but due to changes in application code, was not used recently.