For the complete documentation index, see llms.txt. This page is also available as Markdown.

Monitor resource usage

Report on what your datamart consumes: the cost of each query, and the volume ingested.

Your goal

Understand what your datamart consumes - which queries cost what, and how much data you ingest - so consumption is a number you manage rather than a surprise.

Expected outcome

A consumption report per datamart that you can break down by resource and follow over time.

What is measured

Each datamart resource that leads to write or read access on the platform is tracked, and its consumption is measured - segments, exports, imports, and the rest.

The report gives you:

  • the cost of each query executed on the platform,

  • the amount of data ingested.

Metrics and dimensions

Metric
What it measures

read_count

The number of top-level selectors of an OTQL or GraphQL query, the id selector excluded.

read_size

The number of elements read from the database by the query's selection.

scan_size

The number of elements read from the database to apply the query's filters, its WHERE part.

scan_duration

The time spent applying the query's filters.

SELECT { id activity_events { id } activities { id } } FROM UserPoint has a read_count of 2: activity_events and activities are its two top-level selectors, and id does not count.

Dimension
What it groups or filters by

source

The feature being measured, such as segment computation or export.

sub_source

Context about the source, such as INSERT_PHASE for a segment computation.

resource_id

The resource that led to the read or write operations.

execution_id

One execution of that resource. A resource has several executions.

date_time

When the consumption was measured.

datamart_id

The datamart the resource belongs to.

organisation_id

The organisation the datamart is linked to.

community_id

The community the datamart is linked to.

Read the report

1

Open the Query Tool

Go to Data Studio > Query Tool.

2

Select the data source

In the selector above the query builder, which shows OTQL by default, choose Resources usage. See Choose a data source.

3

Pick a metric and a dimension

In Select, choose a metric, such as scan_size. In by, choose a dimension, such as source to see which feature consumes the most, or resource_id to find the costliest resource.

4

Set the period

In for, choose the period to analyse. It defaults to Last 7 days.

5

Filter the report

Optional. Click Add a filter, choose a dimension in Filter on, and the value it equals - a single source, for instance.

6

Run the report

Click Run. Use New Value to add a metric, such as scan_duration beside scan_size, and New series to compare another request in the same tab.

The endpoint is a mediarithmics data cube, like the other monitoring APIs.

Field
Type
Description

metrics

Array

The metrics to retrieve. Required.

dimensions

Array

The dimensions to group by. Required.

dimension_filter_clauses

Object

Filters on dimensions before calculating the metric. Required.

date_ranges

Array

The periods to analyse - each with a start_date and an end_date. Required.

The response is a report view with columns_headers and rows, in the same shape as the other data cubes. The available dimensions and metrics are the ones listed above.

The full request and response, and the operations that list the available metrics and dimensions, are in the Platform Monitoring API reference.

Retention

These aggregated statistics are kept for two years.

What to do with it

1

Find the expensive queries

Query cost is where consumption usually concentrates. A segment recomputed nightly on an expensive query costs every night.

2

Check ingestion against your plan

Compare the volume ingested with what your tracking plan intends. An unexpected volume usually means an event is being sent more often than designed - not that usage grew.

3

Review what is indexed

Every indexed property costs index size and query performance. If consumption grows without a use case behind it, the schema is the place to look. See Schema definition.

4

Compare against the documented limits

Several platform limits are defaults that can be raised, and knowing your actual consumption is what makes that conversation concrete. See Platform limits.

Last updated

Was this helpful?