For the complete documentation index, see llms.txt. This page is also available as Markdown.

Monitor collection volume

Count the elements indexed in each collection of your datamart, day by day.

Your goal

Know how many elements are indexed in each collection of your datamart - UserPoints, events, profiles, segments - and watch how those counts move.

Expected outcome

A daily count per collection you can trend, and a way to tell an indexing problem apart from an ingestion problem.

What is counted, and what is not

This is the complement to ingestion monitoring: ingestion tells you what arrived, collection volume tells you what is queryable.

The queries behind the numbers

The platform runs one count per collection, so you can reproduce any figure yourself:

Collection
OTQL query

UserPoint

SELECT @count {} FROM UserPoint

UserAccount

SELECT @count {} FROM UserAccount

UserEmail

SELECT @count {} FROM UserEmail

UserAgent

SELECT @count {} FROM UserAgent

UserProfile

SELECT @count {} FROM UserProfile

UserActivity

SELECT @count {} FROM UserActivity

UserEvent

SELECT @count {} FROM UserEvent

UserSegment

SELECT @count {} FROM UserSegment

UserChoice

SELECT @count {} FROM UserChoice

UserScenario

SELECT @count {} FROM UserScenario

The query is adapted automatically when the runtime schema uses the @Mirror directive.

Metrics and dimensions

Metric
What it counts

count

The number of elements indexed in a collection.

Without the date_time dimension, count is the average of every value computed over the period, not a daily figure.

Dimension
What it groups or filters by

collection

The indexed collection, from the table above.

date_time

When the count was computed.

organisation_id

The organisation the datamart is linked to.

community_id

The community the datamart is linked to.

Read the report

1

Open the Query Tool

Go to Data Studio > Query Tool.

2

Select the data source

In the selector above the query builder, which shows OTQL by default, choose Collection volumes. See Choose a data source.

3

Pick the metric and a dimension

In Select, choose count. In by, choose collection for one count per collection, or date_time to follow a count day by day.

4

Set the period

In for, choose the period to analyse. It defaults to Last 7 days.

5

Filter the report

Optional. Click Add a filter, choose a dimension in Filter on, and the value it equals - a single collection, for instance.

6

Run the report

Click Run. Use New series to compare another request in the same tab.

The endpoint is a mediarithmics data cube, like the other monitoring APIs.

Field
Type
Description

metrics

Array

The metrics to retrieve. Required.

dimensions

Array

The dimensions to group by. Required.

dimension_filter_clauses

Object

Filters on dimensions before calculating the metric. Required.

date_ranges

Array

The periods to analyse. Required.

The response returns a report view with columns_headers and rows - for example date_time, datamart_id, collection, count.

The full request and response, and the operations that list the available metrics and dimensions, are in the Platform Monitoring API reference.

Retention

These aggregated statistics are kept for two years.

Long enough for year-on-year comparison, unlike ingestion monitoring's three months.

Observation
What to look at

A collection has no volume at all

It is not in your schema. See Schema definition.

UserPoint grows faster than your real user base

Identifiers are being minted rather than matched - a format problem or a missing registry. See Resolve identities.

UserPoint drops sharply

An identifier association import merged more than you expected, or cleaning rules removed users with no remaining activity or profile.

UserActivity or UserEvent plateaus

Collection stopped, or a cleaning rule now expires as fast as data arrives. Cross-check with ingestion monitoring.

UserChoice is low before you enforce a privacy wall

You are about to drop data. See Manage consent & Processing activities.

Volume grew but a new indexed property returns nothing

Adding an index only covers data arriving afterwards. See what happens when you change a schema.

Last updated

Was this helpful?