Monitor collection volume
Count the elements indexed in each collection of your datamart, day by day.
Your goal
Know how many elements are indexed in each collection of your datamart - UserPoints, events, profiles, segments - and watch how those counts move.
Expected outcome
A daily count per collection you can trend, and a way to tell an indexing problem apart from an ingestion problem.
What is counted, and what is not
This is the complement to ingestion monitoring: ingestion tells you what arrived, collection volume tells you what is queryable.
The queries behind the numbers
The platform runs one count per collection, so you can reproduce any figure yourself:
UserPoint
SELECT @count {} FROM UserPoint
UserAccount
SELECT @count {} FROM UserAccount
UserEmail
SELECT @count {} FROM UserEmail
UserAgent
SELECT @count {} FROM UserAgent
UserProfile
SELECT @count {} FROM UserProfile
UserActivity
SELECT @count {} FROM UserActivity
UserEvent
SELECT @count {} FROM UserEvent
UserSegment
SELECT @count {} FROM UserSegment
UserChoice
SELECT @count {} FROM UserChoice
UserScenario
SELECT @count {} FROM UserScenario
Metrics and dimensions
count
The number of elements indexed in a collection.
collection
The indexed collection, from the table above.
date_time
When the count was computed.
organisation_id
The organisation the datamart is linked to.
community_id
The community the datamart is linked to.
Read the report
Select the data source
In the selector above the query builder, which shows OTQL by default, choose Collection volumes. See Choose a data source.
The endpoint is a mediarithmics data cube, like the other monitoring APIs.
metrics
Array
The metrics to retrieve. Required.
dimensions
Array
The dimensions to group by. Required.
dimension_filter_clauses
Object
Filters on dimensions before calculating the metric. Required.
date_ranges
Array
The periods to analyse. Required.
The response returns a report view with columns_headers and rows - for example date_time, datamart_id, collection, count.
The full request and response, and the operations that list the available metrics and dimensions, are in the Platform Monitoring API reference.
Retention
Long enough for year-on-year comparison, unlike ingestion monitoring's three months.
What the trends tell you
A collection has no volume at all
It is not in your schema. See Schema definition.
UserPoint grows faster than your real user base
Identifiers are being minted rather than matched - a format problem or a missing registry. See Resolve identities.
UserPoint drops sharply
An identifier association import merged more than you expected, or cleaning rules removed users with no remaining activity or profile.
UserActivity or UserEvent plateaus
Collection stopped, or a cleaning rule now expires as fast as data arrives. Cross-check with ingestion monitoring.
UserChoice is low before you enforce a privacy wall
You are about to drop data. See Manage consent & Processing activities.
Volume grew but a new indexed property returns nothing
Adding an index only covers data arriving afterwards. See what happens when you change a schema.
Related pages
Last updated
Was this helpful?