Cleaning rules
Define how each datamart expires events and profiles, with actions, filters, and priorities.
Your goal
Make the datamart expire events and profiles on its own, according to a policy you can state and defend.
Expected outcome
A set of LIVE rules whose combination produces the retention you intended - for every event and profile, on every channel and compartment.
What a cleaning rule is
Cleaning rules define how each datamart manages the deletion of user data. They apply to profiles and events.
When an object enters the datamart, the platform computes its expiration timestamp from the rules that apply. That timestamp becomes part of the object's properties and cannot be modified.
How expiry is evaluated
An event or profile expires if it has not been modified before its expiration timestamp:
Profile
The last modification timestamp, $last_modified_ts.
Event
The creation timestamp, $ts - an event cannot be updated.
When an object expires it stops appearing in queries and in the interface. It still exists in the passive database until it is removed by a scheduled operation.
The rule types
Event-based (USER_EVENT_CLEANING_RULE)
Events
All events, or specific channels, activity types, or event names.
Profile-based (USER_PROFILE_CLEANING_RULE)
Profiles
All profiles, or a compartment.
Every rule has a status (DRAFT, LIVE, or ARCHIVED), an action (KEEP or DELETE), a duration, and a set of filters.
KEEP- the object should stay in the datamart for the duration.DELETE- the object should be removed after the duration has expired.
Cleaning rules are edited in Navigator, in your datamart settings.
Priority: how the combination resolves
These rules decide the outcome:
KEEPrules take priority overDELETErules.Only
LIVErules are taken into account when computing an expiration timestamp.
Lifecycle
Rules are created as
DRAFT. They cannot be createdLIVEorARCHIVED.While
DRAFT, you can change the rule's properties - except its type.You move a rule to
LIVEwith an edit request. OnceLIVE, a rule cannot be edited; only its status can change, toARCHIVED.Once
ARCHIVED, a rule cannot be changed or deleted.Only
DRAFTrules can be deleted. ALIVErule has to be archived instead.
Rule properties
id
String
The cleaning rule identifier.
action
Enum
KEEP or DELETE. Only DELETE is allowed for USER_PROFILE_CLEANING_RULE.
archived
Boolean
Makes a rule disappear from the interface. Can only be set to true when the status is already ARCHIVED.
status
Enum
DRAFT, LIVE, or ARCHIVED, in that order only.
datamart_id
String
The associated datamart.
type
Enum
USER_EVENT_CLEANING_RULE or USER_PROFILE_CLEANING_RULE.
life_duration
ISO 8601 duration
The duration associated with the rule - for example P65D.
channel_filter
String
Event rules only. Optional. The channel the rule applies to.
activity_type_filter
Enum
Event rules only. Optional. SITE_VISIT, APP_VISIT, TOUCH, DISPLAY_AD, or EMAIL.
compartment_filter
String
Profile rules only. Optional. The compartment the rule applies to.
Content filter
An event cleaning rule can carry a content filter:
filter
String
The value of the filter.
content_type
Enum
The target of the filter. Only EVENT_NAME_FILTER is allowed for now.
This is how you give a specific event name its own retention.
Designing the set
Start from the requirement, not the rules
State the retention per data class first - see Data retention.
Verify
Every rule you rely on is
LIVE, notDRAFT.There is a
LIVEevent-based rule with aDELETEaction.For each data class, walking the rules by hand produces the expiry you intended.
Data collected before the rules existed is handled another way - the rules will not touch it.
Related pages
Last updated
Was this helpful?