For the complete documentation index, see llms.txt. This page is also available as Markdown.

Cleaning rules

Define how each datamart expires events and profiles, with actions, filters, and priorities.

Your goal

Make the datamart expire events and profiles on its own, according to a policy you can state and defend.

Expected outcome

A set of LIVE rules whose combination produces the retention you intended - for every event and profile, on every channel and compartment.

What a cleaning rule is

Cleaning rules define how each datamart manages the deletion of user data. They apply to profiles and events.

When an object enters the datamart, the platform computes its expiration timestamp from the rules that apply. That timestamp becomes part of the object's properties and cannot be modified.

How expiry is evaluated

An event or profile expires if it has not been modified before its expiration timestamp:

Object
Field compared against the expiration timestamp

Profile

The last modification timestamp, $last_modified_ts.

Event

The creation timestamp, $ts - an event cannot be updated.

When an object expires it stops appearing in queries and in the interface. It still exists in the passive database until it is removed by a scheduled operation.

The rule types

Type
Applies to
Can be restricted to

Event-based (USER_EVENT_CLEANING_RULE)

Events

All events, or specific channels, activity types, or event names.

Profile-based (USER_PROFILE_CLEANING_RULE)

Profiles

All profiles, or a compartment.

Every rule has a status (DRAFT, LIVE, or ARCHIVED), an action (KEEP or DELETE), a duration, and a set of filters.

  • KEEP - the object should stay in the datamart for the duration.

  • DELETE - the object should be removed after the duration has expired.

Only DELETE is allowed for profile cleaning rules. KEEP exists for events, where it is used to protect a subset from a broader DELETE.

Cleaning rules are edited in Navigator, in your datamart settings.

Priority: how the combination resolves

These rules decide the outcome:

  • KEEP rules take priority over DELETE rules.

  • Only LIVE rules are taken into account when computing an expiration timestamp.

Worked examples

Example 1 - these rules apply to an incoming event:

  • KEEP for 60 days

  • KEEP for 180 days

  • DELETE after 150 days

The event is kept for 180 days, then deleted. The longest KEEP wins over the DELETE.

Example 2 - these rules apply to an incoming event:

  • KEEP for 60 days

  • DELETE after 150 days

The event is deleted after 150 days. The KEEP is shorter than the DELETE, so it changes nothing.

Example 3 - these rules apply to an incoming profile:

  • DELETE after 10 days

  • DELETE after 150 days

The profile is deleted after 10 days. Among DELETE rules, the shortest wins.

Lifecycle

  • Rules are created as DRAFT. They cannot be created LIVE or ARCHIVED.

  • While DRAFT, you can change the rule's properties - except its type.

  • You move a rule to LIVE with an edit request. Once LIVE, a rule cannot be edited; only its status can change, to ARCHIVED.

  • Once ARCHIVED, a rule cannot be changed or deleted.

  • Only DRAFT rules can be deleted. A LIVE rule has to be archived instead.

Rule properties

Property
Type
Description

id

String

The cleaning rule identifier.

action

Enum

KEEP or DELETE. Only DELETE is allowed for USER_PROFILE_CLEANING_RULE.

archived

Boolean

Makes a rule disappear from the interface. Can only be set to true when the status is already ARCHIVED.

status

Enum

DRAFT, LIVE, or ARCHIVED, in that order only.

datamart_id

String

The associated datamart.

type

Enum

USER_EVENT_CLEANING_RULE or USER_PROFILE_CLEANING_RULE.

life_duration

ISO 8601 duration

The duration associated with the rule - for example P65D.

channel_filter

String

Event rules only. Optional. The channel the rule applies to.

activity_type_filter

Enum

Event rules only. Optional. SITE_VISIT, APP_VISIT, TOUCH, DISPLAY_AD, or EMAIL.

compartment_filter

String

Profile rules only. Optional. The compartment the rule applies to.

Content filter

An event cleaning rule can carry a content filter:

Property
Type
Description

filter

String

The value of the filter.

content_type

Enum

The target of the filter. Only EVENT_NAME_FILTER is allowed for now.

This is how you give a specific event name its own retention.

Designing the set

1

Start from the requirement, not the rules

State the retention per data class first - see Data retention.

2

Write the baseline DELETE rule

One event-based DELETE covering all events, at your default retention.

3

Add KEEP rules for what must live longer

Filtered by channel, activity type, or event name. Remember that a KEEP shorter than the applicable DELETE has no effect.

4

Add profile rules per compartment

DELETE only, and remember the shortest one wins.

5

Publish, and re-check the combination

Only LIVE rules count. Walk one representative object of each class through the set and confirm the resulting expiry is what you intended - you cannot change it after the object is written.

Verify

  • Every rule you rely on is LIVE, not DRAFT.

  • There is a LIVE event-based rule with a DELETE action.

  • For each data class, walking the rules by hand produces the expiry you intended.

  • Data collected before the rules existed is handled another way - the rules will not touch it.

Last updated

Was this helpful?