> For the complete documentation index, see [llms.txt](https://docs.mediarithmics.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mediarithmics.io/setup-and-monitor/collect-and-manage-your-data/clean-and-delete-data/cleaning-rules.md).
# Cleaning rules
Define how each datamart expires events and profiles, with actions, filters, and priorities.
## Your goal
Make the datamart expire events and profiles on its own, according to a policy you can state and defend.
## Expected outcome
A set of `LIVE` rules whose combination produces the retention you intended - for every event and profile, on every channel and compartment.
## What a cleaning rule is
**Cleaning rules define how each datamart manages the deletion of user data.** They apply to two objects: **profiles** and **events**.
When an object enters the datamart, the platform **computes its expiration timestamp** from the rules that apply. That timestamp becomes part of the object's properties and **cannot be modified**.
```mermaid
flowchart LR
A["A UserProfile or UserEvent enters the datamart"]
B["Which cleaning rules apply to it?"]
C["Compute the expiration timestamp from the rules and their priority"]
D["Store the expiration timestamp in the object"]
A --> B --> C --> D
```
{% hint style="danger" %}
This is why **cleaning rules are non-retroactive**: changing the rules only affects **future** incoming events and profiles.
{% endhint %}
### How expiry is evaluated
An event or profile expires if it has not been modified before its expiration timestamp:
| Object | Field compared against the expiration timestamp |
| ----------- | ----------------------------------------------------------- |
| **Profile** | The last modification timestamp, `$last_modified_ts`. |
| **Event** | The creation timestamp, `$ts` - an event cannot be updated. |
**When an object expires it stops appearing in queries and in the interface.** It still exists in the passive database until it is removed by a scheduled operation.
## The two types
| Type | Applies to | Can be restricted to |
| ------------------------------------------------ | ---------- | ----------------------------------------------------------------- |
| **Event-based** (`USER_EVENT_CLEANING_RULE`) | Events | All events, or specific channels, activity types, or event names. |
| **Profile-based** (`USER_PROFILE_CLEANING_RULE`) | Profiles | All profiles, or a compartment. |
Every rule has a **status** (`DRAFT`, `LIVE`, or `ARCHIVED`), an **action** (`KEEP` or `DELETE`), a duration, and a set of filters.
* `KEEP` - the object should **stay** in the datamart for the duration.
* `DELETE` - the object should be **removed** after the duration has expired.
{% hint style="info" %}
Only `DELETE` is allowed for profile cleaning rules. `KEEP` exists for events, where it is used to protect a subset from a broader `DELETE`.
{% endhint %}
Cleaning rules are edited in the Navigator, in your datamart settings.
## Priority: how the combination resolves
Two rules decide the outcome:
* **`KEEP` rules take priority over `DELETE` rules.**
* **Only `LIVE` rules** are taken into account when computing an expiration timestamp.
Three worked examples
**Example 1** - these rules apply to an incoming event:
* `KEEP` for 60 days
* `KEEP` for 180 days
* `DELETE` after 150 days
The event is kept for **180 days**, then deleted. The longest `KEEP` wins over the `DELETE`.
**Example 2** - these rules apply to an incoming event:
* `KEEP` for 60 days
* `DELETE` after 150 days
The event is deleted after **150 days**. The `KEEP` is shorter than the `DELETE`, so it changes nothing.
**Example 3** - these rules apply to an incoming profile:
* `DELETE` after 10 days
* `DELETE` after 150 days
The profile is deleted after **10 days**. Among `DELETE` rules, the shortest wins.
## Lifecycle
```mermaid
stateDiagram-v2
[*] --> DRAFT: created
DRAFT --> DRAFT: properties edited (not the type)
DRAFT --> LIVE: edit request
DRAFT --> [*]: deleted
LIVE --> ARCHIVED: status change only
```
* Rules are created as `DRAFT`. They cannot be created `LIVE` or `ARCHIVED`.
* While `DRAFT`, you can change the rule's properties - **except its type**.
* You move a rule to `LIVE` with an edit request. **Once `LIVE`, a rule cannot be edited**; only its status can change, to `ARCHIVED`.
* Once `ARCHIVED`, a rule cannot be changed or deleted.
* **Only `DRAFT` rules can be deleted.** A `LIVE` rule has to be archived instead.
{% hint style="warning" %}
**At any moment there should always be a `LIVE` event-based rule with a `DELETE` action.** To archive the one you have, create another first.
{% endhint %}
## Rule properties
| Property | Type | Description |
| ---------------------- | ----------------- | ----------------------------------------------------------------------------------------------------------- |
| `id` | String | The cleaning rule identifier. |
| `action` | Enum | `KEEP` or `DELETE`. Only `DELETE` is allowed for `USER_PROFILE_CLEANING_RULE`. |
| `archived` | Boolean | Makes a rule disappear from the interface. Can only be set to `true` when the status is already `ARCHIVED`. |
| `status` | Enum | `DRAFT`, `LIVE`, or `ARCHIVED`, in that order only. |
| `datamart_id` | String | The associated datamart. |
| `type` | Enum | `USER_EVENT_CLEANING_RULE` or `USER_PROFILE_CLEANING_RULE`. |
| `life_duration` | ISO 8601 duration | The duration associated with the rule - for example `P65D`. |
| `channel_filter` | String | **Event rules only.** Optional. The channel the rule applies to. |
| `activity_type_filter` | Enum | **Event rules only.** Optional. `SITE_VISIT`, `APP_VISIT`, `TOUCH`, `DISPLAY_AD`, or `EMAIL`. |
| `compartment_filter` | String | **Profile rules only.** Optional. The compartment the rule applies to. |
{% hint style="warning" %}
`channel_filter` and `activity_type_filter` must be **compatible** when both are set. You cannot target an app channel and set the activity type to `DISPLAY_AD`, for instance.
{% endhint %}
### Content filter
An event cleaning rule can carry a content filter:
| Property | Type | Description |
| -------------- | ------ | ---------------------------------------------------------------------- |
| `filter` | String | The value of the filter. |
| `content_type` | Enum | The target of the filter. Only `EVENT_NAME_FILTER` is allowed for now. |
This is how you give a specific event name its own retention.
## Designing the set
{% stepper %}
{% step %}
### Start from the requirement, not the rules
State the retention per data class first - see [Data retention](/setup-and-monitor/plan-and-integrate/set-up-the-environment/manage-data-retention.md).
{% endstep %}
{% step %}
### Write the baseline `DELETE` rule
One event-based `DELETE` covering all events, at your default retention.
{% endstep %}
{% step %}
### Add `KEEP` rules for what must live longer
Filtered by channel, activity type, or event name. Remember that a `KEEP` shorter than the applicable `DELETE` has no effect.
{% endstep %}
{% step %}
### Add profile rules per compartment
`DELETE` only, and remember the shortest one wins.
{% endstep %}
{% step %}
### Publish, and re-check the combination
Only `LIVE` rules count. Walk one representative object of each class through the set and confirm the resulting expiry is what you intended - you cannot change it after the object is written.
{% endstep %}
{% endstepper %}
## Verify
* Every rule you rely on is `LIVE`, not `DRAFT`.
* There is a `LIVE` event-based rule with a `DELETE` action.
* For each data class, walking the rules by hand produces the expiry you intended.
* Data collected before the rules existed is handled another way - the rules will not touch it.
## Related pages
Data retention
Deciding the policy, and the fixed platform windows.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://docs.mediarithmics.io/setup-and-monitor/collect-and-manage-your-data/clean-and-delete-data/cleaning-rules.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.