Build a custom feed plugin
Build an audience segment external feed to push segments to a partner the platform does not connect to.
Your goal
Push audience segments to a system no existing connector covers, by writing and deploying an audience segment external feed plugin.
Expected outcome
A LIVE plugin version your users can attach to a segment, sending data to your partner reliably and within their rate limits.
Before you start
Read Plugin architecture. The plugin, version, worker, instance, and property model is what the rest of this page assumes.
Confirm no existing connector fits - check the Connectors section first.
Decide how the partner wants the data: unit updates, batches, files, or a pixel shown in the browser.
mediarithmics also builds feeds and makes them available to every customer: discuss your need with your Account Manager first.
The shapes of a feed
Server-side stream
Your plugin receives user segment updates and calls the partner's API.
Pixel
A pixel is shown to users in the segment, in the browser.
A feed can also write files instead of calling an API, through the file delivery service - see File destination. If it does, set the Use file delivery parameter on the version; it is optional and defaults to false.
Create and deploy
The plugin type is AUDIENCE_SEGMENT_EXTERNAL_FEED for a server-side feed, or AUDIENCE_SEGMENT_TAG_FEED for a pixel feed.
Create the plugin
In Computing Console, click New Plugin and configure:
Plugin type
One of ACTIVITY_ANALYZER, ATTRIBUTION_PROCESSOR, AUDIENCE_SEGMENT_EXTERNAL_FEED, AUDIENCE_SEGMENT_TAG_FEED, COMPUTED_FIELD_FUNCTION, DISPLAY_AD_RENDERER, ML_FUNCTION, SCENARIO_CUSTOM_ACTION.
Organisation
Defines availability. 1 or 100 makes the plugin accessible to all organisations; any other value restricts it to that organisation and its sub-organisations.
Group ID
Categorises plugins, typically by customer and type: com.mycustomer.plugintype for a customer-specific plugin, com.mediarithmics.plugintype for a shared one.
Artifact ID
A unique identifier for your plugin. Must not be the same as the group ID.
Feeds built by mediarithmics use group_id com.mediarithmics.audience.externalfeed and an artifact_id of the form <partner>-connector.
Or by API:
Create a version
Version
1.0.0, 1.2.0, or 1.2.0-rcl - three digits with an optional suffix.
Engine type
The system where the plugin is executed.
Plugin properties
The context sent to your plugin. provider and name are mandatory.
The body carries version_id, plugin_properties, optionally maintainerId, and optionally file_delivery_activated - false by default.
The version starts as DRAFT: editable, and not available to customers.
Add configuration files
A configuration file stores credentials, connection parameters, or technical settings for a specific customer, organisation, or datamart - API keys, authentication tokens, endpoint URLs. It keeps configuration adaptable without modifying the plugin code.
In the interface, they are under Technical Configuration on the plugin version: Add a Technical Configuration, give it a name, and define the content in JSON.
By API:
Add the layout
Audience segment feeds need a layout, the JSON and translation files behind the form users fill in when they add the feed. See Layouts for feeds.
Set up authentication, if the partner needs it
To let your users authenticate their own partner account when they create a destination, set the authentication type on the version and implement the authentication methods. See Authentication for feeds.
Declare external services
If your code calls external APIs - a CRM, a partner API, a recommendation engine - you must whitelist them.
url
The host of the service, for example https://graph.facebook.com.
technical_name
Deprecated: enter any string.
The response includes an enabled field, and it may be false.
Deploy the build
In the interface, open the Overview tab of your version and click Deploy the version. For an engine type of HOSTED_CONTAINERS, enter your build tag; the status becomes DEPLOYING while containers are created, then TESTING.
By API:
target_build_tag
The build tag you retrieved after your build.
action
DEPLOY_PLUGIN for a new version, or UPGRADE_ALL_CONTAINERS to update the code of an existing version.
Check the deployment
For HOSTED_CONTAINERS plugins, verify that the version is deployed in running containers - if no containers are running, the version is not usable.
In Computing Console, select your plugin and version, and open the Deployment tab to see the name and number of running containers.
By API:
You can also update the containers with a new build tag, which changes the code without creating a new version. Keep this for minor updates, such as debugging the existing version.
Move it to LIVE
TESTING is visible to organisation administrators only, and users cannot create instances from it. Once validated, set the version to LIVE. See the version lifecycle.
Code, build and test
Ask your Account Manager for the starter project of a feed plugin if you do not already have it. It uses the Plugin SDK, which builds the routes the platform calls: you override the methods below and make sure they return the right response.
A plugin project has this general structure:
src- the plugin code:index.ts- the root declaration of your plugin.MyPluginImpl.ts- extends the right Plugin SDK class.tests- the tests.Other folders and files as you wish.
build.sh- the build script.package.jsonrun.sh- the run script.
Build the project and run the unit tests from its root folder:
Testing relies on unit tests. Aim at maximising code coverage, and mock the inbound queries your plugin can receive.
The methods to implement
instanceContextBuilder
Once when the feed is instantiated, and again from the other methods when you want to refresh the context.
Builds the instance context in one place - properties, credentials, configuration files, API clients - and validates the feed properties and the selected identifiers. Extend the context returned by super.instanceContextBuilder(feedId) with what your code needs.
onExternalSegmentCreation
Synchronously, when the feed is created.
Checks whether the audience already exists on the partner platform, and creates it if not. Returns a status (ok, error, retry or no_eligible_identifier), an optional message, and a visibility (PRIVATE or PUBLIC).
onExternalSegmentConnection
To establish the connection with the segment on the partner platform.
Checks that the segment is available on the partner platform.
onUserSegmentUpdate
On each update of a user in the segment.
Sends the user identifiers, for insertion or deletion, to the partner. With batching or file delivery, it returns the lines to add to the batch or the file instead.
onTroubleshoot
From the feed's troubleshooting section, with the action FETCH_DESTINATION_AUDIENCE.
Returns data displayed in the troubleshooting section: as much as possible to compare the identifiers received, stored and used by the partner platform.
onBatchUpdate
Only when you batch the calls to the partner API or send a flat file.
Sends the batch the platform built from the onUserSegmentUpdate responses. See Batching for feeds.
onDynamicPropertyValuesQuery
From the feed creation form, for DYNAMIC_SELECT fields.
Returns, for each property_technical_name, an enum of label and value pairs suggested from the partner solution. One dynamic property may depend on another, read from the properties already set in the form.
onDestinationAudienceDeletion
When a user deletes a feed together with its destination audience, on a version where Delete audience in partner platform is enabled.
Deletes the audience on the partner platform. Returns a status (ok, not_found or error) and an optional message. See Let users delete the destination audience.
instanceContextBuilder, onExternalSegmentCreation, onExternalSegmentConnection, onUserSegmentUpdate and onTroubleshoot are mandatory. If the feed manages authentication, it also implements the methods in Authentication for feeds.
Let users delete the destination audience
When users delete a feed, they can choose to delete the audience it created on the partner platform as well - see Delete a feed. Your plugin offers that choice only once both steps below are done.
Implement onDestinationAudienceDeletion
The method receives the request - feed_id, datamart_id, segment_id, and feed_destination_id when the feed uses a destination's credentials - the instance context, and those credentials when there are any. It deletes the audience on the partner platform and returns a status:
status
Return it when
What the platform does
ok
The audience is deleted.
Stops synchronising the feed, then deletes it.
not_found
The partner platform says the audience does not exist.
The same as ok: the audience is already gone.
error
The deletion failed.
Keeps the feed, and shows an error to the user.
Return not_found, not error, when the audience is already absent.
Write the plugin so it survives a second instance
The same version is attached by your users to several segments, each creating an instance. Your code must read its instance rather than assume one configuration:
Find the instance ID in the request body.
Fetch - and cache - the properties of that instance.
Process according to that configuration.
The Plugin SDK does this for you. See Plugin architecture for the property model.
Respect the partner's limits
Partner APIs have their own rate limits, and a plugin that does not handle the resulting errors well will fail in production. Configure a throttling on the plugin version - calls per second, concurrency, and timeout - rather than relying on the partner to be forgiving. See Plugin architecture.
If the partner needs batched input rather than one user at a time, see Batching for feeds.
Watch it in production
Logs work as for every plugin: see Watch it in production. Prefix every log line with the segment, feed and datamart IDs, so you can filter down to one feed.
Feed statistics, and the troubleshooting tab your onTroubleshoot method feeds, are covered in Track a feed (stats and troubleshoot).
Verify
The version reaches
LIVE, with running containers.Every external host your code calls is declared and enabled.
Two instances of the same version, configured differently, behave differently.
A test segment reaches the partner with the expected payload.
Throttling keeps you inside the partner's limits under load.
Related pages
Last updated
Was this helpful?