Sitecore Data Exchange Framework

Sitecore recently released the newest addition to its integration possibilities – the Data Exchange Framework. What it is it good for? Let us have a look!

First of all, why does the Data Exchange Framework matter? Till now, Sitecore had options to get data in and out of the platform, but a single, clear, configurable framework, applicable to a multitude of use cases was missing. And that is exactly the purpose of the Data Exchange Framework.

You can think of the Data Exchange Framework as standalone ETL (Extract Transform Load) tool that extracts data from the source and loads them into the target. All of it is fully configurable from the Sitecore backend, but can run completely outside of the Sitecore process.

Best of all, Sitecore can be the source or target location, but it does not have to! Imagine that you need to do a product sync between your ERP and a Digital Asset Management system, you can still set everything up, using the Data Exchange Framework.

Use Cases Where To Use Data Exchange Framework

So, where exactly can you use the Framework? There are many scenarios, where you could want to sync data in and out of Sitecore – most typically synchronize your CRM data with Sitecore’s xDB. There is a connector for Microsoft Dynamics CRM that builds on top of the Data Exchange Framework and it is available as Sitecore Dynamics CRM Connect.

But you can create any custom connector that you prefer and load data from other systems, like CRM and ERP systems, other content management systems, or even flat files. Ever needed to update daily hundreds of entries out of an Excel file? Then the Data Exchange Framework is your tool of choice.

What Is The Anatomy Of The Data Exchange Framework

Since the Data Exchange Framework lives completely outside Sitecore, it does not work on top of the standard Processors and Pipelines of Sitecore. However, it has a similar architecture that you can use to your liking.

Process (the engine)

Pipelines – Pipelines are collections of instructions with no properties, but with context. Even if they live outside Sitecore, they are configured with Sitecore items. There is a component called “Converter” that makes sure, the configuration performed in Sitecore and stored as Sitecore items is properly translated into configurations understandable by the Data Exchange Framework.

Pipeline Steps – Like Sitecore uses Processors, the Data Exchange Framework uses Pipeline Steps. They represent a single unit of work and use pipeline-context for inter-step communication.

Pipeline Batches – Whenever you need to run multiple pipelines in a certain order you use Pipeline Batches. Pipeline Batches are used especially in bulk operations, like synchronizations of large quantity of CRM entries into Sitecore.

Data (the payload)

Endpoint – is the place where to take data from and where to write them to. It represents a data source with a unique API.

Mapping (the transformation)

Value Accessors – used to read (Value Readers) and write (Value Writers) – They read and write data from the objects exposed over the endpoints.

Value Mapping – It is used to “translate” data exposed from the read endpoint to the written endpoint. Custom logic can be applied to do appropriate transformations.

Configuration

Provider – The provider is the physical implementation that makes use of the Data Exchange Framework and is aimed at a specific purpose. The Dynamics CRM Connect is an example of such provider.

Tenant – Is a unique set of configurations for a Provider. Say, you have two Dynamics CRM accounts – one for your sales dept. and one for your service dept. You would still use a single Provider for connecting Sitecore with Dynamics CRM, but it would run as two separate Tenants, never interfering.

Running The Process

In order to run the data transfer through the Data Exchange Framework, you need to set up a Runner. This can be done in three ways:

  1. Manually
  2. Scheduled
  3. via API

You can run the process by Sitecore (using a job or the API) or completely standalone. If the synchronizations run out of the Sitecore process, the initialize pipeline cannot be used. Instead the components would need to be initialized. Furthermore, such standalone components cannot  depend on the Sitecore context

Closing Thoughts

The Data Exchange Framework is a separate component available for download. This is the new data exchange platform, based on which Sitecore is evolving its current integration strategy (like the CRM Connect).

You can learn more detail in the excellent documentation available at http://integrationsdn.sitecore.net/DataExchangeFramework/v1.1/

4 thoughts on “Sitecore Data Exchange Framework”

  1. Do you think that the Data Exchange Framework is going to supersede Sitecore.Services.Client for accessing items within Sitecore (e.g., for SPA applications)?

    1. Hi Corey,
      thanks for the comment! I think that the aim of the Data Exchange Framework is slightly different. It is mainly aimed at synchronizing (or transferring for that matter) data from source to target with transformations to it, rather than providing Sitecore content directly to consumers like an SPA.

    1. Hi Michal,

      I would say, it highly depends on the intended usage. If you need Data Decision platforms integration for on-the-fly data, then this might not be the right tool. But if you would like data extraction and loading to/from Sitecore to DMP/DSP for batch operations, then this is the right tool.

Leave a Reply

Your email address will not be published. Required fields are marked *