API Reference

The Data Talks API provides a way for our customers to integrate and send data from multiple systems through a single endpoint. The supported integration can be enabled or disabled for customers, and the way data is organized and sent to the different systems is controlled via mappings. This documentation aims to describe and explain what the currently supported integrations are, how to authenticate and use the API, and how to define the mappings for an integration.

OpenAPI (Swagger) Specification #

https://api.datatalks.se/index.html

API Endpoints #

v3 #

Requests and Responses #

datatalksId - optional parameter field (see Basic Concepts section).

Profiles – Example POST Request #

{
"type": "person",
"null": "main_group",
"data":{
		"customer_id":12345,
		"first_name":"Juliana",
		"last_name":"Crane",
		"email":"juliana.crane@fake.com",
		"age":32,
		"postal_code":"111 17",
		"address":"Kungsbroplan 3",
		"city":"Stockholm",
		"country":"Sweden",
		"is_member":"Yes",
		},
	"datatalksId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

Events – Example POST Request #

{
"type": "purchase_ev",
"null": "main_group",
"data":{
		"event_id":"848ASHC382392AHSJA",
		"event_type":"Purchase",
		"event_datetime":"2021-03-18 11:35:24",
		"customer_id":12345,
		"status":"Confirmed",
		"payment_method":"Debit Card",
		"total_purchase_amount":"123.75",
		"total_purchase_discounted_amount":"112.00",
		"currency":"Euros",
		"online_purchase":"No",
		"Member_points_before_purchase":4561,
		"purchase_points":112
		},
	"datatalksId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

Inventory – Example POST Request #

{
"type": "product_a",
"null": "main_group",
"data":{
		"item_id":"AB-192910",
		"inventory_type":"Product",
		"item_modified":"2020-12-03 17:23:21",
		"name":"Milk",
		"category":"Dairy Products",
		"subcategory":"Cow Milk",
		"brand":"Milky",
		"size":"1.0L",
		"type":"Light"
		},
	"datatalksId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

Response Example #

{
    success = true/false,
    messages = [""],
    requestId = requestId,
    datatalksId = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx,
    key = yourAwsKey,
}

Response Codes #

Batch API #

With v3 of the Data Talks API, you have the opportunity to ingest data in batches to the Data Talks CDP. This will enable faster data ingestion and demand lower effort from your side. 

Using the Data Talks batch processing framework, you are allowed to send a maximum of 50 (and a minimum of 1) items per request.

Further constraints: 

• The ‘Type’ field needs to be specified in the payload according to the Data Point you are about to send data to. This you can find in the Data Onboarding section of the CDP. If you need help, please reach out to your Customer Success Manager.
• Each item needs to have a ‘requestId’ field set.
  • Each item’s ‘requestId’ field value needs to be unique within the batch request.
  • This ‘requestId’ field can only contain alphanumerical or punctuation characters.
    • Correct requestId examples: “My profile”, “my_profile”, “myprofile”
    • Incorrect requestID examples: “My$Profile”, “My\Profile”, “My&Profile”
• The batch itself can have an optional ‘requestId’ field set to help tracking the request.
  • This ‘requestId’ field can only contain alphanumerical or punctuation characters.

Request #

The requests to the batch endpoint should have the following structure:

Endpoint: https://api.datatalks.se/v3/<datapoint>/batch

Auth: <AWS IAM Auth>

Body:

{
  "requestId": "<optional_id>",
  "requestList": [
    {
      "type": "...",
      "requestId": "...",
      "data": {
          "Key1": value1,
          "Key2": value2,
          ...
      }
    },
    ...
  ]
}

Examples of “Key1” can be e.g.: “email”:”john.doe@gmail.com”; “name”:”John Doe”

Response #

The responses from the batch endpoint have the following structure:

{
 "responseList": [
        {
            "dataTalksId": null,
            "success": <bool>,
            "messages": [<List of error messages>],
            "requestId": "<The provided request id from request>",
            "key": null
        },
        ...
    ],
    "responseStatus": <0: Notspecified, 1: Successful, 2: PartiallySuccessful>,
    "success": <bool>,
    "messages": [<error messages related to whole batch request>],
    "requestId": "<guid>",
    "key": "<AWS_KEY>"
}

Response Codes

Batch response is different from the singular API response in two ways:

• The response codes will be slightly different:
  • 5xx errors are the same as for singular requests
  • 4xx errors are more rare and only be thrown when the errors are on the batch level, or if the request is faulty or incomplete
  • 2xx responses will be most common, and they will be returned even if items in the batch get errors. Errors will instead be shown at the item level, where the “success” and “messages” fields describe the status of the item.
• There is a new field “responseStatus”, which contains status information on the job at a batch level. It can obtain one of three values:
  • Notspecified – Returned only in 4xx errors. Returned if there are missing things in the request, e.g:
    • Request ids not being unique,
    • Type not being specified
    • Too many items in request
    • No items in request
  • Successful – returned if all items were successfully processed.
  • PartiallySuccessful – returned if some items were not correctly processed.

Example #

Request #

The following is an example of how the request to the batch endpoint can look:

Endpoint: https://api.datatalks.se/v3/events/batch

Auth: <AWS IAM Auth>

Body:

{
    "requestList": [{
            "type": "My Profile Type",
            "requestId": "My-test-batch-3.1",
            "data": {
                "Email": "john.doe.test.batch.1@datatalks.se",
                "City": "Stockholm",
                "PhoneNumber": "713371336",
                "Name": "John"
            }
        }, {
            "type": "My Profile Type",
            "requestId": "My-test-batch-3.2",
            "data": {
                "Email": "jane.doe.test.batch.2@datatalks.se",
                "City": "Stockholm",
                "PhoneNumber": "713371337",
                "Name": "Jane"
            }
        }, {
            "type": "My Profile Type",
            "requestId": "My-test-batch-3.3",
            "data": {
                "Email": "will.doe.test.batch.3@datatalks.se",
                "City": "Stockholm",
                "PhoneNumber": "713371338",
                "Name": "Will"
            }
        }
    ],
    "requestId": "My-test-batch-3"
}

Response #

Given the above request, the folllowing is an example response:

{
    "responseList": [{
            "dataTalksId": null,
            "success": true,
            "messages": [],
            "requestId": "My-test-batch-3.3",
            "key": null
        }, {
            "dataTalksId": null,
            "success": true,
            "messages": [],
            "requestId": "My-test-batch-3.2",
            "key": null
        }, {
            "dataTalksId": null,
            "success": true,
            "messages": [],
            "requestId": "My-test-batch-3.1",
            "key": null
        }
    ],
    "responseStatus": 1,
    "success": true,
    "messages": [],
    "requestId": "My-test-batch-3",
    "key": "<AWS_KEY>"
}
   

FAQ 

• What if a customer I am already sending data to Data Talks using v3 and I want to change to using batch. Can I just update the procedure without informing Data Talks?
  • Yes, you can go ahead freely. 
• What will “Type” refer to in the batch request?
  • It will have the exact same meaning as for the singular API endpoints.
  • Thus, it will refer to the Datapoint Type Title of the datapoint, i.e. the name that was set for the datapoint in the CDP onboarding page. You will receive these names from your Customer Success Manager. 
• If I receive a 5xx error, what should I do?
  • If a batch request gets a 5xx error, it has to be resent in its entirety to Data Talks Api, because none of it got processed and sent further down the data pipeline.
  • Should you see 5xx response messages other than 502 or 504, please inform your Customer Success Manager, so we can support you the best possible way

Basic Concepts #

Data Talks ID #

Datatalks ID is a UUID (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) that is automatically generated, stored and returned for every entity processed by API. We create a unique Datatalks ID for each profile, but also for each data point type (events etc).

Identifying a customer #

Data Talks should create a Datatalks ID for every customer that arrives in our systems through API, files, or third-party systems. The Datatalks ID is the master id and all potential IDs coming from the source systems should belong to this one as secondary IDs. Different customers can have different requirements when it comes to the Unique Id that they can send to us. Some have email as the unique id while some of them have a CRM id as the unique identifier.

Custom Mappings #

You can describe (map) the entities (Events, Profiles, and Inventories – see Endpoints). As long as it suits the data types that Data Talks CDP accepts, you can define as many fields as you like and the way they will look like. Therefore the following can be defined: the name of the source field, the name of the field to appear in Data Talks CDP, the data type (respecting the limitations), and the format of the data.

Mappings should be defined per each entity (datapoint type) on the onboarding stage, before start using API.

Any unmapped payload field will be ignored by default.

Any missing field will lead to 400 – Bad Request API response. Except for the fields marked as Optional.

Data Validation #

Date #

Dates need to be given in ISO format: YYYY-MM-DD

Datetime: YYYY-MM-DD HH:MM:SS

The date format cannot contain a time zone.

Email #

Email addresses need to be given in following formats (examples below);

someone+tag@somewhere.net
someone@somewhere.com
someone@somewhere.co.uk
someonetag@somewhere.net

Examples of invalid email addresses below, which will be rejected by the API:

mma a@gma.com
“cogwheel the orange”@example.com
fdsa
fdsa@
fdsa@fdsa
fdsa@fdsa

Phone number #

There is no validation of phone numbers done in Data Talks API.

FAQ #

Q: What’s a Hard-ID?

A: This is what makes a person or event unique, and should be the unique denominator using the least amount of attributes. This value cannot be NULL or missing. For instance, the Hard-ID for a profile could be their customer ID (customer_id), for an event it could be the transaction ID (transaction_id). Sometimes the Hard-ID will be a combination of more than one attribute.

Q: What does Type refer to?

A: We use type (profile_type, event_type, inventory_type) to identify what kind of profile, event or inventory data point we are receiving. It’s not an attribute but part of the wrapper. The value of the TYPE field is expected to match the names of the equivalent data point names created in the CDP.

Q: What happens if an email address is rejected by the API?

A: The API caller is informed that there were invalid email addresses in the data.

Q: What happens when the email field is null?

A: As long as the email address isn’t set as Hard-ID*, it will work. If it is however set as Hard-ID, the field cannot be null.

*Hard-ID is the unique identifier for Profiles in your CDP account.

Q: What does optional mean?

A: If an attribute is set as optional, the API won’t fail the payload if the attribute is missing. The missing attribute, however, will be interpreted as set to null.

Q: How does the API treat an empty string value versus a null value?

A: If the data type is not a string an empty string will fail the API call. An empty string is not recommended to represent the lack of data, you need to ensure NULL is used instead.

Authentication #

To build the Authorization header you will need AWS IAM account credentials – ACCESS_KEY_ID and SECRET_ACCESS_KEY. When requesting access to our API an account will be created for you by our team and the credentials will be sent to you.

The authentication of the API is implemented via AWS IAM. When making an API request you will have to authenticate via AWS Signature Version 4. Here is an example of an Authorization header done with AWS Signature Version 4:

Authorization: AWS4-HMAC-SHA256
Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/execute-api/aws4_request,
SignedHeaders=host;range;x-amz-date,
Signature=fe5f80f77d5fa3beca038a248ff027d0445342fe2855ddc963176630326f1024

To build the Authorization header you will need an AWS IAM account credentials – ACCESS_KEY_ID and SECRET_ACCESS_KEY. When requesting access to our API an account will be created for you by our team and the credentials will be sent to you.

Implementing Authentication #

There is an AWS SDK for the most popular development stacks (.NET, Java, Javascript, Ruby, Python, etc.). Some of the SDKs might not provide the Signature Version 4 out of the box, so you might need to implement it by yourself. Browse the articles under this section to see examples of signing requests with Signature Version 4.

Additional Resources #

Signature Version 4 Signing Process

v2 #

Salesmanago Integration (v2) #

If the payload is supposed to be sent to Salesmanago, groupKey field must be filled (see v1 documentation).

A group key is our internal way to group your integrations. This is useful if you have multiple brands or markets and have different accounts in the integration systems and you want to send separate API requests to them. In case you don’t have such, you will have only one group_key to work with. Make sure that the group_key is sent in every request.

API v1 #

https://datatalks.atlassian.net/wiki/spaces/PI/pages/875102209/Datatalks+API

Staging API #

If you want to test API out in a safe environment you can use the staging endpoints:

Sales Manago