Data Shipping

📘

Note

Data Shipping requires the Business or Enterprise plan.

Overview

The data shipping feature delivers click, conversion, and impression data direct to an Amazon AWS S3 bucket. This enables you to ingest large amounts of data without making requests to the Kevel reporting API. You may also perform advanced analytics on the raw data beyond what is currently available in the Kevel platform.

Please note: Once Data Shipping is configured, you will see data from new requests and events. We do not offer backfill of prior data.

Integration Steps

❗️

Warning

You should use our CloudFormation template or Terraform module below to properly set up data shipping. The template will create your S3 bucket for you.

Cloudformation

Kevel's CloudFormation template for data shipping will create:

• An S3 bucket for your Kevel data shipping logs
• A bucket policy authorizing Kevel to create objects in the bucket and list the objects the bucket contains. ListBucket permission is required to ensure all data shipping files are uploaded correctly.

1. Notify your account manager that you are interested in data shipping, and which AWS region you would like your S3 bucket to be in. Your account should be on the Business or Enterprise plan.
2. Choose a name for the CloudFormation stack: STACK_NAME, and a name for the new S3 bucket: BUCKET_NAME.
3. Get our CloudFormation template and use the AWS CLI to create the CloudFormation stack:

$ wget https://static.adzerk.net/docs/cloudformation-templates/create-datashipping-s3-bucket.yaml
$ aws cloudformation create-stack \
  --stack-name $STACK_NAME \
  --parameters ParameterKey=BucketName,ParameterValue=$BUCKET_NAME \
  --template-body file://create-datashipping-s3-bucket.yaml
  #Check output for support team
  aws cloudformation describe-stacks --stack-name $STACK_NAME  | grep Output

👍

Information

Use brew install wget to install wget on Mac OS.

4. Notify Kevel support when the bucket has been successfully created, or if you have run into any issues. To complete the integration, Kevel needs to know the following information:

👍

Information

The actual bucket name must have the prefix 'kevel-dataship-'
The Cloud formation Template takes care of adding this prefix.
Per AWS S3 guidelines, the bucket must be all lower case.

Terraform

Download terraform module here for use in your terraform project, also includes an example directory for a barebones terraform template that just makes the bucket. You will need to provide us with the same information - bucket name, aws region, kevel networkid - as in the cloudformation example above.

About Data Shipping Files

Data shipping returns a JSON object of an event (impression, click, or conversion).

For impressions, each ad serving engine will deliver a file approximately once per minute. These files are delivered as gzipped .log files.

For clicks and events data, all partners receiving data shipping will receive continuous shipping data.

📘

Note

If you use the AWS UI to download data shipping files from S3, the data shipping files will not be gzipped, although their file extension suggests they are. This is because your browser is unzipping the file content as it is downloaded. Downloading from your S3 bucket via other methods will return a gzipped file.

JSON Object

🚧

Caution

Certain fields, such as Location, may not be included as part of the JSON at all if there is no data to associate with the fields.

{
   "Id":"e1cf155248e541ba853b2ed482efd4ed",
   "CreatedOn":"/Date(1430156219349)/",
   "AdTypeId":5,
   "AuctionBids":1,
   "BrandId":15408,
   "BurnIn":0,
   "CampaignId":22318,
   "Categories":[

   ],
   "ChannelId":7731,
   "CreativeId":57609,
   "CreativePassId":82224,
   "Datacenter":true,
   "DeliveryMode":2,
   "Device":{  
      "brandName":"Google",
      "modelName":"Chrome",
      "osRawVersion":"0",
      "osMajorVersion":0,
      "osMinorVersion":0,
      "browser":"Chrome Desktop",
      "browserRawVersion":"63.0",
      "browserMajorVersion":63,
      "browserMinorVersion":0,
      "formFactor":"desktop",
      "deviceIdentified":true
   },
   "Ecpm":0,
   "FirstChannelId":7731,
   "GdprComputed":false,
   "GdprIsEU":false,
   "GdprSource":"none",
   "ImpressionCount":1,
   "IsNoTrack":false,
   "IsTrackingCookieEvents":false,
   "IsPublisherPayoutExempt":false,
   "IsValidUA":true,
   "Keywords":"",
   "Location": {
      "CountryCode": "US",
      "CountryName": "United States",
      "Region": "VA",
      "City": "Ashburn",
      "PostalCode": "20147",
      "MetroCode": 511,
      "DmaCode": 511,
      "AreaCode": 703
    },
   "MatchingKeywords":[

   ],
   "NetworkId":4161,
   "PassId":41596,
   "PhantomCreativePassId":0,
   "PlacementName":"3423",
   "PhantomPassId":0,
   "PriorityId":19301,
   "Price":"0",
   "RateType":1,
   "Revenue":0,
   "RemoteIP":"0.0.0.0",
   "RtbBids": 4,
   "SiteId":22032,
   "TimeZoneId":"America/New_York",
   "User":{
      "Key":"ue1-9040e0cc7c6a45afbf2ca1f40a67b75c"
   },
   "UserAgent":"",
   "UserKey":"ue1-9040e0cc7c6a45afbf2ca1f40a67b75c",
   "ZoneId":0
}