Data Shipping

🚧

Caution

Data Shipping requires the Business or Enterprise plan.

Overview

The data shipping feature delivers click, conversion, and impression data direct to an Amazon AWS S3 bucket. This enables you to ingest large amounts of data without making requests to the Kevel reporting API. You may also perform advanced analytics on the raw data beyond what is currently available in the Kevel platform.

Integration Steps

❗️

Warning

You should use the CloudFormation template below to properly set up data shipping. The template will create your S3 bucket for you.

Kevel's CloudFormation template for data shipping will create:

  • An S3 bucket for your Kevel data shipping logs
  • A bucket policy authorizing Kevel to create objects in the bucket and list the objects the bucket contains. ListBucket permission is required to ensure that all data shipping files are uploaded correctly.
  1. Notify your account manager know that you are interested in data shipping, and which AWS region you would like your S3 bucket to be in. Your account should be on the Business or Enterprise plan.
  2. Choose a name for the CloudFormation stack: STACK_NAME, and a name for the new S3 bucket: BUCKET_NAME.
  3. Get our CloudFormation template and use the AWS CLI to create the CloudFormation stack:
$ wget https://static.adzerk.net/docs/cloudformation-templates/create-datashipping-s3-bucket.yaml
$ aws cloudformation create-stack \
  --stack-name $STACK_NAME \
  --parameters ParameterKey=BucketName,ParameterValue=$BUCKET_NAME \
  --template-body file://create-datashipping-s3-bucket.yaml
  #Check output for support team
  aws cloudformation describe-stacks --stack-name $STACK_NAME  | grep Output

👍

Information

Use brew install wget to install wget on Mac OS.

  1. Notify Kevel support when the bucket has been successfully created, or if you have run into any issues. To complete the integration, Kevel needs to know the following information:

Info

Example

S3 bucket name (not the URL or another identifier)

mysite-kevel-rawdata

The AWS region the S3 bucket is in

us-east-1

Your Kevel network ID

12345

👍

Information

**The actual bucket name will receive the prefix 'kevel-dataship-'
above example will produce a S3 bucket named: 'kevel-dataship-mysite-rawdata'.
Contact Kevel support if another naming convention is needed.

Data shipping will return a JSON object of an event (impression, click, or conversion). An example impression object is shown below.

For impressions, each ad serving engine will deliver a file approximately once per minute. These files are delivered as gzipped .log files.

For clicks and events data, each gzipped file will be delivered once per day at the end of the day.

📘

Note

If you use the AWS UI to download data shipping files from S3, the data shipping files will not be gzipped, although their file extension suggests they are. This is because your browser is unzipping the file content as it is downloaded. Downloading from your S3 bucket via other methods will return a gzipped file.

JSON Object

🚧

Caution

Certain fields, such as Location, may not be included as part of the JSON at all if there is no data to associate with the fields.

{
   "Id":"e1cf155248e541ba853b2ed482efd4ed",
   "CreatedOn":"/Date(1430156219349)/",
   "AdTypeId":5,
   "AuctionBids":1,
   "BrandId":15408,
   "BurnIn":0,
   "CampaignId":22318,
   "Categories":[

   ],
   "ChannelId":7731,
   "CreativeId":57609,
   "CreativePassId":82224,
   "Datacenter":true,
   "DeliveryMode":2,
   "Device":{  
      "brandName":"Google",
      "modelName":"Chrome",
      "osRawVersion":"0",
      "osMajorVersion":0,
      "osMinorVersion":0,
      "browser":"Chrome Desktop",
      "browserRawVersion":"63.0",
      "browserMajorVersion":63,
      "browserMinorVersion":0,
      "formFactor":"desktop",
      "deviceIdentified":true
   },
   "Ecpm":0,
   "FirstChannelId":7731,
   "GdprComputed":false,
   "GdprIsEU":false,
   "GdprSource":"none",
   "ImpressionCount":1,
   "IsNoTrack":false,
   "IsTrackingCookieEvents":false,
   "IsPublisherPayoutExempt":false,
   "IsValidUA":true,
   "Keywords":"",
   "Location": {
      "CountryCode": "US",
      "CountryName": "United States",
      "Region": "VA",
      "City": "Ashburn",
      "PostalCode": "20147",
      "MetroCode": 511,
      "DmaCode": 511,
      "AreaCode": 703
    },
   "MatchingKeywords":[

   ],
   "NetworkId":4161,
   "PassId":41596,
   "PhantomCreativePassId":0,
   "PlacementName":"3423",
   "PhantomPassId":0,
   "PriorityId":19301,
   "Price":"0",
   "RateType":1,
   "Revenue":0,
   "RemoteIP":"0.0.0.0",
   "RtbBids": 4,
   "SiteId":22032,
   "TimeZoneId":"America/New_York",
   "User":{
      "Key":"ue1-9040e0cc7c6a45afbf2ca1f40a67b75c"
   },
   "UserAgent":"",
   "UserKey":"ue1-9040e0cc7c6a45afbf2ca1f40a67b75c",
   "ZoneId":0
}