Relevancy Score

📘

Summary

With Kevel's Relevancy Score functionality, you can directly influence ad selection in an auction using your own relevancy or quality score data. Kevel's model can also produce the score for your network. Kevel combines the relevancy score with each ad's eCPM to calculate an Ad Rank, which the Ad Decision Engine uses to rank ads.

What is a relevancy score?

Relevancy score is a Decision API capability that adjusts how each ad ranks in an auction. You can supply the score from your own relevancy or quality data, or Kevel's model can produce it for your network.

For example, say you've built a self-serve advertising system on top of Kevel where bidders compete in an auction. Some bidders may bid a high CPC for clicks, but their ads are not relevant to the user, or they are poor quality, or they otherwise do not perform well.

By assigning a relevancy score, you can choose how to reward ads that are on-message and popular with viewers while requiring irrelevant ads to pay more to be eligible to serve. An ad with a higher relevancy score will win an auction over similarly priced ads.

Example use cases:

• Using your own click/conversion propensity models to influence the auction so that ads that the user is likely to click on receive an increased likelihood of selection
• Providing an artificial boost smaller advertisers in auctions to allow them to compete with high-budget advertisers
• Punishing promoted items that have poor organic performance
• Punishing low-margin products so that they need to bid more to be selected

In Kevel, relevancy scores are integers between 1 and 1000. The lower the relevancy score, the lower the ad will be ranked in the auction.

📘

Note

An ad that does not have a defined relevancy score will have a default relevancy score of 500. As such, ads that have a relevancy score of less than 500 are considered to be punished, whereas ads that have a relevancy score of over 500 are seen as favourable over similarly priced ads.

Using relevancy scores

A relevancy score comes from one of two sources:

• Your own scores, supplied in either of the two ways below.
• Kevel's model, which produces personalized scores from your network's own activity. See Kevel's relevancy model.

There are two ways to supply your own scores:

• In the Decision API request, per placement. Use this when the score depends on the user or the context of the request, for example a click propensity score for this shopper on this page. This is described below.
• Stored on the Ad or Flight, applied to every request. Use this when the score is the same for every request, for example a product quality or performance score maintained in your Catalog. See Storing relevancy scores on ads and flights.

Scores supplied in a request take precedence over stored scores. If Kevel's model is enabled for your network, stored scores are not used: the model supplies the score instead, and scores supplied in a request still take precedence. See Kevel's relevancy model.

Setting relevancy scores in a request

You can set relevancy scores in the Decision API request in each placement, mapped against:

• On creative template fields/values
• On an AdvertiserID / CampaignId / FlightID / AdId

Mapping relevancy scores to creative template fields and values

If you are using creative templates to store creative metadata (such as product attributes), you can use a creative template field and define how relevant its values are. For example, assuming your creative template had a field ctCategoryName - you may want to define relevancy scores that represent how likely you believe the user is to engage with each category.

"relevancy": {
  "idAttribute": {
    "ctCategoryName": {
      "shoes": 831,
      "shirts": 645,
      "jeans": 505,
      "sportswear": 402
    }
  }
}

In the example above, any ad that has a ctCategoryName that contains one of the following values shoes, shirts, jeans, or sportswear will receive a boost or penalty in the auction process.

Relevancy scores don't require the use of AdQuery, although the two are often used in combination with each other, particularly for Auction as a service use cases, as demonstrated below:

"adQuery": {
  "ctProductId": {
    "in": [
      "123",
      "234",
      "345",
      "456",
      "567"
    ]
  }
},
"relevancy": {
  "idAttribute": {
    "ctProductId": {
      "123": 832,
      "234": 645,
      "345": 505,
      "456": 811,
      "567": 484
    }
  }
}

Mapping relevancy score to entity IDs (advertiser, flight, and so on)

Alternatively, you can map relevancy scores to one of the following entity IDs: advertiserId, campaignId, flightId, adId.

For example:

"relevancy": {
  "idAttribute": {
    "adId": {
      "123456": 403,
      "234567": 901,
      "345678": 133,
      "456789": 482
    }
  }
}

📘

Note

We do not suggest sending relevancy score data for multiple idAttributes - only a single attribute will be used for relevancy score calculations.

Storing relevancy scores on ads and flights

Instead of sending a score with every ad request, you can store a relevancy score on the Ad or the Flight. The ad server applies the stored score to every auction the Ad competes in, unless the request supplies scores for that placement or Kevel's model is enabled for your network. This suits scores that do not vary by user or request context, such as a product quality or performance score you maintain in your Catalog.

Enabling an auction priority for relevancy scores

Stored scores are only used once the auction priority has been enabled for them. Create or update the priority via the API to include the following object:

"RelevancyScoreConfig": {
  "components": {
    "custom": 5
  },
  "aggregation": 2
}

components sets the metric(s) used to calculate relevancy scores. Currently only one metric is available, custom, which reads the score stored on the ad or flight. Its value sets the aggregator for the component.

aggregation sets how multiple components are combined into a single relevancy score. Only one metric is available, so set it to an integer between 1 and 5.

Aggregators

Each component uses an aggregator to compute a relevancy score when scores have been set on both a flight and its ad. Set the aggregator as an enum.

Setting the score

Add the CustomRelevancyScore parameter to the flight or ad object when creating or updating it. Scores are integers between 1 and 1000.

curl -X POST -H "X-Adzerk-ApiKey:<API-KEY>" https://api.kevel.co/v1/flight -d 'flight={"Name":"Relevancy Score","StartDate":"1/1/2015","CampaignId":123456,"PriorityId":123456,"GoalType":2,"IsActive":true,"Price":1,"Impressions":100,"RateType":2,"NoEndDate":true,"IsUnlimited":false,"CustomRelevancyScore":500}'

curl -X POST -H "X-Adzerk-ApiKey:<APIKEY>" https://api.kevel.co/v1/flight/12345/creative -d 'creative={"CampaignId":12345,"Creative":{"Id":123456},"FlightId":12345,"IsActive":true,"CustomRelevancyScore":500}'

🚧

Requirements

Storing relevancy scores must be enabled for your network, and the auction priority must be configured as described above. Until both are in place, ads are scored at the default of 500. Stored scores are not used while Kevel's model is enabled for your network. If your Ads are created from a Catalog using an ad template, the score can be mapped onto the Ad directly, so it stays in step with your feed. Contact your account manager to enable this functionality.

📘

Note

Relevancy scores are used in auctions. Ads served through a lottery priority are selected by weight, and relevancy scores do not apply to them.

Kevel's relevancy model

Kevel's model produces a personalized relevancy score for each ad from your network's own activity, favoring the ads a shopper is more likely to engage with. The score enters the auction the same way as a score you supply. Ad Rank is the ad's eCPM multiplied by its relevancy score, and second price clearing works as described in Relevancy Score and Revenue.

🚧

Availability

Kevel's model is enabled per network. Contact your account manager to enable it or switch it off. Neither step changes your ad requests.

The source of the score is decided per placement. A placement that sends relevancy scores (relevancy.idAttribute) is ranked on your scores alone. Ads without a score take the default of 500, and Kevel's model is not used for that placement. A placement without relevancy scores is left to Kevel's model. To keep a placement off the model, send your own scores on it. A request can carry several placements, and each follows this rule on its own. A multi-winner placement is a single auction, so all of its winners are ranked by the same source.

Stored scores are not used while the model is enabled. This applies to all of your traffic. To keep applying a score of your own, send it in the request for the placements where it matters. Those placements are then ranked on your scores instead of the model's.

Shoppers without consent do not get personalized scores. When a request is subject to GDPR and the shopper has not given consent, the model does not use personalized relevancy scores for them. Nothing about them is used to train it. See GDPR Compliance and Consent Settings for how to pass consent.

To check which source ranked a placement, use includeRelevancyData. Set "includeRelevancyData": true in the request, as described in Reporting on Relevancy Score. The relevancy object on each selected ad in the Decision API response then carries type, which is kevel when the placement was left to Kevel's model. It also carries scored, which is true when the model scored that decision. Other fields in the object are diagnostic and can change.

Relevancy Score and Revenue

Unlike standard auctions, auctions that use relevancy scores will choose the ad with the highest AdRank:

AdRank = eCPM * RelevancyScore

First price auctions

In a first price auction, revenue-generating events will record the full price of the ad.

Second price auctions

In a second price auction, impressions chosen using relevancy score will record the clear price of the ad. This is defined differently from other second price auctions:

Clear Price = (Ad Rank of Runner Up / Winner's Relevancy Score) + $0.01

Reporting on Relevancy Score

Relevancy score data is available in data shipping as the RelevancyScore field, on the auction, selection and impression logs.

To audit how a score affected an auction, use the auction log. It records RelevancyScore alongside Rank (eCPM * RelevancyScore) for each bid, so you can see both the score applied to an ad and the ranking it produced. This is the reliable place to confirm the score that was applied. It records the score, not where the score came from. To see the source for a placement, use includeRelevancyData, described below. It adds relevancy details to the Decision API response only, and nothing is added to the logs. A score mapped onto a creative template field is echoed back in the Decision API response whether or not it was applied to ad selection.

The Decision API can also return information regarding the relevancy score calculation for each ad that has been selected to serve. To enable this, please contact your account manager. You must then specify "includeRelevancyData": true in the Decision API request body.

Example data:

"relevancy": {
    "type": "perId",
    "attributeName": "ctProductId",
    "attributeId": "3",
    "rank": 1515,
    "orderedAttributes": [
        "ctProductId"
    ],
    "scores": {
        "ctProductId": {
            "1": 0.832,
            "2": 0.645,
            "3": 0.505
        }
    }
}

📘

Note

Site relevancy is not available at decision request time. This scoring can be set up to give sites a different weight. See details here.

Legacy Relevancy Scores

If you have used Relevancy scores prior to 20th November 2022 - you may want to refer to the legacy relevancy score documentation.