Relevancy Score

📘

Summary

With Kevel's Relevancy Score functionality, you can directly influence the ad selection process in an auction using your own relevancy/quality score data. Kevel then uses your supplied relevancy score data in combination with eCPM number to calculate an Ad Rank, which is used by the Ad Decision Engine to rank ads.

What is a relevancy score?

Relevancy score is a Decision API capability that allows you to directly influence ad selection in an auction using your own relevancy/quality score data.

For example, say you've built a self-serve advertising system on top of Kevel where bidders compete in an auction. Some bidders may bid a high CPC for clicks, but their ads are not relevant to the user, or they are poor quality, or they otherwise do not perform well.

By assigning a relevancy score, you can choose how to reward ads that are on-message and popular with viewers while requiring irrelevant ads to pay more to be eligible to serve. An ad with a higher relevancy score will win an auction over similarly priced ads.

Example use cases:

• Using your own click/conversion propensity models to influence the auction so that ads that the user is likely to click on receive an increased likelihood of selection
• Providing an artificial boost smaller advertisers in auctions to allow them to compete with high-budget advertisers
• Punishing promoted items that have poor organic performance
• Punishing low-margin products so that they need to bid more to be selected

In Kevel, relevancy scores are integers between 1 and 1000. The lower the relevancy score, the lower the ad will be ranked in the auction.

📘

Note

An ad that does not have a defined relevancy score will have a default relevancy score of 500. As such, ads that have a relevancy score of less than 500 are considered to be punished, whereas ads that have a relevancy score of over 500 are seen as favourable over similarly priced ads.

Using relevancy scores

There are two ways to supply a relevancy score:

• In the Decision API request, per placement. Use this when the score depends on the user or the context of the request — for example a click propensity score for this shopper on this page. This is described below.
• Stored on the ad or flight, applied to every request. Use this when the score is the same for every request — for example a product quality or performance score maintained in your catalog. See Storing relevancy scores on ads and flights.

Scores supplied in a request take precedence over stored scores.

Setting relevancy scores in a request

You can set relevancy scores in the Decision API request in each placement, mapped against:

• On creative template fields/values
• On an AdvertiserID / CampaignId / FlightID / AdId

Mapping relevancy scores to creative template fields and values

If you are using creative templates to store creative metadata (such as product attributes), you can use a creative template field and define how relevant its values are. For example, assuming your creative template had a field ctCategoryName - you may want to define relevancy scores that represent how likely you believe the user is to engage with each category.

"relevancy": {
  "idAttribute": {
    "ctCategoryName": {
      "shoes": 831,
      "shirts": 645,
      "jeans": 505,
      "sportswear": 402
    }
  }
}

In the example above, any ad that has a ctCategoryName that contains one of the following values shoes, shirts, jeans, or sportswear will receive a boost or penalty in the auction process.

Relevancy scores don't require the use of AdQuery, although the two are often used in combination with each other, particularly for Auction as a service use cases, as demonstrated below:

"adQuery": {
  "ctProductId": {
    "in": [
      "123",
      "234",
      "345",
      "456",
      "567"
    ]
  }
},
"relevancy": {
  "idAttribute": {
    "ctProductId": {
      "123": 832,
      "234": 645,
      "345": 505,
      "456": 811,
      "567": 484
    }
  }
}

Mapping relevancy score to entity IDs (advertiser, flight, and so on)

Alternatively, you can map relevancy scores to one of the following entity IDs: advertiserId, campaignId, flightId, adId.

For example:

"relevancy": {
  "idAttribute": {
    "adId": {
      "123456": 403,
      "234567": 901,
      "345678": 133,
      "456789": 482
    }
  }
}

📘

Note

We do not suggest sending relevancy score data for multiple idAttributes - only a single attribute will be used for relevancy score calculations.

Storing relevancy scores on ads and flights

Instead of sending a score with every ad request, you can store a relevancy score on the ad or the flight. The ad server applies the stored score to every auction the ad competes in. This suits scores that do not vary by user or request context, such as a product quality or performance score you maintain in your catalog.

Enabling an auction priority for relevancy scores

Stored scores are only used once the auction priority has been enabled for them. Create or update the priority via the API to include the following object:

"RelevancyScoreConfig": {
  "components": {
    "custom": 5
  },
  "aggregation": 2
}

components sets the metric(s) used to calculate relevancy scores. Currently only one metric is available, custom, which reads the score stored on the ad or flight. Its value sets the aggregator for the component.

aggregation sets how multiple components are combined into a single relevancy score. Only one metric is available, so set it to an integer between 1 and 5.

Aggregators

Each component uses an aggregator to compute a relevancy score when scores have been set on both a flight and its ad. Set the aggregator as an enum.

Setting the score

Add the CustomRelevancyScore parameter to the flight or ad object when creating or updating it. Scores are integers between 1 and 1000.

curl -X POST -H "X-Adzerk-ApiKey:<API-KEY>" https://api.kevel.co/v1/flight -d 'flight={"Name":"Relevancy Score","StartDate":"1/1/2015","CampaignId":123456,"PriorityId":123456,"GoalType":2,"IsActive":true,"Price":1,"Impressions":100,"RateType":2,"NoEndDate":true,"IsUnlimited":false,"CustomRelevancyScore":500}'

curl -X POST -H "X-Adzerk-ApiKey:<APIKEY>" https://api.kevel.co/v1/flight/12345/creative -d 'creative={"CampaignId":12345,"Creative":{"Id":123456},"FlightId":12345,"IsActive":true,"CustomRelevancyScore":500}'

🚧

Requirements

Storing relevancy scores must be enabled for your network, and the auction priority must be configured as described above. Until both are in place, ads are scored at the default of 500. Additionally, If your ads are created from a catalog using an ad template, the score can be mapped onto the ad directly, so it stays in step with your feed. Contact your account manager to fully enable this functionality.

📘

Note

Relevancy scores are used in auctions. Ads served through a lottery priority are selected by weight, and relevancy scores do not apply to them.

Relevancy Score and Revenue

Unlike standard auctions, auctions that use relevancy scores will choose the ad with the highest AdRank:

AdRank = eCPM * RelevancyScore

First price auctions

In a first price auction, revenue-generating events will record the full price of the ad.

Second price auctions

In a second price auction, impressions chosen using relevancy score will record the clear price of the ad. This is defined differently from other second price auctions:

Clear Price = (Ad Rank of Runner Up / Winner's Relevancy Score) + $0.01

Reporting on Relevancy Score

Relevancy score data is available in data shipping as the RelevancyScore field, on the auction, selection, impression, click, conversion and custom event logs.

To audit how a score affected an auction, use the auction log. It records RelevancyScore alongside Rank (eCPM * RelevancyScore) for each bid, so you can see both the score applied to an ad and the ranking it produced. This is the reliable place to confirm a stored score is being used: a score mapped onto a creative template field is echoed back in the Decision API response whether or not it was applied to ad selection.

The Decision API can also return information regarding the relevancy score calculation for each ad that has been selected to serve. To enable this, please contact your account manager. You must then specify "includeRelevancyData": true in the Decision API request body.

Example data:

"relevancy": {
    "attributeName": "ctProductId",
    "attributeId": "3",
    "rank": 1515,
    "orderedAttributes": [
        "ctProductId"
    ],
    "scores": {
        "ctProductId": {
            "1": 0.832,
            "2": 0.645,
            "3": 0.505
        }
    }
}

📘

Note

Site relevancy is not available at decision request time. This scoring can be set up to give sites a different weight. See details here.

Legacy Relevancy Scores

If you have used Relevancy scores prior to 20th November 2022 - you may want to refer to the legacy relevancy score documentation, which also covers overriding stored scores per request.