24:null 25:["$","title",null,{"children":"Transparency in Programmatic Advertising Marketplaces | paavanb | Observable"}] 2a:T5ab,md` Here's what you may see as a buyer on an ad exchange: \`\`\` Hello, I am Seller 156512 on pubmatic.com, and I have a video spot for a user on CNN.com \`\`\` This is a high level example of a typical bid request on an ad exchange. It claims to be a seller with ad space on CNN.com, supposedly triggered by a user visiting the site. If you win the bid against other buyers, your ad will be shown to this user. Do you buy? There are many ways you can be deceived here: 1. This may be a malicious seller. Instead of inventory on CNN.com, they are actually selling a spot on an unscrupulous website. 1. This is a reseller that has unwittingly purchased fraudulent inventory further up the supply chain. This is not going to CNN.com. 1. This seller is arbitraging their inventory, selling a Display spot as Video for a higher CPM. Your advertisement will not be given the space you think it will. 1. This is not a real user, and it is fake traffic meant to generate revenue. Human eyes will not see your ad. Of course, if you are an advertiser you will see none of this. You will directly or indirectly partner with Demand Side Platforms (DSPs) to handle all of this for you, since thousands of transactions need to be handled every second. But without transparency, it is very difficult to determine the **lineage** of the inventory and whether it is **trustworthy**. How much of your ad spend actually went to human eyes on the sites you wanted? `2b:T62b,md` ## Ads.txt With so little information, there is not much we can ask. The simplest question we need answered is **"Is this seller authorized to sell inventory on CNN.com?"** Let us ask the publisher. The publisher is well-positioned to answer this question truthfully, since answering dishonestly would allow fraudulent or otherwise low-quality sellers to sell their inventory and reduce their CPM. Thus, [ads.txt was born](https://iabtechlab.com/ads-txt/): a standard that allows publishers to publicly document accounts that are authorized to sell their inventory on various ad exchanges, in a file called \`ads.txt\` hosted on their domains. Here's the ads.txt for our example CNN.com: \`\`\` adtech.com, 12345, DIRECT pubmatic.com, 156512, RESELLER \`\`\` The first line is a DIRECT record, implying that CNN.com controls seller 12345 on the exchange adtech.com and is selling inventory directly with that account. The second line is a RESELLER record, implying that CNN.com has allowed seller 156512 to sell their inventory on the exchange pubmatic.com. They might be selling it directly, as when websites authorize sellers to sell their inventory in foreign countries, or reselling inventory from another seller further up the chain. Suppose then a buyer sees the following bid request: \`\`\` Hello, I am Seller 54321 on pubmatic.com, and I have a video spot for a user on CNN.com \`\`\` By cross-checking the ads.txt, the buyer can see that this seller is _not_ authorized to sell CNN.com inventory. It may be fraudulent, and therefore should not be purchased. `2c:T442,md` ### Blind Spots Ads.txt protects buyers from the most shameless domain spoofing attacks, like when a malicious seller lies about the domain: a buyer can check the ads.txt for that domain and see that the account ID is not authorized. But can a buyer be deceived even if it is? It turns out that ads.txt relies on resellers to do their due diligence when signing on publishers, because [a particularly clever form of domain spoofing](https://www.admonsters.com/kill-all-the-resellers/) uncovered by DoubleVerify took advantage of the fact that some don't. If an unscrupulous site signs on with the same resellers as the legitimate sites that they are targeting, the reseller might use the _same account ID_ to sell both sites' inventory. This allows the unscrupulous site to spoof the legitimate site and claim to sell their inventory through the reseller. When a buyer checks the ads.txt, the ID appears to be valid. It's up to the publisher to ensure that the resellers they authorize are trustworthy enough to defend against this type of fraud, or else their CPM may be reduced. `2d:T683,md` ## Sellers.json Ads.txt is not meant to be the one solution for all types of ad fraud. Instead, it is meant to be used in conjunction with other standards to improve transparency in the network as a whole. Just as we can trust publishers to document their authorized sellers, so too we can trust Supply Side Platforms (SSP's) to document the sellers they have authorized to transact on their platforms. [This brings us \`sellers.json\`](https://iabtechlab.com/sellers-json/): a file that an SSP can make public to detail every single seller on the platform with their id, their domain, possibly even their name, and their type. Sellers.json categorizes sellers by three types: * \`"PUBLISHER"\` - The inventory sold by this entity is owned by the entity, and the SSP pays them directly. * \`"INTERMEDIARY"\` - The inventory sold is not owned by the entity, _or_ the SSP does not pay them directly. * \`"BOTH"\` - The entity is both authorized as a \`PUBLISHER\` and \`INTERMEDIARY\`. With sellers.json, we can shine a light on the "dark pool sales house" shared by Breitbart.com and RT.com. By checking [\`appnexus.com/sellers.json\`](https://appnexus.com/sellers.json), we can find seller 1899: \`\`\` { "seller_id": "1899", "seller_type": "BOTH", "domain": "zedo.com", "name": "Zedo (ZINC)" } \`\`\` From this we can glean that the seller is Zedo, which sells inventory on appnexus.com as \`BOTH\` a \`PUBLISHER\` and an \`INTERMEDIARY\`. But since Zedo does not own Breitbart or RT, we can conclude that Zedo is selling their inventory as an intermediary. The ads.txt files of Breitbart.com and RT.com should have labeled Zedo as a \`RESELLER.\` `2e:T4be,md` The third component in the IAB transparency ecosystem is the OpenRTB SupplyChain object, introduced in the [OpenRTB 3.0 spec](https://github.com/InteractiveAdvertisingBureau/openrtb/blob/88283c5175312a1cae93a4f4b2a12fd4dddc4670/OpenRTB%20v3.0%20FINAL.md). With the SupplyChain object, we can finally have complete visibility into the entire supply chain involved in the sale of a given bid request. Each SupplyChainNode specified in a SupplyChain object includes the exchange on which the transaction was completed and the seller id (\`sid\`) involved. Here's a sample SupplyChain object: \`\`\` { "ver": "1.0", "complete": 1, "nodes": [ { "asi": "adtech.com", "sid": "12345" "rid": "BidRequestId1", "hp": 1 }, { "asi": "pubmatic.com", "sid": "156512", "rid": "BidRequestId2", "hp": 1 } ] } \`\`\` With this information, not only can we verify the final seller using ads.txt and sellers.json, but _every seller in the supply chain back to the publisher_. This information is invaluable for determining the provenance of bid requests. Without it, fraud can be introduced anywhere in the supply chain without the buyer knowing it. `2f:T85c,md` [Ads.cert](https://github.com/InteractiveAdvertisingBureau/openrtb/blob/f0c4bd968ad9aea75bf467a0d45efdbc6a2d3bc6/ads.cert:%20Signed%20Bid%20Requests%201.0%20BETA.md) is the latest standard developed by IAB, currently in beta. It will allow publishers to cryptographically sign the bid request they issue, which prevents sellers down the chain from altering any of the signed fields. The implications of such a guarantee are difficult to overstate. Today, a publisher with an ads.txt is like Rolex maintaining a database of authorized sellers for their watches. But if a seller is duped into buying a fraudulent watch, it could still be sold as authentic to buyers. That's why Rolex issues a Certificate of Authenticity with each watch: a buyer interested in a Rolex from a reseller just needs to ask for the certificate for a guarantee that the watch is legitimate. Ads.cert is like a certificate of authenticity passed along with every bid request, proving to the buyer that none of the signed fields were tampered with. But unlike certificates, which scammers can buy to sell with fake Rolexes, every signature is unique to the bid request it is created for. So buyers receiving a bid request can cryptographically verify that the initial bid request was legitimately issued by the publisher, and then check all sellers against sellers.json and ads.txt. **This makes some types of domain spoofing obsolete**: you cannot create a signed bid request if you do not own the digital key used to make it, which is private to the ad server. Of course, the ad server could be compromised and made to issue fraudulent bid requests, but this would have to be done on a server-by-server basis. This still doesn't obviate the need for ads.txt. In the world of physical products, a certificate of authenticity is sufficient, since the buyer wants to purchase an authentic product but does not necessarily care from whom. But in adtech, the buyer is being given the _right to serve an ad on the publisher's site_. If publishers don't carefully manage their reseller relationships, the ads served on their websites could hurt their reputation.`30:[] 28:{"id":"4d95e8232cd4f1c7","slug":"transparency-in-programmatic-advertising-marketplaces","title":"Transparency in Programmatic Advertising Marketplaces","thumbnail":"78bb32e3646c414714007133b0ba5e0ff59b2b6a012132e587aeedceab141c2f","access_level":"public","create_time":"$D2021-02-06T20:52:15.393Z","update_time":"$D2021-03-12T18:07:21.165Z","view_time":null,"first_public_time":"$D2021-02-07T22:15:35.781Z","fork":false,"stars":0,"starred":false,"owner":{"type":"team","id":"14c2157156d3ecd6","login":"paavanb","name":"","avatar_url":"https://avatars-next.observableusercontent.com/aa8b8c737e3953f3bd4486da2676f3421a66f87415e722e54f2dc52f26befbf0","tier":"starter_2024"},"version":785,"publish_version":785,"role":null,"authored":false,"authors":[{"type":"individual","id":"fbc05d7557757ff9","login":"paavanb","name":"","avatar_url":"https://avatars-next.observableusercontent.com/aa8b8c737e3953f3bd4486da2676f3421a66f87415e722e54f2dc52f26befbf0","tier":null}],"members":[{"type":"individual","id":"fbc05d7557757ff9","login":"paavanb","name":"","avatar_url":"https://avatars-next.observableusercontent.com/aa8b8c737e3953f3bd4486da2676f3421a66f87415e722e54f2dc52f26befbf0","tier":null,"role":"editor"}],"comments":[{"id":"acc49d14a37a463c","reply_to_id":null,"content":"hello comment","create_time":"$D2021-02-08T23:32:10.948Z","update_time":null,"resolved":true,"user":{"type":"individual","id":"183412c75050475c","login":"gabriel","name":"Gabriel Altay","avatar_url":"https://avatars-next.observableusercontent.com/2e98f1f39c17ae57bc039d8e70fb00d381bb0995826a5cfc7d9ea4eb8c9895ce","tier":null}},{"id":"d473e793ad41edef","reply_to_id":"acc49d14a37a463c","content":"Hello!","create_time":"$D2021-02-08T23:32:17.163Z","update_time":null,"resolved":true,"user":{"type":"individual","id":"fbc05d7557757ff9","login":"paavanb","name":"","avatar_url":"https://avatars-next.observableusercontent.com/aa8b8c737e3953f3bd4486da2676f3421a66f87415e722e54f2dc52f26befbf0","tier":null}}],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":0,"collection":{"id":"f7a05b8062c4400d","slug":"industrial","title":"Industrial","owner":{"id":"f35c755083683fe5","login":"observablehq","name":"Observable"}},"type":"notebook","body":{"title":"Transparency in Programmatic Advertising Marketplaces","cells":[{"id":0,"value":"md`# Transparency in Programmatic Advertising Marketplaces`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":5,"value":"md`The world of programmatic advertising is a chaotic place. Ad exchanges allow buyers to bid on individual impressions in real-time, serving highly tailored ads based on the type of user visiting a publisher's website. Bid requests may pass through multiple sellers on multiple exchanges before a user is served an ad. But how can advertisers be sure that the inventory they are purchasing is legitimate?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":11,"value":"md`## The Wild West`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":15,"value":"$2a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":37,"value":"md`\n## Visualizing the Supply Chain\n\nThe supply chain for a bid request is much like the supply chain for physical products, except the \"product\" is human attention. Like any other product, it can be bought and resold multiple times in multiple marketplaces before reaching a customer. In adtech, the marketplaces are ad exchanges, while the eventual customer is a brand placing ads. Since brands do not transact on programmatic marketplaces directly, they will usually work with an ad agency that will coordinate with DSPs for them.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":44,"value":"mermaid`\n graph TD;\n A((\"👀
CNN.com\")) -- \"Display Spot\" --> B(adtech.com: Seller 12345);\n B --> C(pubmatic.com: Seller 156512);\n C-->D((Ad Agency));\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":48,"value":"md`Without transparency, almost none of this is visible. As the buyer, you only see the last step: the seller on pubmatic claiming to have a spot on CNN.com. Whom did they get it from? Was the inventory modified in any way? Who is this seller? It is impossible to verify on your own, because all you see is this:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":97,"value":"mermaid`\n graph TD;\n C(pubmatic.com: Seller 156512) -- \"Display Spot\" --> D((Ad Agency));\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":93,"value":"$2b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":180,"value":"$2c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":123,"value":"md`\n### Trusting the Publisher\n\nHow far can we trust the publisher to be truthful in their ads.txt? It is in their interest to leave out illegitimate sellers, since the revenue earned from fraudulent inventory does not make it back to the publisher. However, there is one area where it can be in the publisher's interest to lie: whether the account is \\`DIRECT\\` or a \\`RESELLER\\`.\n\nDirect accounts tend to be preferred in programmatic marketplaces, since buyers are ostensibly purchasing inventory directly from the publisher. Reseller accounts are less preferred, since the provenance of their inventory is unknown: it could have been relabeled along the supply chain, or purchased from fraudulent sellers. Reseller accounts net a lower CPM for this reason and yield lower returns for publishers.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":128,"value":"md`\n#### Dark Pool Sales Houses\n\nThis misalignment of incentives has led to the rise of so-called [\"dark pool sales houses\"](https://medium.com/@thezedwards/breitbart-com-is-partnering-with-rt-com-other-sites-via-mislabeled-advertising-inventory-6e7e3b5c3318). Publishers will, knowingly or not, relabel \\`RESELLER\\` accounts as \\`DIRECT\\` and obscure the provenance of the inventory sold by those accounts. A buyer checking the ads.txt may pay more believing they are buying direct inventory, when in fact they are purchasing from a reseller. The reseller involved in this fraud will be capable of selling the less-than-premium inventory at a higher CPM, and pass on that revenue to the publisher.\n\nOne such dark pool sales house is a reseller shared by both Breitbart.com and RT.com. They both share the following ads.txt record:\n\n\\`\\`\\`\nappnexus.com, 1899, DIRECT\n\\`\\`\\`\n\nBreitbart and RT are not owned by the same entity, and therefore should not be sharing a \\`DIRECT\\` account. How can we find out what's going on?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":134,"value":"$2d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":708,"value":"md`\n## OpenRTB SupplyChain Object\n\nTogether, \\`ads.txt\\` and \\`sellers.json\\` can shine a light on the seller in the last step in the supply chain, but what about previous steps? What if the final seller is authorized, but they purchased inventory from an unauthorized seller?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":711,"value":"mermaid`\n graph TD;\n A((Unknown)) --> B(\"adtech.com: Seller 12345
❌ cnn.com/ads.txt
✅ adtech.com/sellers.json\");\n B --> C(\"pubmatic.com: Seller 156512
✅ cnn.com/ads.txt
✅ pubmatic.com/sellers.json\");\n C -->D((Ad Agency));\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":757,"value":"md`_Above: An ad agency believes it is transacting with an authorized seller, but the seller unwittingly purchased that inventory from an unauthorized seller._`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":161,"value":"$2e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":210,"value":"md`\n## Ads.cert\n\nAfter verifying the legitimacy of the sellers in the supply chain, the buyer needs further confirmation that the bid request was not altered in any way. A common form of ad fraud involves a seller relabeling inventory as a different type, such a relabeling Video inventory as Pre-Roll. A buyer believes that they are purchasing a premium above-the-fold spot above an article, when in fact they are purchasing a video spot of the wrong size elsewhere on the page. This is so common that one study demonstrated [up to 70% of video inventory](https://digiday.com/marketing/display-video-arbitrage/) may be mislabeled in this manner. Although the SupplyChain object brings transparency into the sellers involved in the sale of a bid request, there is no way to determine whether this bid request was modified.\n\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":643,"value":"mermaid`\n graph TD;\n A((\"👀
CNN.com\")) -- \"Display Spot
💰\" --> B(adtech.com: Seller 12345);\n B -- \"Video Spot
💰💰\" --> C(pubmatic.com: Seller 156512);\n C-->D((Ad Agency));\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":688,"value":"md`_Above: adtech.com profits off from the arbitrage of selling a \"display\" spot as \"video\"._`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":632,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":254,"value":"md`\n## Putting It All Together\n\nAny one of these standards alone is insufficient for detecting fraudulent inventory. But together, they have the potential to drive out the most prevalent types of ad fraud today.\n\n* Ads.cert allows a buyer to determine if the bid request is authentic,\n* the SupplyChain object surfaces the entire lineage of the bid request,\n* sellers.json allows us to identify exactly who each of the sellers are, and\n* ads.txt tells us if they were authorized by the publisher to sell their inventory in the first place.\n\nSo instead of seeing a bid request like this, as in the beginning of the document:\n\n\\`\\`\\`\nHello, I am Seller 156512 on pubmatic.com, and I have a video spot for a user on CNN.com\n\\`\\`\\`\n\nwith all four standards in place, buyers will soon be able to see this:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":289,"value":"mermaid`\n graph TD;\n A((\"👀
CNN.com\")) -- \"✅ Display Spot\"--> B(\"adtech.com: Seller 12345
✅ cnn.com/ads.txt
✅ adtech.com/sellers.json\");\n B -- \"✅ Unmodified\" --> C(\"pubmatic.com: Seller 156512
✅ cnn.com/ads.txt
✅ pubmatic.com/sellers.json\");\n C -- \"✅ Unmodified\" -->D((Ad Agency));\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":429,"value":"md`Without OpenRTB 3.0 and Ads.cert, buyers only have ads.txt and sellers.json at their disposal, and are largely flying blind. This means that they are susceptible to:\n* Certain types of [domain spoofing](https://headerbidding.co/domain-spoofing/), \n* Fraudulent inventory sold further up the supply chain, and \n* Modified or otherwise misrepresented inventory.\n\nAs the OpenRTB 3.0 spec gains adoption across the adtech space, we can expect to see the rate of these types of ad fraud to decline massively. \n\n### Caveat\nIt is important to note that since these standards ultimately only focus on the ad exchanges, fraud introduced elsewhere will remain unaffected. For example, botfarms and malware generating fake impressions are difficult to distinguish from real users, and none of the aforementioned standards can help with that.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":52,"value":"md`## Appendix`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":56,"value":"import {mermaid} from \"@matthewkenny/mermaid-js\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"}],"files":"$W30","stdlib":"1","titling":"auto"}} 29:"$28:body"