2d:null 2e:["$","title",null,{"children":"Rarity in Slashes | Clément Renaud | Observable"}] 33:T550,md`___ ## A word about rarity Rarity can be defined and calculated in many different ways. A rare thing is by definition uncommon, which often makes it distinctive from the rest of a group. But something can be unusual either for having a rather unique characteristic or for presenting a unique combination of common characteristics. #### Defining rarity is tricky Even before starting to crunch numbers, we can see how this can get complicated. Here I take a simple approach: first consider each defining trait and then explore their combinations (in the various tokens). #### Why separate palettes from traits Note that we explore the palettes separately from the traits, first because it makes sense to separate colors and shapes and second because they are kind of evenly distributed so don't hold significant value for studying rarity. #### A simple "rarity score" I decided to follow a simple scoring algorithm described in this [blog post](https://raritytools.medium.com/ranking-rarity-understanding-rarity-calculation-methods-86ceaeb9b98c) by the founder of [rarity.tools](https://rarity.tools), explained as follow: > *[Rarity Score for a Trait Value] = 1 / ([Number of Items with that Trait Value] / [Total Number of Items in Collection])* > > *The total Rarity Score for an NFT is the sum of the Rarity Score of all of it’s trait values.* `34:Tc31,palettes = [ ["0x101010","0xc5c3c6","0x46494c","0x4c5c68","0x1985a1"], ["0x20bf55","0x0b4f6c","0x292a73","0xfbfbff","0x757575"], ["0x06aed5","0x086788","0x303030","0xcccccc","0xdd1c1a"], ["0x909090","0xf2dc9b","0x6b6b6b","0x260101","0x0d0d0d"], ["0xffa822","0x3a9efd","0x3e4491","0x292a73","0x1a1b4b"], ["0x1b3da6","0x26488c","0x2372d9","0x62abd9","0xf2f557"], ["0x2d3142","0x4f5d75","0xbfc0c0","0xffffff","0x292a73"], ["0xf6511d","0xffb400","0x00a6ed","0x292a73","0x0d2c54"], ["0xdddddd","0x404099","0x929bac","0x013d6f","0x071e44"], ["0x3f0259","0xf2e205","0x606060","0xf2ebdc","0xf6511d"], ["0x302840","0x1f1d59","0x3e518c","0x808080","0xdddddd"], ["0x606060","0xbfc0c0","0x348aa7","0x525174","0x513b56"], ["0xef476f","0xffd166","0x06d6a0","0x118ab2","0x073b4c"], ["0x0b132b","0x1c2541","0x3a506b","0x5bc0be","0x6fffe9"], ["0xbce784","0x5dd39e","0x348aa7","0x525174","0x513b56"], ["0x000000","0x14213d","0xfca311","0xe5e5e5","0xffffff"], ["0x114b5f","0x028090","0xe4fde1","0x456990","0xf45b69"], ["0xdcdcdd","0xc5c3c6","0x46494c","0x4c5c68","0x1985a1"], ["0x22223b","0x4a4e69","0x9a8c98","0xc9ada7","0xf2e9e4"], ["0x3d5a80","0x98c1d9","0xe0fbfc","0xee6c4d","0x293241"], ["0x06aed5","0x086788","0xf0c808","0xfff1d0","0xdd1c1a"], ["0x011627","0xf71735","0x41ead4","0xfdfffc","0xff9f1c"], ["0x13293d","0x006494","0x247ba0","0x1b98e0","0xe8f1f2"], ["0xcfdbd5","0xe8eddf","0xf5cb5c","0x242423","0x333533"], ["0xffbf00","0xe83f6f","0x2274a5","0x32936f","0xffffff"], ["0x540d6e","0xee4266","0xffd23f","0x3bceac","0x0ead69"], ["0xffa69e","0xfaf3dd","0xb8f2e6","0xaed9e0","0x5e6472"], ["0x8a00d4","0xd527b7","0xf782c2","0xf9c46b","0xe3e3e3"], ["0x272643","0xffffff","0xe3f6f5","0xbae8e8","0x2c698d"], ["0x361d32","0x543c52","0xf55951","0xedd2cb","0xf1e8e6"], ["0x122c91","0x2a6fdb","0x48d6d2","0x81e9e6","0xfefcbf"], ["0x27104e","0x64379f","0x9854cb","0xddacf5","0x75e8e7"], ["0xe0f0ea","0x95adbe","0x574f7d","0x503a65","0x3c2a4d"], ["0xffa822","0x134e6f","0xff6150","0x1ac0c6","0xdee0e6"], ["0xd9d9d9","0xa6a6a6","0x8c8c8c","0x595959","0x262626"], ["0xa6032f","0x022873","0x035aa6","0x04b2d9","0x05dbf2"], ["0xa6a6a6","0x737373","0x404040","0x262626","0x0d0d0d"], ["0x0f5cbf","0x072b59","0x0f6dbf","0x042940","0x72dbf2"], ["0x0b132b","0x1c2541","0x3a506b","0x5bc0be","0x6fffe9"], ["0x000000","0x14213d","0xfca311","0xe5e5e5","0xffffff"], ["0x22223b","0x4a4e69","0x9a8c98","0xc9ada7","0xf2e9e4"], ["0x3d5a80","0x98c1d9","0xe0fbfc","0xee6c4d","0x293241"], ["0x011627","0xf71735","0x41ead4","0xfdfffc","0xff9f1c"], ["0xd8dbe2","0xa9bcd0","0x58a4b0","0x373f51","0x1b1b1e"], ["0x13293d","0x006494","0x247ba0","0x1b98e0","0xe8f1f2"], ["0xcfdbd5","0xe8eddf","0xf5cb5c","0x242423","0x333533"], ["0x97151f","0xdfe5e5","0x176c6a","0x013b44","0x212220"], ["0xfef7ee","0xfef000","0xfb0002","0x1c82eb","0x190c28"] ].map(d => d.map(c => getRgb(c)))35:[{"id":"cf076bcebb9b4590cc8a3b8e7668978f5839a0b39f525f090fd6ab32c4e660bef6b4a295246c95b4643c97ba0d968970cb749e2bb831109a29421ceaf8909370","href":"https://static.observableusercontent.com/files/cf076bcebb9b4590cc8a3b8e7668978f5839a0b39f525f090fd6ab32c4e660bef6b4a295246c95b4643c97ba0d968970cb749e2bb831109a29421ceaf8909370","name":"slashes_03.png","type":"image/png","lastModified":1648939228028.195,"size":1291758},{"id":"ba4ad4a52c5ac0ceddf331e418601a2b8fe1a865979ffafd9e37b52fd3c60dbc86cd9e211e3cef2276ba816962e1a2601bdc7f768a25c6a750145da42b0679a7","href":"https://static.observableusercontent.com/files/ba4ad4a52c5ac0ceddf331e418601a2b8fe1a865979ffafd9e37b52fd3c60dbc86cd9e211e3cef2276ba816962e1a2601bdc7f768a25c6a750145da42b0679a7","name":"slashes@2.csv","type":"text/csv","lastModified":1648813416134.012,"size":57003}] 31:{"id":"27ac0e5aaf508112","slug":"slashes-technical-analysis","title":"Rarity in Slashes","thumbnail":"a6c0224d2067372fcf62e14de844c42eb0304d98193c4f3326330ff76fecfe59","access_level":"public","create_time":"$D2022-04-01T11:39:48.280Z","update_time":"$D2022-05-04T08:58:41.196Z","view_time":null,"first_public_time":"$D2022-04-04T08:07:22.975Z","has_unpublished_edits":null,"fork":{"id":"ee9e4fb45c009de1","slug":"search-table","title":"Search + Table","owner":{"id":"f35c755083683fe5","login":"observablehq","name":"Observable"}},"stars":3,"starred":false,"owner":{"type":"team","id":"866b7439c266bbe5","login":"clemsos","name":"Clément Renaud","avatar_url":"https://avatars-next.observableusercontent.com/08d03128c2d7bf6ffc580efc85edec0c40b5d91826b746db27920b996240c8f0","tier":"starter_2024"},"version":1527,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"4b23a90c6da12bac","login":"clemsos","name":"Clément Renaud","avatar_url":"https://avatars-next.observableusercontent.com/08d03128c2d7bf6ffc580efc85edec0c40b5d91826b746db27920b996240c8f0","tier":null}],"members":[{"type":"individual","id":"4b23a90c6da12bac","login":"clemsos","name":"Clément Renaud","avatar_url":"https://avatars-next.observableusercontent.com/08d03128c2d7bf6ffc580efc85edec0c40b5d91826b746db27920b996240c8f0","tier":null,"role":"editor"},{"type":"individual","id":"cae924adff874d09","login":"makio135","name":"Lionel Radisson","avatar_url":"https://avatars-next.observableusercontent.com/1ebff675951134ccb48ffd1faaeabdecafe0daf8d168685c768f506435e400f0","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":null,"type":"notebook","body":{"title":"Rarity in Slashes","cells":[{"id":1428,"value":"md`# Rarity in Slashes\n## a first technical analysis`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1422,"value":"FileAttachment(\"slashes_03.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"This is a technical analysis of rarity patterns in the [Slashes](https://makio135.com/slashes) project that [@makio135](https://observablehq.com/@makio135/creative-coding) and I released earlier this year.\n\nAs all existing 1024 pieces have been generated and minted using an Ethereum contract, it can be difficult to have an overview of the entire collection. The characteristics of the outputs were not entirely predictable. Therefore some straightforward analysis can provide us with a better understanding of simple questions such as: what are the rarest traits found in the Slashes collection, which tokens are the most \"unique\" token - even though they are all unique, etc. \n\n*NB: there is a csv file attached to this notebook containing all the metadata for the 1024 slashes directly exported from the contract*","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1116,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":123,"value":"md`___\n## Traits\n\nThere are ${colNames.length - 2} different traits in Slashes (excluding the palettes) that are used to generate randomness. The different traits are :\n\n${ \nObject.keys(traits)\n .filter(d => ![\"Palette ID\", \"token ID\"].includes(d))\n .map(d => `- ${d} (*${typeof data[0][d]}*) \\n`)\n}\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1469,"value":"md`\n\n### The rarest traits\n\nBased on a simple analysis by percentage of token that have a specific trait, here are the rarest traits :\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1297,"value":"Inputs.table(\n rarest_traits.filter(d => ![\"Palette ID\", \"Background Opacity\"].includes(d.name)), \n {\n columns: [\"name\", \"value\", \"percent\", \"count\", \"tokens\"],\n width: {\n \"value\": \"5%\",\n },\n rows : rarest_traits.filter(d => d.name !== \"Palette ID\").length,\n header: {\n \"name\": \"Trait\", \n \"value\" : \"Value\", \n \"percent\": \"Percentage\",\n \"count\" : \"Count\", \n \"tokens\": \"Some slashes\"\n },\n format: { \n value: d => typeof d === 'string' ? d.replace(\"|\", \"/\") : d,\n palette: d => displayPalette(d),\n tokens: d => html`${\n d.slice(0,5)\n .map(t => t[\"token ID\"])\n .map(id => `${id}`)\n .join(', ')\n }`,\n percent: d => f(d)\n }\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":477,"value":"colNames = Object.keys(data[0])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":485,"value":"traits = colNames\n.map( c => data.map(d => d[c]))\n.map((k,i) => {\n const unique = Array.from(new Set(k))\n const count = unique.map(v => k.filter(d => d === v).length)\n const percents = count.map(c => c / k.length).reduce((res, d, i) => ({...res, [unique[i]] : d }), {})\n \n return { \n name: colNames[i],\n min : d3.min(k), \n max : d3.max(k), \n values : k,\n count: count.reduce((res, d, i) => ({...res, [unique[i]] : d }), {}),\n unique,\n percents,\n score: count.map(c => 1 / c / data.length).reduce((res, d, i) => ({...res, [unique[i]] : d }), {}),\n avg: d3.sum(k) / k.length\n }\n})\n.reduce((res, d, i) => ({...res, [d.name] : d}), {})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":619,"value":"rarest_traits = Object.keys(traits)\n .filter(k => ![\"token ID\"].includes(k))\n .map(k => {\n const { score } = traits[k]\n return Object.keys(score).map(s => [k, s, score[s]])\n})\n.flat()\n.map(([name, value, score]) => ({ \n name, \n value: typeof data[0][name] === 'number' ? parseInt(value) : value, \n score \n}))\n.map(d => ({\n ...d,\n count: traits[d.name].count[d.value],\n percent: traits[d.name].percents[d.value],\n v: parseInt(d.value),\n type: typeof data[0][d.name],\n tokens: hasTrait(d.name, d.value)\n}))\n// .filter(d => d.name !== \"Palette ID\")\n.sort((a,b) => b.score - a.score)\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":887,"value":"md`___\n## Palettes\n\nAltogether there are ${palettes.length} palettes available in Slashes. The distribution of the palettes varies btw ${f(rarestPalettes[0].percent)} and ${f(rarestPalettes[rarestPalettes.length-1].percent)} of the tokens that shares a similar palette.\n\nBelow are all the palettes, sorted by the number of times they were used when minting tokens.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1165,"value":"viewof nbPalettesShown = \nInputs.range([0, palettes.length], {\n value: 5, \n step: 1, \n label: \"The number of palettes to display\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1237,"value":"Inputs.table(\n rarestPalettes, \n {\n columns: [\"palette\", \"value\", \"count\", \"percent\", \"tokens\"],\n height: \"100%\",\n width: {\n \"value\": \"5%\",\n \"palette\": \"40%\"\n },\n rows: nbPalettesShown,\n format: { \n value: d => md`**${d}**`,\n palette: d => displayPalette(d),\n tokens: d => html`${\n d.slice(0,5)\n .map(t => t[\"token ID\"])\n .map(id => `${id}`)\n .join(', ')\n }`,\n percent: d => f(d)\n }\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":930,"value":"rarestPalettes = rarest_traits\n .filter(d => d.name === \"Palette ID\")\n .sort((a,b) => b - a)\n .map(d => ({\n ...d,\n tokenIds : hasTrait(d.name, parseInt(d.value)).map(t => t[\"token ID\"]),\n palette: palettes[parseInt(d.value)]\n}))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":894,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1447,"value":"md`___\n## Tokens\n\nNow let's see which could be considered the \"rarest pieces\". \n\n### Average percentage\n\nMy first \"naive\" approach was to calculate the percentage for each trait, then get the average value for each token. However, this approach only take into account the overall rarity of a token, but does not really consider either a token have one or a few very rare traits. \n\n> List of rarest token by average percentage: ${rarest(avg_percents)}...\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1511,"value":"html`

\n ${avg_percents.filter((d, i) => i < 9).map(d => `

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":646,"value":"// percentage\nrarity_percent = data.map(d => (\n Object.keys(d)\n .filter(d => d !== \"token ID\")\n .map(k => traits[k].percents[d[k]])\n))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":598,"value":"avg_percents = rarity_percent.map((token, i) => ({\n avg_percent: d3.sum(token) / token.length,\n tokenId : i + 1\n}))\n.sort((a,b) => a.avg_percent - b.avg_percent )","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1452,"value":"md`\n\n### Statistical rarity\n\nThis second approach also relies on percentage, but instead of averaging them we multiply them. This was described in a newsletter by [Adam Chekroud](https://achekroud.substack.com/p/d8f13e0a-ae26-40e4-bff2-beacb2916ff0?s=r) but does not appear to have real statistical basis despite its name and can lead to weird results.\n\n> List of token by \"statistical rarity\" (mult of percentage): ${rarest(stat_rarity)}...\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1500,"value":"html`

\n ${stat_rarity.filter((d, i) => i < 9).map(d => `

`)}\n

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":640,"value":"stat_rarity = rarity_percent\n.map((token, i) => {\n const f = d3.format(\".2f\");\n const values = token.map(d => f(d))\n return {\n values,\n stat: values.reduce((res, d) => res * d, 1),\n tokenId : i + 1\n }\n})\n.sort((a,b) => a.stat - b.stat )","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1459,"value":"md`\n\n### Rarity score\n\nThe founder of [rarity.tools](https://rarity.tools) describes in this [blog post](https://raritytools.medium.com/ranking-rarity-understanding-rarity-calculation-methods-86ceaeb9b98c) the following scoring algorithm:\n\n> *[Rarity Score for a Trait Value] = 1 / ([Number of Items with that Trait Value] / [Total Number of Items in Collection])*\n>\n> *The total Rarity Score for an NFT is the sum of the Rarity Score of all of it’s trait values.*\n\nThis has the advantage to weight the trait in the overall collection. We will use the following for now, as this seems to have become a reference in the NFT space so far.\n\n> List of token by \"rarity score\": ${rarest(rarity_score)}...\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1514,"value":"html`

\n ${rarity_score.filter((d, i) => i < 9).map(d => `

`)}\n