2b:null 2c:["$","title",null,{"children":"Understanding Word Vectors | KarlieDoesNotExist | Observable"}] 31:T44d,md`Modeling animals in this way has a few other interesting properties. For example, you can pick an arbitrary point in "animal space" and then find the animal closest to that point. If you imagine an animal of size 25 and cuteness 30, you can easily look at the space to find the animal that most closely fits that description: the chicken. Reasoning visually, you can also answer questions like: what's halfway between a chicken and an elephant? Simply draw a line from "elephant" to "chicken," mark off the midpoint and find the closest animal. (According to our chart, halfway between an elephant and a chicken is a horse.) You can also ask: what's the *difference* between a hamster and a tarantula? According to our plot, it's about seventy five units of cute (and a few units of size). The relationship of "difference" is an interesting one, because it allows us to reason about *analogous* relationships. In the chart below, I've drawn an arrow from "tarantula" to "hamster" (in blue): ${html`

${await FileAttachment("animal-space.png").image()}`} `32:T5e7,md`You can understand this arrow as being the relationship between a tarantula and a hamster, in terms of their size and cuteness (i.e., hamsters and tarantulas are about the same size, but hamsters are much cuter). In the same diagram, I've also transposed this same arrow (this time in red) so that its origin point is "chicken." The arrow ends closest to "kitten." What we've discovered is that the animal that is about the same size as a chicken but much cuter is... a kitten. To put it in terms of an analogy:

    Tarantulas are to hamsters as chickens are to kittens.

A sequence of numbers used to identify a point is called a *vector*, and the kind of math we've been doing so far is called *linear algebra.* (Linear algebra is surprisingly useful across many domains: It's the same kind of math you might do to, e.g., simulate the velocity and acceleration of a sprite in a video game.) A set of vectors that are all part of the same data set is often called a *vector space*. The vector space of animals in this section has two *dimensions*, by which I mean that each vector in the space has two numbers associated with it (i.e., two columns in the spreadsheet). The fact that this space has two dimensions just happens to make it easy to *visualize* the space by drawing a 2D plot. But most vector spaces you'll work with will have more than two dimensions—sometimes many hundreds. In those cases, it's more difficult to visualize the "space," but the math works pretty much the same. `33:T751,md`## Language with vectors: colors So far, so good. We have a system in place—albeit highly subjective—for talking about animals and the words used to name them. Let's talk about another vector space that has to do with language: the vector space of colors. Colors are often represented in computers as vectors with three dimensions: red, green, and blue. Just as with the animals in the previous section, we can use these vectors to answer questions like: which colors are similar? What's the most likely color name for an arbitrarily chosen set of values for red, green and blue? Given the names of two colors, what's the name of those colors' "average"? We'll be working with this [color data](https://github.com/dariusk/corpora/blob/master/data/colors/xkcd.json) from the [xkcd color survey](https://blog.xkcd.com/2010/05/03/color-survey-results/). The data relates a color name to the RGB value associated with that color. [Here's a page that shows what the colors look like](https://xkcd.com/color/rgb/). A few notes before we proceed: * The linear algebra functions implemented below (\`addv\`, \`meanv\`, etc.) are slow, potentially inaccurate, and shouldn't be used for "real" code — they're included so that beginner programmers can understand how these kinds of functions work behind the scenes. Use [numjs](https://github.com/nicolaspanel/numjs) or a similar library for fast and accurate math in JavaScript. * If you're interested in perceptually accurate color math in JS, consider using the [colormath library](https://www.npmjs.com/package/color-math). * *Aside: It should also be noted that python is really the most common/powerful language for machine-learning. We're using JS here only so that examples will run in the browser* Now load the JSON color data:`34:T67c,md`That's all well and good for color words, which intuitively seem to exist in a multidimensional continuum of perception, and for our animal space, where we've written out the vectors ahead of time. But what about... arbitrary words? Is it possible to create a vector space for all English words that has this same "closer in space is closer in meaning" property? To answer that, we have to back up a bit and ask the question: what does *meaning* mean? No one really knows, but one theory popular among computational linguists, computer scientists and other people who make search engines is the [Distributional Hypothesis](https://en.wikipedia.org/wiki/Distributional_semantics), which states that: Linguistic items with similar distributions have similar meanings. What's meant by "similar distributions" is *similar contexts*. Take for example the following sentences: It was really COLD yesterday. It will be really WARM today, though. It'll be really HOT tomorrow! Will it be really COOL Tuesday? According to the Distributional Hypothesis, the words \`cold\`, \`warm\`, \`hot\` and \`cool\` must be related in some way (i.e., be close in meaning) because they occur in a similar context, i.e., between the word "really" and a word indicating a particular day. (Likewise, the words \`yesterday\`, \`today\`, \`tomorrow\` and \`Tuesday\` must be related, since they occur in the context of a word indicating a temperature.) In other words, according to the Distributional Hypothesis, a word's meaning is just a big list of all the contexts it occurs in. Two words are closer in meaning if they share contexts.`35:Tebe,md` ${html`

${await FileAttachment("best-of-times.png").image()}`} The spreadsheet has one column for every possible context, and one row for every word. The values in each cell correspond with how many times the word occurs in the given context. The numbers in the columns constitute that word's vector, i.e., the vector for the word \`of\` is [0, 0, 0, 0, 1, 0, 0, 0, 1, 0] Because there are ten possible contexts, this is a ten dimensional space! It might be strange to think of it, but you can do vector arithmetic on vectors with ten dimensions just as easily as you can on vectors with two or three dimensions, and you could use the same distance formula that we defined earlier to get useful information about which vectors in this space are similar to each other. In particular, the vectors for \`best\` and \`worst\` are actually the same (a distance of zero), since they occur only in the same context (\`the ___ of\`): [0, 0, 0, 1, 0, 0, 0, 0, 0, 0] Of course, the conventional way of thinking about "best" and "worst" is that they're *antonyms*, not *synonyms*. But they're also clearly two words of the same kind, with related meanings (through opposition), a fact that is captured by this distributional model. ### Contexts and dimensionality Of course, in a corpus of any reasonable size, there will be many thousands if not many millions of possible contexts. It's difficult enough working with a vector space of ten dimensions, let alone a vector space of a million dimensions! It turns out, though, that many of the dimensions end up being superfluous and can either be eliminated or combined with other dimensions without significantly affecting the predictive power of the resulting vectors. The process of getting rid of superfluous dimensions in a vector space is called [dimensionality reduction](https://en.wikipedia.org/wiki/Dimensionality_reduction), and most implementations of count-based word vectors make use of dimensionality reduction so that the resulting vector space has a reasonable number of dimensions (say, 100—300, depending on the corpus and application). The question of how to identify a "context" is itself very difficult to answer. In the toy example above, we've said that a "context" is just the word that precedes and the word that follows. Depending on your implementation of this procedure, though, you might want a context with a bigger "window" (e.g., two words before and after), or a non-contiguous window (skip a word before and after the given word). You might exclude certain "function" words like "the" and "of" when determining a word's context, or you might [lemmatize](https://en.wikipedia.org/wiki/Lemmatisation) the words before you begin your analysis, so two occurrences with different "forms" of the same word count as the same context. These are all questions open to research and debate, and different implementations of procedures for creating count-based word vectors make different decisions on this issue. ### GloVe vectors But you don't have to create your own word vectors from scratch! Many researchers have made downloadable databases of pre-trained vectors. One such project is Stanford's [Global Vectors for Word Representation (GloVe)](https://nlp.stanford.edu/projects/glove/). We'll use a compressed (150 dimensions) version of these vectors for the rest of this tutorial. ## Word vectors Okay, let's have some fun with real word vectors. We're going to use the GloVe vectors mentioned above to creatively analyze and manipulate the text of Bram Stoker's *Dracula*. The following cell loads the language model, parses the input text, and creates a list of unique words (or tokens) in the text, as an array of strings. `36:T52a,md`---------- ## Further resources * [Word2vec](https://en.wikipedia.org/wiki/Word2vec) is another procedure for producing word vectors which uses a predictive approach rather than a context-counting approach. [This paper](http://clic.cimec.unitn.it/marco/publications/acl2014/baroni-etal-countpredict-acl2014.pdf) compares and contrasts the two approaches. (Spoiler: it's kind of a wash.) * If you want to train your own word vectors on a particular corpus, the popular Python library [gensim](https://radimrehurek.com/gensim/) has an implementation of Word2Vec that is relatively easy to use. [There's a good tutorial here.](https://rare-technologies.com/word2vec-tutorial/) * When you're working with vector spaces with high dimensionality and millions of vectors, iterating through your entire space calculating cosine similarities can be a drag. I use [Annoy](https://pypi.python.org/pypi/annoy) to make these calculations faster, and you should consider using it too. *Again, this notebook is a port of Allison Parrish's original [Python/Jupyter version](https://gist.github.com/aparrish/2f562e3737544cf29aaf1af30362f469), created for my students working with Observable and JavaScript. All credit should go to Allison. You can check out more of [her work here](https://www.decontextualize.com/).* `37:T200e,embeddings=function(e){var t={};function r(o){if(t[o])return t[o].exports;var n=t[o]={i:o,l:!1,exports:{}};return e[o].call(n.exports,n,n.exports,r),n.l=!0,n.exports}return r.m=e,r.c=t,r.d=function(e,t,o){r.o(e,t)||Object.defineProperty(e,t,{enumerable:!0,get:o})},r.r=function(e){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},r.t=function(e,t){if(1&t&&(e=r(e)),8&t)return e;if(4&t&&"object"==typeof e&&e&&e.__esModule)return e;var o=Object.create(null);if(r.r(o),Object.defineProperty(o,"default",{enumerable:!0,value:e}),2&t&&"string"!=typeof e)for(var n in e)r.d(o,n,function(t){return e[t]}.bind(null,n));return o},r.n=function(e){var t=e&&e.__esModule?function(){return e.default}:function(){return e};return r.d(t,"a",t),t},r.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)},r.p="",r(r.s=2)}([function(e,t){e.exports=tf},function(e,t,r){var o,n=function(){var e=String.fromCharCode,t="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=",r="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-$",o={};function n(e,t){if(!o[e]){o[e]={};for(var r=0;r>>8,r[2*o+1]=a%256}return r},decompressFromUint8Array:function(t){if(null==t)return s.decompress(t);for(var r=new Array(t.length/2),o=0,n=r.length;o>=1}else{for(n=1,o=0;o>=1}0==--p&&(p=Math.pow(2,f),f++),delete i[l]}else for(n=a[l],o=0;o>=1;0==--p&&(p=Math.pow(2,f),f++),a[u]=h++,l=String(c)}if(""!==l){if(Object.prototype.hasOwnProperty.call(i,l)){if(l.charCodeAt(0)<256){for(o=0;o>=1}else{for(n=1,o=0;o>=1}0==--p&&(p=Math.pow(2,f),f++),delete i[l]}else for(n=a[l],o=0;o>=1;0==--p&&(p=Math.pow(2,f),f++)}for(n=2,o=0;o>=1;for(;;){if(v<<=1,m==t-1){d.push(r(v));break}m++}return d.join("")},decompress:function(e){return null==e?"":""==e?null:s._decompress(e.length,32768,function(t){return e.charCodeAt(t)})},_decompress:function(t,r,o){var n,s,a,i,c,u,l,p=[],h=4,f=4,d=3,v="",m=[],g={val:o(0),position:r,index:1};for(n=0;n<3;n+=1)p[n]=n;for(a=0,c=Math.pow(2,2),u=1;u!=c;)i=g.val&g.position,g.position>>=1,0==g.position&&(g.position=r,g.val=o(g.index++)),a|=(i>0?1:0)*u,u<<=1;switch(a){case 0:for(a=0,c=Math.pow(2,8),u=1;u!=c;)i=g.val&g.position,g.position>>=1,0==g.position&&(g.position=r,g.val=o(g.index++)),a|=(i>0?1:0)*u,u<<=1;l=e(a);break;case 1:for(a=0,c=Math.pow(2,16),u=1;u!=c;)i=g.val&g.position,g.position>>=1,0==g.position&&(g.position=r,g.val=o(g.index++)),a|=(i>0?1:0)*u,u<<=1;l=e(a);break;case 2:return""}for(p[3]=l,s=l,m.push(l);;){if(g.index>t)return"";for(a=0,c=Math.pow(2,d),u=1;u!=c;)i=g.val&g.position,g.position>>=1,0==g.position&&(g.position=r,g.val=o(g.index++)),a|=(i>0?1:0)*u,u<<=1;switch(l=a){case 0:for(a=0,c=Math.pow(2,8),u=1;u!=c;)i=g.val&g.position,g.position>>=1,0==g.position&&(g.position=r,g.val=o(g.index++)),a|=(i>0?1:0)*u,u<<=1;p[f++]=e(a),l=f-1,h--;break;case 1:for(a=0,c=Math.pow(2,16),u=1;u!=c;)i=g.val&g.position,g.position>>=1,0==g.position&&(g.position=r,g.val=o(g.index++)),a|=(i>0?1:0)*u,u<<=1;p[f++]=e(a),l=f-1,h--;break;case 2:return m.join("")}if(0==h&&(h=Math.pow(2,d),d++),p[l])v=p[l];else{if(l!==f)return null;v=s+s.charAt(0)}m.push(v),p[f++]=s+v.charAt(0),s=v,0==--h&&(h=Math.pow(2,d),d++)}}};return s}();void 0===(o=function(){return n}.call(t,r,t,e))||(e.exports=o)},function(e,t,r){"use strict";r.r(t);var o=r(1),n=r(0);const s=function(e,t){var r=JSON.parse(o.decompressFromBase64(e));return n.tensor(r.vectors,r.shape,t)};r.d(t,"loadModel",function(){return i});class a{constructor(e,t,r){this.vocabulary=r,this.centroids=t,this.codes=e}_getVector(e){var t=this.vocabulary.indexOf(e);if(-1===t)return n.zeros([this.codes.shape[1]*this.centroids.shape[2]]);var r=this._getSearchVector(t),o=n.range(0,this.codes.shape[1],1,"int32"),s=n.stack([o,r],-1);return n.gatherND(this.centroids,s).flatten()}_getSearchVector(e){return this.codes.gather([e]).as1D()}_transformSequence(e,t){for(var r=[],o=0;o x(d.cuteness))\n .attr(\"cy\", d => y(d.size))\n .attr(\"r\", 20)\n .style(\"opacity\", .8);\n\n svg\n .append(\"g\")\n .attr(\"font-family\", \"sans-serif\")\n .attr(\"font-size\", 14)\n .selectAll(\"text\")\n .data(data)\n .join(\"text\")\n .attr(\"dy\", \"0.35em\")\n .attr(\"text-anchor\", \"middle\")\n .attr(\"x\", d => x(d.cuteness))\n .attr(\"y\", d => y(d.size))\n .text(d => d.name);\n\n svg\n .append(\"text\")\n .attr(\"x\", (width - (margin.left + margin.right)) / 2 + 50)\n .attr(\"y\", margin.top - 20)\n .attr(\"text-anchor\", \"middle\")\n .style(\"font-size\", \"16px\")\n .style(\"font-style\", \"bold\")\n .text(\"ANIMAL SPACE\");\n\n return svg.node();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":109,"value":"md`The plot shows us that the closest animal to the capybara is the panda bear (again, in terms of its subjective size and cuteness). One way of calculating how \"far apart\" two points are is to find their Euclidean distance. (This is simply the length of the line that connects the two points.) For points in two dimensions, Euclidean distance can be calculated with the basic distance function:`\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":113,"value":"distance2d = function(p1, p2){\n\tlet dx = p2[0] - p1[0];\n\tlet dy = p2[1] - p1[1];\n\treturn Math.sqrt(dx * dx + dy * dy);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":132,"value":"md`So, the distance between \"capybara\" (70, 30) and \"panda\" (74, 40):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":137,"value":"distance2d(animal('capybara'), animal('panda bear')) // panda and capybara","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":150,"value":"md`... is less than the distance between \"tarantula\" and \"elephant\":`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":144,"value":"distance2d(animal('tarantula'), animal('elephant')) // tarantula and elephant","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":157,"value":"$31","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":386,"value":"$32","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":403,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":411,"value":"colorData = (await fetch(\"https://raw.githubusercontent.com/dariusk/corpora/master/data/colors/xkcd.json\")).json()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":415,"value":"md`The following function converts colors from hex format (\\`#1a2b3c\\`) to a tuple of integers \\`([26, 43, 60]\\`):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":426,"value":"hexToInt = function(s) {\n return [\n parseInt(s.slice(1, 3), 16),\n parseInt(s.slice(3, 5), 16),\n parseInt(s.slice(5, 7), 16)\n ];\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":428,"value":"colors = {\n let data = {};\n colorData.colors.forEach(item => data[item.color] = hexToInt(item.hex));\n return data;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":473,"value":"md`Testing it out:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":481,"value":"colors['olive']","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":486,"value":"colors['red']","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":487,"value":"colors['black']","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":489,"value":"md`\n### Vector math\n\nBefore we keep going, we'll need some functions for performing basic vector \"arithmetic.\" These functions will work with vectors in spaces of any number of dimensions.\n\nThe first function returns the Euclidean distance between two points:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":492,"value":"distance = function(coord1, coord2) { \n return Math.sqrt(coord1.reduce((acc, e, i) => acc + Math.pow(e - coord2[i], 2), 0));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":494,"value":"distance([10, 1], [5, 2]);","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":497,"value":"md`The \\`subtractv\\` function subtracts one vector from another:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":507,"value":"subtractv = function(coord1, coord2) { \n return coord1.map((e,i) => e - coord2[i]);\n}\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":515,"value":"subtractv([10, 1], [5, 2])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":502,"value":"md`The \\`addv\\` function subtracts one vector from another:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":509,"value":"addv = function(coord1, coord2) { \n return coord1.map((e,i) => e + coord2[i]);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":520,"value":"addv([10, 1], [5, 2])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":524,"value":"md`And the \\`meanv\\` function takes a list of vectors and finds their mean or average:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":526,"value":"meanv = function(coords) { \n let result = [];\n for (let item in coords[0]) {\n let total = coords.reduce( (acc, arr) => acc += arr[item], 0);\n result.push(total / coords.length);\n }\n return result;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":527,"value":"meanv([[0, 1], [2, 2], [4, 3]])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":529,"value":"md`Just as a test, the following cell shows that the distance from \"red\" to \"green\" is greater than the distance from \"red\" to \"pink\":`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":532,"value":"distance(colors['red'], colors['green']) > distance(colors['red'], colors['pink'])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":595,"value":"md`### Finding the closest item\n\nJust as we wanted to find the animal that most closely matched an arbitrary point in cuteness/size space, we'll want to find the closest color name to an arbitrary point in RGB space. The easiest way to find the closest item to an arbitrary vector is simply to find the distance between the target vector and each item in the space, in turn, then sort the list from closest to farthest. The \\`closest()\\` function below does just that. By default, it returns a list of the ten closest items to the given vector.\n\n*Note: Calculating \"closest neighbors\" like this is fine for the examples in this notebook, but unmanageably slow for vector spaces of any appreciable size. As your vector space grows, you'll want to move to a faster solution, like the JS version of [Annoy](https://www.npmjs.com/package/annoy).*`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":601,"value":"closest = function(space, coord, n=10) {\n let sorted = Object.keys(space).sort((a, b) => \n distance(space[a],coord) - distance(space[b],coord));\n return sorted.slice(0,n);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":602,"value":"closest(colors, colors.red)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":620,"value":"md`... or the ten colors closest to (150, 60, 150):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":622,"value":"closest(colors, [150, 60, 150])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":627,"value":"md`### Color magic\nThe magical part of representing words as vectors is that the vector operations we defined earlier appear to operate on language the same way they operate on numbers. For example, if we find the word closest to the vector resulting from subtracting \"red\" from \"purple,\" we get a series of \"blue\" colors:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":629,"value":"closest(colors, subtractv(colors['purple'], colors['red']))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":632,"value":"md`This matches our intuition about RGB colors, which is that purple is a combination of red and blue. Take away the red, and blue is all you have left.\n\nYou can do something similar with addition. What's blue plus green?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":633,"value":"closest(colors, addv(colors['blue'], colors['green']))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":636,"value":"md`That's right, it's something like turquoise or cyan! What if we find the average of black and white? Predictably, we get gray:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":638,"value":"// the average of black and white: medium grey\nclosest(colors, meanv([colors['black'], colors['white']]))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":643,"value":"md`Just as with the tarantula/hamster example from the previous section, we can use color vectors to reason about relationships between colors. In the cell below, finding the difference between \"pink\" and \"red\" then adding it to \"blue\" seems to give us a list of colors that are to blue what pink is to red (i.e., a slightly lighter, less saturated shade):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":645,"value":"{\n // an analogy: pink is to red as X is to blue\n let pinkToRed = subtractv(colors['pink'], colors['red'])\n return closest(colors, addv(pinkToRed, colors['blue']))\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":657,"value":"md`Another example of color analogies: Navy is to blue as true green/dark grass green is to green:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":658,"value":"{\n // another example: \n let navyToBlue = subtractv(colors['navy'], colors['blue'])\n return closest(colors, addv(navyToBlue, colors['green']))\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":661,"value":"md`The examples above are fairly simple from a mathematical perspective but nevertheless *feel* magical: they're demonstrating that it's possible to use math to reason about how people use language.\n\n### Interlude: A Love Poem That Loses Its Way`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":666,"value":"{\n let lines = [];\n let red = colors['red'];\n let blue = colors['blue'];\n for (let i = 0; i < 14; i++) {\n let rednames = closest(colors, red);\n let bluenames = closest(colors, blue);\n lines.push(`Roses are ${rednames.shift()}, violets are ${bluenames.shift()}`);\n red = colors[rednames[Math.floor(Math.random() * rednames.length)]];\n blue = colors[bluenames[Math.floor(Math.random() * bluenames.length)]];\n }\n return md`

${lines.join(\"\\r\\n\")}

`;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":764,"value":"md`### Doing bad digital humanities with color vectors\n\nWith the tools above in hand, we can start using our vectorized knowledge of language toward academic ends. In the following example, I'm going to calculate the average color of Bram Stoker's *Dracula*, which we can load from [here](http://www.gutenberg.org/cache/epub/345/pg345.txt).`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":766,"value":"avgColor = {\n let doc = await gutenFetch('http://www.gutenberg.org/cache/epub/345/pg345.txt');\n let words = RiTa.tokenize(doc), dracColors = [];\n words.forEach((w,i) => colors[w.toLowerCase()] \n && dracColors.push(colors[w.toLowerCase()]));\n return meanv(dracColors);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":874,"value":"closest(colors, avgColor)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":876,"value":"md`On the other hand, here's what we get when we average the colors of Charlotte Perkins Gilman's classic *The Yellow Wallpaper*, from [here](http://www.gutenberg.org/cache/epub/1952/pg1952.txt). The result definitely reflects the content of the story, so maybe we're on to something here.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":881,"value":"{\n let doc = await gutenFetch('http://www.gutenberg.org/cache/epub/1952/pg1952.txt');\n let words = RiTa.tokenize(doc), dracColors = [];\n words.forEach((w,i) => colors[w.toLowerCase()] \n && dracColors.push(colors[w.toLowerCase()]));\n let avgColor = meanv(dracColors);\n return closest(colors, avgColor);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":891,"value":"md`Exercise for the reader: Use the vector arithmetic functions to rewrite a text, making it...\n* more blue (i.e., add \\`colors['blue']\\` to each occurrence of a color word); or\n* more light (i.e., add \\`colors['white']\\` to each occurrence of a color word); or\n* darker (i.e., attenuate each color. You might need to write a vector multiplication function to do this one right.)\n\n## Distributional semantics\n\nIn the previous section, the examples are interesting because of a simple fact: colors that we think of as similar are \"closer\" to each other in RGB vector space. In our color vector space, or in our animal cuteness/size space, you can think of the words identified by vectors close to each other as being *synonyms*, in a sense: they sort of \"mean\" the same thing. They're also, for many purposes, *functionally identical*. Think of this in terms of writing, say, a search engine. If someone searches for \"mauve trousers,\" then it's probably also okay to show them results for, say,`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":902,"value":"closest(colors, colors['mauve']).map(cname => cname + \" trousers\");","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":910,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":922,"value":"md`## Word vectors by counting contexts\n\nSo how do we turn this insight from the Distributional Hypothesis into a system for creating general-purpose vectors that capture the meaning of words? Maybe you can see where I'm going with this. What if we made a *really big* spreadsheet that had one column for every context for every word in a given source text. Let's use a small source text to begin with, such as this excerpt from Dickens:\n\n It was the best of times, it was the worst of times.\n\nSuch a spreadsheet might look something like this:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":964,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1167,"value":"tokens = RiTa.tokens(await gutenFetch('http://www.gutenberg.org/cache/epub/345/pg345.txt'));\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1024,"value":"md`You can see the vector of any word in vocabulary list like so:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1180,"value":"gloVe.getVector(\"cheese\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1376,"value":"md`For the sake of convenience, the following function gets the vector of a given string from spaCy's vocabulary:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1383,"value":"vec = function(word) { return gloVe.getVector(word) }","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1194,"value":"md`\n### Cosine similarity and finding closest neighbors\n\nThe cell below defines a function \\`cosine()\\`, which returns the [cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) of two vectors. Cosine similarity is another way of determining how similar two vectors are, which is more suited to high-dimensional spaces. [See the Encyclopedia of Distances](http://www.uco.es/users/ma1fegan/Comunes/asignaturas/vision/Encyclopedia-of-distances-2009.pdf) for more information and even more ways of determining vector similarity.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1217,"value":"cosine = function(v1, v2) {\n\t\tlet vec1 = tf.tensor1d(v1);\n\t\tlet vec2 = tf.tensor1d(v2);\n\t\tlet dot = vec1.dot(vec2).asScalar();\n\t\tlet sim = dot.div(vec1.norm(2)).div(vec2.norm(2));\n\t\treturn 1 - sim.dataSync()[0];\n\t}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1212,"value":"cosine(vec('dog'), vec('puppy')) < cosine(vec('trousers'), vec('octopus'))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1244,"value":"md`The following cell defines a function that iterates through a list of tokens and returns the token whose vector is most similar to a given vector.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1389,"value":"gloVeClosest = async function(tokenList, vector, n=10) {\n let tensor = tf.tensor1d(vector);\n let neighbors = await gloVe._getNearestNeighbors(tensor, n*100, true);\n return neighbors.filter(t => tokenList.includes(t.word)).slice(0, n).map(n => n.word);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1347,"value":"md`Using this function, we can get a list of synonyms, or words closest in meaning (or distribution, depending on how you look at it), to any arbitrary word in spaCy's vocabulary. In the following example, we're finding the words in Dracula closest to \"basketball\":`\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1254,"value":"gloVeClosest(tokens, vec(\"basketball\"))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1369,"value":"md`### Fun with GloVe, Dracula, and vector arithmetic\n\nNow we can start doing vector arithmetic and finding the closest words to the resulting vectors. For example, what word is closest to the halfway point between day and night?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1371,"value":"gloVeClosest(tokens, meanv([vec(\"day\"), vec(\"night\")]))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1420,"value":"md`Variations of \\`night\\` and \\`day\\` are still closest, but after that we get words like \\`evening\\` and \\`morning\\`, which are indeed halfway between day and night!\n\nHere are the closest words in _Dracula_ to \"wine\":\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1422,"value":"gloVeClosest(tokens, vec(\"wine\"))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1426,"value":"md`If you subtract \"alcohol\" from \"wine\" and find the closest words to the resulting vector, you're left with simply a lovely dinner:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1427,"value":"gloVeClosest(tokens, subtractv(vec(\"wine\"), vec(\"alcohol\"))) // hmmm, something off here","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1434,"value":"md`The closest words to \"water\":`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1438,"value":"gloVeClosest(tokens, vec(\"water\"))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1440,"value":"md`But if you add \"frozen\" to \"water,\" you get \"ice\":`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1444,"value":"gloVeClosest(tokens, addv(vec(\"water\"),vec(\"frozen\")))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1498,"value":"md`You can even do analogies! For example, the words most similar to \"grass\":`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1499,"value":"gloVeClosest(tokens, vec(\"grass\"))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1503,"value":"md`If you take the difference of \"blue\" and \"sky\" and add it to grass, you get the analogous word (\"green\"):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1505,"value":"{\n // analogy: blue is to sky as X is to grass\n let blueToSky = subtractv(vec(\"blue\"), vec(\"sky\"))\n return gloVeClosest(tokens, addv(blueToSky, vec(\"grass\")))\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1511,"value":"md`## Sentence similarity\nTo get the vector for a sentence, we simply average its component vectors, like so:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1530,"value":"sentVec = function(s) {\n let words = RiTa.tokenize(s);\n return meanv(words.map(w => gloVe.getVector(w)));\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1542,"value":"sentences = RiTa.sentences(await gutenFetch('http://www.gutenberg.org/cache/epub/345/pg345.txt'));","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1538,"value":"gloVeClosestSent = function(space, inputStr, n=10) {\n let inputVec = sentVec(inputStr);\n let dists = {};\n console.time(\"cos\");\n space = space.slice(0,500);\n space.forEach(x => dists[x] = cosine(sentVec(x), inputVec));\n console.timeEnd(\"cos\");\n return Object.keys(dists).sort((a,b) => dists[a] - dists[b]).slice(0,n);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1553,"value":"md`Below you can see the sentences in *Dracula* closest in meaning to \"My favorite food is strawberry ice cream.\" (I won't run it live here as its too slow in the browser)\n\n~~~js\n gloVeClosestSent(sentences, \"My favorite food is strawberry ice cream.\")\n~~~\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1674,"value":"result = [ \n \"This, with some cheese and a salad and a bottle of…Tokay, of which I had two glasses, was my supper.\",\n \"There is not even a toilet glass on my table, and …bag before I could either shave or brush my hair.\",\n \"I dined on what they called \\\"robber steak\\\"--bits o…re, in the simple style of the London cat's meat!\",\n \"I had for breakfast more paprika, and a sort of po…very excellent dish, which they call \\\"impletata.\\\"\",\n \"There was everywhere a bewildering mass of fruit b… under the trees spangled with the fallen petals.\",\n \"I had for dinner, or rather supper, a chicken done…with red pepper, which was very good but thirsty.\",\n \"After breakfast I did a little exploring in the castle.\",\n \"I set to and enjoyed a hearty meal.\",\n \"When I had dressed myself I went into the room whe…e kept hot by the pot being placed on the hearth.\",\n \"There is a flask of slivovitz (the plum brandy of …) underneath the seat, if you should require it.\\\"\"\n]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1688,"value":"md`For doing real working with sentence vectors, you will likely want to use some kind of sentence encoder, for example the *Universal Sentence Encoder* which gives the following results: `","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1806,"value":"[\n 'I had for dinner, or rather supper, a chicken done up some way with red pepper, which was very good but thirsty.',\n 'I had for breakfast more paprika, and a sort of porridge of maize flour which they said was \"mamaliga,\" and egg-plant stuffed with forcemeat, a very excellent dish, which they call \"impletata.\"',\n 'Breakfast was a strange meal to us all.',\n 'I never liked garlic before, but to-night it is delightful!',\n 'I set to and enjoyed a hearty meal.',\n 'It is not only because it is all sweet to me, but because you have been, and are, very dear to me.',\n 'We had a happy surprise at breakfast.',\n 'Give me some water, my lips are dry; and I shall try to tell you.',\n 'I have grown quite fond of the garlic, and a boxful arrives for me every day from Haarlem.',\n 'Wet my lips with brandy again.'\n ]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1810,"value":"md`If you want to experiment further, you can find a light version of the *Universal Sentence Encoder* in JavaScript [here](https://github.com/tensorflow/tfjs-models/tree/master/universal-sentence-encoder).`\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1685,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":25,"value":"md`------------\n## Appendix\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"import {mdTable} from '@uwdata/data-utilities'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1708,"value":"data = [\n {'name': 'kitten', 'cuteness': 95, 'size': 15},\n {'name': 'hamster', 'cuteness': 80, 'size': 8},\n {'name': 'tarantula', 'cuteness': 8, 'size': 3},\n {'name': 'puppy', 'cuteness': 90, 'size': 20},\n {'name': 'crocodile', 'cuteness': 5, 'size': 40},\n {'name': 'dolphin', 'cuteness': 60, 'size': 45},\n {'name': 'panda bear', 'cuteness': 75, 'size': 40},\n {'name': 'lobster', 'cuteness': 2, 'size': 15},\n {'name': 'capybara', 'cuteness': 70, 'size': 30},\n {'name': 'elephant', 'cuteness': 65, 'size': 90},\n {'name': 'mosquito', 'cuteness': 1, 'size': 1},\n {'name': 'goldfish', 'cuteness': 25, 'size': 2},\n {'name': 'horse', 'cuteness': 50, 'size': 50},\n {'name': 'chicken', 'cuteness': 25, 'size': 15}\n]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1705,"value":"animal = function(name) { return data.filter(d => d.name === name).map(o => [o.cuteness, o.size])[0] }","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":806,"value":"RiTa = require('rita@2.4.86');","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1150,"value":"gloVe = await embeddings.loadModel(\n \"https://raw.githubusercontent.com/mb-14/embeddings.js/master/assets/word-embeddings.json\"\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1566,"value":"tf = require('https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@3.3.0/dist/tf.min.js')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":791,"value":"gutenFetch = async function(url) {\n return fetch('https://observable-cors.glitch.me/' + url, {\n headers: {\n Accept: '*/*',\n 'Accept-Language': 'en-US,en;q=0.5',\n Referer: 'https://beta.observablehq.com/',\n },\n cache: 'no-store',\n mode: 'cors',\n redirect: 'follow',\n }).then(response => {\n if (!response.ok) throw new Error(response.status);\n return response.text();\n }).catch(error => {return error})\n};","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":8,"value":"d3 = require(\"d3@6\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":43,"value":"x = d3.scaleLinear()\n .domain(d3.extent(data, d => d.cuteness)).nice()\n .range([margin.left, width - margin.right])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":572,"value":"xAxis = g => g\n .attr(\"transform\", `translate(0,${height - margin.bottom})`)\n .call(d3.axisBottom(x).ticks(width / 80))\n .call(g => g.select(\".domain\").remove())\n .call(g => g.append(\"text\")\n .attr(\"x\", (width-(margin.left+margin.right))/2+80) \n .attr(\"y\", margin.bottom - 24)\n .attr(\"fill\", \"currentColor\")\n .attr(\"text-anchor\", \"end\")\n .text(\"CUTENESS\"))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":42,"value":"y = d3.scaleLinear()\n .domain([1, 100]).nice()\n .range([height - margin.bottom, margin.top])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":583,"value":"yAxis = g => g\n .attr(\"transform\", `translate(${margin.left}, 0)`)\n .call(d3.axisLeft(y))\n .call(g => g.select(\".domain\").remove())\n .call(g => g.append(\"text\")\n .attr(\"x\", -margin.left)\n .attr(\"y\", height/2-20)\n .attr(\"fill\", \"currentColor\")\n .attr(\"text-anchor\", \"start\")\n .text(\"SIZE\"))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":96,"value":"grid = g => g\n .attr(\"stroke\", \"currentColor\")\n .attr(\"stroke-opacity\", 0.1)\n .call(g => g.append(\"g\")\n .selectAll(\"line\")\n .data(x.ticks())\n .join(\"line\")\n .attr(\"x1\", d => 0.5 + x(d))\n .attr(\"x2\", d => 0.5 + x(d))\n .attr(\"y1\", margin.top)\n .attr(\"y2\", height - margin.bottom))\n .call(g => g.append(\"g\")\n .selectAll(\"line\")\n .data(y.ticks())\n .join(\"line\")\n .attr(\"y1\", d => 0.5 + y(d))\n .attr(\"y2\", d => 0.5 + y(d))\n .attr(\"x1\", margin.left)\n .attr(\"x2\", width - margin.right));","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":32,"value":"margin = ({top: 50, right: 400, bottom: 65, left: 55})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1454,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":33,"value":"height = 400","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W38","stdlib":"1"}} 30:"$2f:body"