2e:null 2f:["$","title",null,{"children":"Workshop Notebook: Machine Learning for Hackers (FullStack) | Athan | Observable"}] 34:Teaa,md` **Why machine learning (and, more generally, scientific computing) in JavaScript?** - _Speed_: JavaScript is fast. - _Rendering Engine_: web browsers are performant, highly optimized view engines. - _Community_: JavaScript has one of the largest and most diverse developer communities. - _Ubiquity_: JavaScript is ubiquitous. - _Distribution_: because JavaScript is ubiquitous, the need for installing additional languages and tooling is often unnecessary. - _Package Management_: Node.js package management is superior to anything available in other numeric computing environments. **What are the use cases for numeric computing in JavaScript?** Fundamentally, the use cases for numerical and scientific computing in JavaScript are the same as for any other commonly used language for numerical and scientific computation, such as MATLAB, R, Python, and Julia. For example, - perform some sort of statistical analysis, such as computing summary statistics or hypothesis testing. - train and apply a statistical model (e.g., assign a probability as to whether \`A\` has characteristic \`B\` given an observation \`Q\`). - cluster observations into a set of \`N\` distinct groups. - determine the similarity of two or more datasets (i.e., can we reasonably distinguish \`X\` from \`Y\`?). - assign a likelihood to unexpected events (i.e., all things being equal, how often should we expect to see a particular outcome were we to repeat our experiment hundreds, thousands or millions of times?). - perform simulations involving alternative models (e.g., what would the data look like if we change model parameters \`a\` and \`b\`?). - extract key "features" accounting for the most variation in observed values. - translate text from one language to another. - generate synthetic speech. - rank documents according to their relevance (e.g., to build a search engine). - recognize and classify objects in images (e.g., does image \`I\` contain a cat or a dog?). - predict growth and financial returns. - compute the trajectory of spacecraft and flying objects. - determine optimal ticket prices and staffing requirements to maximize revenue. - parse, transform, filter, and aggregate data. - add, subtract, multiply, and divide numbers. In addition to the above, JavaScript has additional use cases by virtue of its being the _lingua franca_ of the web. - offline computation in web applications (i.e., an entirely client-side machine learning library does not require a network connection in order to execute commands as is the case, e.g., in the Jupyter notebook computation model). - rapid prototyping and visualization. - interactive explanations. The popularity of Node.js due to its ease-of-use and growing ubiquity in the HTTP networking stack affords Node.js additional use cases such as - a rapidly prototyped and demo-able Node.js server endpoint which performs natural language processing (NLP) and uses the Amazon Alexa API for a chat bot. - a serverless application which performs a machine learning computation without needing to install and bundle all of, e.g., Python, NumPy, and SciPy, thus saving time, money, and resources (all by virtue of Node.js' dominance in serverless cloud offerings, such as AWS Lambda, Google Cloud, and Microsoft Azure). - browserless model computation done entirely in JavaScript. While other languages and platforms exist which _may_ be better suited for specific use cases, particularly those requiring bare metal performance or involving massive data sets, when considered in totality, JavaScript (and Node.js) provides a comparable, if not better, environment for numerical and scientific computation than other competitor environments. `35:T45a,md` Outline: - Sign-up/Sign-in (requires GitHub account!) - Starting from a blank notebook - Creating and deleting cells - Multi-line code blocks - Reactivity - Creating interactive elements - Writing mathematical equations (LaTeX) - Loading packages - Help - Forking notebooks ObservableHQ allows you to easily create interactive JavaScript notebooks for data analysis, visualization, and exploration. You can start by signing in via GitHub. Go to https://beta.observablehq.com/, click the "Sign in" button and then "Sign in with GitHub". After signing in, click the "+ New" button to create an empty notebook. An empty notebook will appear for you to edit. Under "My Notebooks", you will find a list of all your notebooks. Progress is automatically saved when cells are executed. If you do not have a GitHub account and don't want to create one, you can still complete all workshop exercises. The "Scratchpad" provides an empty workbook where you can run any code without having to sign in, and you can also edit existing notebooks. However, you **won't** be able to save your progress. `36:T4f1,md` A _model_ is an attempt to understand and represent a particular phenomenon (e.g., page clicks, human heights, solar flares, etc) via a specific representation, be it architectural (e.g., a scaled model of a building), biological (e.g., a description of gene regulation), or mathematical (e.g., mapping home size to its selling price). A model typically consists of what are deemed essential elements, making qualifying assumptions and abstractions. When we refer to a _mathematical model_, we refer to a collection of symbols and expressions which describe relationships between variable quantities. The expressions are typically general enough that they include parameters whose values we seek to infer from observed data. But how do we even build a model in the first place? The truth is that building a model is part art and part science and entails an iterative process in which one is forced to make, clarify, and refine assumptions about the underlying structure of reality. And while we'd like to say that there exists a single template and standard by which we make those choices and refinements and how we explain them, no such standard exists. We must rely on a combination of common sense, domain expertise, and rigorous testing and evaluation. `37:T121e,md` Over time, as you encounter new datasets and problems, you'll build an arsenal of potential statistical models, gaining an intuition as to their strengths, general applicability, and, perhaps most importantly, limitations. Undoubtedly, what you'll find is that some of the fundamental building blocks of these models are _probability distributions_. Before the advent of modern digital computers, scientists observed real-world phenomenon, recorded their observations, and noticed that certain mathematical shapes kept recurring. To approximate these shapes, scientists wrote mathematical equations whose parameters could be estimated from the data. A classic example is the normal distribution, a bell-shaped curve commonly referred to as the Gaussian distribution, which is used to approximate the distribution of human heights. While, in reality, there exists an infinite number of probability distributions, we often encounter a more limited set of canonical (named) distributions, which are simply a set of shapes capable of being described mathematically and which occur often enough that someone decided they deserved names. For example, the normal distribution may be expressed ${tex.block`N(x|\mu,\sigma) \sim \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}`} where the parameter ${tex`\mu`} is the mean and, as a normal distribution is symmetric, specifies the distribution center. The parameter ${tex`\sigma`} is the standard deviation, which quantifies how spread out (wide) a distribution is. When we want to describe a phenomenon with a normal distribution, these parameters are realized as actual numbers, whose values we estimate from observed data in a manner similar to when we computed the sample mean above. Probability distributions come in two flavors: **discrete** and **continuous**. A _discrete probability distribution_ describes the probability of occurrence for each value of a discrete random variable, where a _discrete random variable_ is a random variable that has countable values (e.g., a list of nonnegative integers). A _continuous probability distribution_ describes the probability of occurrence for a range of continuous random variables, where a _continuous random variable_ is a random variable whose set of possible values is infinite and uncountable. To help familiarize ourselves with common probability distributions, let's complete the following exercises. **Exercise 1**: create an interactive widget immediately following this cell which plots the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/normal/pdf) (PDF) for a [normal distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/normal) and includes two sliders, one for ${tex`\mu`} and another for ${tex`\sigma`}, which, when updated, updates the plot. A boilerplate [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) instance has already been provided below. > **Hint**: to generate a linearly spaced array for the \`x\` values (i.e., the values at which to evaluate the PDF), see [linspace](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/math/utils/linspace) (available as [\`stdlib.linspace(a, b, n)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/math/utils/linspace) in this notebook). > **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/normal/pdf) for a [normal distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/normal) is available as [\`stdlib.base.dists.normal.pdf(x, \`${tex`\mu`}, ${tex`\sigma`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/normal/pdf) in this notebook. > **Hint**: for interactive HTML elements, see the documentation on [tagged template literals](https://beta.observablehq.com/@mbostock/standard-library) in Observable. > **Hint**: you may find it useful to fix the x-axis limits by setting the \`xMin\` and \`xMax\` [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) properties. `38:T43c,md` **Exercise 7**: repeat the previous exercise for the [beta distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/beta/pdf) with parameters ${tex`\alpha`} and ${tex`\beta`}. ${tex.block`f(x|\alpha,\beta)= \begin{cases} \frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha) + \Gamma(\beta)}{x^{\alpha-1}(1-x)^{\beta-1}} & \text{ for } x \in (0,1) \\ 0 & \text{ otherwise } \end{cases}`} > **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/beta/pdf) for a [beta distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/beta) is available as [\`stdlib.base.dists.beta.pdf(\`${tex`x`}, ${tex`\alpha`}, ${tex`\beta`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/beta/pdf) in this notebook. `39:T41a,md` **Exercise 8**: repeat the previous exercise for the [Weibull distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/weibull/pdf) with parameters ${tex`k`} and ${tex`\lambda`}. ${tex.block`f(x|\lambda,k) = \begin{cases} \frac{k}{\lambda}\left (\frac{x}{\lambda} \right)^{k-1}e^{-(x/\lambda)^k} & x \geq 0 \\ 0 & x < 0\end{cases}`} > **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/weibull/pdf) for a [Weibull distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/weibull) is available as [\`stdlib.base.dists.weibull.pdf(\`${tex`x`}, ${tex`k`}, ${tex`\lambda`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/weibull/pdf) in this notebook. `3a:T407,md` **Exercise 10**: repeat the previous exercise for the [Poisson distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/poisson/pmf) with parameter ${tex`\lambda`}. ${tex.block`f(x|\lambda) = P(X=x|\lambda) = \begin{cases} \tfrac{\lambda^x}{x!}e^{-\lambda} & \text{ for } x = 0,1,2,\ldots \\ 0 & \text{ otherwise} \end{cases}`} > **Hint**: the function to evaluate the [probability mass function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/poisson/pmf) for a [Poisson distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/poisson) is available as [\`stdlib.base.dists.poisson.pmf(\`${tex`x`}, ${tex`\lambda`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/poisson/pmf) in this notebook. `3b:T53c,md` A _statistical hypothesis_ is a testable hypothesis concerning an observed process modeled according to a set of random variables. A _statistical hypothesis test_ is a statistical inference method in which two (or more) datasets are compared based on their statistical properties. Commonly, such a test involves the statistical comparison of two datasets or of a _sample_ against a simulated dataset generated according to an idealized statistical model. Often a _statistical hypothesis_ proposes a statistical relationship between two (or more) datasets and serves as an alternative to a **null hypothesis** that proposes no relationship. A comparison is considered _statistically significant_ if the relationship between the datasets has a high likelihood of being due to a real systematic difference and a low likelihood of occurring by chance alone. Whether a relationship is likely is determined according to a threshold probability--the _significance level_. To make hypothesis testing more concrete, imagine the following scenario: you have two landing pages for your website and would like to know which one is more effective in converting visitors. Suppose you have collected the following two data sets, where \`1\` denotes that a visitor signed up and \`0\` denotes that a visitor left the page without taking any action. `3c:T92b,md` Provided the proportions are different, is the difference also statistically significant? Let's find out! A common statistical hypothesis test for testing whether two datasets are different is the [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2). > A [_t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) is any statistical hypothesis test in which the test statistic follows a [Student's _t_-distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/t/pdf) under the **null hypothesis**. When dataset sample sizes and population variances may **not** be equal, the test statistic for determining whether the population means of two datasets differ may be expressed as ${tex.block`t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}`} where ${tex`\bar{x}_i`} is the sample mean, ${tex`s_i`} is the unbiased estimate of the sample variance, and ${tex`n_i`} is the number of observations in dataset ${tex`i`}. **Exercise 12**: perform a two-sample [_t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) for the datasets introduced above. > **Hint**: the function to perform a two-sample [_t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) is available as [\`stdlib.ttest2(\`${tex`x`}, ${tex`y`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) in this notebook. > **Hint**: the returned test result has a \`rejected\` property which is a boolean indicating whether the **null hypothesis** of the test was rejected. The **null hypothesis**, often denoted ${tex`H_0`}, denotes the case where a difference is **not** statistically significant. The **alternative hypothesis**, often denoted ${tex`H_1`}, denotes the case where a difference **is** statistically significant. > **Hint**: for pretty-printed output, use the \`print()\` method, which will print the exact definition of the **alternative hypothesis**. `3d:T460,md` **Exercise 13**: by default, [\`ttest2()\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) and the other [stdlib](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats) statistical tests test for a non-directional effect. This is called a _two-sided_ test. When you have prior beliefs about which group has a larger effect, you should consider running a _one-sided_ test. For illustration purposes, rerun the test for the landing pages using one-sided tests and observe how the alternative hypothesis and test result change. Keep in mind, however, that you should choose a test direction **before** looking at the data. > **Hint**: to run a one-sided [_t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2), set the \`alternative\` option when invoking [\`stdlib.ttest2()\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2). `3e:Ta93,md` **Challenge 1**: create an interactive widget immediately following this cell for simulating landing page data and performing a two-sample [_t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) on the simulated datasets. The widget should contain four sliders: one for adjusting the number of simulated visitors to the first landing page ${tex`n_1`}, a second for adjusting the number of simulated visitors to the second landing page ${tex`n_2`}, another for adjusting the probability ${tex`p_1`} that a user converts when visiting the first landing page, and a fourth for adjusting the probability ${tex`p_2`} that a user converts when visiting the second landing page. > **Hint**: the probabilities ${tex`p_1`} and ${tex`p_2`} should be both limited to the interval [0,1]. > **Hint**: to generate binary pseudorandom numbers (i.e., random values which can be either 0 or 1), you can generate a [uniformly distributed pseudorandom number](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/randu) on the interval [0,1) and apply a threshold corresponding to probability ${tex`p`}. If a pseudorandom number is greater than ${tex`p`}, then the binary pseudorandom number is 1 and otherwise 0. The function for generating [uniformly distributed pseudorandom numbers](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/randu) on the interval [0,1) is available as [\`stdlib.base.random.randu()\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/randu) in this notebook. > **Hint**: to generate reproducible pseudorandom number sequences, you may want to create a seeded pseudorandom number generator. > **Hint**: to sanity check the simulation, [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) the simulated datasets (sorted in descending order) and visually confirm that the relative proportion of conversions accords with expectation. A boilerplate [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) instance has already been provided below. > **Hint**: for pretty-printed output, use the \`print()\` method, which will print the exact definition of the **alternative hypothesis**. > **Hint**: for interactive HTML elements, see the documentation on [tagged template literals](https://beta.observablehq.com/@mbostock/standard-library) in Observable. `3f:T9fb,md` **Challenge 2**: generate a [scatterplot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) which explores how many visitors we would need to sample in order to ascertain whether a difference in conversion proportions is statistically significant. To do so, for a fixed probability ${tex`p_1`} and a varying probability ${tex`p_2`}, where, for simplicity, ${tex`p_2 >= p_1`}, systematically generate simulated datasets of varying length ${tex`n`} until you find a statistically significant difference (i.e., rejection of the **null hypothesis**) according to a two-sample two-sided [_t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2). For each ${tex`\Delta p = p_1 - p_2`}, store the length at which a statistically significant difference was found, and [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) ${tex`|\Delta p| = |p_1 - p_2|`} against ${tex`n`}. A boilerplate [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) instance has already been provided below. > **Hint**: for simplicity, set ${tex`p_1 = 0.1`} and then vary ${tex`p_2`} on the interval ${tex`[p_1,1]`}. > **Hint**: to generate a linearly spaced array for the ${tex`p_2`} (and/or ${tex`\Delta p`}) values, see [linspace](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/math/utils/linspace) (available as [\`stdlib.linspace(a, b, n)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/math/utils/linspace) in this notebook). > **Hint**: in order to avoid infinite loops, you will likely want to set a maximum ${tex`n`} at which, if a statisticially significant difference is **not** found, you stop searching. This is particularly relevant when ${tex`\Delta p \approx 0`}. Where appropriate, you can omit such occurrences from plotted results. > **Hint**: in order to make results more robust, you may want to consider repeating each dataset simulation and corresponding two-sample two-sided [_t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) a specified number of times and then calculating the average length at which a statistically significant difference was found. `40:T10bd,md` We can decide whether a **null hypothesis** should be rejected by using the [_p_-value](https://en.wikipedia.org/wiki/P-value) of a statistical test. A smaller [_p_-value](https://en.wikipedia.org/wiki/P-value) indicates that a **null hypothesis** fails to adequately explain the data. As a decision rule, we reject the **null hypothesis** whenever the [_p_-value](https://en.wikipedia.org/wiki/P-value) is lower than the chosen significance level ${tex`\alpha`}, which, by convention, is often set to \`0.05\`. Recall that ${tex`\alpha`} denotes the probability that you reject the **null hypothesis** even though the **null hypothesis** is true. By setting ${tex`\alpha = 0.05`}, we are basically saying that we are content with erroneously rejecting the **null hypothesis** in 5% of tests. Why is ${tex`\alpha`} equal to this error probability? We can mathematically prove that, if the **null hypothesis** is true, then the [_p_-value](https://en.wikipedia.org/wiki/P-value) follows a [uniform distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/uniform/pdf) on the interval ${tex`[0,1]`}. Hence, over the course of many experiments, we would expect 5% of [_p_-values](https://en.wikipedia.org/wiki/P-value) to be smaller than ${tex`0.05`}. Instead of a mathematical proof, let's simulate some data to confirm that this is, in fact, the case. **Exercise 14**: create an interactive widget immediately following this cell for generating an array whose elements are values drawn from a [normal distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/normal/pdf) with mean ${tex`\mu`} and standard deviation ${tex`\sigma`}. The widget should include three sliders: one for adjusting the number of elements in the simulated array, another for adjusting ${tex`\mu`}, and a third for adjusting ${tex`\sigma`}. And further, the widget should update a [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) displaying the generated numbers. For initial values, use ${tex`\mu = 0`} and ${tex`\sigma = 3`}. A boilerplate [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) instance has already been provided below. > **Hint**: the function to generate [pseudorandom numbers](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/normal) drawn from a [normal distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/normal) is available as [\`stdlib.base.random.normal(\`${tex`\mu`}, ${tex`\sigma`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/normal) in this notebook. > **Hint**: generating an array of random variates can be done in a single line using a combination of the PRNG \`factory()\` method and [inmap](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/inmap) (available as [\`stdlib.inmap(collection, fcn)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/inmap) in this notebook). > **Hint**: to better understand how adjusting parameters affects results, you may want to specify a pseudorandom number generator seed, thus allowing reproducible generation of pseudorandom numbers. > **Hint**: based on the range of possible values for ${tex`\mu`} and ${tex`\sigma`}, you may want to explicitly set the y-axis limits via the \`yMin\` and \`yMax\` [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) properties. Doing so will provide visual constancy which may better convey how adjusting ${tex`\mu`} and ${tex`\sigma`} affects simulated output. > **Hint**: for interactive HTML elements, see the documentation on [tagged template literals](https://beta.observablehq.com/@mbostock/standard-library) in Observable. `41:T5d4,md` **Challenge 3**: combining the solutions for the previous two exercises, create an interactive widget immediately following this cell for exploring the relationship between dataset length ${tex`n`} and the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) which are less than ${tex`\alpha`}. The widget should include three sliders: one for adjusting ${tex`\mu`}, another for adjusting ${tex`\sigma`}, and a third for adjusting ${tex`\alpha`}. Upon adjusting a slider, the widget should 1. generate \`M\` simulated datasets of length ${tex`n`} (where ${tex`n`} is varies over an implementation-defined interval). 2. perform a one-sample [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest) for each simulated dataset. 3. calculate the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) which less than ${tex`\alpha`}. 4. update a [scatterplot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) displaying the relationship between dataset length and the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) less than ${tex`\alpha`}. Let ${tex`n`} exist on the interval (0,1000] and \`M = 100\`. A boilerplate [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) instance has already been provided below. `42:T86e,md` To formally test whether computed [_p_-values](https://en.wikipedia.org/wiki/P-value) are drawn from a [uniform distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/uniform/pdf) on the interval [0,1], we can use the [Kolmogorov-Smirnov test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/kstest), a statistical hypothesis test which tests a sample against a specified continuous reference distribution. **Exercise 16**: using the same approach as in previous exercises, generate \`M\` simulated datasets. For each simulated dataset, perform a one-sample [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest), which tests the hypothesis that the true mean of the distribution from which we drew pseudorandom numbers is equal to zero. Store the [_p_-value](https://en.wikipedia.org/wiki/P-value) of each test in an array of length \`M\`. Perform a [Kolmogorov-Smirnov test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/kstest) to determine whether the [_p_-values](https://en.wikipedia.org/wiki/P-value) follow a [uniform distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/uniform/pdf). > **Hint**: the function to perform a [Kolmogorov-Smirnov test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/kstest) is available as [\`stdlib.kstest(\`${tex`x`}, ${tex`y`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/kstest) in this notebook. > **Hint**: the [\`stdlib.kstest(\`${tex`x`}, ${tex`y`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/kstest) documentation examples demonstrate how to test against a ${tex`U(0,1)`} distribution. `43:Td83,md` The [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest) is based on the assumption that data originates from a [normal distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/normal/pdf). If this assumption is violated, [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest) results may not be trusted under certain circumstances. **Exercise 17**: create an interactive widget immediately following this cell for generating an array whose elements are values drawn from a [gamma distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/gamma/pdf) with shape parameter ${tex`\alpha`} and rate parameter ${tex`\beta`}. The widget should include three sliders: one for adjusting the number of elements in the simulated array, another for adjusting ${tex`\alpha`}, and a third for adjusting ${tex`\beta`}. And further, the widget should update a [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) displaying the generated numbers. For initial values, use ${tex`\alpha = 1.0`} and ${tex`\beta = 2.0`}. A boilerplate [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) instance has already been provided below. > **Hint**: the function to generate [pseudorandom numbers](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/gamma) drawn from a [gamma distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/gamma) is available as [\`stdlib.base.random.gamma(\`${tex`\alpha`}, ${tex`\beta`}\`)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/gamma) in this notebook. > **Hint**: generating an array of random variates can be done in a single line using a combination of the PRNG \`factory()\` method and [inmap](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/inmap) (available as [\`stdlib.inmap(collection, fcn)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/inmap) in this notebook). > **Hint**: to better understand how adjusting parameters affects results, you may want to specify a pseudorandom number generator seed, thus allowing reproducible generation of pseudorandom numbers. > **Hint**: based on the range of possible values for ${tex`\alpha`} and ${tex`\beta`}, you may want to explicitly set the y-axis limits via the \`yMin\` and \`yMax\` [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) properties. Doing so will provide visual constancy which may better convey how adjusting ${tex`\alpha`} and ${tex`\beta`} affects simulated output. > **Hint**: for interactive HTML elements, see the documentation on [tagged template literals](https://beta.observablehq.com/@mbostock/standard-library) in Observable. `44:T49b,md` **Exercise 17**: using the same approach as in the previous exercise, generate \`M\` simulated datasets. For each simulated dataset, perform a one-sample [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest), which tests the hypothesis that the true mean of the distribution from which we drew pseudorandom numbers is equal to ${tex`\alpha \cdot \beta`} (where ${tex`\alpha`} is the shape parameter and ${tex`\beta`} is the scale parameter of a [gamma distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/gamma/pdf)). Store the [_p_-value](https://en.wikipedia.org/wiki/P-value) of each test in an array of length \`M\`. Calculate the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) which are less than a specified significance level. > **Hint**: to specify the mean to test against when performing a one-sample [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest), set the \`mu\` option. `45:T45a,md` Instead of using a [_p_-value](https://en.wikipedia.org/wiki/P-value), we can alternatively use a confidence interval for the parameter of interest. Confidence intervals are often used to convey uncertainty in model estimates and are useful even when one does not have a concrete hypothesis to test. For a ${tex`1-\alpha`} confidence interval, one has the guarantee that, for a large number of such confidence intervals, ${tex`1-\alpha`} of them will contain the true parameter. **Exercise 18**: verify the above guarantee by generating \`M\` simulated datasets of length ${tex`n`} whose values are drawn from a [normal distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/random/base/normal) with ${tex`\mu = 0`}. For each dataset, perform a one-sample [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest) and extract the confidence interval from the test results. Count the number of times in which the confidence interval contains ${tex`\mu`}. `46:T6f8,md` All too often, you encounter newspaper articles and blogs presenting the results of some newly published study which announce an association between eating some kind of food and a desired health outcome. Chances are high that these bold announcements are untenable and incapable of replication in controlled studies. By mining data sets for correlations between different variables, especially those of high dimensionality, we'll inevitably "discover" correlations. Even if variables are all independent, simply due to the sheer number of comparisons that can be made, one is going to find significant correlations by chance alone. Some time ago, the good people at FiveThirtyEight wrote an [article](http://fivethirtyeight.com/features/you-cant-trust-what-you-read-about-nutrition) on the shady practice of _p-hacking_, which refers the practice of applying data mining methods in order to obtain significant research findings. As an exploratory step to generate hypotheses, such an approach has its merits, but, more often than not, people present their findings without disclosing the process of how they were obtained or accounting for the multitude of hypothesis tests that were conducted. In this section, we'll get our own hands dirty trying to find spurious correlations in the FiveThirtyEight [data](http://fivethirtyeight.com/features/you-cant-trust-what-you-read-about-nutrition). As we'll discover, obtaining essentially meaningless results is relatively easy. To help mitigate the risk of spurious correlations, we'll discuss some remedies we can employ when faced with the task of running a large number of hypothesis tests. The hope is that our findings will serve as a reminder that we must **always** be cautious when dredging through massive amounts of data. `47:T423,md` For our analysis, we'll use two-sample [Student's _t_-tests](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) to check whether respondents with different characteristics exhibit significant differences in terms of food intake. For example, we could check whether those respondents with diabetes consume more green salad than those respondents without diabetes. **Exercise 20**: split the nutrition dataset into two groups. One group should include those respondents with diabetes and the other group should include those respondents without diabetes. > **Hint**: a function to ["bifurcate"](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/bifurcate-by) a collection according to a predicate function is available as [\`stdlib.bifurcateBy(collection, predicate)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/bifurcate-by) in this notebook. `48:T77e,md` One problem with the [Bonferroni correction](https://en.wikipedia.org/wiki/Bonferroni_correction) is that the adjusted significance level is overly conservative. Accordingly, its use often coincides with a large increase in the false negative rate. As an alternative to the [Bonferroni correction](https://en.wikipedia.org/wiki/Bonferroni_correction), we can use the the Benjamini–Hochberg procedure, which controls the [False Discovery Rate (FDR)](https://en.wikipedia.org/wiki/False_discovery_rate) at a signficiance level ${tex`\alpha`}. The Benjamini–Hochberg procedure is as follows: 1. Order the [_p_-values](https://en.wikipedia.org/wiki/P-value) in ascending order and denote each [_p_-value](https://en.wikipedia.org/wiki/P-value) as ${tex`P_{(1)} \ldots P_{(m)}`}. 2. For a given ${tex`\alpha`}, find the largest ${tex`k`} such that ${tex`P_{(k)} \leq {\frac {k}{m}}\alpha`}. 3. Reject the **null hypothesis** for the ${tex`k`} hypotheses corresponding to the ${tex`k`} lowest [_p_-values](https://en.wikipedia.org/wiki/P-value). Graphically, the procedure corresponds to the following: 1. Plot the ordered [p-values](https://en.wikipedia.org/wiki/P-value) ${tex`P_{(k)}`} against ${tex`k`}. 2. Draw a line through the origin with slope ${tex`\frac{\alpha}{m}`}. 3. Declare as significant all hypotheses that are to the left of where the points cross the line. **Exercise 25**: [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) the ordered [_p_-values](https://en.wikipedia.org/wiki/P-value) from your analysis of the nutrition data. How many significant results do you obtain when controlling the FDR for ${tex`\alpha = 0.05`}? A boilerplate [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) instance has already been provided below. `49:T5df,md` A typical method for determining whether an e-mail is spam or non-spam is to examine an e-mail's content and check whether an e-mail contains "spammy" words (i.e., words which have a higher likelihood of being used in spam e-mails than non-spam e-mails). > **Question**: what words do you typically associate with spam e-mails? However, not all words are spammy. In fact, we might expect that spam and non-spam e-mails will use many of the same words, such as the articles "a", "an", and "the", along with other commonly used (in this case) English words. Such commonly used words have low informational value when determining whether an e-mail is spam, so, as part of pre-processing, we'll want to remove those words from e-mail content. To this end, we'll load a list of English (as our e-mails are written in English) ["stopwords"](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/datasets/stopwords-en). > **Aside**: Removing stopwords is common when building a bag-of-words representation (the technique used in this notebook). This practice was more common in the past when memory considerations played a larger role, but the results of a number of other statistical models also benefit from the removal of stopwords. See [stdlib](https://github.com/stdlib-js/stdlib) for additional language [stopword lists](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/datasets). `4a:T43e,md` We can improve the \`extractBody()\` function by performing additional text processing. **Exercise 30:** amend the \`extractBody()\` function to (1) convert text to lowercase such that a word is treated the same no matter its case, (2) expand contractions (e.g., "don't" => "do not"), and (3) remove interpunctuation. > **Hint:** the function to convert text to lowercase is available as [\`stdlib.lowercase(str)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/string/lowercase) in this notebook. > **Hint**: the function to expand contractions is available as [\`stdlib.expandContractions(str)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/nlp/expand-contractions) in this notebook. > **Hint**: the function to remove interpunctuation is available as [\`stdlib.removeInterpunctuation(str)\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/string/remove-punctuation) in this notebook. `4b:T402,md` Our *Naive Bayes* model relies on a so-called [bag-of-words](https://en.wikipedia.org/wiki/Bag-of-words_model) representation of the documents in our data set. A large part of building a statistical model is the selection of variables that are useful in predicting the outcome of interest. In our case, we will use the words appearing in the texts, but observe that we may throw away useful information by doing so. For example, we might speculate that spam mails contain more capitalized or uppercased words in order to grab the attention of the recipients. > **Aside:** in natural language processing, a [bag-of-words](https://en.wikipedia.org/wiki/Bag-of-words_model) model is a simplified representation of a text document, which disregards word order and grammar and only keeps count of how often a word appears in a particular document. To generate a [bag-of-words](https://en.wikipedia.org/wiki/Bag-of-words_model) representation, we need to tokenize our texts (i.e., split them up into pieces called "tokens"). `4c:T770,md` * * * > **Mathematical Formulation of the Naive Bayes Model** > Naive Bayes is a classification algorithm for estimating the conditional probability that an observation belongs to a certain class given a set of features **x**. When used to model text documents, text document words are usually used as features, but this is not a requirement. > The model derives its name from the use of [Bayes' Rule](https://en.wikipedia.org/wiki/Bayes%27_theorem) and the fact that the model is based on the "naive" assumption that the presence of any feature does not influence the probabilities of observing any other feature (i.e., the probability of observing an observation's features given class *c*, ${tex`P(\mathbf{x}|c)`}, can be factorized as the product over all the individual feature probabilities). > In our case, we may write this as > ${tex.block`P(\mathbf{x}|c) = \prod_w P(w|c)`} > where the product is computed over all words in a respective document. [Bayes' Rule](https://en.wikipedia.org/wiki/Bayes%27_theorem) allows us to then rearrange the conditioning such that we can calculate the class probabilities given the observed features > ${tex.block`P(c|\mathbf{x}) = \frac{P(\mathbf{x}|c) P(c)}{P(\mathbf{x})}`} > Our decision rule as to whether an email should be classified as spam is straightforward. We can drop the denominator and calculate > ${tex.block`P(\mathbf{x}|c) P(c) = \prod_w P(w|c) P(c)`} > for each class, assigning the observation to the class for which the conditional probability is larger. > How do we obtain these probabilities? We estimate them by their empirical frequencies. We replace ${tex`P(c)`} by the percentage of documents from the respective class in our data corpus, and ${tex`P(w|c)`} by its relative frequency (i.e., the number of times the word appeared in documents of class *c* divided by the overall number of tokens in that class). * * * `4d:T6ed,md` * * * > **Laplace Smoothing** > Estimating word probabilities ${tex`P(w|c)`} by empirical frequencies is problematic because, for any word not appearing in a document of the respective class, we would estimate the word probability as zero. Thus, when classifying a new document containing a word not present in class' corpus, the document's classification probability would also be zero, even though the document text may otherwise look considerably similar to the documents from that class. > One approach to overcoming this problem is to simply drop words that are unknown to the training corpus. However, a better approach is _Laplace Smoothing_, in which every word, seen or unseen, is assigned a non-zero probability. Assigning a non-zero probability is accomplished by adding a value ${tex`\alpha`} to every count, such that the probability estimates change from > ${tex.block`\hat P(w|c) = \frac{\operatorname{count}(w,c)}{\sum_{w \text{ in } V} \operatorname{count}(w,c) } = \frac{\operatorname{count}(w,c)}{N_c}`} > to > ${tex.block`\hat P(w|c) = \frac{\operatorname{count}(w,c)}{\sum_{w \text{ in } V} \operatorname{count}(w,c) }= \frac{\operatorname{count}(w,c) + \alpha}{N_c+\alpha|V|}`} > where ${tex`\operatorname{count}(w,c)`} is the number of times a word *w* appears in the documents of class *c*, ${tex`N_c`} is the total number of tokens of the respective class, and ${tex`|V|`} is the overall size of the vocabulary. Often, ${tex`\alpha`} is chosen to be one. > _Laplace Smoothing_ is preferable when compared to simply discarding unknown tokens because, while unknown tokens should decrease the probability of belonging to a class, these tokens should **not** cause a new document to have zero probability of belonging that class. * * * `4e:T464,function classifyEmail( text ) { var nospamScore; var spamScore; var tokens; var body; var i; // Extract the e-mail body: body = extractBody( text ); // Convert the body to a set of tokens: tokens = stdlib.tokenize( body ); // Compute the baseline probability that an e-mail is spam: spamScore = stdlib.base.ln( groupProbs[ "spam" ] ); // Update the probability that an e-mail is spam based on the probability that an e-mail is spam given a specific token: for ( i = 0; i < tokens.length; i++ ) { spamScore += stdlib.base.ln( wordSpamProb( tokens[i] ) ); } // Compute the baseline probability that an e-mail is non-spam: nospamScore = stdlib.base.ln( groupProbs[ "nospam" ] ); // Update the probability that an e-mail is non-spam based on the probability that an e-mail is non-spam given a specific token: for ( i = 0; i < tokens.length; i++ ) { nospamScore += stdlib.base.ln( wordNospamProb( tokens[i] ) ); } // If the spam score is greater than the non-spam score, we classify the e-mail as spam: return ( spamScore > nospamScore ) ? 'spam' : 'nospam'; }4f:T47c,plt40 = { clusters; // cell dependency simulate; // cell dependency simulate_data_pt; // cell dependency incrkmeans; // cell dependency centroid_initialization; // cell dependency var colors; var opts; var plt; var X; var Y; var i; X = data[ 0 ].slice(); Y = data[ 1 ].slice(); // TODO: extract centroid locations from the cluster results object and append coordinates to the respective coordindate arrays // Define plot options: opts = { 'x': X, 'y': Y, 'title': 'k-means clustering', 'xLabel': 'x', 'yLabel': 'y', 'xMin': -10.0, 'xMax': 10.0, 'yMin': -10.0, 'yMax': 10.0, 'lineStyle': 'none', 'symbols': [ 'closed-circle' ], 'symbolsOpacity': [], 'xRug': false, 'yRug': false, 'width': 600, 'height': 600 }; for ( i = 0; i < k; i++ ) { opts.symbolsOpacity.push( 0.1 ); } opts.symbolsOpacity.push( 1.0 ); // Create a new plot: plt = stdlib.plot( opts ); // Ensure that centroids are black: colors = plt.colors; if ( colors.length >= k+1 ) { colors[ k ] = '#000000'; } plt.colors = colors; return plt; }50:T67b,function train_model() { var predictions; var optimizer; var yrange; var ynorm; var ymin; var ymax; var loss; var out; var a; var b; var c; var d; var f; var i; // We first initialize the coefficients to random values: a = tf.variable( tf.randomNormal( [], 0.0, 0.1 ) ); b = tf.variable( tf.randomNormal( [], 0.0, 0.1 ) ); c = tf.variable( tf.randomNormal( [], 0.0, 0.1 ) ); d = tf.variable( tf.randomNormal( [], 0.0, 0.1 ) ); // Next, we define a optimization function which uses stochastic gradient descent: optimizer = tf.train.sgd( learning_rate ); // We then normalize the `y` values such that they exist on the interval [0,1]: ymin = poly_data.y.min(); ymax = poly_data.y.max(); yrange = ymax.sub( ymin ); ynorm = poly_data.y.sub( ymin ).div( yrange ); // Define the polynomial: y = ax^3 + bx^2 + cx + d f = x => a.mul( x.pow( tf.scalar( 3, 'int32' ) ) ).add( b.mul( x.square() ) ).add( c.mul( x ) ).add( d ); // Define the objective function as the mean squared error (MSE) between actual and predicted values: loss = ( preds, label ) => preds.sub( label ).square().mean(); // Train the model: for ( i = 0; i < iterations; i++ ) { optimizer.minimize( () => loss( f( poly_data.x ), ynorm ) ); } // Retrieve the predicted polynomial: out = f( poly_data.x ); // "Un-normalize" the output data (as we normalized the input data): predictions = out.mul( yrange ).add( ymin ); // We need to manually dispose variables as we are continuously creating and disposing models: a.dispose(); b.dispose(); c.dispose(); d.dispose(); return predictions; }51:[] 32:{"id":"2a580806a0046aa9","slug":"workshop-notebook-machine-learning-for-hackers-fullstack","title":"Workshop Notebook: Machine Learning for Hackers (FullStack)","thumbnail":"c4cebc0119f9f46335e7b7a1729888fb15dfdeda408d83b3fd54289ef6d1a3ea","access_level":"public","create_time":"$D2018-06-26T04:00:23.873Z","update_time":"$D2019-08-13T22:26:03.086Z","view_time":null,"first_public_time":"$D2018-07-12T14:33:10.307Z","has_unpublished_edits":null,"fork":false,"stars":12,"starred":false,"owner":{"type":"team","id":"305fcf2f11e07649","login":"kgryte","name":"Athan","avatar_url":"https://avatars-next.observableusercontent.com/62abcb0f1a226f03a1138db7fec8a58afd71e37ca89254954af372e75fb90a73","tier":"starter_2024"},"version":1185,"publish_version":1185,"role":null,"authored":false,"authors":[{"type":"individual","id":"7b908ca3d7dc4524","login":"kgryte","name":"Athan","avatar_url":"https://avatars-next.observableusercontent.com/62abcb0f1a226f03a1138db7fec8a58afd71e37ca89254954af372e75fb90a73","tier":null}],"members":[{"type":"individual","id":"7b908ca3d7dc4524","login":"kgryte","name":"Athan","avatar_url":"https://avatars-next.observableusercontent.com/62abcb0f1a226f03a1138db7fec8a58afd71e37ca89254954af372e75fb90a73","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":85,"collection":null,"type":"notebook","body":{"title":"Workshop Notebook: Machine Learning for Hackers (FullStack)","cells":[{"id":0,"value":"md`# Workshop Notebook: Machine Learning for Hackers (FullStack)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":25,"value":"md`\nIn this workshop notebook, we will explore statistical inference and machine learning.\n\n> _Statistical inference_ is defined as the development of procedures, methods, and theorems that allow for extraction of information and meaning from data generated by stochastic (i.e., random) processes.\n\n> _Machine learning_ is defined as the use of statistical techniques to give computers the ability to \"learn\" (i.e., progressively improve performance for a specific task) without being explicitly programmed.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":28,"value":"md`\n### Table of Contents\n\n- Motivation\n- How to Use This Notebook\n- Modeling\n - Probability Distributions\n- Hypothesis Testing\n - p-values\n - Confidence Intervals\n- p-hacking\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":378,"value":"md`\n* * *\n\n## Motivation\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":380,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":399,"value":"md`\n* * *\n\n## Using This Notebook\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":402,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":404,"value":"md`\n**Creating and deleting cells**\n\nClick on the insert button (+) between cells in the left margin to create a new cell. You can also click Shift-Enter to render the result of a given cell. Each cell renders the result of executed JavaScript code. Clicking the caret (>) of any cell will open up the associated code block (which you can freely edit!).\n\n**Exercise**: Insert some text as a ES6 template string and render the result by pressing the \"▶︎\" button.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":409,"value":"md`\nHolding \"Shift\" and clicking the insert button allows one remove the given cell and to merge its contents into the adjacent cell (see how the + button changes its shape once \"Shift\" is pressed). You can also remove a cell by clicking the trash can icon in the right margin.\n\nYou can move cells using Alt-Cmd-Up and Alt-Cmd-Down or click on the arrows in the right margin.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":788,"value":"md`\n**Multi-line code blocks**\n\nOften, notebook cells consist of a single statement. For those times when you want to consolidate multiple statements into a single cell, you **must** wrap the statements in brackets \\`{}\\`, similar to how one might write an anonymous nullary multi-line fat-arrow function. For example,\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":792,"value":"{\n var s = 0;\n var i;\n for ( i = 0; i < 10; i++ ) {\n s += i; \n }\n return i;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":796,"value":"md`\nSimilar to single-line cells, multi-line cells can be \"named\" and, similarly, multi-line cell return values can be referenced from other cells within the notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":798,"value":"intro_sum = {\n var s = 0;\n var i;\n for ( i = 0; i < 10; i++ ) {\n s += i; \n }\n return i;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":802,"value":"intro_sum","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":811,"value":"md`\nA return value can be a literal, such as a string or number, or a reference to an array, object, function, generator, promise, et cetera.\n\n**Exercise**: create a named multi-line cell which computes a product rather than a sum.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":823,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":411,"value":"md`\n**Reactivity**\n\nYou can create named cells in order to reference their values throughout a notebook as follows:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":413,"value":"color = 'red'","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":416,"value":"html`

The text background should be red.

`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":420,"value":"md`\nA cell referencing other cells is automatically re-evaluated whenever the referenced value changes.\n\n**Exercise**: change the \\`color\\` cell and click the play button to save the the new value. The red background should change to the chosen color.\n\nObservable notebooks are designed such that, when a cell changes, only cells referencing the updated cell are re-evaluated. The order in which cells are defined does **not** matter for the execution order.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":425,"value":"md`\n**Creating interactive elements**\n\nTo create interactive elements in a notebook, we can use the \\`viewof\\` operator to expose the value of an HTML input element. For example, we can allow users to choose a text background color as follows:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":427,"value":"viewof color_input = html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":431,"value":"html`

This text should have a color background.

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":435,"value":"md `\nSome other useful inputs are\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":437,"value":"viewof range_input = html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":440,"value":"range_input","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":443,"value":"viewof text_input = html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":446,"value":"md`Hello ${text_input || 'stranger'}, how are you doing today?`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":449,"value":"md`\nIf you create an HTML input cell without using the \\`viewof\\` operator, then the return value will be a DOM element.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":452,"value":"number_input = html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":455,"value":"number_input","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":458,"value":"md`\nUsing a DOM element directly may be useful if you want to access or change an input element's properties. If an input element is created without using \\`viewof\\`, in order to access an input element's value, you must create another cell which uses built-in notebook functionality to extract the value:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":459,"value":"Generators.input( number_input )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":467,"value":"md`\n**Writing mathematical equations (LaTeX)**\n\nTo write mathematical equations, use the \\`tex\\` tagged template literal.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":468,"value":"tex`f(x) = x^2 + 4x - 2`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":475,"value":"md`\n**Exercise**: write your own mathematical equation.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":477,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":480,"value":"md`\n**Loading packages**\n\nTo use [stdlib](https://github.com/stdlib-js/stdlib) in an Observable notebook, we can load one of the distributable [files](https://github.com/stdlib-js/stdlib/tree/develop/dist) intended for use in browser environments.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":483,"value":"stdlib = require( 'https://unpkg.com/@stdlib/stdlib@0.0.43' )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":490,"value":"md` \n> **Aside:** \"unpkg\" is a content delivery network to serve files from npm. Use a require path like\n\\`unpkg.com/:package@:version/:file\\` to load the chosen file from the specified package. If no file is specified, \"unpkg\" will resolve the package according to the package's \\`package.json\\` file.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":498,"value":"md`\n**Exercise**: try loading an npm package for use in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":500,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":509,"value":"md`\n**Help**\n\nAll [stdlib](https://github.com/stdlib-js/stdlib) functionality used in this notebook has associated documentation which can be displayed using the \\`help( alias )\\` function.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":511,"value":"help( 'random.iterators.normal' )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":780,"value":"md`\n> **Aside**: when providing an alias to \\`help( alias )\\`, be sure to **exclude** the \\`stdlib.\\` prefix.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":521,"value":"md`\n**Exercise**: display the help for creating an [ndarray](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ndarray/ctor) (alias: \\`ndarray\\`).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":524,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":492,"value":"md`\n**Forking notebooks**\n\nWhen viewing an Observable notebook, click on the \"Fork\" button in the top navigation to create a copy of the notebook. Similar to the act of forking a repository on GitHub, your notebook fork will appear in \"My Notebooks\". Changes made to forked notebooks are saved, whereas changes you make to someone else's notebooks are **not** saved.\n\n**Exercise**: fork this notebook! :)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":37,"value":"md`\n* * *\n\n## Modeling\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":39,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"md`\n### Probability Distributions\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":43,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":45,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":48,"value":"plt1 = stdlib.plot({\n 'x': [ /* insert your `x` values */ ], // e.g., 'x': [ x ]\n 'y': [ /* insert your PDF values */ ], // e.g., 'y': [ pdf ]\n 'xLabel': 'x',\n 'yLabel': 'pdf',\n 'yMin': 0.0,\n 'width': 600\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":50,"value":"html`${plt1.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":53,"value":"md`\n**Exercise 2**: repeat the previous exercise for the [Cauchy Distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/cauchy/pdf) with parameters ${tex`x_0`} and ${tex`\\gamma`}.\n\n${tex.block`f(x|\\gamma,x_0)=\\frac{1}{\\pi\\gamma\\,\\left[1 + \\left(\\frac{x-x_0}{\\gamma}\\right)^2\\right]}\\!`}\n\n> **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/cauchy/pdf) for a [Cauchy distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/cauchy) is available as [\\`stdlib.base.dists.cauchy.pdf(\\`${tex`x`}, ${tex`x_0`}, ${tex`\\gamma`}\\`)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/cauchy/pdf) in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":55,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":58,"value":"md`\n**Exercise 3**: repeat the previous exercise for the [Student's _t_-distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/t/pdf) with degrees of freedom ${tex`\\nu`}.\n\n${tex.block`f(x|\\nu) = \\frac{1} {\\sqrt{\\nu}\\,B\\left( \\tfrac{1}{2}, \\tfrac{\\nu}{2} \\right )} \\left(1+\\frac{x^2}{\\nu} \\right)^{-\\frac{\\nu+1}{2}}`}\n\n> **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/t/pdf) for a [Student's _t_-distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/t) is available as [\\`stdlib.base.dists.t.pdf(x, \\` ${tex`\\nu`}\\`)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/t/pdf) in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":60,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":63,"value":"md`\n**Exercise 4**: repeat the previous exercise for the [F distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/f/pdf) with parameters ${tex`d_1`} and ${tex`d_2`}.\n\n${tex.block`f(x| d_1,d_2) = \\frac{\\sqrt{\\frac{(d_1\\,x)^{d_1}\\,\\,d_2^{d_2}} {(d_1\\,x+d_2)^{d_1+d_2}}}} {x\\,\\mathrm{B}\\!\\left(\\frac{d_1}{2},\\frac{d_2}{2}\\right)}`}\n\n> **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/f/pdf) for an [F distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/f) is available as [\\`stdlib.base.dists.f.pdf(\\`${tex`x`}, ${tex`d_1`}, ${tex`d_2`}\\`)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/f/pdf) in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":65,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":67,"value":"md`\n**Exercise 5**: repeat the previous exercise for the [${tex`\\chi^2`} distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/chisquare/pdf) with degrees of freedom ${tex`k`}.\n\n${tex.block`f(x|k) = {\\frac{1}{2^{\\frac {k}{2}}\\Gamma\\left({\\frac{k}{2}}\\right)}}\\;x^{{\\frac {k}{2}}-1}e^{-{\\frac {x}{2}}}`}\n\n> **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/chisquare/pdf) for a [${tex`\\chi^2`} distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/chisquare) is available as [\\`stdlib.base.dists.chisquare.pdf(\\`${tex`x`}, ${tex`k`}\\`)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/chisquare/pdf) in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":70,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":73,"value":"md`\n**Exercise 6**: repeat the previous exercise for the [gamma distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/gamma/pdf) with parameters ${tex`\\alpha`} and ${tex`\\beta`}.\n\n${tex.block`f(x|\\alpha,\\beta)=\\frac{\\beta^\\alpha}{\\Gamma(\\alpha)} x^{\\alpha \\,-\\, 1} e^{- \\beta x }`}\n\n> **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/gamma/pdf) for a [gamma distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/gamma) is available as [\\`stdlib.base.dists.gamma.pdf(\\`${tex`x`}, ${tex`\\alpha`}, ${tex`\\beta`}\\`)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/gamma/pdf) in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":75,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":77,"value":"$38","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":80,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":83,"value":"$39","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":85,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":89,"value":"md`\n**Exercise 9**: repeat the previous exercise for the [exponential distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/exponential/pdf) with parameter ${tex`\\lambda`}.\n\n${tex.block`f(x|\\lambda) = \\begin{cases} \\lambda e^{-\\lambda x} & x \\ge 0 \\\\ 0 & x < 0 \\end{cases}`}\n\n> **Hint**: the function to evaluate the [probability density function](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/exponential/pdf) for an [exponential distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/exponential) is available as [\\`stdlib.base.dists.exponential.pdf(\\`${tex`x`}, ${tex`\\lambda`}\\`)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/exponential/pdf) in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":92,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":95,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":97,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":100,"value":"md`\n* * *\n\n## Hypothesis Testing\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":102,"value":"$3b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":104,"value":"dataset_page1 = [\n 1, 0, 0, 0, 0, 0, 1, 0, 0, 1,\n 0, 0, 0, 1, 0, 0, 0, 1, 0, 0,\n 0, 1, 1, 0, 0, 0, 0, 0, 0, 0\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":107,"value":"dataset_page2 = [\n 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 0, 0, 0, 0, 0, 0, 1, 0, 0, 0,\n 0, 0, 0, 0, 0, 0, 0, 1, 0, 0\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":111,"value":"md`\nA cursory look at the data suggests that the first page is more effective at converting users. This observation may, in fact, be correct; however, we'd undoubtedly prefer a means for statistical verification and certainly a means which does not rely on us having to manually inspect relative proportions. To this end, let's see if we can statistically confirm our belief that the **higher proportion** of users who signed up on the first page compared to the second page is **not** due to chance alone.\n\n**Exercise 11**: calculate the proportion of users who signed up per page.\n\n> **Hint**: for a binary random variable (i.e., a random variable which can assume only one of two values; e.g., \\`0\\` and \\`1\\`), calculating the **mean** gives you the proportion of \"\\`1\\`\" observations. This is the **only** instance in which calculating a numerical [summary statistic](https://en.wikipedia.org/wiki/Summary_statistics) for [categorical data](https://en.wikipedia.org/wiki/Categorical_variable) is appropriate.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":114,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":117,"value":"md`\n> **Question**: are the proportions the same or different? If the proportions are different, how much larger is one proportion than the other (e.g., \"100x larger\")? And if different, do you feel more or less confident, based on the relative difference, that the first page is \"superior\" to the second with regard to user conversion? Why? (feel free to create a cell below containing your thoughts)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":119,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":121,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":124,"value":"md`\n> **Question**: based on the test results, do we conclude that the first page or the second page is more effective at converting visitors?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":126,"value":"md`\nWhen all test assumptions are satisfied, the significance level of a test, commonly denoted by the Greek letter ${tex`\\alpha`}, informs you as to the probability that you reject the **null hypothesis** even though the **null hypothesis** is true. Hypothesis testing gives us only this guarantee and does **not** provide any guarantees when we fail to reject a false **null hypothesis**.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":128,"value":"md`\n> **Question**: observe that the test decision has an inherent uncertainty; i.e., a small probability exists that you might have reached a false conclusion. How large is this probability?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":130,"value":"$3d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":132,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":136,"value":"md`\n> **Question**: when A/B testing websites, which is preferred: a two-sided or a one-sided statistical hypothesis test? Why? (feel free to create a cell below containing your thoughts)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":139,"value":"$3e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":141,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":144,"value":"pltc1 = stdlib.plot({\n 'x': [ /* insert your `x` values */ ], // e.g., 'x': [ stdlib.linspace(0,1,n1), stdlib.linspace(0,1,n2) ]\n 'y': [ /* insert your pseudorandom numbers */ ], // e.g., 'y': [ sorted_dataset1, sorted_dataset2 ]\n 'xLabel': 'proportion',\n 'yLabel': 'converted',\n 'yMin': -0.1,\n 'yMax': 1.1,\n 'width': 600\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":146,"value":"html`${pltc1.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":149,"value":"$3f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":151,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":155,"value":"pltc2 = stdlib.plot({\n 'x': [ /* insert your `|p1-p2|` values */ ], // e.g., 'x': [ delta_p ]\n 'y': [ /* insert your lengths */ ], // e.g., 'y': [ n ]\n 'xLabel': 'Δp',\n 'yLabel': 'n',\n 'xMin': 0.0,\n 'xMax': 1.0,\n 'yMin': 0,\n 'width': 600,\n 'lineStyle': [ 'none' ],\n 'symbols': [ 'open-circle' ]\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":158,"value":"html`${pltc2.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":163,"value":"md`\n> **Question**: what conclusions can you draw regarding the relationship between ${tex`n`} and ${tex`\\Delta p`}?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":187,"value":"md`\n### Simulation Study\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":189,"value":"$40","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":192,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":195,"value":"plt14 = stdlib.plot({\n 'x': [ /* insert your `x` values */ ], // e.g., 'x': [ stdlib.linspace(-10,10,n) ]\n 'y': [ /* insert your pseudorandom numbers */ ], // e.g., 'y': [ rvs ]\n 'xLabel': 'x',\n 'yLabel': 'y',\n 'width': 600\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":199,"value":"html`${plt14.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":202,"value":"md`\n**Exercise 15**: using the same approach as in the previous exercise, generate \\`M\\` simulated datasets. For each simulated dataset, perform a one-sample [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest), which tests the hypothesis that the true mean of the distribution from which we drew pseudorandom numbers is equal to zero. Store the [_p_-value](https://en.wikipedia.org/wiki/P-value) of each test in an array of length \\`M\\`. Calculate the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) which are less than ${tex`\\alpha`}. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":204,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":207,"value":"md`\n> **Question**: is the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) reasonably close to ${tex`\\alpha`}, as we would expect?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":209,"value":"$41","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":211,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":215,"value":"pltc3 = stdlib.plot({\n 'x': [ /* insert your `n` values */ ], // e.g., 'x': [ stdlib.linspace(1,n,n) ]\n 'y': [ /* insert your proportions */ ], // e.g., 'y': [ y ]\n 'xLabel': 'n',\n 'yLabel': 'proportion',\n 'width': 600,\n 'yMin': 0.0,\n 'yMax': 1.1,\n 'lineStyle': [ 'none' ],\n 'symbols': [ 'open-circle' ]\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":219,"value":"html`${pltc3.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":221,"value":"md`\n> **Question**: what conclusions can you draw regarding the relationship between ${tex`n`} and the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) which are less than ${tex`\\alpha`}?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":223,"value":"$42","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":225,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":228,"value":"md`\n> **Question**: do the [_p_-values](https://en.wikipedia.org/wiki/P-value) follow a [uniform distribution](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/base/dists/uniform/pdf)? How does varying \\`M\\` affect your conclusions?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":230,"value":"$43","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":232,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":235,"value":"plt16 = stdlib.plot({\n 'x': [ /* insert your `x` values */ ], // e.g., 'x': [ stdlib.linspace(-10,10,n) ]\n 'y': [ /* insert your pseudorandom numbers */ ], // e.g., 'y': [ rvs ]\n 'xLabel': 'x',\n 'yLabel': 'y',\n 'width': 600\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":239,"value":"html`${plt16.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":242,"value":"$44","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":244,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":248,"value":"md`\n> **Question**: is the proportion of [_p_-values](https://en.wikipedia.org/wiki/P-value) reasonably close to the specified significance level, as we would expect? Why or why not?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":251,"value":"md`\n> **Question**: what happens as you vary dataset length?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":253,"value":"md`\n### Confidence Intervals\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":255,"value":"$45","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":257,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":261,"value":"md`\n> **Question**: are you able to verify that the confidence intervals contain the value ${tex`\\mu`} around ${tex`1-\\alpha`} percent of the time?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":266,"value":"md`\n> **Question**: does varying dataset length affect your conclusions? Why or why not?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":268,"value":"md`\n> **Aside**: while in reality we are not going to repeat a specific test multiple times, we are likely to conduct many different tests over a long period of time. If we conduct them all at a significance level ${tex`\\alpha`}, we can be certain that ${tex`1-\\alpha`} of them contain the true value (provided all test assumptions are valid).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":270,"value":"md`\n* * *\n\n## p-hacking\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":274,"value":"$46","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":276,"value":"md`\n**Exercise 19**: as a first step, take some time to familiarize yourself with the FiveThirtyEight dataset, which is available in this notebook as \\`nutrition_data\\`. The data originates from a [survey](https://en.wikipedia.org/wiki/Food_frequency_questionnaire) (referred to as the \"[food frequency questionnaire](https://en.wikipedia.org/wiki/Food_frequency_questionnaire)\") completed by respondents regarding their eating habits and personal characteristics.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":278,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":282,"value":"md`\n> **Question**: how many respondents are there? Is this data quantitative, qualitative, or a mixture of both?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":285,"value":"md`\n> **Question**: based on this dataset, what sorts of hypotheses might you want to test? (feel free to create a list in a new cell below)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":287,"value":"$47","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":289,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":292,"value":"md`\n**Exercise 21**: for each group, extract the frequency of green salad consumption (key: \\`GREENSALADFREQ\\`).\n\n> **Hint**: be sure to convert the frequencies to integer values.\n\n> **Hint**: extracting frequencies can be done with two one-liners via [inmap](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/inmap) (available as [\\`stdlib.inmap(collection, fcn)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/inmap) in this notebook). Just be sure to slice (i.e., copy) the raw (bifurcated) data **before** invocation!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":294,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":297,"value":"md`\n**Exercise 22**: perform a two-sample two-sided [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2) on the two frequency collections.\n\n> **Hint**: for pretty-printed output, use the \\`print()\\` method, which will print the exact definition of the **alternative hypothesis**.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":299,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":302,"value":"md`\n> **Question**: is green salad consumption significantly different between the two groups?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":304,"value":"md`\n> **Aside**: a few points to remember when _p_-hacking:\n\n> - Even if no real association exists, when using a significance level of 5% (as is standard), you expect to detect an average of five significant results per 100 tests, thus corresponding to a false positive rate of 5%.\n> - Even if you were to detect a true effect, the question still persists as to whether that effect is of any practical significance.\n> - As we explored above, as sample sizes become larger, hypothesis tests become better at detecting even very small differences across groups.\n> - Finally, we should keep in mind that correlation does **not** imply causation, as a confounding factor may exist that affects both variables and is thus responsible for the observed difference!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":306,"value":"md`\n**Exercise 23**: as we did above for green salad consumption frequency, iterate over all 1066 frequency and quantity responses for those respondents with and without diabetes. For each set of responses, perform a two-sample [Student's _t_-test](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/stats/ttest2). Record whether the test result is significant. Upon analysis completion, print a list of statistically significant results (e.g., \"those with diabetes are less likely to eat green salad\").\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":308,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":311,"value":"md`\n> **Question**: did you find any interesting statistically significant differences between the two groups? If so, which differences are most surprising?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":313,"value":"md`\n**Challenge 4**: repeat the previous exercise for the remaining personal characteristics (e.g., whether someone is left-handed, has a particular belly button, owns a dog, etc).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":315,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":318,"value":"md`\n> **Question**: did you find any interesting statistically significant differences between groups? If so, which differences are most surprising?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":320,"value":"md`\n**Challenge 5**: repeat the previous exercise for all personal characteristic combinations (e.g., has diabetes vs owns a dog).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":322,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":325,"value":"md`\n> **Question**: did you find any interesting statistically significant differences between groups? If so, which differences are most surprising?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":327,"value":"md`\n> **Question**: can you think of any reasons why the previous analysis might be flawed? If so, what are those flaws? (feel free to write your thoughts in a new cell below)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":329,"value":"md`\n### Bonferroni Correction\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":331,"value":"md`\nOne solution for the problem of multiple comparisons is the [Bonferroni correction](https://en.wikipedia.org/wiki/Bonferroni_correction). The procedure is straightforward: instead of testing each individual hypothesis at a significance level of ${tex`\\alpha`}, test each at ${tex`\\tfrac{\\alpha}{m}`}, where ${tex`m`} is the number of hypotheses. One can show that the family-wise error rate (FWER; i.e., the probability of rejecting **at least** one true **null hypothesis**) is less than ${tex`\\alpha`}.\n\n**Exercise 24**: had you used a [Bonferroni correction](https://en.wikipedia.org/wiki/Bonferroni_correction) in previous exercises, at what significance level should you have performed your tests?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":333,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":336,"value":"md`\n### Controlling the False Discovery Rate (FDR)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":338,"value":"$48","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":340,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":343,"value":"plt25 = stdlib.plot({\n 'x': [ /* insert your `x` values */ ], // e.g., 'x': [ stdlib.linspace(1,1066,1066), stdlib.linspace(1,1066,1066) ]\n 'y': [ /* insert your y values and p-values */ ], // e.g., 'y': [ line_with_slope_alpha_over_m, pvals ]\n 'xLabel': 'x',\n 'yLabel': 'P_k',\n 'width': 600\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":345,"value":"html`${plt25.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":534,"value":"md`\n* * *\n\n## Spam Classifier\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":536,"value":"md`\nSpam e-mails arrive in our mailboxes each day. Determining whether an e-mail spam is what we call a classification task, since the task consists of assigning an observation into one of two (or more) categories (spam/no spam). We will explore building our own spam classifier using a simple, but well-performing, statistical technique called Naive Bayes.\n\nFor this purpose, we will make use of the \\`SPAM_ASSASSIN\\` data set that comes with [stdlib](https://github.com/stdlib-js/stdlib).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":538,"value":"spam = datasets( 'SPAM_ASSASSIN' )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":553,"value":"md`\n**Exercise 26**: take a few minutes to expore the data set (you might also want to read its [documentation](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/datasets/spam-assassin).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":556,"value":"md`\nTo evaluate the performance of a statistical model, common practice is to set aside a test set of observations for which we know the true class, but which we hold out and do not use while building our model. This ensures that we get an unbiased estimate of how well the model will generalize once we evaluate the model on our test set. The e-mails from the \\`easy-ham-1\\` and \\`spam-2\\` batches will serve as our training set from which we will build a model that can predict whether a new e-mail should be classified as spam. The e-mails from \\`spam-1\\`, \\`hard-ham-1\\`, and \\`easy-ham-2\\` will constitute our test set.\n\n**Exercise 27**: create an array containing e-mails belonging to our training set.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":561,"value":"training = {\n // Your code here.\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":564,"value":"md`\n**Exercise 28**: create an array containing e-mails belonging to our test set.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":568,"value":"test = {\n // Your code here.\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":578,"value":"$49","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":589,"value":"STOPWORDS = datasets( 'STOPWORDS_EN' )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":573,"value":"md`\nWhile examining the e-mails, you may have noticed that an e-mail always includes meta-information before the e-mail's content. To remove this meta-information, stopwords, and other unwanted character sequences, we'll define the following function:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":575,"value":"function extractBody( email ) {\n // Remove the meta-information before two initial line breaks:\n var text = email.match( /[\\r\\n]{2}([\\s\\S]*)$/ )[ 1 ];\n \n // Remove numbers and other special characters:\n text = stdlib.replace( text, /[0-9\\-\\+]/g, '' );\n \n // Remove common words such as \"the\" or \"and\":\n text = stdlib.removeWords( text, STOPWORDS );\n \n return text;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":600,"value":"md`\n**Exercise 29:** to convince yourself as to the efficacy of \\`extractBody()\\`, try providing the function spam texts and inspect the results.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":604,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":607,"value":"$4a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":610,"value":"md`\n**Challenge 6:** build a widget which provides an input field and displays the processed text for a selected e-mail. For added difficulty, allow users to specify which text processing steps \\`extractBody()\\` can perform and study the effects of different text processing techniques.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":612,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":615,"value":"$4b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":620,"value":"md`\n**Exercise 31:** tokenize the training set e-mails using the [\\`stdlib.tokenize()\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/nlp/tokenize) function available in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":629,"value":"function getTokens( email ) {\n // 1) Extract the e-mail body and attach it as `body` property to `email`:\n \n \n // 2) Tokenize the e-mail body and save the tokens inside of the `tokens` property of `email`:\n \n \n return email;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":633,"value":"tokenized_training = stdlib.inmap( [ /* replace with training set array */ ], getTokens )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":639,"value":"$4c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":649,"value":"md`\nBefore building our model, let's first explore the frequency differences for the tokens in the spam and non-spam groups and find which tokens help us discriminate between the two classes.\n\n**Exercise 32**: divide the training data into two groups, \\`spam\\` and \\`nospam\\`.\n\n> **Hint**: to perform a \"group-by\" operation, use the [\\`groupBy()\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/group-by) function available in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":662,"value":"function groupSpam( email ) {\n // Your code here. \n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":654,"value":"grouped = stdlib.groupBy( tokenized_training, groupSpam )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":657,"value":"md`\n**Exercise 33**: consolidate group tokens such that we have a list of **all** tokens that appear in the spam and non-spam texts, respectively.\n\n> **Hint:** to \"pluck\" data from an object array, use the [\\`stdlib.pluck(arr, prop)\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/pluck) function.\n\n> **Hint:** to convert an array-of-arrays to a single \"flat\" array, use the [\\`stdlib.flattenArray()\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/flatten-array) available in this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":666,"value":"tokens = {\n return {\n 'spam': [ /* your code here */ ],\n 'nospam': [ /* your code here */ ]\n };\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":690,"value":"md`\n**Exercise 34:** generate an object which contains the token frequencies for both the spam and non-spam texts.\n\n> **Hint:** to count the number of times an element occurs in an array, use the [\\`stdlib.countBy()\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/count-by) function available in this notebook.\n\n> **Hint:** supply the [\\`stdlib.identity()\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/identity) function as the second argument to [\\`stdlib.countBy()\\`](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/utils/count-by) to group the array elements by their values.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":696,"value":"freqs = {\n return {\n 'spam': { /* your code here */ },\n 'nospam': { /* your code here */ }\n };\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":701,"value":"md`\nWe can now explore the absolute frequencies of the tokens in the two groups.\n\n**Exercise 35:** think of a few words that you would expect to show up much more frequently in spam e-mails, and check whether the frequencies support your hypothesis.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":703,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":706,"value":"md`\nNote, however, that the absolute frequencies may be misleading since the groups are not equally represented in our training set. \\`spam-2\\` contains \\`1396\\` e-mails, while \\`easy-ham-1\\` contains \\`2500\\` e-mails. Accordingly, we are more interested in relative frequencies, thus accounting for the different group sizes. For later use in this notebook, we'll store the relative group frequencies as follows:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":710,"value":"groupProbs = {\n var total = 1396 + 2500; // total e-mails\n return {\n 'spam': 1396 / total,\n 'nospam': 2500 / total\n };\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":714,"value":"md`\nTo convert token counts to relative frequencies, we must divide the token counts by the total number of tokens in the respective group. Let's define the following two functions for this purpose.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":716,"value":"function wordSpamProb( word ) {\n var nk = freqs.spam[ word ] || 0;\n return ( nk ) / ( tokens.nospam.length );\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":719,"value":"function wordNospamProb( word ) {\n var nk = freqs.nospam[ word ] || 0;\n return ( nk ) / ( tokens.nospam.length );\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":722,"value":"md`\n**Exercise 36**: invoke \\`wordSpamProb\\` and \\`wordNospamProb\\` for the words you examined in the previous exercise. What would happen if we were to call either of the functions for a word that did not occur in the respective group? Feel free to write down your thoughts in a separate cell below.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":726,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":729,"value":"md`\nFor our classifier to work for documents which contain words unknown to our corpus, we will have to change how we estimate word probabilities by making use of a technique called _Laplace Smoothing_. In its simplest form, Laplace Smoothing consists of adding a count of one to all words, including for those words which do not appear in documents of a given class.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":733,"value":"$4d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":742,"value":"md`\n**Exercise 37:** the number of unique tokens in our training data set is \\`76,709\\` (saved as variable *vocabSize*), modify \\`wordSpamProb()\\` and \\`wordNoSpamProb()\\` to use Laplace Smoothing.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":744,"value":"vocabSize = 76709","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":747,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":750,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":753,"value":"md`\nOur e-mail classification function can now be implemented in a relatively straightforward manner. Instead of directly evaluating ${tex`\\prod_w P(w|c)`}, which is prone to underflow since many of the probabilities will be small, we'll take the natural logarithm and calculate the sum ${tex`\\sum_w \\ln P(w|c)`} instead. Taking the logarithm is a monotonic transformation, which does not change the rank order of the untransformed values. Therefore, predictions made from the model are **not** affected by this transformation.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":755,"value":"$4e","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":758,"value":"md`\n**Exercise 38**: test classification on a few of the emails from the \\`test\\` set. Do we make the correct predictions?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":760,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":763,"value":"md`\n**Challenge 7:** assess the performance of our model by calcuating the percentage of emails that are correctly classified in both the training and test sets. What do you observe?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":764,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":768,"value":"md`\n**Challenge 8:** can you think of other features besides words which could help discriminate between spam and non-spam e-mails (e.g., the number of capitalized words or \\`$\\` signs). Test whether your assumption is correct by checking for a difference in the training set data.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":772,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":775,"value":"md`\nThe process of identifying and creating new features that are predictive of an outcome is often called _feature engineering_. Note that the Naive Bayes classifier is not restricted to words, but we could include these other features, as well!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":836,"value":"md`\n* * *\n\n## k-means Clustering\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":838,"value":"md`\n_k_-means clustering is an unsupervised machine learning technique for partitioning data into \\`k\\` clusters. In the next set of exercises, we will explore [incremental _k_-means clustering](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ml/incr/kmeans), which is useful in streaming/on-line processing contexts, as the technique allows us to update cluster results as data becomes available.\n\nFor the purposes of the following exercises, we're going to simulate 2-dimensional data containing 5 clusters. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":857,"value":"k = 5","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":886,"value":"ndims = 2","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":860,"value":"md`\nWe'll define the true cluster properties, consisting of a cluster's center position (mean) and associated spread (standard deviation).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":843,"value":"cluster_properties = new Float64Array([\n 0.0, 1.0, 0.0, 1.0, // meanX, stdevX, meanY, stdevY\n -5.0, 1.0, 5.0, 1.0,\n 5.0, 1.0, 5.0, 1.0,\n 5.0, 1.0, -5.0, 1.0,\n -5.0, 1.0, -5.0, 1.0\n ])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":846,"value":"md`\nWe'll expose the true cluster properties via an [ndarray](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ndarray/ctor).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":854,"value":"clusters = {\n var ctor = stdlib.ndarray( 'float64', ndims, { 'codegen': false } );\n var shape = [ k, ndims*2 ];\n var strides = [ ndims*2, 1 ];\n var offset = 0;\n var order = 'row-major';\n var mat = ctor( cluster_properties, shape, strides, offset, order );\n return mat;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":915,"value":"md`\nTo help us plot cluster results, we'll store simulated data point coordinates for each cluster in coordinate arrays.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":917,"value":"function init_coord_array( k ) {\n var out;\n var i;\n \n out = [];\n for ( i = 0; i < k; i++ ) {\n out.push( [] );\n }\n return out;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":929,"value":"function reset_coord_array( arr ) {\n var i;\n for ( i = 0; i < arr.length; i++ ) {\n arr[ i ].length = 0; \n }\n return arr;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":920,"value":"data = {\n var out;\n var i;\n out = [];\n for ( i = 0; i < ndims; i++ ) {\n out.push( init_coord_array( k ) );\n }\n return out;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":864,"value":"md`\nNext, in order to simulate cluster data, we'll create pseudorandom number generators (PRNGs) for generating pseudorandom numbers drawn from 2-d uncorrelated normal distributions.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":866,"value":"simulate = {\n var ctor = stdlib.ndarray( 'generic', ndims, { 'codegen': false } );\n var randn = ctor( new Array( k*ndims ), [ k, ndims ], [ ndims, 1 ], 0, 'row-major' );\n for ( var i = 0; i < k; i++ ) {\n for ( var j = 0; j < ndims; j++ ) {\n randn.set( i, j, stdlib.base.random.normal.factory( clusters.get( i, j*2 ), clusters.get( i, (j*2)+1 ) ) );\n }\n }\n return randn;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":949,"value":"md`\nTo simulate a single data point, we'll define a function which returns a random data point vector.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":951,"value":"simulate_data_pt = {\n var vec;\n var v;\n \n // Define a reusable data vector to avoid unnecessary GC thrashing:\n vec = stdlib.ndarray( 'float64', 1, { 'codegen': false } );\n v = vec( new Float64Array( ndims ), [ ndims ], [ 1 ], 0, 'row-major' );\n \n return simulator;\n \n function simulator() {\n var c;\n var r;\n var i;\n \n // Pick a cluster from which to sample:\n c = stdlib.base.random.discreteUniform( 0, k-1 );\n \n // Generate a random cluster data point:\n for ( i = 0; i < ndims; i++ ) {\n r = simulate.get( c, i )();\n v.set( i, r );\n \n // Append the simulated data point to the relevant coordinate array:\n data[ i ][ c ].push( r );\n }\n // Return the simulated data point:\n return v;\n }\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":880,"value":"md`\nNow that we have logic for simulating 2-dimensional Gaussians, we can initialize an [incremental _k_-means clustering accumulator](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ml/incr/kmeans). We'll first specify the number of data points we want to use for centroid initialization (i.e., the process of calculating the initial set of centroid positions, where a centroid is defined as a cluster center). In general, the more data points used for initialization, the better _k_-means will perform; however, the more data points used for initialization, the greater the computational and memory demands. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":897,"value":"init_npts = 100","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":901,"value":"md`\nNext, we'll create the [accumulator](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ml/incr/kmeans).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":882,"value":"incrkmeans = {\n clusters; // cell dependency\n simulate; // cell dependency\n simulate_data_pt; // cell dependency\n \n var opts = {\n 'metric': 'euclidean',\n 'init': [ 'kmeans++', init_npts ]\n };\n return stdlib.incrkmeans( k, ndims, opts );\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":905,"value":"md`\nLet's \"warm-up\" the accumulator by simulating just enough data points to allow for centroid initialization.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":907,"value":"centroid_initialization = {\n var v;\n var i;\n \n // Reset the data coordinate arrays...\n for ( i = 0; i < ndims; i++ ) {\n reset_coord_array( data[ i ] );\n }\n // Simulate data points and update the accumulator...\n for ( i = 0; i <= init_npts; i++ ) {\n v = simulate_data_pt();\n incrkmeans( v );\n }\n // Return initial cluster results:\n return incrkmeans();\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":912,"value":"md`\n**Exercise 39**: inspect initial cluster results. Do they make sense?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1005,"value":"''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1003,"value":"md`\n**Exercise 40**: generate a [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) which displays initial centroid locations relative to simulated data. To help get you started with [plotting](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor), a boilerplate plot cell is provided.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":994,"value":"$4f","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":997,"value":"html`${plt40.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1013,"value":"md`\n**Exercise 41**: update the accumulator (and the plot) by simulating additional data points. How much do the centroids move over time? Write your observations in a cell below.\n\n> **Hint**: you may want to simulate additional data points one-by-one, perhaps by using an interactive button which you can click.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1033,"value":"md`\n(your thoughts here)\n`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1017,"value":"md`\n**Exercise 42**: reduce the number of data points used for centroid initialization and rerun the simulation. How well does clustering perform? Write your observations in a cell below.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1036,"value":"md`\n(your thoughts here)\n`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1022,"value":"md`\n**Exercise 43**: instead of [\\`kmeans++\\`](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ml/incr/kmeans), try using [sampling](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ml/incr/kmeans) for centroid initialization. How does sampling compare to [\\`kmeans++\\`](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ml/incr/kmeans) initialization? How does changing the number of data points used for centroid initialization affect clustering performance when using [sampling](https://github.com/stdlib-js/stdlib/tree/1ca03f761500b5a6da2b0aa0e9f150b4e4fdfaae/lib/node_modules/%40stdlib/ml/incr/kmeans)? Write your observations in a cell below.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1039,"value":"md`\n(your thoughts here)\n`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1025,"value":"md`\n**Exercise 44**: adjust cluster properties and investigate clustering performance. If you adjust the means such that the clusters are closer together or farther apart and hold the cluster standard deviations constant, does clustering improve or worsen? If you adjust the standard deviations such that the clusters are tighter/more disperse and hold the cluster means constant, does clustering improve or worsen? Write your observations in a cell below.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1042,"value":"md`\n(your thoughts here)\n`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1045,"value":"md`\n**Challenge 9**: instead of simulating 2-dimensional Gaussians, try simulating data points drawn from a different continuous distribution and repeat the exercises above. Write your observations in a cell below.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1047,"value":"md`\n(your thoughts here)\n`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1053,"value":"md`\n* * *\n\n## Neural Networks\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1055,"value":"md`\nArtificial neural networks are computing systems loosely based on biological neural networks. Their rise in the pantheon of machine learning algorithms is driven by their relatively straightforward implementation and suitability for parallelization (computing a neural network consists largely of matrix multiplications which can readily implemented and run on GPUs).\n\nTo familiarize ourselves with neural networks, we will use **tensorflow.js**, a JavaScript-based library for training and applying neural networks.\n\nThe fundamental data structure exposed by tensorflow.js is an immutable \\`tensor\\`.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1056,"value":"{ \n // Create a 2x2 matrix:\n var matrix = tf.tensor2d([\n [ 1.0, 2.0 ],\n [ 3.0, 4.0 ]\n ]);\n \n // Resolve tensor values:\n var data = await matrix.data();\n \n // Similar to the `ndarray` data type exposed by stdlib, TF tensors have associated meta data:\n return {\n 'data': data,\n 'shape': matrix.shape,\n 'dtype': matrix.dtype,\n 'rank': matrix.rank\n };\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1066,"value":"md`\nNeural networks excel in curve fitting, particularly in high-dimensional data containing nonlinearities. As a first foray into training a neural network, we'll try and fit the following polynomial\n\n${tex.block`y = ax^3 + bx^2 + cx + d + \\mathcal{N}(0, \\sigma^{2})`}\n\nwhere ${tex`\\mathcal{N}(0, \\sigma^{2})`} is a noise term and with coefficients\n\n- \\`a = ${a}\\`\n- \\`b = ${b}\\`\n- \\`c = ${c}\\`\n- \\`d = ${d}\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1074,"value":"viewof a = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1084,"value":"viewof b = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1089,"value":"viewof c = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1099,"value":"viewof d = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1116,"value":"md`\nand noise level ${tex`\\sigma = ${noise_level}`}.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1119,"value":"viewof noise_level = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1109,"value":"md`\nWe'll begin by defining a function for simulating polynomial data\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1112,"value":"function polysim() {\n var npts;\n var sa;\n var sb;\n var sc;\n var sd;\n var x;\n var y;\n \n npts = 100;\n \n // Declare TF scalars:\n sa = tf.scalar( a );\n sb = tf.scalar( b );\n sc = tf.scalar( c );\n sd = tf.scalar( d );\n \n // Generate the `x` data:\n x = tf.randomUniform( [ npts ], -1.0, 1.0 );\n\n // Evaluate the polynomial at the `x` values:\n y = sa.mul( x.pow( tf.scalar( 3, 'int32' ) ) )\n .add( sb.mul( x.square() ) )\n .add( sc.mul( x ) )\n .add( sd )\n .add( tf.randomNormal( [ npts ], 0, noise_level ) );\n\n return [ x, y ]; \n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1128,"value":"md`\nWe can now use our simulation function to simulate polynomial data.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1107,"value":"poly_data = {\n // We need to use `tidy()` in order to clean-up unused GPU memory:\n var data = tf.tidy( polysim );\n \n yield {\n 'x': data[ 0 ],\n 'y': data[ 1 ]\n };\n \n // Whenever this cell is re-evaluated, we need to dispose the previous dataset:\n try {\n yield invalidation;\n } finally {\n data[ 0 ].dispose();\n data[ 1 ].dispose();\n }\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1130,"value":"md`\n**Exercise 45**: [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) the simulated polynomial data. To help get you started with [plotting](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor), a boilerplate plot function call is provided.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1134,"value":"plt45 = stdlib.plot({\n 'x': [ /* your code here */ ], // e.g., 'x': [ x ]\n 'y': [ /* your code here */ ], // e.g., 'y': [ y ]\n 'xLabel': 'x',\n 'yLabel': 'y',\n 'yMin': -5.0,\n 'yMax': 5.0,\n 'xMin': -1.0,\n 'xMax': 1.0,\n 'width': 600,\n 'symbols': [ 'closed-circle', 'none' ],\n 'symbolsOpacity': 0.3,\n 'lineStyle': [ 'none', '-' ]\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1136,"value":"html`${plt45.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1142,"value":"md`\nTo train the model and learn the polynomial coefficients, we'll first define the number of training iterations\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1149,"value":"viewof iterations = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1151,"value":"iterations","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1155,"value":"md`\nand a learning rate\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1157,"value":"viewof learning_rate = html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1160,"value":"learning_rate","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1165,"value":"md`\nWe are now ready to define our training function.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1144,"value":"$50","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1167,"value":"md`\nWe are now ready to train our model!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1169,"value":"model_output = {\n var predictions;\n var py;\n var x;\n var y;\n \n predictions = tf.tidy( train_model );\n \n x = await poly_data.x.data();\n y = await poly_data.y.data();\n py = await predictions.data();\n \n return [ x, y, py ];\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1176,"value":"md`\n**Exercise 46**: [plot](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor) the simulated polynomial data along with the polynomial fit. How well did we do? To help get you started with [plotting](https://github.com/stdlib-js/stdlib/tree/a5fc046e1a13d5938d45b8cb70b45fe14fbbe5ed/lib/node_modules/%40stdlib/plot/ctor), a boilerplate plot function call is provided.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1178,"value":"plt46 = stdlib.plot({\n 'x': [ /* your code here */ ], // e.g., 'x': [ x ]\n 'y': [ /* your code here */ ], // e.g., 'y': [ y ]\n 'xLabel': 'x',\n 'yLabel': 'y',\n 'yMin': -5.0,\n 'yMax': 5.0,\n 'xMin': -1.0,\n 'xMax': 1.0,\n 'width': 600,\n 'symbols': [ 'closed-circle', 'none' ],\n 'symbolsOpacity': 0.3,\n 'lineStyle': [ 'none', '-' ]\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1181,"value":"html`${plt46.render('html')}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1183,"value":"md`\n**Exercise 47**: update the coefficients, noise level, and model training parameters. How do your changes affect the quality of fit?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":21,"value":"md`\n* * *\n\n## Conclusions\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":347,"value":"md`\nIn this notebook, you achieved the following:\n\n- Learned how to use Observable notebooks.\n- Developed a sense as to why JavaScript is a suitable environment for scientific computing.\n- Performed simulations, helping motivate why simulation is important.\n- Created plots both for visualizing results and for verifying data shape and simulation behavior.\n- Learned about statistical significance and quantifying uncertainty.\n- Gained an intuition as to best practice and as to why one should always regard correlations with suspicion.\n- Implemented a machine learning classification algorithm for discriminating between spam and non-spam e-mails.\n- Used _k_-means clustering to partition data into clusters.\n- Trained a neural network to learn polynomial coefficients.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":14,"value":"md`\n* * *\n\n## Notes\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":16,"value":"md`\n- This notebook uses [stdlib](https://github.com/stdlib-js/stdlib), a standard library for JavaScript and Node.js, with an emphasis on numerical and scientific computing applications. [stdlib](https://github.com/stdlib-js/stdlib) contains over 2,000 functions for mathematics, statistics, data processing, streams, and more.\n- If you are interested in learning more about [stdlib](https://github.com/stdlib-js/stdlib), be sure to check out the GitHub [repository](https://github.com/stdlib-js/stdlib) and to follow updates on [Twitter](https://twitter.com/stdlibjs). :)\n- This notebook is part of a workshop on machine learing in JavaScript at [FullStack London 2018](https://skillsmatter.com/conferences/9815-fullstack-2018-the-conference-on-javascript-node-and-internet-of-things), held in London, UK, 11-13 July 2018.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":8,"value":"md`\n* * *\n\n## Appendix\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":10,"value":"md`\nAn appendix containing functionality used in running this notebook.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":543,"value":"stdlib_version = '0.0.43'","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":486,"value":"/* stdlib = require( 'https://unpkg.com/@stdlib/stdlib@${stdlib_version}' ) */","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":363,"value":"stdlib_flat_help = require( \"https://unpkg.com/@stdlib/stdlib@0.0.43/dist/stdlib-flat-help.min.js\" )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":360,"value":"HELP = stdlib_flat_help.help()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":370,"value":"function help( alias ) {\n var txt;\n if ( HELP.hasOwnProperty( alias ) ) {\n txt = HELP[ alias ];\n } else {\n txt = 'Error: unrecognized alias. Ensure that the provided alias is spelled correctly, including capitalization.';\n }\n return md`\\`\\`\\`${txt}\\`\\`\\``;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":367,"value":"help( 'incrkmeans' )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":541,"value":"datasets = {\n var pkg = await require( \"https://unpkg.com/@stdlib/stdlib@\"+stdlib_version+\"/dist/stdlib-datasets-tree.min.js\" );\n return pkg.datasets;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":349,"value":"nutrition_data = {\n var d3 = await require('https://d3js.org/d3.v5.min.js');\n return d3.csv('https://raw.githubusercontent.com/fivethirtyeight/data/master/nutrition-studies/raw_anonymized_data.csv');\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1060,"value":"tf = require( '@tensorflow/tfjs@0.12.1' )","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"md`\n* * *\n\n## License\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"md`\nCopyright (c) 2018 The Stdlib Authors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W51","stdlib":"1","titling":"auto"}} 33:"$32:body"