25:null 26:["$","title",null,{"children":"CSE 412 Section 10: Machine Learning and Visualization in Javascript | cse412 | Observable"}] 2b:T4ec,md` ## Machine Learning Overview Machine learning is often classified along two axes: **supervised** versus **unsupervised**, and **discrete** versus **continuous** outputs. *Supervised* machine learning is when we train a model to predict an output variable from input variables, while *unsupervised* machine learning is when we don't have any specific prediction goals, and are just trying to discover patterns in the data. Four of the main categories of machine learning are as follows; - **Classification** - supervised, discrete - **Regression** - supervised, continuous - **Clustering** - unsupervised, discrete - **Dimensionality reduction** - unsupervised, continuous Machine learning has also been divided into "traditional" or "classical" methods on one hand, and "deep learning" on the other. We will start with traditional machine learning, and then maybe take a brief look at deep learning. ## Loading required libraries Javascript is not traditionally a language used for machine learning, but there is increasing interest in running machine learning models in the browser (privacy is one reason, among many others). The **[MLJS](https://github.com/mljs/ml)** library is one of the more common libraries for "traditional" machine learning. `2c:T4a6,// plot confusion matrices { const knn_table = makeCMTable(data_split[3], knn_results); const nb_table = makeCMTable(data_split[3], nb_results.map(x => ids_to_classes[x])); const dt_table = makeCMTable(data_split[3], dt_results.map(x => ids_to_classes[x])); const knn_plot = vl.mark('rect') .data(knn_table) .encode( vl.x().fieldN('pred_label'), vl.y().fieldN('true_label'), vl.color().fieldQ('frac'), vl.tooltip([vl.fieldN('pred_label'), vl.fieldN('true_label'), vl.fieldQ('frac')]), ) .title('KNN confusion matrix'); const nb_plot = vl.mark('rect') .data(nb_table) .encode( vl.x().fieldN('pred_label'), vl.y().fieldN('true_label'), vl.color().fieldQ('frac'), vl.tooltip([vl.fieldN('pred_label'), vl.fieldN('true_label'), vl.fieldQ('frac')]), ) .title('NB confusion matrix'); const dt_plot = vl.mark('rect') .data(dt_table) .encode( vl.x().fieldN('pred_label'), vl.y().fieldN('true_label'), vl.color().fieldQ('frac'), vl.tooltip([vl.fieldN('pred_label'), vl.fieldN('true_label'), vl.fieldQ('frac')]), ) .title('DT confusion matrix'); return vl.hconcat(knn_plot, nb_plot, dt_plot).render(); }2d:T517,// These are sentence examples from a wikipedia comments dataset that was used to train similar models (not this model). // see: https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/ lines = ["Stop undoing my edits or die!", // a "toxic" comment "UNBLOCK ME OR I'LL GET MY LAWYERS ON TO YOU FOR BLOCKING MY CONSTITUTIONAL RIGHT TO FREE SPEECH", // another "toxic" comment "Hi, good day.", // a nontoxic comment "Thanks for entering this article.", // another nontoxic comment "idiot did u ever read anything?", // marked as toxic, but try changing the 'o' in 'idiot' to a '0'. "Considering the vile and perverse anti-Egyptian racism spewed on this talk page on a regular basis, which is all over many Arab countries, especially the Gulf, it is relevant to examine it on an article about Arabs.", // this is marked as a "toxic" comment in the training set. "Who the hell r u and why do u care neway it was just a bit of fun. Get a life", "I hate you.", // it's marked as toxic, but are there contexts where this is not toxic? "I'm working abroad in Australia, and was surprised to find out the green sSkittles over here seem to be Apple flavour.", "F F F F F F F" // this is a pathological case. ]2e:[] 29:{"id":"657b9fc7831bf270","slug":"cse-412-section-10-machine-learning-and-visualization-in-ja","title":"CSE 412 Section 10: Machine Learning and Visualization in Javascript","thumbnail":"bf547f07c043976ea1afde7596d79bbe48066a8a0a74cf7270adccaa8c172357","access_level":"public","create_time":"$D2021-06-03T02:46:20.524Z","update_time":"$D2021-06-03T15:03:22.893Z","view_time":null,"first_public_time":"$D2021-06-03T02:46:26.916Z","fork":{"id":"bfa190c4a75a6aed","slug":"cse-412-section-10-machine-learning-and-visualization-in-ja","title":"CSE 412 Section 10: Machine Learning and Visualization in Javascript","owner":{"id":"0d266bbc8b38cf0a","login":"yjzhang","name":"Yue Zhang"}},"stars":1,"starred":false,"owner":{"type":"team","id":"16c5e985f8389c16","login":"cse412","name":"","avatar_url":"https://avatars-next.observableusercontent.com/e212536a9dc2df98bc8c2baec0ece23b782e9d064e97e67ede01a5ca9a933fa9","tier":"starter_2024"},"version":1022,"publish_version":1022,"role":null,"authored":false,"authors":[{"type":"individual","id":"9461267a8818334e","login":"cse412","name":"","avatar_url":"https://avatars-next.observableusercontent.com/e212536a9dc2df98bc8c2baec0ece23b782e9d064e97e67ede01a5ca9a933fa9","tier":null}],"members":[{"type":"individual","id":"9461267a8818334e","login":"cse412","name":"","avatar_url":"https://avatars-next.observableusercontent.com/e212536a9dc2df98bc8c2baec0ece23b782e9d064e97e67ede01a5ca9a933fa9","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":3,"collection":{"id":"dfcea94f61fa37af","slug":"uw-cse-412-quiz-sections","title":"UW CSE 412 Quiz Sections","owner":{"id":"16c5e985f8389c16","login":"cse412","name":""}},"type":"notebook","body":{"title":"CSE 412 Section 10: Machine Learning and Visualization in Javascript","cells":[{"id":127,"value":"md`# CSE 412 Section 10: Machine Learning and Visualization in Javascript`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":114,"value":"md`\nIn this section, we will be discussing machine learning in javascript. First, we will be introducing basic machine learning principles using the ml.js library, and running it on simple datasets. Next, we will introduce deep learning in the browser using tensorflow.js, and try out some pre-built models for computer vision and natural language processing.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":153,"value":"$2b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"ML = require('ml@5.0.0/dist/ml.js')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":106,"value":"import {vl} from '@vega/vega-lite-api'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":180,"value":"// code from https://github.com/mljs/ml/blob/master/examples/leafDataset/knn.js\nfunction shuffleArray(array) {\n for (var i = array.length - 1; i > 0; i--) {\n var j = Math.floor(Math.random() * (i + 1));\n var temp = array[i];\n array[i] = array[j];\n array[j] = temp;\n }\n return array;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":249,"value":"function calculateAccuracy(predicted, expected) {\n var accurate_count = 0;\n for (var i = 0; i < predicted.length; i++) {\n if (predicted[i] == expected[i]) {\n accurate_count += 1;\n }\n }\n return accurate_count/predicted.length;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":120,"value":"md`## Loading data\n\nWe will start by working with the much-loved cars dataset, as was used in a number of the earlier sections.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":169,"value":"cars = (await require('vega-datasets@1'))['cars.json']()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1022,"value":"keys = ['Cylinders', 'Displacement', 'Horsepower', 'Weight_in_lbs', 'Acceleration', 'Miles_per_Gallon', 'Name', 'Origin']","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1021,"value":"md`### Null Values\n\nIf you take a look into the data, you'll see that a number of the fields are null. In Javascript math, null values are treated as 0, which might not be what we want, and could lower the accuracy of downstream tasks.\n\nBelow, we count the number of null values in each column:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1020,"value":"null_col_count = {\n const keys = ['Cylinders', 'Displacement', 'Horsepower', 'Weight_in_lbs', 'Acceleration', 'Miles_per_Gallon'];\n var null_counts = {};\n for (var k of keys) {\n null_counts[k] = 0;\n for (var row of cars) {\n if (row[k] == null) {\n null_counts[k] += 1;\n }\n }\n }\n return null_counts;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1019,"value":"md`There are 6 rows that have a null Horsepower value and 8 rows that have a null Miles_per_gallon. We can create a new data table that does not have any null values:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1018,"value":"cars_filtered = cars.filter(row => {\n for (var k of keys) {\n if (row[k] == null) {\n return false;\n }\n }\n return true;\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":0,"value":"md`## Classification\n\nClassification is when we are trying to predict a discrete variable, given some data. Here, we will be training a model to predict the country of origin, using the miles per gallon, cylinders, displacement, etc.\n\nFirst, we will be processing the data to only include the columns that we are concerned about: the input, which are the numerical variables, and the output, which is Origin.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":135,"value":"cars_data = { \n var cars_data = cars_filtered.map((row) =>\n [[row.Cylinders, row.Displacement, row.Horsepower, row.Weight_in_lbs, row.Acceleration, row.Miles_per_Gallon],\n row.Origin, row.Name]);\n return cars_data;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":197,"value":"md`As usual in supervised machine learning problems, we will be splitting our dataset into a **training dataset**, used to train the classifier, and a **test dataset**, a subset of the data not used for training that is used to evaluate the model.\n\nAlso, we will be splitting the data into input variables (data about the car model) and output variables (the country of origin). The usual convention in machine learning is to call the array of input variables X, and the output variables Y.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":202,"value":"data_split = {\n var cars_shuffled = shuffleArray(cars_data);\n const train_count = 0.8*cars_shuffled.length; // we will be using 80% of the data for training\n var X_train = cars_shuffled.slice(0, train_count).map(x => x[0]);\n var X_test = cars_shuffled.slice(train_count).map(x => x[0]);\n var Y_train = cars_shuffled.slice(0, train_count).map(x => x[1]);\n var Y_test = cars_shuffled.slice(train_count).map(x => x[1]);\n var names_train = cars_shuffled.slice(0, train_count).map(x => x[2]);\n var names_test = cars_shuffled.slice(train_count).map(x => x[2]); \n return [X_train, X_test, Y_train, Y_test, names_train, names_test];\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":214,"value":"md`\nSupervised machine learning consists of **training** and **prediction** stages. Training refers to when we fit the model to the training dataset, and prediction is when we query the model with new, unseen data.\n\nWe will first try using a **[K-nearest neighbors](https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm)**, or **kNN** classifier model. In the training stage, this model simply stores all the training data. In the prediction stage, this classifier finds the k closest points to the test points within the training set, and outputs the class that is most common among the k closest points.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":216,"value":"knn = new ML.KNN(data_split[0], data_split[2], {k:5});","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":220,"value":"md`Now, we can try testing the trained model on the test dataset.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":226,"value":"knn_results = knn.predict(data_split[1])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":231,"value":"knn_accuracy = calculateAccuracy(knn_results, data_split[3])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":241,"value":"md`This will change based on the random train-test split, but the knn accuracy should be between 50 and 70%.\n\nHowever, raw accuracy might not be the best metric, since there might be an imbalance in the number of points belonging to each class (i.e., there might be very few European cars, which is the case for our dataset). The predicted values for our kNN classifier almost never return Europe. Is that good or bad?\n\nIn order to investigate how accuracy varies for different classes, we can plot a **[confusion matrix](https://en.wikipedia.org/wiki/Confusion_matrix)**, which is a heatmap that compares true labels to predicted labels. This is a very useful visualization for evaluating classification models. Click on the cells below to see the code (it's a bit complicated).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":846,"value":"// Function to generate a confusion matrix table (this is necessary for vega-lite).\nfunction makeCMTable(true_labs, pred_labs) {\n var cm = ML.ConfusionMatrix.fromLabels(true_labs, pred_labs);\n var table = [];\n var labels = new Set(true_labs);\n var i, j = 0;\n var row_sums = cm.matrix.map(x => x.reduce((a, b) => a+b, 0));\n for (var c1 of labels) {\n for (var c2 of labels) {\n i = cm.labels.indexOf(c1);\n j = cm.labels.indexOf(c2);\n table.push({'true_label': c1, 'pred_label': c2,\n 'count': cm.matrix[i][j], 'frac': cm.matrix[i][j]/row_sums[i]});\n }\n }\n return table;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":794,"value":"// As you might have realized, data processing is one of the hardest problems in machine learning.\n// Also, it might be easier to create graphs such as this in plotly or even matplotlib or seaborn,\n// because those tools are better for visualizing matrices rather than data tables.\n{\n var table = makeCMTable(data_split[3], knn_results);\n return vl.mark('rect')\n .data(table)\n .encode(\n vl.x().fieldN('pred_label'),\n vl.y().fieldN('true_label'),\n vl.color().fieldQ('frac'),\n vl.tooltip([vl.fieldN('pred_label'), vl.fieldN('true_label'), vl.fieldQ('frac')]),\n )\n .render();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":830,"value":"md`Each box shows the fraction of cars belonging to that true label that were classified as the predicted label. Your results might vary due to the random train/test split, but from where I'm looking, while most USA-made cars were classified correctly, most cars made in Japan and Europe were classified incorrectly! Can we do better than this?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":788,"value":"md`\n\nThere are many other classification models that we can potentially use. For example, we can try the **[Naive Bayes](https://en.wikipedia.org/wiki/Naive_Bayes_classifier)** and **[Decision Tree](https://en.wikipedia.org/wiki/Decision_tree_learning)** classifiers:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":260,"value":"// A Gaussian Naive Bayes assumes that all variables are Gaussian-distributed for each class.\nnb = new ML.NaiveBayes.GaussianNB()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":506,"value":"// we have to convert the classes to integers first before we use the Naive Bayes or decision tree classifiers.\nclasses_to_ids = {\n return {'USA': 0, 'Japan': 1, 'Europe': 2};\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":857,"value":"ids_to_classes = {\n return {0: 'USA', 1: 'Japan', 2: 'Europe'};\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":494,"value":"nb_results = {\n // exercise for the reader?\n nb.train(data_split[0], data_split[2].map(x => classes_to_ids[x]));\n return nb.predict(data_split[1]);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":519,"value":"nb_accuracy = calculateAccuracy(data_split[3].map(x => classes_to_ids[x]), nb_results)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":495,"value":"// There are a lot of potential options for the decision tree classifier - see https://github.com/mljs/decision-tree-cart\ndt = new ML.DecisionTreeClassifier()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":498,"value":"dt_results = {\n // exercise for reader?\n dt.train(data_split[0], data_split[2].map(x => classes_to_ids[x]));\n return dt.predict(data_split[1]);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":512,"value":"dt_accuracy = calculateAccuracy(data_split[3].map(x => classes_to_ids[x]), dt_results)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":608,"value":"md`This might change randomly based on the train-test split, but I'm seeing the decision tree accuracy as higher than both the KNN and Naive Bayes models. Does this mean that the decision tree classifier is the best classifier so far for this data?\n\nLet's plot confusion matrices for all three classifiers (click the cell to see the code):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":843,"value":"$2c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":863,"value":"md`Your results will vary due to the random train-test split. Based on what I see here, the kNN classifier has very high accuracy for USA and low (25%) accuracies for Japan and Europe, the NB classifier has about 50% accuracy for all three classes, and the DT classifier has high accuracies for USA and Europe and a lower accuracy for Japan. But this might vary significantly. Can one of them be said to be the best classifier?\n\nOften in classification, the \"best\" classifier depends on the goals of your application. We might value certain types of errors more than other errors.\n\nThere are other metrics beyond raw accuracy such as [ROC](https://en.wikipedia.org/wiki/Receiver_operating_characteristic), [precision-recall](https://en.wikipedia.org/wiki/Precision_and_recall), F1 score, and others, that try to consider different kinds of errors. But this is a brief overview, so we won't cover them for now.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":171,"value":"md`## Regression\n\nRegression is when we are trying to predict a continuous variable. In the cars example, we will be trying to predict the Miles_per_Gallon given its Cylinders, Horsepower, Displacement, and Weight_in_lbs.\n\nAs with classification, we will be processing the cars dataset to only include the relevant input and output variables, and then splitting the data into train and test sets.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":268,"value":"regression_data_split = { \n var cars_data = cars_filtered.map((row) =>\n [[row.Cylinders, row.Displacement, row.Horsepower, row.Weight_in_lbs], [row.Miles_per_Gallon], row.Name]);\n // note: for MultivariateLinearRegression the output has to be an array\n var cars_shuffled = shuffleArray(cars_data);\n const train_count = 0.8*cars_shuffled.length; // we will be using 80% of the data for training\n var X_train = cars_shuffled.slice(0, train_count).map(x => x[0]);\n var X_test = cars_shuffled.slice(train_count).map(x => x[0]);\n var Y_train = cars_shuffled.slice(0, train_count).map(x => x[1]);\n var Y_test = cars_shuffled.slice(train_count).map(x => x[1]);\n var names_train = cars_shuffled.slice(0, train_count).map(x => x[2]);\n var names_test = cars_shuffled.slice(train_count).map(x => x[2]);\n return [X_train, X_test, Y_train, Y_test, names_train, names_test];\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":272,"value":"md`Our model will be a multivariate **linear regression**. Basically, this tries to find a linear equation for the output variable in terms of the input variables. This is done by minimizing the squared error between the predicted and actual outputs. To learn more about the math, see the wikipedia page on [ordinary least squares](https://en.wikipedia.org/wiki/Ordinary_least_squares).`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":281,"value":"lr = new ML.MultivariateLinearRegression(regression_data_split[0], regression_data_split[2])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":284,"value":"lr_results = lr.predict(regression_data_split[1])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":293,"value":"md`In order to evaluate the accuracy of the multivariate regression, we can plot the predictions for the test set against the true values on a graph:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":303,"value":"{\n // this code creates a new data table with the predicted and actual values.\n // making this plot would probably be easier in plotly or matplotlib.\n var temp_data = [];\n for (var i = 0; i < lr_results.length; i++) {\n temp_data.push({'name': regression_data_split[5][i],\n 'mpg_actual': regression_data_split[3][i][0],\n 'mpg_predicted': lr_results[i][0]});\n }\n return vl.markPoint()\n .data(temp_data)\n .encode(\n vl.x().fieldQ('mpg_actual'),\n vl.y().fieldQ('mpg_predicted'),\n vl.tooltip().fieldN('name')\n )\n .render()\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":330,"value":"md`From the graph, the predicted values look like they are pretty similar to the actual values. However, if we want to quantitatively evaluate the accuracy of our regression, we need a metric. The most commonly used metric for regression is [r^2 (r-squared)](https://en.wikipedia.org/wiki/Coefficient_of_determination), which is calculated below. The value of r^2 is usually between 0 and 1, with 1 being a perfect fit and 0 equivalent to always predicting the mean.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":441,"value":"SS = require('simple-statistics')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":314,"value":"// the r^2 function in simple-statistics doesn't actually work for our case so we'll write our own.\nr_squared = {\n var data_mean = SS.mean(regression_data_split[3].map(x=>x[0]));\n var tss = 0;\n var res = 0;\n for (var i = 0; i < lr_results.length; i++) {\n tss += Math.pow(regression_data_split[3][i][0] - data_mean, 2);\n res += Math.pow(regression_data_split[3][i][0] - lr_results[i][0], 2);\n }\n return 1 - res/tss;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":447,"value":"md`Interpretations of the r^2 value will vary depending on the application, but usually a value of 0.6 is considered 'pretty good' for a heterogeneous dataset such as this.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":477,"value":"md`## Clustering\n\nClustering is kind of like the unsupervised corrolary to classification. Instead of trying to predict a label, we try to partition the data into a number of subsets based on the characteristics of the data. Here, we'll be clustering the cars data based on Cylinders, Horsepower, Displacement, Weight_in_lbs, Acceleration, and Miles_per_Gallon.\n\nAgain, we start by processing the data to keep the relevant columns, but there won't be a train-test split this time.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":539,"value":"clustering_data = { \n var cars_data = cars_filtered.map((row) =>\n [row.Cylinders, row.Displacement, row.Horsepower, row.Weight_in_lbs, row.Acceleration, row.Miles_per_Gallon]);\n // note: for MultivariateLinearRegression the output has to be an array\n return cars_data;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":541,"value":"md`We will be using the **[k-means](https://en.wikipedia.org/wiki/K-means_clustering)** algorithm, which is probably the most common and basic algorithm used for clustering. It works by: 1) randomly selecting k points as cluster centers, 2) assigning every point to the cluster with the closest center, 3) finding new cluster centers by taking the averages of the points in each cluster, and repeating steps 2-3 until convergence.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":545,"value":"km_results = {\n // we will be using 3 clusters for now.\n var km = ML.KMeans(clustering_data, 3);\n return km;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":556,"value":"md`We'll visualize and evaluate the clustering in the next section.\n\n## Dimensionality Reduction\n\nDimensionality reduction is kind of like the unsupervised version of regression. As the name suggests, we are trying to reduce a complicated dataset to a smaller set of dimensions, while still preserving the relationships between the points. It is often used for visualization of high-dimensional datasets. In this example, we want to reduce the cars dataset, which has 6 numerical dimensions, down to 2 dimensions.\n\nWe can use the same data matrix we created in the clustering section.\n\nOur first dimensionality reduction method will be **PCA**, or **principal components analysis**. Basically, the goal of PCA is to find new axes in the data that explain most of the variation in the dataset. The first axis will always explain the most variation, and then the second, and so on.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":564,"value":"pca = new ML.PCA(clustering_data)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":566,"value":"pca_results = pca.predict(clustering_data)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":577,"value":"md`Even though we are calling 'predict', note that the \"test\" dataset is the exact same as the \"training\" dataset.\n\nNext, we'll extract the first two columns of PCA, and plot them along with the clusters seen previously.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":568,"value":"pca_data = {\n var d1 = pca_results.getColumn(0);\n var d2 = pca_results.getColumn(1);\n var temp_data = [];\n for (var i = 0; i < cars_filtered.length; i++) {\n temp_data.push({'name': cars_filtered[i].Name,\n 'd1': d1[i],\n 'd2': d2[i],\n 'cluster': km_results.clusters[i]});\n }\n return temp_data;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":575,"value":"vl.markPoint()\n .data(pca_data)\n .encode(\n vl.x().fieldQ('d1'),\n vl.y().fieldQ('d2'),\n vl.color().fieldN('cluster'),\n vl.tooltip().fieldN('name'),\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":584,"value":"md`It might not be immediately obvious how to interpret this. Basically, PCA provides an easy way of representing a multi-dimensional dataset in a 2-dimensional plane. Both the x-axis and y-axis here represent linear combinations of the variables in the dataset, so we see the effects of all six of the variables here in only two dimensions.\n\nThe clustering seems to correlate pretty well with the PCA plot, as the different clusters separate out neatly. In general, evaluating an unsupervised clustering is a difficult problem because a \"good\" clustering is hard to define.\n\nWe can try to plot the other variables on the PCA plot, to better see the trends:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":596,"value":"// There's probably a better way to do this with arquero or something like that\njoint_data = {\n var temp_data = [];\n for (var i = 0; i < cars_filtered.length; i++) {\n var c = cars_filtered[i];\n c.d1 = pca_data[i].d1;\n c.d2 = pca_data[i].d2;\n temp_data.push(c);\n }\n return temp_data;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":885,"value":"md`Select a variable below to visualize on the graph:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":587,"value":"viewof field_input = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":591,"value":"{\nconst p1 = vl.markPoint()\n .data(joint_data)\n .encode(\n vl.x().fieldQ('d1'),\n vl.y().fieldQ('d2'),\n vl.color().fieldQ(field_input),\n vl.tooltip([vl.fieldN('Name'), vl.fieldQ(field_input)]),\n );\n \n const p2 = vl.markPoint()\n .data(joint_data)\n .encode(\n vl.x().fieldQ('d1'),\n vl.y().fieldQ('d2'),\n vl.color().fieldN('Origin'),\n vl.tooltip([vl.fieldN('Name'), vl.fieldN('Origin')]),\n );\n return vl.hconcat(p1, p2).render();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":605,"value":"md`The cars on the left of the graph are generally smaller Japanese and European cars with lower horsepower, fewer cylinders, and lower weight, while cars on the right of the graph are all American cars, which seem to be bigger cars with higher horsepower, more cylinders, higher weight, and so on. It is not immediately obvious what the y-axis represents.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":628,"value":"pca.getExplainedVariance()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":633,"value":"md`If we look at the explained variance, we see that 99.7% of the variance in the dataset is explained by the first PC dimension, or the x-axis in the plot.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":927,"value":"md`With PCA, we can project new data onto the same PCA axes. We can use this to visualize the results of our earlier country classifiers:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":928,"value":"classification_pca_data = {\n // 1. project test data data onto PCA axes\n var data_vars = data_split[1].map(x => x.slice(0, 6));\n var data_pca = pca.predict(data_vars);\n // 2. create new data table with pca results\n var d1 = data_pca.getColumn(0);\n var d2 = data_pca.getColumn(1);\n var temp_data = [];\n for (var i = 0; i < knn_results.length; i++) {\n temp_data.push({'name': data_split[5][i],\n 'd1': d1[i],\n 'd2': d2[i],\n 'true_lab': data_split[3][i],\n 'knn_lab': knn_results[i],\n 'nb_lab': ids_to_classes[nb_results[i]],\n 'dt_lab': ids_to_classes[dt_results[i]]});\n }\n return temp_data;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":962,"value":"md`Use the dropdown below to select the classifier or labels.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":945,"value":"viewof label_input = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":953,"value":"vl.markPoint()\n .data(classification_pca_data)\n .encode(\n vl.x().fieldQ('d1'),\n vl.y().fieldQ('d2'),\n vl.color().fieldN(label_input),\n vl.tooltip([vl.fieldN('name'), vl.fieldN(label_input)]),\n )\n .render()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":968,"value":"md`The shape of the decision boundaries between different classes should tell you something about how the classifiers work.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":99,"value":"md`# Deep Learning and TensorflowJS`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":139,"value":"md`Deep learning is a subset of machine learning defined by its use of **neural networks**, a computing abstraction based on a notion of composable mathematical operations, or \"neurons\". As with \"classical\" machine learning, deep learning can be used for classification, regression, dimensionality reduction, and clustering. These days, deep learning has the best performance on complex, human-interpretable data types such as images, sounds, or text.\n\nWe don't have much time to do a deep dive, so we'll just be showing a few interesting pre-trained models. If you're interested in learning more, there are tons of deep learning tutorials available online, such as [fast.ai](https://www.fast.ai/) and [deeplearning.ai](https://www.deeplearning.ai/programs/).\n\n\n[TensorflowJS](https://www.tensorflow.org/js) is a deep learning library for javascript, a port of the popular tensorflow library. It is capable of running the entire machine learning pipeline, from model construction to training and testing, in the browser.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":478,"value":"md`## Image classification\n\nImage classification was one of the first applications that drove deep learning research. Long story short, it turned out that [convolutional neural networks](https://en.wikipedia.org/wiki/Convolutional_neural_network) are really good at learning image representations. Much of the research on image classifcation has used the [ImageNet](https://www.image-net.org/) dataset, which provides the category labels used here.\n\nWe don't have time here to talk about the details of these models; feel free to read about them in the links provided if you're interested.\n\nHere, we will be using the [mobilenet](https://ai.googleblog.com/2017/06/mobilenets-open-source-models-for.html) model, an image classification model designed for low-resource computing environments such as phones.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":146,"value":"mobilenet = (await require('@tensorflow-models/mobilenet@0.2')).load()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":482,"value":"md`\nFeel free to upload any image, preferably a small image of a single object. Then, run 'mobilenet.classify(image)', and check out the results by clicking the arrow to the left of \"Array\".\n\nSome example images from the imagenet dataset can be found at https://github.com/EliSchwartz/imagenet-sample-images and https://github.com/ajschumacher/imagen/tree/master/imagen. For images that are hard to classify, see some of the examples from [this paper](https://arxiv.org/pdf/1907.07174.pdf).\n\nOne of the advantages of machine learning in javascript is that this runs entirely within the browser, with in your computer. No data is uploaded to a server, which is a plus for privacy.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":406,"value":"viewof image_input = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":430,"value":"image_data = Files.url(image_input)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":410,"value":"image = {\n var image = new Image();\n image.src = image_data;\n return image\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":924,"value":"md`Note: this cell might not work immediately after uploading an image. You might have to run this cell again.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":396,"value":"mobilenet.classify(image)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":658,"value":"md`Do these seem like good classifications to you?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":672,"value":"md`## Sentence classification\n\nAnother common application of deep learning is in natural language processing. In this simple example, we will be using a pre-trained model to classify sentences by their \"toxicity\", trying to identify toxic comments and such. A description of the model and training data is available at https://github.com/tensorflow/tfjs-models/tree/master/toxicity.\n\nNote: There are a significant number of potential biases and issues with applying a model like this to real-world moderation scenarios, including [vulnerability to simple changes in text](https://arxiv.org/pdf/1808.09115v2.pdf), [not taking into account context or intent of comments](https://www.technologyreview.com/2017/02/24/153573/its-easy-to-slip-toxic-language-past-alphabets-toxic-comment-detector/), and [racial bias](https://homes.cs.washington.edu/~msap/pdfs/sap2019risk.pdf). Can an AI judge whether someone is \"reclaiming\" a slur or using it in a derogatory way? Often times, even humans don't know how to judge that!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":40,"value":"toxicity = require('@tensorflow-models/toxicity')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":51,"value":"classifier = await toxicity.load(0.9)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":67,"value":"$2d","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":79,"value":"results = await classifier.classify(lines)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":987,"value":"md`We can plot the toxicity of the above example sentences:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":990,"value":"{\n const toxicity_results = [];\n for (var i = 0; i < lines.length; i++) {\n toxicity_results.push({'line': lines[i], 'toxicity': results[6].results[i].probabilities[1]});\n }\n // add custom sentence\n toxicity_results.push({'line': sentence_input, 'toxicity': sentence_toxicity_chance});\n return vl.mark('bar')\n .data(toxicity_results)\n .encode(\n vl.x().fieldQ('toxicity'),\n vl.y().fieldN('line'),\n vl.tooltip([vl.fieldN('line'), vl.fieldQ('toxicity')])\n ).render();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":760,"value":"md`Try to copy in a sentence in the box below, and see what the toxicity classifier returns! You can try using the above examples or try your own. The toxicity of this sentence will be plotted in the chart above.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":749,"value":"viewof sentence_input = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":763,"value":"html`

Probability of toxic sentence: ${sentence_toxicity_chance}

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":752,"value":"sentence_results = await classifier.classify([sentence_input])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":754,"value":"sentence_toxicity_chance = sentence_results[6].results[0].probabilities[1]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":875,"value":"md`\n# Additional Resources\n\nThe machine learning ecosystem in javascript is still relatively immature compared with languages such as python. [scikit-learn](https://scikit-learn.org/stable/tutorial/basic/tutorial.html) is the premier \"classical\" machine learning library in python. For deep learning, check out [tensorflow](https://www.tensorflow.org/) and [pytorch](https://pytorch.org/).\n\nFor more machine learning visualization examples in Python, see the colab notebook from the previous quarter of this class: https://colab.research.google.com/drive/1RGDFGCh4nPZkg3fgLCAA5s2bi00DfsuQ\n\nFor a much more detailed tutorial on tensorflowjs, see https://observablehq.com/@zaidalyafeai/an-intractive-introduction-to-tensorflow-js\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":651,"value":"md`# Acknowledgements\n\nThis notebook was written by [Yue Zhang](https://github.com/yjzhang) for CSE 412, Spring 2021 at the University of Washington.\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":437,"value":"md`# Scratch area`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W2e","stdlib":"1","titling":"auto"}} 2a:"$29:body"