25:["$","title",null,{"children":"Getting started with TensorFlow.js (TF.js) | Jake Peters | Observable"}] 49:T904,## Motivation for client-side deployment with TF.js The main motivation for a Python programmer, like me, to learn TF.js is to deploy the machine learning models I develop in TensorFlow into production in the user's browser, i.e. at the client-side. This production design has several benefits including— 1. Zero server costs: since the model is deployed on the client-side, there is no cost of running the model in a server. 2. Zero installation: the browser sets up the environment required to run the machine learning model. The user does not have to install anything to be able to run the model. The developer does not need to maintain a Docker image and interact with multiple languages. 3. Zero latency: once the model is loaded on a client's browser, the predictions are local and therefore there is no latency from sending messages to a remote server. In fact, the predictions can be made offline. 4. User privacy: since the user queries are not sent to a remote sever, it preserves user privacy. This can be particularly important in sensitive applications like in medicine and finance. 5. Scale: since the model travels to the user, it instantly scales to the demand. There is no need for server load balancing. 6. GDPR compliance: since the code travels and not the data, such a deployment automatically conforms to the [GDPR regulations](https://gdpr-info.eu/). 7. FAIR: such a deployment adheres to the [FAIR (findable, accessible, interoperable, and reusable) principles](https://arxiv.org/abs/2206.06159). The model is *findable* with a unique, persistent URL, where it is hosted. It is *accessible* via OAuth protocols or under some open source licensing. It is *interoperable* because any system that supports a browser can run the TF.js model. These factors make the model highly *reusable*. 8. Federated learning: such a design enables the development of [federated learning](https://ai.googleblog.com/2017/04/federated-learning-collaborative.html) systems, where the code travels and not the data. 9. Prevalence: [JavaScript is by and far the most popular programming language](https://octoverse.github.com/#top-languages-over-the-years). Therefore, such a deployment can trigger rapid adoption and development of ideas. 4a:T987,## Caveats of client-side deployment For someone moving from server-side programming in TensorFlow to client-side programming with TF.js, there are a few caveats to remember. 1. Model size: while the browser can handle models of sizes up to 500 MB, the user experience greatly suffers from large model sizes. [TF.js recommends model sizes of up to 30 MB](https://github.com/tensorflow/tfjs/blob/master/tfjs-converter/README.md#faq). [A quick look at a list of model sizes of popular neural network architectures in Keras](https://keras.io/api/applications/#available-models) shows us that many popular models are not suited for client-side deployment. 2. Performance: the client-side isn't suited for compute-heavy tasks. Server-side builds of TensorFlow, i.e. the C++ TensorFlow build accessible via the Python/Node.js APIs, are better optimized for heavy compute. So, client-side TF.js is often preferable for inference (prediction) and not for model development. Model development often involves extensive and convoluted big data preprocessing pipelines and GPU- or TPU-intensive training. While TF.js supports both [WebGL](https://www.npmjs.com/package/@tensorflow/tfjs-backend-webgl) and the newer [WebGPU](https://www.npmjs.com/package/@tensorflow/tfjs-backend-webgpu) backends, they are implemented in the [OpenGL Shading Language](https://en.wikipedia.org/wiki/OpenGL_Shading_Language). As a result, there is a significant overhead to prepare the data for GPU training. The memory management during GPU training is also not optimal. Comparatively, the C++ build of TensorFlow (accessible via the Python/Node.js APIs) directly use the CUDA libraries of Nvidia to access the GPU resource. As a result, they are more optimal. 3. No feedback loop: since the model is being queried at the client-side, we cannot collect queried data for continuous learning from the clients to improve our models. We cannot detect [concept drift](https://en.wikipedia.org/wiki/Concept_drift). Also, we would have to rely upon proxy key performance indicators to determine how well the new versions of our models are performing. Note that the caveats listed above pertain to client-side deployment and not TF.js since [TF.js also supports server-side deployment via Node.js](https://www.tensorflow.org/js/guide/nodejs). So, if you are a JavaScript developer, TF.js is still the best place to start your machine learning journey. 4b:T9b3,## TensorFlow Python API vs. TF.js TF.js excels on the production side. However, [TensorFlow' Python API](https://www.tensorflow.org/api_docs/python/tf) is still significantly better for cutting-edge research in artificial intelligence. It is due to the following reasons— 1. The Python API has a lot more functionalities than the TF.js API. 2. The Python API comes with many additional TensorFlow extensions including— [TensorFlow Addons](https://www.tensorflow.org/addons/), [TensorFlow Datasets](https://www.tensorflow.org/datasets/), [TensorFlow Decision Forests](https://www.tensorflow.org/decision_forests), [TensorFlow Federated](https://www.tensorflow.org/federated/), [TensorFlow Graphics](https://www.tensorflow.org/graphics/), [TensorFlow Hub](https://www.tensorflow.org/hub) (also available for TF.js), [model optimization libraries](https://www.tensorflow.org/model_optimization), [neural structured learning](https://www.tensorflow.org/neural_structured_learning), and [TensorFlow Probability](https://www.tensorflow.org/probability). 3. Many ground-breaking research papers use [PyTorch](https://pytorch.org/), TensorFlow's Python API, and [JAX](https://github.com/google/jax). Reusing them is complicated with Node.js. Tinkering with them requires Python knowledge. So, a Python virtual environment is better suited for the task. In fact, many virtual machines in Google Cloud, Microsoft Azure, Amazon Web Services, Colab, Kaggle, etc. come preinstalled with such environments. You can still move models built with [TensorFlow's Python API to TF.js](https://codelabs.developers.google.com/codelabs/tensorflowjs-convert-python-savedmodel#0) or [JAX to TF.js](https://blog.tensorflow.org/2022/08/jax-on-web-with-tensorflowjs.html). These Python models can also be converted and then run using the [ONNX runtime in JavaScript](https://onnxruntime.ai/). However, both these conversions (to TF.js or ONNX) is limited because not all models can be ported this way. This is mainly because not all Python API functionalities are available in their JavaScript counterparts. **Exception**: I would also like to point out that there are several sub-domains of artificial intelligence were cutting-edge research can be done on the client-side. This is particularly true for applied machine learning and [human computer interaction](https://en.wikipedia.org/wiki/Human%E2%80%93computer_interaction). In these cases, TF.js is again a good framework to have in your toolkit.4c:[] 29:{"id":"566d5c4557a59a21","slug":null,"title":"Getting started with TensorFlow.js (TF.js)","thumbnail":"1f70ff1671bf4b5461fcdfef06fe864c259912098983fdf2a38eab5ecfa3b428","access_level":"public","create_time":"$D2022-10-26T14:38:58.066Z","update_time":"$D2022-10-26T14:38:58.066Z","view_time":null,"first_public_time":"$D2022-10-26T14:38:58.066Z","has_unpublished_edits":null,"fork":{"id":"2c7039295c47c9c1","slug":"getting-started-with-tensorflow-js","title":"Getting started with TensorFlow.js (TF.js)","owner":{"id":"7c99fef7c866716c","login":"jeyabbalas","name":"Jeya Balaji Balasubramanian"}},"stars":0,"starred":false,"owner":{"type":"team","id":"9032d35f25565273","login":"jacobmpeters","name":"Jake Peters","avatar_url":"https://avatars-next.observableusercontent.com/f202fdaf3b32dd88cf45172d227f7fd7c88c28921ebf383030178a842d6b366d","tier":"starter_2024","bio":"I'm an evolutionary biologist by training. I currently work as a data scientist at the National Cancer Institute.","followed_by_you":false,"follows_you":false},"version":479,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"fab7cea82e14ca8c","login":"jacobmpeters","name":"Jake Peters","avatar_url":"https://avatars-next.observableusercontent.com/f202fdaf3b32dd88cf45172d227f7fd7c88c28921ebf383030178a842d6b366d","tier":null,"bio":"I'm an evolutionary biologist by training. I currently work as a data scientist at the National Cancer Institute.","followed_by_you":false,"follows_you":false}],"members":[{"type":"individual","id":"fab7cea82e14ca8c","login":"jacobmpeters","name":"Jake Peters","avatar_url":"https://avatars-next.observableusercontent.com/f202fdaf3b32dd88cf45172d227f7fd7c88c28921ebf383030178a842d6b366d","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Getting started with TensorFlow.js (TF.js)","cells":[{"id":0,"value":"# Getting started with TensorFlow.js (TF.js)\n\n[TensorFlow.js](https://www.tensorflow.org/js) brings the [TensorFlow](https://www.tensorflow.org/) open source machine learning framework to JavaScript.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":49,"value":"$49","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":51,"value":"$4a","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":354,"value":"$4b","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":410,"value":"## Hands-on example: Toxicity classifier\n\nLet's get started by trying to run a ready-made TF.js model in this notebook. We will try running the [Toxicity classifier](https://github.com/tensorflow/tfjs-models/tree/master/toxicity). The model takes as input an English sentence and determines if the sentence is toxic. It predicts on 7 types of toxicity: identity attack, insult, obscenity, severe toxicity, sexually explicit, threat, and toxicity.\n\nThe model's `.load()` method can take two optional arguments: `threshold` and `labels`. Threshold sets a cut-off for each of the different types of toxicity. If the model thinks the probability of that toxicity being true exceeds this threshold, it predicts that to be true. This value defaults to `0.85` if unspecified. The labels argument allows the user to only predict a subset of the 7 toxicity types. We will leave them both unspecified. You can try to pass some values to see how it affects the model output.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":223,"value":"toxicityModel = toxicity.load()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":447,"value":"Enter a statement below to be passed as a query into the toxicity classifier.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":274,"value":"viewof sentence = Inputs.text({label: \"Enter a sentence to evaluate its toxicity: \", \n placeholder: \"Enter your sentence here\", \n value: \"This is stupid\", submit: true})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":450,"value":"Examine the object below, which is returned by the TF.js model as the prediction for the above query.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":261,"value":"prediction = toxicityModel.classify([sentence])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":454,"value":"We take the toxicity prediction output by the model and plot it below, for users to interact with it and to let them evaluate the strengths and weaknesses of this model. Their feedback here can help us data scientists to improve the model.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":294,"value":"{\n let x = prediction.map(d => d.label);\n let y = prediction.map(d => d.results[0].probabilities[1]*100.0);\n \n let trace = {\n x: x,\n y: y,\n type: 'bar',\n };\n \n let data = [trace];\n\n let threshold = toxicityModel.threshold*100.0;\n\n let layout = {\n title: `Toxicity of: \"${sentence}\"`, \n xaxis: {\n title: {\n text: 'Toxicity type',\n },\n }, \n yaxis: {\n title: {\n text: 'Percentage (%)',\n },\n }, \n shapes: [\n {\n type: 'line',\n x0: -0.5,\n y0: threshold,\n x1: 6.5,\n y1: threshold,\n line: {\n color: 'red',\n width: 2,\n dash: 'dash'\n }\n }\n ]\n };\n\n const div = htl.html`

`;\n Plotly.newPlot(div, data, layout);\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":457,"value":"## Conclusion\nI think, this example demonstrates where [Mike Bostock's vision with the D3.js library (and eventually Observable) intersects with TF.js](https://medium.com/@mbostock/a-better-way-to-code-2b1d2876a3a0). Providing visual representation of the input and output of an AI model can help engage domain experts and allow them to provide useful feedback to us data scientists. This way, we keep the complex machinery of the AI model behind the scenes and let the domain experts only focus on what is important— the user experience.\n\nFrom an AI research standpoint, this can help immensely reduce the [research debt](https://distill.pub/2017/research-debt/) to be paid for getting into the field of machine learning. User interaction can help people from diverse backgrounds make lasting contributions to the development of machine learning and artificial intelligence.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":420,"value":"**Time to explore**: to get your hands dirty with running other ready-made TF.js models, please pick and run a model from the repository here: https://github.com/tensorflow/tfjs-models.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":4,"value":"# Appendix","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":106,"value":"toxicity = require('@tensorflow-models/toxicity')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":291,"value":"Plotly = require(\"https://cdn.plot.ly/plotly-2.14.0.min.js\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W4c","stdlib":"1"}} 24:null