2c:null 2d:["$","title",null,{"children":"Limitations of AI | Sandraviz | Observable"}] 32:Tf3d,### Reasons for errors in AI - Generally speaking, what are your thoughts ? #### Probability One is simply that the algorithms typically rely on probability, because they make so many predictions, it’s likely that some will be wrong, just because there’s always a chance that they’ll be off. The likelihood of errors depends on a lot of factors, including the amount and quality of the data used to train the algorithms, the specific type of machine-learning method chosen (for example, deep learning, which uses complex mathematical models, versus classification trees that rely on decision rules), and whether the system uses only explainable algorithms (meaning humans can describe how they arrived at their decisions), which may not allow it to maximize accuracy. #### Change in environment The environment in which machine learning operates may itself evolve or differ from what the algorithms were developed to face. While this can happen in many ways, two of the most frequent are concept drift and covariate shift. - Before having a deeper look, any idea what these two concepts refer to? ##### Concept drift - Change over time The relationship between the inputs the system uses and its outputs isn’t stable over time or may be misspecified. Consider a machine-learning algorithm for stock trading. If it has been trained using data only from a period of low market volatility and high economic growth, it may not perform well when the economy enters a recession or experiences turmoil—say, during a crisis like the Covid-19 pandemic. As the market changes, the relationship between the inputs and outputs, for example between how leveraged a company is and its stock returns also may change. Similar misalignment may happen with credit-scoring models at different points in the business cycle. - Are there any other examples which cross your mind? ##### Covariate shifts - Change in the population Covariate shifts occur when the data fed into an algorithm during its use differs from the data that trained it. This can happen even if the patterns the algorithm learned are stable and there’s no concept drift. For example, a medical device company may develop its machine-learning-based system using data from large urban hospitals. But once the device is out in the market, the medical data fed into the system by care providers in rural areas may not look like the development data. The urban hospitals might have a higher concentration of patients from certain sociodemographic groups who have underlying medical conditions not commonly seen in rural hospitals. Such disparities may be discovered only when the device makes more errors while out in the market than it did during testing. Now, let's imagine a rather simple setting. Say you train a deep learning modell to detect the race of a dog by using billions of dog pictures. Then you request a classification of a cat photo. - How can the algorithm give a true answer to this? - Are there any other examples which cross your mind? #### Complexity The third reason machine learning can make inaccurate decisions has to do with the complexity of the overall systems it’s embedded in. Consider a device used to diagnose a disease on the basis of images that doctors input—such as IDx-DR, which identifies eye disorders like diabetic retinopathy and macular edema and was the first autonomous machine-learning-based medical device authorized for use by the U.S. Food and Drug Administration. The quality of any diagnosis depends on how clear the images provided are, the specific algorithm used by the device, the data that algorithm was trained with, whether the doctor inputting the images received appropriate instruction, and so on. With so many parameters, it’s difficult to assess whether and why such a device may have made a mistake, let alone be certain about its behavior.33:T576,## Appendix ### Web links - [AI Is All Around You](https://okai.brown.edu/chapter1.html) - [Facebook apology as AI labels black men 'primates'](https://www.bbc.com/news/technology-58462511) - [A visual introduction to machine learning](http://www.r2d3.us/visual-intro-to-machine-learning-part-1/) - [Model Tuning and the Bias-Variance Tradeoff](http://www.r2d3.us/visual-intro-to-machine-learning-part-2/) ### Books

[The Alignment Problem](https://brianchristian.org/the-alignment-problem/) Brian Christian ISBN-13: 978-0393635829 ### Research papers [When Machine Learning Goes Off the Rails](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3881422). Boris Babic. Independent I. Glenn Cohen. Harvard Law School 2021 [Three Machine Learning Solutions to the Bias-Variance Dilemma](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3588594). Marcos López de Prado. Cornell University. Operations Research & Industrial Engineering. 2020 [Artificial Intelligence Based Suicide Prediction](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3324874). Mason Marks Harvard Law School; Yale Law School; University of New Hampshire Franklin Pierce School of Law; Leiden Law School, Center for Law and Digital Technologies 201934:T558,35:[{"id":"b36eaa18a669ffd4dc5c96947fc3639e7114a9b43435de9b298d3b2b19b10fc9cb6374eb0e50ca123d234df735ae6f50a67152a626b274edcc4fa66e169b3849","href":"https://static.observableusercontent.com/files/b36eaa18a669ffd4dc5c96947fc3639e7114a9b43435de9b298d3b2b19b10fc9cb6374eb0e50ca123d234df735ae6f50a67152a626b274edcc4fa66e169b3849","name":"love.jpeg","type":"image/jpeg","lastModified":1630593622470.007,"size":76968},{"id":"a3f0c5fa50cfc63d097ab3ce74e1a8061d5c718f1dc114dfb2c1763365d0c643393244ae5d22e003d28dbe929d728f0a4b930557dcf73bc0ee48690979bb9190","href":"https://static.observableusercontent.com/files/a3f0c5fa50cfc63d097ab3ce74e1a8061d5c718f1dc114dfb2c1763365d0c643393244ae5d22e003d28dbe929d728f0a4b930557dcf73bc0ee48690979bb9190","name":"FB.png","type":"image/png","lastModified":1631545870120.517,"size":7873305},{"id":"2fb7f85d6a9a1a6d4bb75c10af3c0f441aefd5703f6778eccd5b83e77f2ce5eda7d80874a6515769f2d4077c717788b59bdf1048e66e752aa9b118c3c02ecd59","href":"https://static.observableusercontent.com/files/2fb7f85d6a9a1a6d4bb75c10af3c0f441aefd5703f6778eccd5b83e77f2ce5eda7d80874a6515769f2d4077c717788b59bdf1048e66e752aa9b118c3c02ecd59","name":"training.jpg","type":"image/jpeg","lastModified":1632126697402.834,"size":101195},{"id":"69271f4f9f4acd46f0406142153e7abca3eded27bc21edb8aeb28fa192b51a564f901d2523a87085225a1e2d48fad43aa9a695528196f26bfe41f494e1ba5def","href":"https://static.observableusercontent.com/files/69271f4f9f4acd46f0406142153e7abca3eded27bc21edb8aeb28fa192b51a564f901d2523a87085225a1e2d48fad43aa9a695528196f26bfe41f494e1ba5def","name":"transferir (1).jpeg","type":"image/jpeg","lastModified":1632130159230.372,"size":18244},{"id":"37e5e95f7aa4a86863424286494057f070d235a0f0f29690b11cfdd791a9e109340bad5ed45b7f351e55508c970300db0b743828e327953feb6030279148d0e0","href":"https://static.observableusercontent.com/files/37e5e95f7aa4a86863424286494057f070d235a0f0f29690b11cfdd791a9e109340bad5ed45b7f351e55508c970300db0b743828e327953feb6030279148d0e0","name":"bias_vairance.png","type":"image/png","lastModified":1632130465366.431,"size":273511},{"id":"dcb8107fbdb98e0f1c973375ff8934cf42a3ffc1c2f88a2ab25731f0d479c6223f948a1a51deb1303d153cad65555f921feb434fa309c02f6375396d1b568b12","href":"https://static.observableusercontent.com/files/dcb8107fbdb98e0f1c973375ff8934cf42a3ffc1c2f88a2ab25731f0d479c6223f948a1a51deb1303d153cad65555f921feb434fa309c02f6375396d1b568b12","name":"wrong lable.png","type":"image/png","lastModified":1632140702835.977,"size":2860691},{"id":"0e21723b071ece4d4f85ea131032d5c528dd2a056d3e0c5cc76f47f39f8e7779aa735c21352872003d0345a757bdae0fb3eb023996cc8f27b045dc57c39b8175","href":"https://static.observableusercontent.com/files/0e21723b071ece4d4f85ea131032d5c528dd2a056d3e0c5cc76f47f39f8e7779aa735c21352872003d0345a757bdae0fb3eb023996cc8f27b045dc57c39b8175","name":"ai.jpg","type":"image/jpeg","lastModified":1632143124578.735,"size":154709},{"id":"7da358cefc0f968175efad60fcbc1b19c8b90eff0414b17b6ca2ffe2035f16d8b2d240f09255401c952a1143f55c769c24e146e921b965c34daafe3b1fc2d876","href":"https://static.observableusercontent.com/files/7da358cefc0f968175efad60fcbc1b19c8b90eff0414b17b6ca2ffe2035f16d8b2d240f09255401c952a1143f55c769c24e146e921b965c34daafe3b1fc2d876","name":"align.jpeg","type":"image/jpeg","lastModified":1632154749153.976,"size":95351}] 30:{"id":"fe4c72b0f9bf01af","slug":"limitations-of-ai","title":"Limitations of AI","thumbnail":"3f3d7a678469bc3af6a75d9a6932b21b0dcb35f9598e0a038ceb64a9fbea0180","access_level":"public","create_time":"$D2021-09-02T09:54:47.019Z","update_time":"$D2023-11-28T12:31:36.542Z","view_time":null,"first_public_time":"$D2021-09-02T10:04:18.692Z","has_unpublished_edits":null,"fork":true,"stars":5,"starred":false,"owner":{"type":"team","id":"6661932e6ac85aef","login":"sandraviz","name":"Sandraviz","avatar_url":"https://avatars-next.observableusercontent.com/e6a2480f89a40dcfbe75f19d5081ae2296848971c4b934ed0499fd3b7f1973eb","tier":"starter_2024"},"version":1654,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"3d1beb3a90fbfe98","login":"sandraviz","name":"Sandraviz","avatar_url":"https://avatars-next.observableusercontent.com/9b19f1a074469dcaafa9bd24c7b08f4bc969889b551fe70c209497f1d03c6514","tier":null}],"members":[{"type":"individual","id":"3d1beb3a90fbfe98","login":"sandraviz","name":"Sandraviz","avatar_url":"https://avatars-next.observableusercontent.com/9b19f1a074469dcaafa9bd24c7b08f4bc969889b551fe70c209497f1d03c6514","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":4,"collection":{"id":"eab5d593efbf8501","slug":"ai","title":"AI ","owner":{"id":"6661932e6ac85aef","login":"sandraviz","name":"Sandraviz"}},"type":"notebook","body":{"titling":"auto","title":"Limitations of AI","cells":[{"id":0,"value":"Decision_Support = md`# Limitations of AI\n\nIn 2021 and 2022 I taught remotely at the **University Arcada (Finland)** responsible and ethical uses of machine learning practice, which is a fundamental precondition to trustworthy development of AI. You can find all the material and links in this [Github Repository](https://github.com/sandravizz/Decision-support-systems-verification-Python).\n\n\n**Premature optimization is the root of all evil.** - Donald Knuth \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1222,"value":"

","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":181,"value":"Index1 = toc(\"h2, h3, h4, h5\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":528,"value":"QUIZ = md`## Current state of AI\n\nLet's start by looking at the current state of AI in our world. \n\n[

](https://okai.brown.edu/chapter1.html)\n\n### Quiz \n\n- Which kind of applications come to your mind when you think about AI?\n- Which one is your favorite and why?\n- What is Artificial Narrow Intelligence ANI?\n- What do you think will be the estimated revenue in AI by 2025?\n- What is an algorithm again?\n- What is the difference between AI and Machine Learning?\n- Why is Machine Learning important?\n\nGreat, we can understand easily the importance and excitment about AI and Ml. \n\n**Now** let's have a look at the \"costs\" of this development. What can go wrong, what are the consequences and which are options to handle these unwanted outcomes.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1476,"value":"## When ML goes wrong\n\nProducts and services that rely on machine learning, computer programs that constantly absorb new data and adapt their decisions in response don’t always make ethical or accurate choices as we can see in the following example. \n\n[

](https://www.bbc.com/news/technology-58462511)\n\n- What went wrong? \n- Was it really an error, technically speaking? \n- What is the real problem? \n- If you were Facebook, how would you handle the situation?\n\n*\"We disabled the entire topic-recommendation feature as soon as we realised this was happening so we could investigate the cause and prevent this from happening again.\"* Facebook \n\n- How might this investigation look like? Any idea?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1196,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1330,"value":"## Bias-Variance tradeoff dilema\n\nFrom a more general perspective, we've seen consequences and causes of errors in ML. Now let's look at a typical error dilemma in ML modelling, the Bias-Variance Tradeoff. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1451,"value":"[

](https://observablehq.com/@sandravizmad/bias-vs-variance)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1342,"value":"## Use case: decision trees\n\nNow we are going to have a deeper look at decision trees and how the Bias-Variance trade-off is applied there. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1398,"value":"[

](https://observablehq.com/@sandravizmad/decision-trees)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1417,"value":"## Social Suicide Prediction (FB)\n\nErrors regarding the decision whether a house is based in SF or NY have pretty undramatic consequences. \n\nWhat about real use cases, where the same ML techniques are applied to people's mental health situations?\n\nWhat harm does an error produces in these cases? ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1426,"value":"[

](https://observablehq.com/@sandravizmad/social-suicide-prediction)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1572,"value":"## The alignment problem\n\nWe've seen above that errors are not avoidable. The relevant question is when is an error unacceptable. \n\nHow can we ensure that ML models capture our values and understand what we intend and want? \n\nThis has emerged as one of the most central and urgent scientific questions in the field of computer science and it has the name: the alignment problem. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1350,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1355,"value":"### Code","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":18,"value":"$34","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":179,"value":"import {toc} from \"@nebrius/indented-toc\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W35","stdlib":"1"}} 31:"$30:body"