2e:null 2f:["$","title",null,{"children":"Data Science for aspiring professionals - Introduction | Srinivas Rampalli | Observable"}] 34:[] 32:{"id":"87ef5e69252fb0c0","slug":"data-science-for-aspiring-professionals","title":"Data Science for aspiring professionals - Introduction","thumbnail":"54e704161ce89bc21db778916c84aa01194aefb1c2bf590c1e5af79c2dcf5026","access_level":"public","create_time":"$D2020-02-07T17:08:24.987Z","update_time":"$D2020-04-19T06:44:43.468Z","view_time":null,"first_public_time":"$D2020-04-13T17:34:31.864Z","has_unpublished_edits":null,"fork":false,"stars":1,"starred":false,"owner":{"type":"team","id":"57ba91783a6a250f","login":"rampalli-github","name":"Srinivas Rampalli","avatar_url":"https://avatars-next.observableusercontent.com/fd315dac01a3e313b27f55bbf9a5919d1d36c3b775e19582abf8da9a3db82dc4","tier":"starter_2024"},"version":142,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"64df3cbc722323a9","login":"rampalli-github","name":"Srinivas Rampalli","avatar_url":"https://avatars-next.observableusercontent.com/fd315dac01a3e313b27f55bbf9a5919d1d36c3b775e19582abf8da9a3db82dc4","tier":null}],"members":[{"type":"individual","id":"64df3cbc722323a9","login":"rampalli-github","name":"Srinivas Rampalli","avatar_url":"https://avatars-next.observableusercontent.com/fd315dac01a3e313b27f55bbf9a5919d1d36c3b775e19582abf8da9a3db82dc4","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"title":"Data Science for aspiring professionals - Introduction","cells":[{"id":0,"value":"md`# Data Science for aspiring professionals - Introduction`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"md `## What is Data Science?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":10,"value":"md `Trying to create science from Data. Usual mode of discovery in science is` ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":24,"value":"dot`digraph { observe -> hypothesise -> experiment -> formulate; experiment -> observe; }`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":34,"value":"html `This cycle is a long one. Normally it takes lot of observation to gather enough data to create hypothesis. Historically, many scientists like Kepler or Tycho Brahe had collected information by great observation and discipline. That data was extremely useful in formulating various laws and validating them.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":51,"value":"md `With the advent of computers data accumulation process has been happening in computers in ways that make it easier to process using computers. As internet happened, more data started getting accumulated. As mobiles and internet on mobiles happened, lot more data started getting captured.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":53,"value":"md `In order to analyse the data, appropriate _compute infrastructure_ and _tools_ are also needed. In the last few years (as in 2020) all of these variables have converged and the possibility of doing analysis on data at scale has opened up. Various kinds of tools have come up that deal with different kinds of use cases.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":60,"value":"md ` _It is not that data analysis has been happening recently_, it got more *organised* recently because of all the variables coming together. Many people had insights by looking at data that was publicly available for almost everyone. Let us see a few examples.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":62,"value":"html ``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":64,"value":"md `To start with let us look at this Macworld video of 2001 where Steve Jobs launched _iTunes application on Mac_ . The video has been marked to start at around 52 minutes when he explains the music revolution that was happening at that time. He explains what people are doing with the music cds and how they are making playlists and writing/burning them to empty CDs. The lynchpin in this argument is that in the year 2000 apparently just in US alone **320 million blank CDs ** were sold.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":71,"value":"md `Now that was a simple data point that so many blank CDs are being sold. Further questions were asked on why so many are being sold? What are the people doing with them? The answers were probably conclusive for the field of audio that they were trying to focus on. **The whole idea of Data Science** is to change the typical cycle from *hypothesise -> validate* to **what does the data say? -> oh it says this? -> Is it because of this (hypothesise)? -> validate**`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":77,"value":"md `We are starting with an advantage that the data is already there and *lots of it*. Making an Hypothesis and validating hypothesis could possibly happen without further collection of data, or at least we can restrict the scope of hypothesis to such topics that could be easily validated with the available data.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":81,"value":"html `Another interesting data analysis project we can look at is the airline time performance. http://stat-computing.org/dataexpo/2009/` ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":85,"value":"md `Lots of data about flight arrivals and departures details of commercial flights was analysed by various people to understand the patterns of being ontime or delayed. This data is analysed to understand the patterns and possible variables that impact the arrivals and departures that may end up in a prediction tool.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":98,"value":"md `There are couple of important examples that are some of the best data analysis that I have seen in my experience. One is of the swedish Statistician Professor **Hans Rosling**. His work can be looked at [gapminder](https://www.gapminder.org). There are many talks and videos and tools there that help you look at data in various ways and ask pertinent questions. `","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":101,"value":"md `American Statistician Professor **Edward Tufte** has done some great work. His work can be found [here](https://www.edwardtufte.com/tufte/)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":131,"value":"html `

Additional interesting examples can be found here https://flowingdata.com/

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":110,"value":"md `Understanding levels of any subject usually follow a certain order`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":115,"value":"dot`digraph { \"Being Informed\" -> \"Able to use it\" -> \"Expert User\" -> \"Create It\"; }`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":119,"value":"md `This article is an attempt to touch the first step, to bring awareness about this topic. In the next article, we will try to look at how we can use the tools and already known concepts of data science.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":30,"value":"dot = require(\"@observablehq/graphviz@0.2\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W34","stdlib":"1","titling":"auto"}} 33:"$32:body"