2b:null 2c:["$","title",null,{"children":"Final Project Progress Report | Zhenyu Chen | Observable"}] 31:T4bb,## 1. Project Descriptions **Title:** 2020 H1B Petition Data Analysis **Team Member:** Zhenyu Chen **Introduction:** _Problem:_ The H1B lottery is a process used to select the limited number of H1B visas that are available each year. The lottery system is random and there are more applicants than available visas, leading to a high level of competition. The process can be challenging due to the strict eligibility requirements, the need for a sponsoring employer, and the complex application process. _Importance:_ Analyzing H1B Visa petition data can provide valuable information on the job market in the US, including the demand for specific skills and industries that are more likely to sponsor foreign workers. This can help international students like me to make informed decisions about career paths, target industries, target companies, etc. **Questions:** 1. Are Computer Science related jobs are easier to get H1B sponsored? 2. Does Better economy make employers more willing to sponsor H1B? 3. In which state are employers most likely to sponsor H1B? 4. Do employers in “Blue States” really prefer to sponsor H1B? 5. Is Texas becoming the new Silicon Valley in terms of H1B applications? 32:T93c,## Idea 1: Computer Science (CS) related jobs are more likely to get sponsored by employers ### Idiom 1 Stacked area charts of job titles during 2017 to 2022 ### Made-up Insight 1 In each year during 2017 to 2022, over half of the U.S. employers sponsor H1B visas for employees in CS related jobs, and the proportion of CS related jobs is increasing year by year. ### Data (what) |Data|Type| |---|---| |Used Datasets|H1B Petition Records| |Data Availability|static| |Dataset Type|table, temporal| |Attributes|Type & Direction| |---|---| |year|quantitative, sequential| |SOC_Title|categorical| |h1b_count (derived)|quantitative, sequential| |CS_related (derived)|categorical| A new attribute, named `CS_related` meaning "computer science related", will be derived from the `SOC_Title` attribute. Every `SOC_Title` which includes "computer", "software", or "programmer" will be categorized into `Computer Science Related`, others will be categorized into `Non Computer Science Related`. In the future I might used SOC standard to categorized the data more correctly. Also, the attribute `h1b_count` is derived as well, which is just the tally based on the value of each attribute. > **Note**: SOC means Standard Occupational Classification. It's an international standard for job titles. ### Tasks (why) I1_MT: In each year from 2017 to 2022, compare the number of H1B applicants whose job is computer science related with the number of H1B applicants whose job is NOT computer science related. **(compare, distribution)** I1_ST1: Discover the yearly distribution of job types (attribute: `CS_related`) from 2017 to 2022. **(discover, distribution)** I1_ST2: Summarize the trend of the number of H1B applicants from 2017 to 2022. **(summarize, pattern)** ### How |Attributes|Type & Direction|Channel|Rationale| |---|---|---|---| |year|quantitative, sequential|position, separate & order & (sort of) align|Positions on common scale is the most effective channel for quantitative & sequential attribute.| |CS_related (derived)|categorical|color hues|Color hues is the second most effective channel for categorical attribute.| |h1b_count (number of H1B petitions, derived)|quantitative, sequential|area, express|Positions on common scale is the most effective channel for quantitative & sequential attribute..| |Marks|Attribute| |---|---| |area|h1b_count|33:T70f,## Idea 2: Better economy (low unemployment rate) makes employers more willing to sponsor H1B. ### Idiom 2 A hybrid chart of stacked area chart of job titles and a line chart of unemployment rate during 2017 to 2022 ### Made-up Insight 2 In 2020, the year when the COVID-19 pandemic was most severe in the United States, the unemployment rate also reached a peak in recent 6 years. Meanwhile the total number of H1B petitions is the lowest in 2020. ### Data (what) |Data|Type| |---|---| |Used Datasets|H1B Petition Records, Unemployment Rate| |Data Availability|static| |Dataset Type|table, temporal| |Attributes|Type & Direction| |---|---| |year|quantitative, sequential| |h1b_count (derived)|quantitative, sequential| |unemployment|quantitative, sequential| How I derive `CS_related` & `h1b_count` attributes has been explain in Idea 1, so I won't repeat the explanation again. ### Tasks (why) I2_MT: Discover the correlation between the total number of H1B petitions and the unemployment during 2017 to 2022. **(discover, correlation)** I2_ST1: Compare the number of (attribute: `h1b_count`) H1B petitions in the year when the unemployment rate peaked with other 5 years. **(compare, distribution)** I2_ST2: Lookup the year when the unemployment rate peaked. **(lookup, outlier)** ### How |Attributes|Type & Direction|Channel|Rationale| |---|---|---|---| |year|quantitative, sequential|horizontal position, separate & order & align|Positions on common scale is the most effective channel for quantitative & sequential attribute.| |h1b_count (derived)|quantitative, sequential|vertical position, express|Same as above.| |unemployment|quantitative, sequential|vertical position, express|Same as above.| |Marks|Attribute| |---|---| |point (or maybe line)|unemployment| |line|h1b_count (derived)|34:T48d,hybridBarLineChart = { const svg = d3.create("svg") .attr("viewBox", [0, 0, width+100, height]); svg.append("g") .attr("fill", "steelblue") .attr("fill-opacity", 0.8) .selectAll("rect") .data(idiom2Data) .join("rect") .attr("x", d => x(d.year)) .attr("width", x.bandwidth()) .attr("y", d => y1(d.count)) .attr("height", d => y1(0) - y1(d.count)); svg.append("path") .attr("fill", "none") .attr("stroke", "currentColor") .attr("stroke-miterlimit", 1) .attr("stroke-width", 3) .attr("d", line(idiom2Data)); svg.append("g") .attr("fill", "none") .attr("pointer-events", "all") .selectAll("rect") .data(idiom2Data) .join("rect") .attr("x", d => x(d.year)) .attr("width", x.bandwidth()) .attr("y", 0) .attr("height", height) .append("title") .text(d => `${d.year} ${d.count.toLocaleString("en")} H1B Petition Count ${d.unemployment.toLocaleString("en")} Unemployment Rate`); svg.append("g") .call(xAxis); svg.append("g") .call(y1Axis); svg.append("g") .call(y2Axis); return svg.node(); }35:T68b,## Idea 3: In which state are employers most likely to sponsor H1B? ### Idiom 3 A choropleth map indicating the H1B sponsor rate of all e-verify employers in each U.S. state. > **Note**: E-Verify employers are the employers who can sponsor employees H1B visas. Besides, this whole section (idea 3) is based on the data of 2022. ### Made-up Insight 3 California has the highest H1B sponsor rate in its all e-verify employers in 2022. ### Data (what) |Data|Type| |---|---| |Used Datasets|H1B Petition Records| |Data Availability|static| |Dataset Type|table, geo, temporal| |Attributes|Type & Direction| |---|---| |year|quantitative, sequential| |worksite|categorical| |h1b_count (derived)|quantitative, sequential| |employer_count (number of e-verify employers)|quantitative, sequential| |sponsor_rate (derived)|quantitative, sequential| How I derive `h1b_count` attribute has been explain in Idea 1, so I won't repeat the explanation again. `sponsor_rate`, which is literally "H1B sponsor rate", is derived from by "`h1b_count` / `employer_count`". ### Tasks (why) I3_MT: Locate the state where employers are most willing to sponsor H1B. **(locate, outlier)** I3_ST1: Discover the states where employers are more willing to sponsor. **(discover, features)** ### How |Attributes|Type & Direction|Channel|Rationale| |---|---|---|---| |worksite|categorical|spatial region|It's the most effective channel for categorical attributes.| |sponsor_rate (derived)|quantitative, sequential|color saturation|On a map, where positions are not suitable, the color saturation is the most effective channel.| |Marks|Attribute| |---|---| |area|worksite| |line|h1b_count (derived)|36:T6a8,## Idea 4: Do employers in “Blue States” really prefer to sponsor H1B? ### Idiom 4 A special table with top 10 states ranked by H1B sponsor rate in each year of 2017-2022. ### Made-up Insight 4 Blue States do prefer sponsoring H1B visas during 2017-2022, and California sponsored the most number of H1B visas of all time. ### Data (what) |Data|Type| |---|---| |Used Datasets|H1B Petition Records, Blue or Red States Data| |Data Availability|static| |Dataset Type|table, geo, temporal| |Attributes|Type & Direction| |---|---| |year|quantitative, sequential| |worksite|categorical| |sponsor_rate (derived)|quantitative, sequential| |state_type|categorical| How I derive `h1b_count` & `sponsor_rate` attributes has been explain in Idea 1, so I won't repeat the explanation again. `state_type` is from _"Blue or Red States" Data_ (see 2. Data) and the value is either `red` or `blue`. ### Tasks (why) I4_MT: Compare the number of red states and blue states in the top 10 sponsoring states each year from 2017 to 2022. **(compare, features)** I4_ST1: Explore if there is any state which is often the top 1 state. **(explore, outlier)** I4_ST2: Summarize the trend of top 10 sponsoring states . **(summarize, trends)** ### How |Attributes|Type & Direction|Channel|Rationale| |---|---|---|---| |worksite|categorical|N/A because it's text|N/A| |sponsor_rate (derived)|quantitative, sequential|horizontal position, separate & order & (sort of) align|Positions on common scale is the most effective channel for quantitative & sequential attribute.| |state_type|categorical|color hues|Color hues is the second most effective channel for categorical attribute.| |Marks|Attribute| |---|---| |N/A|N/A|37:T864,## Idea 5: Is Texas becoming the new Silicon Valley in terms of H1B applications? ### Idiom 5 An interactive bubble map showing H1B petition count in each worksite (city) with a slider to control the time. ### Made-up Insight 5 Texas is becoming the new Silicon Valley in terms of the number of cities where employers have filed 10k+ H1B petitions. ### Data (what) |Data|Type| |---|---| |Data Availability|static| |Dataset Type|table, geo, temporal| |Attributes|Type & Direction| |---|---| |year|quantitative, sequential| |h1b_count (derived)|quantitative, sequential| |worksite|categorical| |longitude & latitude (derived)|quantitative, diverging| How I derive `h1b_count` attributes has been explain in Idea 1, so I won't repeat the explanation again. `longitude` & `latitude` are derived using Mapbox API. Each pair of `longitude` & `latitude` are retrieved from the result of searching `worksite` on Mapbox Geocoding API. Learn more from the *A1. Data Pre-Processing* Appendices section at the end. ### Tasks (why) I5_MT: Compare the number of cities with 10k+ H1B petitions in Texas and Bay Area in each year. **(compare, feature)**. I5_ST1: Summarize the trend of H1B petition count by cities in Bay Area and Texas from 2017 to 2022. **(summarize, trends)** I5_ST2: Lookup all the cities in Bay Area and Texas where employers filed 10k+ H1B petitions in each year. **(lookup, feature)** ### How |Attributes|Type & Direction|Channel|Rationale| |---|---|---|---| |year|quantitative, sequential|interactive slider|A dynamic timeline can be great for observing gradual change.| |h1b_count (derived)|quantitative, sequential|area (2D size), separate|On a map, area is the most effective magnitude channel for ordered attribute. | |longitude & latitude (derived)|quantitative, diverging|position, express with geo-projection algorithm|Nothing is better than positions to represent spatial/geological points. It's also the most effective one.| |worksite|categorical|N/A|This attribute isn't visualized. But it derived longitude & latitude.| |Marks|Attribute| |---|---| |point|longitude & latitude (derived)| |area|h1b_count (derived)|38:T7be,## Evaluation Plan I will be running an usability study for Idiom 5, I'll be evaluating tasks: - I5_MT: Compare the number of cities with 10k+ H1B petitions in Texas and Bay Area in each year. **(compare, feature)**. - I5_ST1: Summarize the trend of H1B petition count by cities in Bay Area and Texas from 2017 to 2022. **(summarize, trends)** - I5_ST2: Lookup all the cities in Bay Area and Texas where employers filed 10k+ H1B petitions in each year. **(lookup, feature)** __Script__ Welcome to the study, I'm going to ask these following questions to test if you can understand and utilize this visualization. Please take your time to familiarize yourself with the interactive bubble map and the slider to control the time. Once you're ready, we'll begin the tasks. __Tasks to test__ * Q1. Please identify all the cities where the employers have filed over 10k H1B petitions in 2022. * Q2. Please describe the trend of H1B petition count by cities in Texas and Bay Area from 2017 to 2022. * Q3. Do you think Texas is catching up with Bay Area in terms of H1B petitions, and why? __Expected answers__ * Q1. In California: San Jose, San Francisco, Sunnyvale. In Texas: Austin, Plano, Irving. * Q2. Texas has more and more cities where employers filed over 10k H1B petitions. Cities in Bay Area are relatively stable. * Q3. Yes, I think Texas is catching up with Bay Area. There are more and more cities where H1B petitions are closer to the cities in Bay Area. __Post test questionnaire__ In a scale of 1 to 5, being 1 very ineffective/useless and 5 very effective/useful: 1. How easy was it to understand this visualization? Was it easy to interact with? 2. How effective was the visualization for helping you find the trend in Texas and the Bay Area? 3. How easy was it to compare the number of cities with 10k+ H1B petitions in Texas and the Bay Area in each year? 4. How useful was the slider for controlling the time and observing changes in H1B petition counts?39:T1038,// Copyright 2021 Observable, Inc. // Released under the ISC license. // https://observablehq.com/@d3/stacked-area-chart function StackedAreaChart(data, { x = ([x]) => x, // given d in data, returns the (ordinal) x-value y = ([, y]) => y, // given d in data, returns the (quantitative) y-value z = () => 1, // given d in data, returns the (categorical) z-value marginTop = 20, // top margin, in pixels marginRight = 30, // right margin, in pixels marginBottom = 30, // bottom margin, in pixels marginLeft = 40, // left margin, in pixels width = 640, // outer width, in pixels height = 400, // outer height, in pixels xType = d3.scaleUtc, // type of x-scale xDomain, // [xmin, xmax] xRange = [marginLeft, width - marginRight], // [left, right] yType = d3.scaleLinear, // type of y-scale yDomain, // [ymin, ymax] yRange = [height - marginBottom, marginTop], // [bottom, top] zDomain, // array of z-values offset = d3.stackOffsetDiverging, // stack offset method order = d3.stackOrderNone, // stack order method xFormat, // a format specifier string for the x-axis yFormat, // a format specifier for the y-axis yLabel, // a label for the y-axis colors = d3.schemeTableau10, // array of colors for z yRightFormat = ".0%", // a format specifier for the right y-axis yRightLabel, // a label for the right y-axis } = {}) { // Compute values. const X = d3.map(data, x); const Y = d3.map(data, y); const Z = d3.map(data, z); // Compute default x- and z-domains, and unique the z-domain. if (xDomain === undefined) xDomain = d3.extent(X); if (zDomain === undefined) zDomain = Z; zDomain = new d3.InternSet(zDomain); // create a set that supports Date object as a key // Omit any data not present in the z-domain. const I = d3.range(X.length).filter(i => zDomain.has(Z[i])); // console.log({x, y, z, X, Y, Z, xDomain, zDomain, I}); // Compute a nested array of series where each series is [[y1, y2], [y1, y2], // [y1, y2], …] representing the y-extent of each stacked rect. In addition, // each tuple has an i (index) property so that we can refer back to the // original data point (data[i]). This code assumes that there is only one // data point for a given unique x- and z-value. const series = d3.stack() .keys(zDomain) .value(([x, I], z) => Y[I.get(z)]) .order(order) .offset(offset) (d3.rollup(I, ([i]) => i, i => X[i], i => Z[i])) .map(s => s.map(d => Object.assign(d, {i: d.data[1].get(s.key)}))); // Compute the default y-domain. Note: diverging stacks can be negative. if (yDomain === undefined) yDomain = d3.extent(series.flat(2)); // Construct scales and axes. const xScale = xType(xDomain, xRange); const yScale = yType(yDomain, yRange); const color = d3.scaleOrdinal(zDomain, colors); const xAxis = d3.axisBottom(xScale).ticks(width / 80, xFormat).tickSizeOuter(0); const yAxis = d3.axisLeft(yScale).ticks(height / 50, yFormat); const area = d3.area() .x(({i}) => xScale(X[i])) .y0(([y1]) => yScale(y1)) .y1(([, y2]) => yScale(y2)); const svg = d3.create("svg") .attr("width", width) .attr("height", height) .attr("viewBox", [0, 0, width, height]) .attr("style", "max-width: 100%; height: auto; height: intrinsic;"); svg.append("g") .attr("transform", `translate(${marginLeft},0)`) .call(yAxis) .call(g => g.select(".domain").remove()) .call(g => g.selectAll(".tick line").clone() .attr("x2", width - marginLeft - marginRight) .attr("stroke-opacity", 0.1)) .call(g => g.append("text") .attr("x", -marginLeft) .attr("y", 10) .attr("fill", "currentColor") .attr("text-anchor", "start") .text(yLabel)); svg.append("g") .selectAll("path") .data(series) .join("path") .attr("fill", ([{i}]) => color(Z[i])) .attr("d", area) .append("title") .text(([{i}]) => Z[i]); svg.append("g") .attr("transform", `translate(0,${height - marginBottom})`) .call(xAxis); return Object.assign(svg.node(), {scales: {color}}); }3a:[{"id":"24b2e0492f9312aa1d2ba65d5f447a341cf54d57609c8fb612b7569588180ffefd45ca2fda1b07bf34c3c6704b9c480dd4feac286c65566243428fd951f90c83","href":"https://static.observableusercontent.com/files/24b2e0492f9312aa1d2ba65d5f447a341cf54d57609c8fb612b7569588180ffefd45ca2fda1b07bf34c3c6704b9c480dd4feac286c65566243428fd951f90c83","name":"LCA_FY_2022.csv","type":"text/csv","lastModified":1680500157830.358,"size":651915},{"id":"cac9b3d1f3e7aea64097d1e6a6d0dd8d1d3475b44e4ebc7270748f8137eabd39ccaaf622bb4a9b232a9522bbb489689606f3534cd594e98f43676360a73663b9","href":"https://static.observableusercontent.com/files/cac9b3d1f3e7aea64097d1e6a6d0dd8d1d3475b44e4ebc7270748f8137eabd39ccaaf622bb4a9b232a9522bbb489689606f3534cd594e98f43676360a73663b9","name":"idiom1.csv","type":"text/csv","lastModified":1680697244821.833,"size":503},{"id":"6bf6e307bd5ea16f745cf0b60944d44dd4b639afb932a8d358d79494d7645a6bdc38f890917f355f9542ce47337a21e7fa4077d54499af6c37a109eb2d570927","href":"https://static.observableusercontent.com/files/6bf6e307bd5ea16f745cf0b60944d44dd4b639afb932a8d358d79494d7645a6bdc38f890917f355f9542ce47337a21e7fa4077d54499af6c37a109eb2d570927","name":"idiom1-3.png","type":"image/png","lastModified":1680885984780.66,"size":49088},{"id":"be563b35a7fcbce36e9f271e474185126eabe67ab27cc0752321cb48325754d144ff016b8c511848e1cb340c69c0b3157638b99534f139b29a01202d946b0869","href":"https://static.observableusercontent.com/files/be563b35a7fcbce36e9f271e474185126eabe67ab27cc0752321cb48325754d144ff016b8c511848e1cb340c69c0b3157638b99534f139b29a01202d946b0869","name":"image.png","type":"image/png","lastModified":1680833738663.949,"size":72027},{"id":"416efc5b864c16b2298864abe0522570084dc5a54049d70824b869d7e9f5f101e9679de54af7020b9fcad3219d79626874d068334789c4019e6c853055dc445c","href":"https://static.observableusercontent.com/files/416efc5b864c16b2298864abe0522570084dc5a54049d70824b869d7e9f5f101e9679de54af7020b9fcad3219d79626874d068334789c4019e6c853055dc445c","name":"idiom1-1.png","type":"image/png","lastModified":1680834176260.847,"size":86512},{"id":"8fbc76639fb14a5846326e3d07b5d4c5a3183a77d6f5374d1026930057dc4d54b86af8dee853e12500de6dc7de3b905cae5b3147454795bed321e00df80cb324","href":"https://static.observableusercontent.com/files/8fbc76639fb14a5846326e3d07b5d4c5a3183a77d6f5374d1026930057dc4d54b86af8dee853e12500de6dc7de3b905cae5b3147454795bed321e00df80cb324","name":"idiom1-2.png","type":"image/png","lastModified":1680834176283.116,"size":89507},{"id":"443c9b24ec605140df3762bfd97a4a30e40efe6bed4bb2e183ad7e9e8f84520eda2eac9b158643d6ba7845963b243f7f2cc1cf77e3210f75ed43c21999f41750","href":"https://static.observableusercontent.com/files/443c9b24ec605140df3762bfd97a4a30e40efe6bed4bb2e183ad7e9e8f84520eda2eac9b158643d6ba7845963b243f7f2cc1cf77e3210f75ed43c21999f41750","name":"idiom2-1.png","type":"image/png","lastModified":1680903062311.917,"size":118812},{"id":"e0e60e1a3fe80fbbad30fda82e79846ec1c2daea91d238fce1a388a51a2e33f5c23256f6ee4997f527e7bd8a4abadedd95ba69b79bde06219936d2ca736a7fe1","href":"https://static.observableusercontent.com/files/e0e60e1a3fe80fbbad30fda82e79846ec1c2daea91d238fce1a388a51a2e33f5c23256f6ee4997f527e7bd8a4abadedd95ba69b79bde06219936d2ca736a7fe1","name":"idiom2-2.png","type":"image/png","lastModified":1680903062499.88,"size":119096},{"id":"3157240ca6eafb1245e078c4de05d0c9c50f7a4b29d34ee719261c3a83a47267f5c631c207c371d109f8e35e5a467ae1156b88cbf4d972b509391857729b1b2a","href":"https://static.observableusercontent.com/files/3157240ca6eafb1245e078c4de05d0c9c50f7a4b29d34ee719261c3a83a47267f5c631c207c371d109f8e35e5a467ae1156b88cbf4d972b509391857729b1b2a","name":"idiom2-3.png","type":"image/png","lastModified":1680903062517.607,"size":119351},{"id":"190df961740eab96520d2404cc897d5c4af400e2c9c209842d0fb3bc008b706fe39d9784016b89d6ff2267c250b163c172966b3d020eadcd4f79293dc131f66f","href":"https://static.observableusercontent.com/files/190df961740eab96520d2404cc897d5c4af400e2c9c209842d0fb3bc008b706fe39d9784016b89d6ff2267c250b163c172966b3d020eadcd4f79293dc131f66f","name":"idiom3.png","type":"image/png","lastModified":1680917290085.903,"size":307077},{"id":"ced2bf102146510a4dfa03586ece73e44ad017ec3a6c3638af59af7b5cc099999be36c2004fd7fc32b8620d86a9149bc63fc8cabed848630a117f0e17c944f5e","href":"https://static.observableusercontent.com/files/ced2bf102146510a4dfa03586ece73e44ad017ec3a6c3638af59af7b5cc099999be36c2004fd7fc32b8620d86a9149bc63fc8cabed848630a117f0e17c944f5e","name":"idiom4.png","type":"image/png","lastModified":1680917584039.007,"size":875438},{"id":"9037aa3853387505586e8d9e1935dfabe8166fb2c2682f4e26cb035c2e2227788bd6e37ffecb3c29d3f908016b3a66f0f07bbf5572d7d639334b7d43129fe5a1","href":"https://static.observableusercontent.com/files/9037aa3853387505586e8d9e1935dfabe8166fb2c2682f4e26cb035c2e2227788bd6e37ffecb3c29d3f908016b3a66f0f07bbf5572d7d639334b7d43129fe5a1","name":"idiom5.png","type":"image/png","lastModified":1680925557959.012,"size":657137}] 2f:{"id":"6ad3422c4b68ce96","slug":null,"title":"Final Project Progress Report","thumbnail":"400cc41d74c2dbd1f4be5eb64bc2f83afe438292198fc1b69d5a6ca4084de8aa","access_level":"public","create_time":"$D2023-04-03T05:35:57.830Z","update_time":"$D2023-04-08T04:57:14.729Z","view_time":null,"first_public_time":"$D2023-04-03T05:35:57.830Z","has_unpublished_edits":null,"fork":{"id":"db9af3217c652752","slug":null,"title":"Final Project Proposal","owner":{"id":"f9d2e396ea9c36e7","login":"zychen","name":"Zhenyu Chen"}},"stars":0,"starred":false,"owner":{"type":"team","id":"f9d2e396ea9c36e7","login":"zychen","name":"Zhenyu Chen","avatar_url":"https://avatars-next.observableusercontent.com/bd7f5338dd83e2af26a89fd5f8b3b8f2e371ac3333067f03ed98213201b91502","tier":"starter_2024"},"version":803,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"750bcf843c0f1afd","login":"zychen","name":"Zhenyu Chen","avatar_url":"https://avatars-next.observableusercontent.com/bd7f5338dd83e2af26a89fd5f8b3b8f2e371ac3333067f03ed98213201b91502","tier":null}],"members":[{"type":"individual","id":"750bcf843c0f1afd","login":"zychen","name":"Zhenyu Chen","avatar_url":"https://avatars-next.observableusercontent.com/bd7f5338dd83e2af26a89fd5f8b3b8f2e371ac3333067f03ed98213201b91502","tier":null,"role":"editor"}],"comments":[{"id":"c6794bf5bcc04aba","reply_to_id":null,"content":"Hi Zhenyu, 你可以介绍下是如何归类CS_related的吗","create_time":"$D2023-04-07T02:52:22.774Z","update_time":null,"resolved":true,"user":{"type":"individual","id":"fce8939dac2f5d1a","login":"jinpeng","name":"Jinpeng Li","avatar_url":"https://avatars-next.observableusercontent.com/498aa3d9bf5a2b9e06554aa43e8d999b7808c9ceb759421408080d552d5c3b38","tier":null}},{"id":"66a9143d939cf4ef","reply_to_id":"c6794bf5bcc04aba","content":"目前的办法很土，就是把SOC_Title里有computer/software/programmer字样的就直接判定为CS_related. 之后可能会根据SOC的大分类标准准确的重新分类一下。","create_time":"$D2023-04-07T03:25:39.922Z","update_time":null,"resolved":true,"user":{"type":"individual","id":"750bcf843c0f1afd","login":"zychen","name":"Zhenyu Chen","avatar_url":"https://avatars-next.observableusercontent.com/bd7f5338dd83e2af26a89fd5f8b3b8f2e371ac3333067f03ed98213201b91502","tier":null}},{"id":"6df4f8528035dd31","reply_to_id":"c6794bf5bcc04aba","content":"哈哈，but it's useful，谢谢～","create_time":"$D2023-04-07T03:31:17.651Z","update_time":null,"resolved":true,"user":{"type":"individual","id":"fce8939dac2f5d1a","login":"jinpeng","name":"Jinpeng Li","avatar_url":"https://avatars-next.observableusercontent.com/498aa3d9bf5a2b9e06554aa43e8d999b7808c9ceb759421408080d552d5c3b38","tier":null}}],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"title":"Final Project Progress Report","cells":[{"id":0,"value":"# Final Project Progress Report\nBy [Zhenyu Chen](https://observablehq.com/@zychen) for Professor John Alexis Guerra Gómez' Spring 2023 section of [CS7295: Special Topics in Data Visualization](https://johnguerra.co/classes/data_visualization_spring_2023/). \n\n[Slides](https://docs.google.com/presentation/d/1VC_3P0kb7Xi2iN_6w__OLOegeQjvMlReX3hJZnO4LwY/edit?usp=sharing)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":15,"value":"$31","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":85,"value":"## 2. Data\n### Main Data\nH1B Petition Records (2017-2022): https://www.kaggle.com/datasets/jishnukoliyadan/lca-programs-h1b-h1b1-e3-visa-petitions\n\n### Supporting Data\n\nUnemployment Rate: https://www.statista.com/statistics/193290/unemployment-rate-in-the-usa-since-1990/\n\nE-Verify Data: https://www.e-verify.gov/about-e-verify/e-verify-data/e-verify-usage-statistics\n\n\"Blue or Red States\" Data (election data): https://www.politico.com/2020-election/results/president/\n\nBelow is a small extract of the main data. Unimportant columns are hidden.\n\nAll the dataset type/availability, attributes, are in each idea's data section.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":53,"value":"lca_fy_2022 = await FileAttachment(\"LCA_FY_2022.csv\").csv({typed: \"auto\"})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"lca_fy_2022","database":"$undefined"},{"id":63,"value":"## 3. Example Insights","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":67,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":255,"value":"### Prototypes\nBelow are 2 temporal stacked area charts from 2017 to 2022, for CS-related and non CS-related jobs. The first one is by total number, the second one is by percentage.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":259,"value":"key = Swatches(idiom1.scales.color, {columns: \"180px\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":231,"value":"idiom1 = StackedAreaChart(idiom1Data, {\n x: d => d.year,\n y: d => d.count,\n z: d => d.CS_related,\n xType: d3.scaleSequential,\n xFormat: d3.format(\".0f\"),\n yLabel: \"↑ H1B Petitions\",\n yFormat: d3.format(\"s\"),\n width: 500,\n height: 500\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":437,"value":"idiom1_percentage = StackedAreaChart(idiom1Data, {\n x: d => d.year,\n y: d => d.count,\n z: d => d.CS_related,\n xType: d3.scaleSequential,\n xFormat: d3.format(\".0f\"),\n yLabel: \"↑ H1B Petitions %\",\n yFormat: d3.format(\"%\"),\n width: 500,\n height: 500,\n offset: d3.stackOffsetExpand\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":377,"value":"### Iterations\nAt the very beginning, the scale isn't right.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":380,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":392,"value":"Then I changed the scale, now here's another problem, there are too many ticks on the x-axis. So I need to remove them.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":387,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":394,"value":"Then I tried to add an extra vertical axis of percentages on the right, but it seems not okay to do so. So I removed it.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":353,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":444,"value":"Finally, in order to show proportions of different job titles, I added another percentage stacked area chart. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":446,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":404,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":576,"value":"### Prototypes","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":564,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":578,"value":"### Iterations","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":603,"value":"At the beginning, the tick format of right axis isn't quite okay. It's better to use percentage for unemployment rate.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":596,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":606,"value":"After changing the tick format on the right axis, now the whole chart is too big and some % characters are cut out partially. Besides, there is no label.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":599,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":610,"value":"Finally, I changed the chart size and added labels on the two vertical axes.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":601,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":624,"value":"$35","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":681,"value":"### Mockups","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":684,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":687,"value":"$36","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":694,"value":"## Mockups","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":696,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":740,"value":"$37","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":770,"value":"## Mockups","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":772,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":141,"value":"$38","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":264,"value":"## Appendices","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":266,"value":"### Imports","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":233,"value":"idiom1Data = FileAttachment(\"idiom1.csv\").csv({typed: true})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":271,"value":"$39","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":262,"value":"import {Swatches} from \"@d3/color-legend\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":520,"value":"countAllPetitions = (data, year) => {\n const targetYear = data.filter(d => d.year === year).map(d => d.count)\n return targetYear.reduce((prev, cur) => (prev + cur))\n\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":540,"value":"unemploymentRate = ({2017: 0.044, 2018: 0.039, 2019: 0.037, 2020: 0.081, 2021: 0.053, 2022: 0.036})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":513,"value":"idiom2Data = d3.range(2017, 2023).map(year => ({year, count: countAllPetitions(idiom1Data, year), unemployment: unemploymentRate[year]}))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":489,"value":"margin = ({top: 20, right: 30, bottom: 30, left: 40})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":492,"value":"height = 500","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":494,"value":"y2Axis = g => g\n .attr(\"transform\", `translate(${width - margin.right},0)`)\n .call(d3.axisRight(y2).ticks(null, \"%\"))\n .call(g => g.select(\".domain\").remove())\n .call(g => g.append(\"text\")\n .attr(\"x\", margin.right)\n .attr(\"y\", 10)\n .attr(\"fill\", \"currentColor\")\n .attr(\"text-anchor\", \"end\")\n .text(\"Unemployment Rate\"))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":496,"value":"y1Axis = g => g\n .attr(\"transform\", `translate(${margin.left},0)`)\n .style(\"color\", \"steelblue\")\n .call(d3.axisLeft(y1).ticks(null, \"s\"))\n .call(g => g.select(\".domain\").remove())\n .call(g => g.append(\"text\")\n .attr(\"x\", -margin.left)\n .attr(\"y\", 10)\n .attr(\"fill\", \"currentColor\")\n .attr(\"text-anchor\", \"start\")\n .text(\"H1B Petition Count\"))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":498,"value":"xAxis = g => g\n .attr(\"transform\", `translate(0,${height - margin.bottom})`)\n .call(d3.axisBottom(x)\n .tickValues(d3.ticks(...d3.extent(x.domain()), width / 40).filter(v => x(v) !== undefined))\n .tickSizeOuter(0))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":500,"value":"y2 = d3.scaleLinear()\n .domain(d3.extent(idiom2Data, d => d.unemployment))\n .rangeRound([height - margin.bottom, margin.top])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":502,"value":"y1 = d3.scaleLinear()\n .domain([0, d3.max(idiom2Data, d => d.count)])\n .rangeRound([height - margin.bottom, margin.top])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":504,"value":"x = d3.scaleBand()\n .domain(idiom2Data.map(d => d.year))\n .rangeRound([margin.left, width - margin.right])\n .padding(0.1)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":506,"value":"line = d3.line()\n .x(d => x(d.year) + x.bandwidth() / 2)\n .y(d => y2(d.unemployment))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W3a","stdlib":"1","titling":"auto"}} 30:"$2f:body"