30:Ted9,| Name | Category | Status | Description | API | Examples | | - | - | - | - | - | - | | Sort | Transform | ✔️ | Sorting values in ascending or descending order for ${hyperlink("transform_tables_rearrange", "rearranging tables")}. | `var = ascsort(var)`
`var = descsort(var)` | `input = [3009.4, 2589.1, 2881.3]`
`ascsort(input) = [2589.1, 2881.3, 3009.4]`
`descsort(input) = [3009.4, 2881.3, 2589.1]` | Bin | Transform | ✔️ | Binning values into a set of buckets for ${hyperlink("combine_rows_summarize", "summarizing rows")}. | `var = bin(var, step, lowerbound, upperbound)` | `input = [3009.4, 2589.1, 2881.3]`
`bin(input, 3) = ["[2589.1, 2729.2)", "[2729.2, 2869.3)", "[2869.3, 3009.4]"]`
`bin(input, 3, 2500, 3100) = ["[2500, 2700)", "[2700, 2900)", "[2900, 3100]"]` | Sum | Transform | ✔️ | Computing the sum of values to aggregate them for ${hyperlink("combine_rows_summarize", "summarizing rows")}. | `var = sum(var)` | `input = [3009.4, 2589.1, 2881.3]`
`sum(input) = 8479.8` | Average | Transform | ✔️ | Computing the average of values to aggregate them for ${hyperlink("combine_rows_summarize", "summarizing rows")}. | `var = average(var)` | `input = [3009.4, 2589.1, 2881.3]`
`average(input) = 2826.6` | Count | Transform | ✔️ | Computing the number of values to aggregate them for ${hyperlink("combine_rows_summarize", "summarizing rows")}. | `var = count(var)` | `input = [3009.4, 2589.1, 2881.3]`
`count(input) = 3` | Filter | Transform | ✔️ | Filtering values by given conditions for ${hyperlink("delete_rows", "deleting rows")}. | For categorical variables: `var = filter(var, value_1, value_2, ...)`
For quantitative variables: `var = filter(var, lowerbound, upperbound)` | `input = [3009.4, 2589.1, 2881.3]`
`filter(var, 2600, 2900) = [2881.3]`
`input2 = ["a", "b", "c"]`
`filter(input2, "b", "c") = ["b", "c"]` | Format | Transform | Developing | Formatting values according to a given format, which is usually used for ${hyperlink("transform_columns", "transforming columns")}. | `var = Format(var, flag)` | `input = [3009.4, 2589.1, 2881.3]`
`format(input, "NumberInThousands") = ["3.0k", "2.6k", "2.9k"]` | Unnest | Transform | Developing | Unnesting arrays or sets of values for ${hyperlink("separate_rows", "separating rows")}. | `var = unnest(var)` | `input = [["Alabama", "Alaska", "Columbia"], ["Alabama", "Arizona"]]`
`unnest(input, ',') = ["Alabama", "Alaska", "Columbia", "Arizona"]` | Split | Separate | ✔️ | Splitting a string variable by a given separator into a list of variables for ${hyperlink("separate_columns", "separating columns")}. | `var = split(var, pattern)[index]` | `input = ["A-123", "B-233", "C-190"]`
`split(input, '-')[0] = ["A", "B", "C"]`
`split(input, '-')[1] = ["123", "233", "190"]` | Concatenate | Combine | ✔️ | Concatenating multiple string variables into one by a given delimiter for ${hyperlink("combine_columns", "combining columns")}. | `var = concat(var, delimiter)` | `input = ["a", "b", "c"]`
`concat(input, ',') = "a,b,c"` | Union| Combine | ✔️ | Computing the union of multiple variables for ${hyperlink("combine_tables_extend", "extending tables")} and ${hyperlink("combine_tables_supplement", "supplementing tables")}. | var = union(var_1, var_2) | `input1 = ["Alabama", "Alaska", "Columbia"], input2 = ["Alabama", "Arizona"]`
`union(input1, input2) = ["Alabama", "Alaska", "Columbia", "Arizona"]` | Intersection | Combine | ✔️ | Compute the intersection of multiple variables for ${hyperlink("combine_tables_match", "matching tables")}. | var = intersection(var_1, var_2) | `input1 = ["Alabama", "Alaska", "Columbia"], input2 = ["Alabama", "Arizona"]`
`intersection(input1, input2) = ["Alabama"]`31:Taf1,| | Table | Row | Column | | --------- | ------------------------------------------------------------ | ------------------------------------------------------- | ---------------------------------- | | Create | ✔️${hyperlink("create_tables", "Create Tables")} (Mapping) | ✔️${hyperlink("create_rows", "Create Rows")} (UI) | ✔️${hyperlink("create_columns", "Create Columns")} (Mapping) | | Delete | ✔️${hyperlink("delete_tables", "Delete Tables")} (Mapping) | ✔️${hyperlink("delete_rows", "Delete Rows")} (Filter) | ✔️${hyperlink("delete_columns", "Delete Columns")} (Mapping) | | Transform | ✔️${hyperlink("transform_tables_rearrange", "Rearrange Tables")}
(Mapping / Sort)
✔️${hyperlink("transform_tables_reshape", "Reshape Tables")} (Mapping) | ✔️${hyperlink("transform_rows", "Transform Rows")} (UI) | ✔️${hyperlink("transform_columns", "Transform Columns")} (Format) | | Separate | ✔️${hyperlink("separate_tables_subset", "Subset Tables")}
(Filter + Mapping)
✔️${hyperlink("separate_tables_decompose", "Decompose Tables")}
(Filter + Mapping)
✔️${hyperlink("separate_tables_split", "Split Tables")} (Mapping) | ✔️${hyperlink("separate_rows", "Separate Rows")}
(Unnest + Mapping) | ✔️${hyperlink("separate_columns", "Separate Columns")}
(Split + Mapping) | | Combine | ✔️${hyperlink("combine_tables_extend", "Extend Tables")}
(Union + Mapping)
✔️${hyperlink("combine_tables_supplement", "Supplement Tables")}
(Union + Mapping)
✔️${hyperlink("combine_tables_match", "Match Tables")}
(Intersection + Mapping) | ✔️${hyperlink("combine_rows_summarize", "Summarize Rows")}
(Aggregate + Mapping)
❌${hyperlink("combine_rows_interpolate", "Interpolate Rows")} | ✔️${hyperlink("combine_columns", "Combine Columns")}
(Concatenate + Mapping) | - 11 tasks can be simply completed by either functions, mappings, or directly editing in the user interface and are mainly distributed in the categories of _Create_, _Delete_, and _Transform_. - 6 tasks can be realized by just mapping variables into different channels. - 2 tasks can be realized by just applying derivation functions. - 1 task can be realized by just mapping or applying functions. - 2 tasks can be realized by directly editing in the user interface. - 9 tasks in the categories of _Separate_ and _Combine_ are complex and should be completed by combining functions and mappings. - 1 task cannot be completed currently.32:T497,{ const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: ["4029.3", "3900", "3370.9", "3615"] }); const s = aq.table({ " ": ['2004', '2005'], " Alabama": ["4029.3", "3900"], " Alaska": ["3370.9", "3615"] }); const p = aq.table({ " ": ['[3000, 4000)', '[4000, 5000)'], " Alabama": [1, 1], " Alaska": [2, 0] }); const opt_t = { color: { Year: (i, r) => i < 0 ? null : categories[0], Crime: (i, r) => i < 0 ? null : categories[1], // " 2004": (i, r) => i < 0 ? null : categories[2], // " 2005": (i, r) => i < 0 ? null : categories[3], } }; let table_height = 5; let spec_t = "(State × Year), () ⟶ (Crime)"; let spec_s = "(Year), (State) ⟶ (Crime)"; let spec_p = "(Bin(Crime)), (State) ⟶ (Count(Year))"; return hconcat( toTableView(t.view(opt_t), spec_t), html`

→

`, toTableView(s.view({ color: { " ": i => i < 0 ? null : categories[0], " Alabama": i => i < 0 ? null : categories[1], " Alaska": i => i < 0 ? null : categories[1] } }), spec_s, 20), ); }33:T49c,{ const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: ["4029.3", "3900", "3370.9", "3615"] }); const s = aq.table({ " ": ['2004', '2005'], " Alabama": ["4029.3", "3900"], " Alaska": ["3370.9", "3615"] }); const p = aq.table({ " ": ['[3000, 4000)', '[4000, 5000)'], " Alabama": [1, 1], " Alaska": [2, 0] }); const opt_t = { color: { Year: (i, r) => i < 0 ? null : categories[0], Crime: (i, r) => i < 0 ? null : categories[1], // " 2004": (i, r) => i < 0 ? null : categories[2], // " 2005": (i, r) => i < 0 ? null : categories[3], } }; let table_height = 5; let spec_t = "(State × Year), () ⟶ (Crime)"; let spec_s = "(Year), (State) ⟶ (Crime)"; let spec_p = "(Bin(Crime, ...)), (State) ⟶ (Count(Year))"; return hconcat( toTableView(t.view(opt_t), spec_t), html`

→

`, toTableView(p.view({ color: { " ": i => i < 0 ? null : categories[1], " Alabama": i => i < 0 ? null : categories[0], " Alaska": i => i < 0 ? null : categories[0] } }), spec_p, 20), ); }34:T498,{ const t = aq.table({ Name: ["Nile", "Jean", "Bach", "Peter"], Number: ["Fax:645-8397", "Tel:637-2985", "Fax:559-0178", "Tel:528-1132"] }); const s = aq.table({ "Filter(Name, 'Nile', 'Jean')": ["Nile", "Jean"], Number: ["Fax:645-8397", "Tel:637-2985"] }); const r = aq.table({ "Filter(Name, 'Bach', 'Peter')": ["Bach", "Peter"], Number: ["Fax:559-0178", "Tel:528-1132"] }); const opt = { color: { Name: (i, r) => i < 0 ? null : categories[0], Number: (i, r) => i < 0 ? null : categories[1], "Filter(Name, 'Nile', 'Jean')": (i, r) => i < 0 ? null : categories[2], "Filter(Name, 'Bach', 'Peter')": (i, r) => i < 0 ? null : categories[3] } }; let table_height = 5; let spec_before = "(Name), () ⟶ (Number)"; let spec_s = "(Filter(Name, 'Nile', 'Jean'), () ⟶ (Number))"; let spec_r = "(Filter(Name, 'Bach', 'Peter'), () ⟶ (Number))" return hconcat( toTableView(t.view(opt), spec_before), html`

→

`, toTableView(s.view(opt), spec_s, 20), html`

`, toTableView(r.view(opt), spec_r, 20), ); }35:T475,{ const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1], GDP: [2169.6, 2193, 2002.3, 2004] }); const s = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1] }); const r = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], GDP: [2169.6, 2193, 2002.3, 2004] }); const opt = { color: { Year: (i, r) => i < 0 ? null : categories[0], Crime: (i, r) => i < 0 ? null : categories[1], GDP: (i, r) => i < 0 ? null : categories[2] } }; let table_height = 5; let spec_before = "(State × Year), () ⟶ (Crime + GDP)"; let spec_s = "(State × Year), () ⟶ (Crime)"; let spec_r = "(State × Year), () ⟶ (GDP)"; return hconcat( toTableView(t.view(opt), spec_before), html`

→

`, toTableView(s.view(opt), spec_s), html`

`, toTableView(r.view(opt), spec_r), ); }36:T411,{ const t = aq.table({ Authors: ["Bostock, Ogievetsky, Heer", "Arvind, Dominik, Kanit, Heer"], Papers: ["D3: Data-Driven Documents", "Vega-Lite: A Grammar of Interactive Graphics"], }); const s = aq.table({ Authors: ["Bostock", "Ogievetsky", "Heer", "Arvind", "Dominik", "Kanit"], Papers: ["D3: Data-Driven Documents", "D3: Data-Driven Documents", "D3: Data-Driven Documents, Vega-Lite: A Grammar of Interactive Graphics", "Vega-Lite: A Grammar of Interactive Graphics", "Vega-Lite: A Grammar of Interactive Graphics", "Vega-Lite: A Grammar of Interactive Graphics"], }); const opt = { color: { Papers: (i, r) => i < 0 ? null : categories[1], Authors: (i, r) => i < 0 ? null : categories[0], } }; let table_height = 8; let spec_before = "(Authors), () ⟶ (Papers)"; let spec_after = "(Unnest(Authors)), () => (Papers)"; return hconcat( toTableView(t.view(opt), spec_before, 50), html`

→

`, toTableView(s.view(opt), spec_after), ); }37:T4f6,{ const t = aq.table({ Name_1: ["Nile", "Jean"], Number_1: ["Fax:645-8397", "Tel:637-2985", ] }); const s = aq.table({ Name_2: ["Bach", "Peter"], Number_2: ["Fax:559-0178", "Tel:528-1132"] }); const r = aq.table({ "Union(Name_1, Name_2)": ["Nile", "Jean", "Bach", "Peter"], "Union(Number_1, Number_2)": ["Fax:645-8397", "Tel:637-2985", "Fax:559-0178", "Tel:528-1132"] }); const opt = { color: { Name_1: (i, r) => i < 0 ? null : categories[0], Number_1: (i, r) => i < 0 ? null : categories[1], Name_2: (i, r) => i < 0 ? null : categories[2], Number_2: (i, r) => i < 0 ? null : categories[3], "Union(Name_1, Name_2)": (i, r) => i < 0 ? null : categories[4], "Union(Number_1, Number_2)": (i, r) => i < 0 ? null : categories[5] } }; let table_height = 5; let spec_t = "(Name_1), () ⟶ (Number_1)"; let spec_s = "(Name_2), () ⟶ (Number_2)"; let spec_after = "(Union(Name_1, Name_2)), () ⟶ (Union(Number_1, Number_2))"; return hconcat( toTableView(t.view(opt), spec_t, 20), html`

`, toTableView(s.view(opt), spec_s, 20), html`

→

`, toTableView(r.view(opt), spec_after), ); }38:T44c,{ const t = aq.table({ State_1: ["Alabama", "Alaska"], GDP: ["10044.5", "8897.3", ] }); const s = aq.table({ State_2: ["Alabama", "Alaska"], Crime: ["3933.5", "3507.1"] }); const r = aq.table({ "Union(State_1, State_2)": ["Alabama", "Alaska"], GDP: ["10044.5", "8897.3", ], Crime: ["3933.5", "3507.1"] }); const opt = { color: { State_1: (i, r) => i < 0 ? null : categories[0], State_2: (i, r) => i < 0 ? null : categories[2], GDP: (i, r) => i < 0 ? null : categories[3], Crime: (i, r) => i < 0 ? null : categories[4], "Union(State_1, State_2)": (i, r) => i < 0 ? null : categories[5], } }; let table_height = 3; let spec_t = "(State_1), () ⟶ (GDP)"; let spec_s = "(State_2), () ⟶ (Crime)"; let spec_after = "(Union(State_1, State_2)), () ⟶ (GDP + Crime)"; return hconcat( toTableView(t.view(opt), spec_t), html`

`, toTableView(s.view(opt), spec_s), html`

→

`, toTableView(r.view(opt), spec_after), ); }39:T449,{ const t = aq.table({ State_1: ["Alabama", "Alaska"], GDP: ["10044.5", "8897.3", ] }); const s = aq.table({ State_2: ["Alabama", "New Jersey"], Crime: ["3933.5", "3507.1"] }); const r = aq.table({ "Intersection(State_1, State_2)": ["Alabama"], GDP: ["10044.5"], Crime: ["3933.5"] }); const opt = { color: { State_1: (i, r) => i < 0 ? null : categories[0], State_2: (i, r) => i < 0 ? null : categories[2], GDP: (i, r) => i < 0 ? null : categories[3], Crime: (i, r) => i < 0 ? null : categories[4], "Intersection(State_1, State_2)": (i, r) => i < 0 ? null : categories[5], } }; let table_height = 3; let spec_t = "(State_1), () ⟶ (GDP)"; let spec_s = "(State_2), () ⟶ (Crime)"; let spec_after = "(Intersection(State_1, State_2)), () ⟶ (GDP + Crime)"; return hconcat( toTableView(t.view(opt), spec_t), html`

`, toTableView(s.view(opt), spec_s), html`

→

`, toTableView(r.view(opt), spec_after, 10), ); }3a:T555,{ const t = aq.table({ "Title": ["Table Scraps: An Actionable Framework for Multi-Table Data Wrangling From An Artifact Study of Computational Journalism", "Voyager: Exploratory Analysis via Faceted Browsing of Visualization Recommendations", "Voyager 2: Augmenting Visual Analysis with Partial View Specifications", "Vega-Lite: A Grammar of Interactive Graphics", "Wrangler: interactive visual specification of data transformation scripts"], "Year": [2021, 2016, 2017, 2017, 2011], "Venue": ["IEEE TVCG", "IEEE TVCG", "ACM CHI", "IEEE TVCG", "ACM CHI"], "Author": ["Stephen Kasica, Charles Berret, Tamara Munzner", "Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay, Bill Howe, Jeffrey Heer", "Kanit Wongsuphasawat, Zening Qu, Dominik Moritz, Riley Chang, Felix Ouk, Anushka Anand, Jock Mackinlay, Bill Howe, Jeffrey Heer", "Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, Jeffrey Heer", "Sean Kandel, Andreas Paepcke, Joseph Hellerstein, Jeffrey Heer"], "Citation": [12, 444, 212, 545, 730]}); const opt = { color: { Year: (i, r) => i < 0 ? null : categories[0], Venue: (i, r) => i < 0 ? null : categories[1], Author: (i, r) => i < 0 ? null : categories[2], Citation: (i, r) => i < 0 ? null : categories[3], } }; let table_height = 5; return hconcat( toTableView(t.view(opt), ""), ); }3b:T628,{ const t = aq.table({ "Title": ["Table Scraps: An Actionable Framework for Multi-Table Data Wrangling From An Artifact Study of Computational Journalism", "Voyager: Exploratory Analysis via Faceted Browsing of Visualization Recommendations", "Voyager 2: Augmenting Visual Analysis with Partial View Specifications", "Vega-Lite: A Grammar of Interactive Graphics", "Wrangler: interactive visual specification of data transformation scripts"], "Abbr Title": ["Table Scraps", "Voyager", "Voyager 2", "Vega-Lite", "Wrangler"], "Year": [2021, 2016, 2017, 2017, 2011], "Venue": ["IEEE TVCG", "IEEE TVCG", "ACM CHI", "IEEE TVCG", "ACM CHI"], "Author": ["Stephen Kasica, Charles Berret, Tamara Munzner", "Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay, Bill Howe, Jeffrey Heer", "Kanit Wongsuphasawat, Zening Qu, Dominik Moritz, Riley Chang, Felix Ouk, Anushka Anand, Jock Mackinlay, Bill Howe, Jeffrey Heer", "Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, Jeffrey Heer", "Sean Kandel, Andreas Paepcke, Joseph Hellerstein, Jeffrey Heer"], "Citation": [12, 444, 212, 545, 730]}); const opt = { color: { Year: (i, r) => i < 0 ? null : categories[0], Venue: (i, r) => i < 0 ? null : categories[1], Author: (i, r) => i < 0 ? null : categories[2], Citation: (i, r) => i < 0 ? null : categories[3], "Abbr Title": (i, r) => i < 0 ? null : categories[4], } }; let table_height = 5; return hconcat( toTableView(t.view(opt), "(Title), (), (Split(Title, ':')[0] + Year + Venue + Author + Citation)"), ); }3c:Teb3,The worst-case time complexity for our proposed declarative mapping approach is ${tex`O(cN^{1+a+b}M)`}, where the input tables have ${tex`N`} rows and ${tex`M`} columns in total, and the user-specified mapping have ${tex`a`}, ${tex`b`}, and ${tex`c`} variables in the row, column, and cell channel respectively. Based on the Equation 4 and 5, the mapping can be denoted as ${tex`(v_1 \times v_2 \times ... \times v_a), (v_{a+1} \times ... \times v_{a+b}) \rightarrow (v_{a+b+1} + ... + v_{a+b+c})`}. In particular, the ${tex`a`} and ${tex`b`} cannot both be zero and are usually not very large (${tex`< 5`}) in practice to avoid tables with too many empty cells like Fig. 6 and 7. Therefore, the worst-case time complexity is ${tex`\Omega(N^3M)`} for cross-tabulation (${tex`a > 0`}, ${tex`b > 0`}, and ${tex`c = 1`}) and is ${tex`\Omega(cN^2M)`} for other tables (either ${tex`a = 0`} or ${tex`b = 0`}, and ${tex`c \geq 1`}). We analyze the complexity based on the two phases of the declarative mapping approach: _extract_ data variables and relations and _map_ variables to table channels. - The extract stage scans the input data such that the complexity is ${tex`O(N \times M)`}. - The map stage derives and maps variables to table channels, and the worst-case time complexity is ${tex`O(cN^{1+a+b}M)`}: - The worst-case time complexity of variable derivations is ${tex`O(N^2)`}. Specifically, the functions of variable derivation handles one-dimensional data variables comprising a set of data values, the complexity of most derivation functions is ${tex`O(N)`} except _sort_ (${tex`O(N^2)`}). In addition, the output variables of these functions have at most ${tex`N`} values. One exception is the _unnest_ function, which unnests a variable of nested objects (e.g., array and set) and produces a variable that has more than ${tex`N`} values. Moreover, the unnest function can be applied multiple times on multiple nested objects, and its complexity is hard to determine. Hence, we exclude the unnest function in the subsequent analysis. - Since the proposed approach enumerates the values of the variables mapped to the row and column channels, the worst-case time complexity of mapping variables to the row and column channels is ${tex`O(N^a) (a > b)`} or ${tex`O(N^b) (a \leq b)`}. - The worst-case time complexity of mapping variables to the cell channel is ${tex`O(cN^{1+a+b}M)`}. Specifically, the proposed approach queries the values of variables mapped to the cell channel. On the one hand, the queries require iterating over the input data, and the worst-case time complexity is ${tex`O(N \times M)`}. On the other hand, the cell channel comprises at most ${tex`cN^{a+b}`} cells (${tex`N^{a+b}`} cells for cross-tabulations where ${tex`c = 1`}, and ${tex`cN^a`} or ${tex`cN^b`} cells for other tables where ${tex`b = 0`} or ${tex`a = 0`}). We found that the major bottleneck is mainly from the queries, which takes ${tex`O(N \times M)`} time in each cell in the cell channel. Meanwhile, the traditional data operators fix the operations of cells and only take ${`O(1)`} on each cell. We plan to construct indexes between variables to reduce the complexity of these queries to close to ${`O(1)`} in the future. Since these queries frequently query a variable value by multiple other variables, data cube ${hyperlink("ref1", "[2]")} and related techniques may be appropriate. With the indexes, the worst-case time complexity will be reduced to ${tex`O(cN^{a+b})`}, which is comparable to traditional operators. In addition, the declarative mapping approach will prevent unnecessary computation and operation by skipping the intermediate tables of operation sequence and directly computing the resulting table.3d:T562,The current version of Rigel system supports two input data formats: **JSON** and **CSV**. ### JSON A sample input in JSON format is given as follows. The *name* field represents the name of the input table. The *values* field should be an array indicating records of the table. Each element should be an object corresponding to a record. ```json { "name": "crime", "values": [ { "state": "Alabama", "year": 2004, "crime": 4029.3 }, { "state": "Alabama", "year": 2005, "crime": 3900 }, { "state": "Alaska", "crime": 3370.9, "year": 2004 }, { "state": "Alaska", "crime": 3615, "year": 2005 } ] } ``` ### CSV A sample input in CSV format is given as follows. Note that the first line of the input file should include all the attribute tables, and there should NOT be any blank lines or redundant characters after the final line. ``` State,Year,Crime Alabama,2004,4029.3 Alabama,2005,3900 Alabama,2006,3937 Alabama,2007,3974.9 Alabama,2008,4081.9 District of Columbia,2004,4852.8 District of Columbia,2005,4490 District of Columbia,2006,4653.9 District of Columbia,2007,4916.3 District of Columbia,2008,5104.6 California,2004,3423.9 California,2005,3321 California,2006,3175.2 California,2007,3032.6 California,2008,2940.3 New Jersey,2004,2433 New Jersey,2005,2337 New Jersey,2006,2278.4 New Jersey,2007,2205.5 New Jersey,2008,2293.4 ``` 3e:T7a8,{ const t = aq.table({ Papers: ["D3: Data-Driven Documents","D3: Data-Driven Documents","D3: Data-Driven Documents","Lyra: An Interactive Visualization Design Environment","Lyra: An Interactive Visualization Design Environment", "Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization","Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization","Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization","Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization", "Vega-Lite: A Grammar of Interactive Graphics","Vega-Lite: A Grammar of Interactive Graphics","Vega-Lite: A Grammar of Interactive Graphics","Vega-Lite: A Grammar of Interactive Graphics",], Authors: ["Michael Bostock","Vadim Ogievetsky","Jeffrey Heer","Arvind Satyanarayan","Jeffrey Heer","Arvind Satyanarayan","Ryan Russell","Jane Hoffswell","Jeffrey Heer","Arvind Satyanarayan","Dominik Moritz","Kanit Wongsuphasawat","Jeffrey Heer",], }); const s = aq.table({ Papers: ["D3: Data-Driven Documents", "Lyra: An Interactive Visualization Design Environment", "Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization", "Vega-Lite: A Grammar of Interactive Graphics"], Authors: ["Michael Bostock,Vadim Ogievetsky,Jeffrey Heer", "Arvind Satyanarayan, Jeffrey Heer", "Arvind Satyanarayan, Ryan Russell, Jane Hoffswell, Jeffrey Heer", "Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, Jeffrey Heer"], }); const opt = { color: { Papers: (i, r) => i < 0 ? null : categories[1], Authors: (i, r) => i < 0 ? null : categories[0], } }; let table_height = 16; let spec_before = ""; let spec_after = ""; return hconcat( toTableView(t.view(opt), spec_before), html`

→

`, toTableView(s.view(opt), spec_after, 100), ); }3f:T864,{ const t = aq.table({State: ["Alabama", "Alabama", "Alabama", "Alabama", "Alabama", "District of Columbia", "District of Columbia", "District of Columbia", "District of Columbia", "District of Columbia", "California", "California", "California", "California", "California", "New Jersey", "New Jersey", "New Jersey", "New Jersey", "New Jersey"], "Year": [2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008], "Crime_Rate": [4029.3, 3900, 3937, 3974.9, 4081.9, 4852.8, 4490, 4653.9, 4916.3, 5104.6, 3423.9, 3321, 3175.2, 3032.6, 2940.3, 2433, 2337, 2278.4, 2205.5, 2293.4]}); const s = aq.table({State: ["Alabama", "Alabama", "Alabama", "Alabama", "Alabama", "District of Columbia", "District of Columbia", "District of Columbia", "District of Columbia", "District of Columbia", "California", "California", "California", "California", "California", "New Jersey", "New Jersey", "New Jersey", "New Jersey", "New Jersey"], "Year": [2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008], "Crime_Rate": [4029.3, 3900, 3937, 3974.9, 4081.9, 4852.8, 4490, 4653.9, 4916.3, 5104.6, 3423.9, 3321, 3175.2, 3032.6, 2940.3, 2433, 2337, 2278.4, 2205.5, 2293.4], "Key": ["Alabama2004", "Alabama2005", "Alabama2006", "Alabama2007", "Alabama2008", "District of Columbia2004", "District of Columbia2005", "District of Columbia2006", "District of Columbia2007", "District of Columbia2008", "California2004", "California2005", "California2006", "California2007", "California2008", "New Jersey2004", "New Jersey2005", "New Jersey2006", "New Jersey2007", "New Jersey2008"]}); const opt = { color: { Year: (i, r) => i < 0 ? null : categories[1], "Crime_Rate": (i, r) => i < 0 ? null : categories[0], "Key": (i, r) => i < 0 ? null : categories[5] } }; let table_height = 16; let spec_before = ""; let spec_after = ""; return hconcat( toTableView(t.view(opt), spec_before), html`

→

`, toTableView(s.view(opt), spec_after), ); }40:T4ee,{ const t = aq.table({ Name: ["Niles C.", "Niles C.", "Niles C.", "Jean H.", "Jean H.", "Jean H.", "Jean H.", "Bach J.", "Bach J."], Type: ["Tel", "Tel", "Fax", "Tel", "Tel", "Fax", "Fax", "", ""], Number: ["(800)645-8397", "(800)645-8398", "(907)586-7252", "(918)781-4600", "(918)781-4601", "(918)781-4603", "(918)781-4604", "781-4605", "(918)781-4604"], Usage: ["home", "work", "work", "home", "work", "home", "work", "work", "work"]}); const s = aq.table({ " ": ["Tel", "Tel", "Fax", "Fax"], " ": ["home", "work", "home", "work"], "Nile C.": ["(800)645-8397", "(800)645-8398", " ", "(907)586-7252"], "Jean H.": ["(918)781-6400", "(918)781-6401", "(918)781-6403", "(918)781-6404"] }); const opt = { color: { Type: (i, r) => i < 0 ? null : categories[1], Number: (i, r) => i < 0 ? null : categories[0], Usage: (i, r) => i < 0 ? null : categories[2], "Nile C.": (i, r) => i < 0 ? null : categories[3], "Jean H.": (i, r) => i < 0 ? null : categories[4], } }; let table_height = 10; let spec_before = ""; let spec_after = ""; return hconcat( toTableView(t.view(opt), spec_before), html`

→

`, toTableView(s.view(opt), spec_after, 40), ); }41:T7ef,{ const t = aq.table({"State": ["Alabama", "Alabama", "Alabama", "Alabama", "Alabama", "District of Columbia", "District of Columbia", "District of Columbia", "District of Columbia", "District of Columbia", "California", "California", "California", "California", "California", "New Jersey", "New Jersey", "New Jersey", "New Jersey", "New Jersey"], "Year": [2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008, 2004, 2005, 2006, 2007, 2008], "Crime_Rate": [4029.3, 3900, 3937, 3974.9, 4081.9, 4852.8, 4490, 4653.9, 4916.3, 5104.6, 3423.9, 3321, 3175.2, 3032.6, 2940.3, 2433, 2337, 2278.4, 2205.5, 2293.4]}); const s = aq.table({"Alabama": ["2004", 4029.3], "Alabama ": ["2005", 3900], "Alabama ": ["2006", 3937], "Alabama ": ["2007", 3974.9], "Alabama ": ["2008", 4081.9], "District of Columbia": ["2004", 4852.8], "District of Columbia ": ["2005", 4490], "District of Columbia ": ["2006", 4653.9], "District of Columbia ": ["2007", 4916.3], "District of Columbia ": ["2008", 5104.6], "California": ["2004", 3423.9], "California ": ["2005", 3321], "California ": ["2006", 3175.2], "California ": ["2007", 3032.6], "California ": ["2008", 2940.3], "New Jersey": ["2004", 2433], "New Jersey ": ["2005", 2337], "New Jersey ": ["2006", 2278.4], "New Jersey ": ["2007", 2205.5], "New Jersey ": ["2008", 2293.4]}); const opt = { color: { Year: (i, r) => i < 0 ? null : categories[1], "Crime_Rate": (i, r) => i < 0 ? null : categories[0], "Key": (i, r) => i < 0 ? null : categories[5] } }; const g = t.groupby("State"); const opt_after = { color: (i, r) => i < 0 ? null : categories[1-r] }; let table_height = 16; let spec_before = ""; let spec_after = ""; return hconcat( toTableView(t.view(opt), spec_before), html`

→

`, html`

${toTableView(s.view(opt_after), spec_after)}

`, ); }42:T6ba,toView = { const DEFAULT_LIMIT = 100; const DEFAULT_NULL = value => `${value}`; const tableStyle = 'margin: 0 auto; border-collapse: collapse; width: initial; overflow-x: scroll;'; const cellStyle = 'padding: 1px 5px; white-space: nowrap; overflow-x: hidden; text-overflow: ellipsis; font-variant-numeric: tabular-nums; text-align: center;'; // extend table prototype to provide an HTML table view return function(dt, opt = {}) { // permit shorthand for limit if (typeof opt === 'number') opt = { limit: opt }; // marshal cell color options const color = { ...opt.color }; if (typeof opt.color === 'function') { // if function, apply to all columns dt.columnNames().forEach(name => color[name] = opt.color); } else { // otherwise, gather per-column color options for (const key in color) { const value = color[key]; color[key] = typeof value === 'function' ? value : () => value; } } // marshal CSS styles as toHTML() options const table = `${tableStyle}`; const td = (name, index, row) => { return `${cellStyle} max-width: ${+opt.maxCellWidth || 300}px;` + (color[name] ? ` background-color: ${color[name](index, row)};` : ''); }; opt = { limit: DEFAULT_LIMIT, null: DEFAULT_NULL, ...opt, style: { table, td, th: td } }; // return container div, bind table value to support viewof operator const size = `max-height: ${+opt.height || 270}px`; const style = `${size}; overflow-x: auto; overflow-y: auto;`; const view = html`

${dt.toHTML(opt)}

`; view.value = dt; return view; }; }43:[{"id":"519e9bbb3a7c5ea2899312e80ebacc1500696c5411cac75acc34f4453367f98845e9d64ebe628fd7a4169792c88257c432709fa510b002fe73415e7affe22553","href":"https://static.observableusercontent.com/files/519e9bbb3a7c5ea2899312e80ebacc1500696c5411cac75acc34f4453367f98845e9d64ebe628fd7a4169792c88257c432709fa510b002fe73415e7affe22553","name":"Empty set.png","type":"image/png","lastModified":1656581813888.199,"size":7832}] 2e:{"title":"APPENDIX","cells":[{"id":0,"value":"# APPENDIX","pinned":false,"hidden":false,"mode":"md"},{"id":443,"value":"## A. FUNCTIONS FOR VARIABLE DERIVATION","pinned":false,"hidden":false,"mode":"md"},{"id":27,"value":"$30","pinned":false,"hidden":false,"mode":"md"},{"id":416,"value":"## B. COVERAGE OVER KASICA ET AL.'S FRAMEWORK ${hyperlink(\"ref1\", \"[1]\")}","pinned":false,"hidden":false,"mode":"md"},{"id":447,"value":"$31","pinned":false,"hidden":false,"mode":"md"},{"id":38,"value":"create_tables = md`### Create Tables`","pinned":false,"hidden":false,"mode":"ojs"},{"id":218,"value":"{\n const t = aq.table({ ' ': [' ', ' ', ' ', ' '], ' ': [' ', ' ', ' ', ' '], ' ': [' ', ' ', ' ', ' ']});\n const s = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1]});\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n }\n };\n\n let table_height = 5;\n let spec_before = \"(), () ⟶ ()\";\n let spec_after = \"(State × Year), () ⟶ (Crime)\";\n return hconcat(\n html`

`,\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":131,"value":"create_columns = md`### Create Columns`","pinned":false,"hidden":false,"mode":"ojs"},{"id":524,"value":"{\n const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1] });\n const s = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1], \"GDP\": [2169.6, 2193, 2002.3, 2004]});\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n GDP: (i, r) => i < 0 ? null : categories[2]\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State × Year), () ⟶ (Crime)\";\n let spec_after = \"(State × Year), () ⟶ (Crime + GDP)\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":234,"value":"create_rows = md`### Create Rows`","pinned":false,"hidden":false,"mode":"ojs"},{"id":550,"value":"Insert values into empty cells in a row in the spreadsheet directly.","pinned":false,"hidden":false,"mode":"md"},{"id":243,"value":"delete_tables = md`### Delete Tables`","pinned":false,"hidden":false,"mode":"ojs"},{"id":553,"value":"{\n const s = aq.table({ ' ': [' ', ' ', ' ', ' '], ' ': [' ', ' ', ' ', ' '], ' ': [' ', ' ', ' ', ' ']});\n const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1]});\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n }\n };\n\n let table_height = 5;\n let spec_after = \"(), () ⟶ ()\";\n let spec_before = \"(State × Year), () ⟶ (Crime)\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n html`

`\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":248,"value":"delete_columns = md`### Delete Columns`","pinned":false,"hidden":false,"mode":"ojs"},{"id":250,"value":"{\n const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1], GDP: [2169.6, 2193, 2002.3, 2004] });\n const s = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1] });\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n GDP: (i, r) => i < 0 ? null : categories[2]\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State × Year), () ⟶ (Crime + GDP)\";\n let spec_after = \"(State × Year), () ⟶ (Crime)\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":254,"value":"delete_rows = md`### Delete Rows`","pinned":false,"hidden":false,"mode":"ojs"},{"id":380,"value":"{\n const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1]});\n const s = aq.table({ 'Filter(State, \"Alabama\")': ['Alabama', 'Alabama'], Year: [2004, 2005], Crime: [3009.4, 2589.3] });\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n GDP: (i, r) => i < 0 ? null : categories[2]\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State × Year), () ⟶ (Crime)\";\n let spec_after = \"(Filter(State, 'Alabama') × Year), () ⟶ (Crime)\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after, 20),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":256,"value":"transform_tables_rearrange = md`### Transform Tables - Rearrange`","pinned":false,"hidden":false,"mode":"ojs"},{"id":264,"value":"{\n const t = aq.table({ State: ['New Jersey', 'District of Columbia', 'California', 'Alabama'], Crime: [2433.0, 4852.8, 3423.9, 4029.3] });\n const s = aq.table({ State: ['Alabama', 'California', 'District of Columbia', 'New Jersey'], Crime: [4029.3, 3423.9, 4852.8, 2433.0] });\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State), () ⟶ (Crime)\";\n let spec_after = \"(Ascsort(State)), () ⟶ (Crime)\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":559,"value":"{\n const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1] });\n const s = aq.table({ Year: [2004, 2004, 2005, 2005], State: ['Alabama', 'Alaska', 'Alabama', 'Alaska'], Crime: [3009.4, 3178.2, 2589.3, 2204.1] });\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n GDP: (i, r) => i < 0 ? null : categories[2]\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State × Year), () ⟶ (Crime)\";\n let spec_after = \"(Year × State), () ⟶ (Crime)\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":270,"value":"tranform_tables_reshape = md`### Transform Tables - Reshape`","pinned":false,"hidden":false,"mode":"ojs"},{"id":280,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":601,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":291,"value":"transform_columns = md`### Transform Columns`","pinned":false,"hidden":false,"mode":"ojs"},{"id":289,"value":"{\n const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [3009.4, 2589.3, 3178.2, 2204.1] });\n const s = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], \"Format(Crime, ...)\": [\"3.0k\", \"2.6k\", \"3.2k\", \"2.2k\"] });\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n \"Format(Crime, ...)\": (i, r) => i < 0 ? null : categories[1]\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State × Year), () ⟶ (Crime)\";\n let spec_after = \"(State × Year), () ⟶ (Format(Crime, ...))\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":606,"value":"tranform_rows = md`### Transform Rows`","pinned":false,"hidden":false,"mode":"ojs"},{"id":610,"value":"Editing individual rows in the spreadsheet when raw data contained errors.","pinned":false,"hidden":false,"mode":"md"},{"id":299,"value":"separate_tables_subset = md`### Separate Tables - Subset`","pinned":false,"hidden":false,"mode":"ojs"},{"id":303,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":308,"value":"separate_tables_decompose = md`### Separate Tables - Decompose`","pinned":false,"hidden":false,"mode":"ojs"},{"id":387,"value":"{\n const t = aq.table({ Name: [\"Nile\", \"Jean\"], Number: [\"Fax:645-8397\", \"Tel:637-2985\"] });\n const s = aq.table({ \"Filter(Name, 'Nile')\": [\"Nile\"], Number: [\"Fax:645-8397\"] });\n const r = aq.table({ \"Filter(Name, 'Jean')\": [\"Jean\"], Number: [\"Tel:637-2985\"] });\n const opt = {\n color: {\n Name: (i, r) => i < 0 ? null : categories[0],\n Number: (i, r) => i < 0 ? null : categories[1],\n \"Filter(Name, 'Nile')\": (i, r) => i < 0 ? null : categories[2],\n \"Filter(Name, 'Jean')\": (i, r) => i < 0 ? null : categories[3]\n }\n };\n\n let table_height = 3;\n let spec_before = \"(Name), () ⟶ (Number)\";\n let spec_s = \"(Filter(Name, 'Nile'), () ⟶ (Number))\";\n let spec_r = \"(Filter(Name, 'Jean'), () ⟶ (Number))\"\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_s, 10),\n html`

`,\n toTableView(r.view(opt), spec_r, 10),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":310,"value":"separate_tables_split = md`### Separate Tables - Split`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":399,"value":"$35","pinned":false,"hidden":false,"mode":"ojs"},{"id":317,"value":"separate_rows = md`### Separate Rows`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":404,"value":"$36","pinned":false,"hidden":false,"mode":"ojs"},{"id":312,"value":"separate_columns = md`### Separate Columns`","pinned":false,"hidden":false,"mode":"ojs"},{"id":314,"value":"{\n const t = aq.table({ Name: [\"Nile\", \"Jean\", \"Bach\", \"Peter\"], Number: [\"Fax:645-8397\", \"Tel:637-2985\", \"Fax:559-0178\", \"Tel:528-1132\"] });\n const s = aq.table({ Name: [\"Nile\", \"Jean\", \"Bach\", \"Peter\"], \"Split(Number, ':')[0]\": [\"Fax\", \"Tel\", \"Fax\", \"Tel\"], \"Split(Number, ':')[1])\": [\"645-8397\", \"637-2985\", \"559-0178\", \"528-1132\"] });\n const opt = {\n color: {\n Name: (i, r) => i < 0 ? null : categories[0],\n Number: (i, r) => i < 0 ? null : categories[1],\n \"Split(Number, ':')[0]\": (i, r) => i < 0 ? null : categories[2],\n \"Split(Number, ':')[1])\": (i, r) => i < 0 ? null : categories[3]\n }\n };\n\n let table_height = 5;\n let spec_before = \"(Name), () ⟶ (Number)\";\n let spec_after = \"(Name), () => (Split(Number, ':')[0] + Split(Number, ':')[1])\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":319,"value":"combine_tables_extend = md`### Combine Tables - Extend`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":321,"value":"$37","pinned":false,"hidden":false,"mode":"ojs"},{"id":328,"value":"combine_tables_supplement = md`### Combine Tables - Supplement`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":330,"value":"$38","pinned":false,"hidden":false,"mode":"ojs"},{"id":338,"value":"combine_tables_match = md`### Combine Tables - Match`","pinned":false,"hidden":false,"mode":"ojs"},{"id":344,"value":"$39","pinned":false,"hidden":false,"mode":"ojs"},{"id":1124,"value":"combine_rows_summarize = md`### Combine Rows - Summarize`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":362,"value":"{\n const t = aq.table({ State: ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [\"4029.3\", \"3900\", \"3370.9\", \"3615\"] });\n const s = aq.table({ State: ['Alabama', 'Alaska'], \"Sum(Crime)\": [\"7929.3\", \"6985.9\"], \"Avg(Crime)\": [\"3964.7\", \"3493.0\"] });\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n \"Sum(Crime)\": (i, r) => i < 0 ? null : categories[2],\n \"Avg(Crime)\": (i, r) => i < 0 ? null : categories[3],\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State × Year), () ⟶ (Crime)\";\n let spec_after = \"(State), () ⟶ (Sum(Crime) + Avg(Crime))\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after, 20),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1128,"value":"{\n const t = aq.table({ 'State': ['Alabama', 'Alabama', 'Alaska', 'Alaska'], Year: [2004, 2005, 2004, 2005], Crime: [\"4029.3\", \"3900\", \"3370.9\", \"3615\"] });\n const s = aq.table({ 'Bin(Crime, ...)': ['[3000, 4000)', '[4000, 5000)'], \"Count(Year)\": [3, 1] });\n const opt = {\n color: {\n Year: (i, r) => i < 0 ? null : categories[0],\n Crime: (i, r) => i < 0 ? null : categories[1],\n \"Sum(Crime)\": (i, r) => i < 0 ? null : categories[2],\n \"Count(Year)\": (i, r) => i < 0 ? null : categories[4],\n 'Bin(Crime, ...)': (i, r) => i < 0 ? null : categories[3],\n }\n };\n\n let table_height = 5;\n let spec_before = \"(State × Year), () ⟶ (Crime)\";\n let spec_after = \"(Bin(Crime, ...)), () ⟶ (Count(Year))\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after, 20),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":680,"value":"combine_rows_interpolate = md`### Combine Rows - Interpolate`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":686,"value":"`Interpolate` operations directly retrieve and operate a part of cells, which will be realized in a future version of Rigel with a fine-grained mapping between values and cells instead of the current mapping between variables and table channels.","pinned":false,"hidden":false,"mode":"md"},{"id":350,"value":"combine_columns = md`### Combine Columns`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":354,"value":"{\n const t = aq.table({ Name: [\"Nile\", \"Jean\", \"Bach\", \"Peter\"], Type: [\"Fax\", \"Tel\", \"Fax\", \"Tel\"], Number: [\"645-8397\", \"637-2985\", \"559-0178\", \"528-1132\"] });\n const s = aq.table({ Name: [\"Nile\", \"Jean\", \"Bach\", \"Peter\"], \"Concat(Type, Number, ':')\": [\"Fax:645-8397\", \"Tel:637-2985\", \"Fax:559-0178\", \"Tel:528-1132\"] });\n const opt = {\n color: {\n Name: (i, r) => i < 0 ? null : categories[0],\n Type: (i, r) => i < 0 ? null : categories[1],\n Number: (i, r) => i < 0 ? null : categories[2],\n \"Concat(Type, Number, ':')\": (i, r) => i < 0 ? null : categories[3]\n }\n };\n\n let table_height = 5;\n let spec_before = \"(Name), () ⟶ (Type + Number)\";\n let spec_after = \"(Name), () => (Concat(Type, Number))\";\n return hconcat(\n toTableView(t.view(opt), spec_before),\n html`

→

`,\n toTableView(s.view(opt), spec_after),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":373,"value":"## C. EXTRA USE CASE EXAMPLES","pinned":false,"hidden":false,"mode":"md"},{"id":375,"value":"Consider Jane is given a dataset of publications for analysis. When the analysis starts, Jane has no clear transformation goals in mind, so she prefers to transform the dataset into different forms for insights. Part of the the dataset is as follows:","pinned":false,"hidden":false,"mode":"md"},{"id":720,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs"},{"id":728,"value":"To explore the dataset, jane may transform the above table into the following **nine** forms, and these tables help her understand the dataset from different aspects:","pinned":false,"hidden":false,"mode":"md"},{"id":755,"value":"{\n const t = aq.table({ \"Author\": [\"Stephen Kasica\", \"Charles Berret\", \"Tamara Munzner\", \"Kanit Wongsuphasawat\", \"Dominik Moritz\", \"Anushka Anand\", \"Jock Mackinlay\", \"Bill Howe\", \"Jeffrey Heer\", \"Zening Qu\", \"Riley Chang\", \"Felix Ouk\", \"Arvind Satyanarayan\", \"Sean Kandel\", \" Andreas Paepcke\", \"Joseph Hellerstein\"], \"Citation\": [12, 12, 12, 1201, 1201, 656, 656, 656, 1931, 212, 212, 212, 545, 730, 730, 730, 730]});\n const opt = {\n color: {\n \"Author\": (i, r) => i < 0 ? null : categories[2],\n \"Citation\": (i, r) => i < 0 ? null : categories[3],\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(unnest(Author)), () => (Sum(Citation))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":956,"value":"{\n const t = aq.table({ \"Author\": [\"Stephen Kasica\", \"Charles Berret\", \"Tamara Munzner\", \"Kanit Wongsuphasawat\", \"Dominik Moritz\", \"Anushka Anand\", \"Jock Mackinlay\", \"Bill Howe\", \"Jeffrey Heer\", \"Zening Qu\", \"Riley Chang\", \"Felix Ouk\", \"Arvind Satyanarayan\", \"Sean Kandel\", \" Andreas Paepcke\", \"Joseph Hellerstein\"], \"Count(Title)\": [1, 1, 1, 3, 3, 2, 2, 2, 4, 1, 1, 1, 1, 1, 1, 1, 1]});\n const opt = {\n color: {\n \"Author\": (i, r) => i < 0 ? null : categories[2],\n \"Count(Title)\": (i, r) => i < 0 ? null : categories[4],\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(unnest(Author)), () => (Count(Title))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":962,"value":"{\n const t = aq.table({ \"Bin(Year, 5)\": [\"[2011, 2013)\", \"[2013, 2015)\", \"[2015, 2017)\", \"[2017, 2019)\", \"[2019, 2021]\"], \"Count(Title)\": [1, 0, 1, 2, 1], \"Sum(Citation)\": [12, 0, 444, 757, 730]});\n const opt = {\n color: {\n \" \": (i, r) => i < 0 ? null : categories[2],\n \"Count(Title)\": (i, r) => i < 0 ? null : categories[3],\n \"Sum(Citation)\": (i, r) => i < 0 ? null : categories[4],\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(Bin(Year, 5)), () => (Count(Title) + Sum(Citation))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":976,"value":"{\n const t = aq.table({\" \": [\"[12, 371)\", \"[371, 730]\"], \"Concat(Split(Title, ':')[0])\": [\"Table Scraps,Voyager 2\", \"Voyager,Vega-Lite,Wrangler\"], \"Concat(Split(Author, ',')[0])\": [\"Stephen Kasica,Kanit Wongsuphasawat\", \"Kanit Wongsuphasawat,Arvind Satyanarayan,Sean Kandel\"]});\n const opt = {\n color: {\n \" \": (i, r) => i < 0 ? null : categories[2],\n \"Concat(Split(Author, ',')[0])\": (i, r) => i < 0 ? null : categories[3],\n \"Concat(Split(Title, ':')[0])\": (i, r) => i < 0 ? null : categories[5],\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(Bin(Citation, 2)), () => (Concat(Title, ',') + Concat(Split(Author, ',')[0]))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":732,"value":"{\n const t = aq.table({ \"Ascsort(Year)\": [2011, 2016, 2017, 2021], \"Sum(Citation)\": [730, 444, 757, 12]});\n const opt = {\n color: {\n \"Ascsort(Year)\": (i, r) => i < 0 ? null : categories[4],\n \"Sum(Citation)\": (i, r) => i < 0 ? null : categories[5],\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(Ascsort(Year)), () => (Sum(Citation))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":738,"value":"{\n const t = aq.table({ \"\": [2011, 2016, 2017, 2021], \"IEEE TVCG\": [0, 444, 545, 12], \"ACM CHI\": [730, 0, 212, 0]});\n const opt = {\n color: {\n \"IEEE TVCG\": (i, r) => i < 0 ? null : categories[6],\n \"ACM CHI\": (i, r) => i < 0 ? null : categories[7],\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(Ascsort(Year)), (Venue) => (Sum(Citation))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":743,"value":"$3b","pinned":false,"hidden":false,"mode":"ojs"},{"id":970,"value":"{\n const t = aq.table({ \"Author\": [\"Stephen Kasica\", \"Kanit Wongsuphasawat\", \"Arvind Satyanarayan\", \"Sean Kandel\"], \"Sum(Citation)\": [12 ,656, 545, 730], \"Count(Title)\": [1, 2, 1, 1]});\n const opt = {\n color: {\n \"Author\": (i, r) => i < 0 ? null : categories[2],\n \"Sum(Citation)\": (i, r) => i < 0 ? null : categories[3],\n \"Count(Title)\": (i, r) => i < 0 ? null : categories[5]\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(Split(Author, ',')[0]), () => (Sum(Citation) + Count(Title))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":750,"value":"{\n const t = aq.table({ \"Title\": [\"Table Scraps: An Actionable Framework for Multi-Table Data Wrangling From An Artifact Study of Computational Journalism\", \"Voyager: Exploratory Analysis via Faceted Browsing of Visualization Recommendations\", \"Voyager 2: Augmenting Visual Analysis with Partial View Specifications\", \"Vega-Lite: A Grammar of Interactive Graphics\", \"Wrangler: interactive visual specification of data transformation scripts\"], \"Concat(Venue, Year, ' ')\": [\"IEEE TVCG 2021\", \"IEEE TVCG 2016\", \"ACM CHI 2017\", \"IEEE TVCG 2017\", \"ACM CHI 2011\"]});\n const opt = {\n color: {\n \"Concat(Venue, Year, ' ')\": (i, r) => i < 0 ? null : categories[7],\n }\n };\n\n let table_height = 5;\n return hconcat(\n toTableView(t.view(opt), \"(Title), () => (Concat(Venue, Year, ' '))\"),\n );\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":512,"value":"## D. COMPLEXITY ANALYSIS","pinned":false,"hidden":false,"mode":"md"},{"id":521,"value":"$3c","pinned":false,"hidden":false,"mode":"md"},{"id":517,"value":"## E. INPUT DATA FORMAT","pinned":false,"hidden":false,"mode":"md"},{"id":519,"value":"$3d","pinned":false,"hidden":false,"mode":"md"},{"id":988,"value":"## F. USER STUDY TASKS","pinned":false,"hidden":false,"mode":"md"},{"id":993,"value":"### TASK 1: Aggregate Observations\nSubjects started with data containing papers and authors in a relational format. They should group data records by paper names, aggregate paper authors into a cell, and separate author names with commas.","pinned":false,"hidden":false,"mode":"md"},{"id":996,"value":"$3e","pinned":false,"hidden":false,"mode":"ojs"},{"id":1008,"value":"### Task 2: Generate Unique Key\nWe gave subjects a relational table of housing crime data by years and states and asked them to generate a unique key for each row by concatenating year and state.","pinned":false,"hidden":false,"mode":"md"},{"id":1011,"value":"$3f","pinned":false,"hidden":false,"mode":"ojs"},{"id":1030,"value":"### Task 3: Create Cross-tabulation\nSubjects started with a table comprising people's phone and fax numbers at home and work.\nThey were asked to remove incomplete records and create a cross-tabulation of numbers by people, number types of phone or fax and at home or work.","pinned":false,"hidden":false,"mode":"md"},{"id":1032,"value":"$40","pinned":false,"hidden":false,"mode":"ojs"},{"id":1044,"value":"### Task 4: Transpose Table\nSubjects were required to transpose the same table of housing crime in Task 2 from 20 rows × 3 columns into 3 rows × 20 columns.","pinned":false,"hidden":false,"mode":"md"},{"id":1047,"value":"$41","pinned":false,"hidden":false,"mode":"ojs"},{"id":17,"value":"ref1 = md`## REFERENCES`","pinned":false,"hidden":false,"mode":"ojs"},{"id":871,"value":"[1] S. Kasica, C. Berret, and T. Munzner. Table Scraps: An Actionable Framework for Multi-Table Data Wrangling From An Artifact Study of Computational Journalism. IEEE Transactions on Visualization and Computer Graphics, 27(2):957-966, 2021.\n\n[2] J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Reichart, M. Venkatrao, H. Pirahesh, and F. Pellow. Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross-Tab, and Sub-Totals. Data Mining and Knowledge Discovery, 1(1):29-53, 1997.","pinned":false,"hidden":false,"mode":"md"},{"id":66,"value":"md`## Utilities\n**Note:** The template of this document is revised from [here](https://observablehq.com/@uwdata/an-illustrated-guide-to-arquero-verbs?collection=@uwdata/arquero%EF%BC%89).\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":70,"value":"aq = {\n const aq = await require(`arquero@${aq_version}`);\n\n // load and install any additional packages\n (await Promise.all(aq_packages.map(pkg => require(pkg))))\n .forEach(pkg => aq.addPackage(pkg));\n\n // Add HTML table view method to tables\n aq.addTableMethod('view', toView, { override: true });\n\n return aq;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":128,"value":"$42","pinned":false,"hidden":false,"mode":"ojs"},{"id":172,"value":"toTableView = {\n const specStyle = 'font-size: 14px; font-family: sans-serif; margin: 0 auto; text-align: center';\n return function(rawHTML, spec, top) {\n const newView = html`

${rawHTML}

${spec}

`;\n return newView;\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":120,"value":"aq_version = '5.0.0'","pinned":false,"hidden":false,"mode":"ojs"},{"id":123,"value":"aq_packages = []","pinned":false,"hidden":false,"mode":"ojs"},{"id":87,"value":"function hconcat(...elem) {\n const span = html``;\n\n elem.forEach(e => {\n e.style.display = 'inline-block';\n e.style.verticalAlign = 'top';\n e.style.marginRight = '1em';\n span.appendChild(e);\n });\n\n return span;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":90,"value":"call = (arg, prefix, suffix = '') => {\n const elem = [md`— \\`${prefix}(\\``, arg, md`\\`${suffix})\\` ⟶`];\n const span = html``;\n \n elem.forEach(e => {\n e.style.display = 'inline-block';\n e.style.verticalAlign = 'top';\n e.style.marginRight = '0.4em';\n span.appendChild(e);\n });\n \n return span;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":93,"value":"edge = label => md` — \\`${label}\\` ⟶`","pinned":false,"hidden":false,"mode":"ojs"},{"id":99,"value":"d3 = require('d3@6')","pinned":false,"hidden":false,"mode":"ojs"},{"id":101,"value":"opacity = (c, value) => {\n c = d3.color(c);\n c.opacity = value;\n return c + '';\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":104,"value":"categories = d3.schemeTableau10\n .filter((c, i) => i !== 2)\n .map(c => opacity(d3.color(c).brighter(), 0.3))","pinned":false,"hidden":false,"mode":"ojs"},{"id":106,"value":"sortColors = domain => d3.scaleOrdinal()\n .domain(domain)\n .range(d3.schemeBlues[domain.length].map(c => opacity(c, 0.5)))","pinned":false,"hidden":false,"mode":"ojs"},{"id":108,"value":"groupColors = domain => d3.scaleOrdinal()\n .domain(domain)\n .range(categories)","pinned":false,"hidden":false,"mode":"ojs"},{"id":811,"value":"hyperlink = (referencingCellName, text) => {\n return html`${text} `;\n}","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W43","stdlib":"1","titling":"auto"}