2e:null 2f:["$","title",null,{"children":"[V2.5] Youtube Recommendations & Climate Crisis (work in progress) | Tian Breznik | Observable"}] 34:T8ce,md`# [V2.5] Youtube Recommendations & Climate Crisis (work in progress) This graph shows a network of Youtube recommendations, drawn from 17 search queries related to the climate crisis. The search queries are bi-grams with high tf-idf scores in a corpus of TED Talk transcripts on climate change (curated by TED: https://www.ted.com/topics/climate+change). We simulated a session where a user watches the first recommended videos for each query, and then continues to watch the next recommended videos, always the first in the list, for a series of 20 videos. In the graph below, each **node** represents a video. The **size** represents the number of views. The **color** represents the videos watched during a user session. The more **transparent nodes** represent recommended videos that were not watched, as the simulated user watched only the first recommended video. The data shown below was scraped using *R* and *Rvest* [1] which **disabled user tracking**. Thus it provides a baseline to compare to the recommendations observed for tracked users (e.g., with different search history, or geolocation). We also published a recommendation network obtain with basic user tracking, e.g., where Youtube tracks which videos are clicked and watched (*implicit user feedback*) [4]. We scraped the data using *Selenium* instead of *Rvest* [3]. You are welcome to scrape Youtube yourself, and compare this graph with the recommendations you would get. You can scrape Youtube using open source R scripts: [1] Scrape data without user tracking, using Rvest only: https://github.com/fcalv/Climatedream/blob/master/Scrape/DSS_Youtube_Scrape_pure_Rvest.R [2] Scrape data with user tracking, using Selenium: https://github.com/fcalv/Climatedream/blob/master/Scrape/DSS_Youtube_Scrape_V2.R [3] Format the scraped data into .json data to upload in this notebook. https://github.com/fcalv/Climatedream/blob/master/Scrape/DSS_Youtube_Format_V2.R [4] Notebook of recommendation network without user tracking: https://observablehq.com/@emma-ba/v2-youtube-recommendations-climate-crisis-sample-data-wit *Disclaimer: These scripts are not production-ready, beware they might contain flaws. The scripts and this D3 visualization are updated as we progress.* `35:T5a85,chart = { ///// DATA const nodes = data.nodes.map(d => Object.create(d)); const links = data.links.map(d => Object.create(d)); // Filters on data var min_occ = 1; var min_occ_link = 1; var min_occ_iter_new = 0; var min_occ_iter = 0; var show_l1 = true; var show_l2 = true; var show_l34 = true; ///// DOM ROOT ELEMENTS const width = 2000; const height = 2000; const svg = d3.create("svg") .attr("viewBox", [-width / 2, -height / 2, width, height]) .attr('style','background: #fff;'); const g = svg.append("g"); var legend_svg = svg.append('g') .attr('class','legend_div') .attr('active','no') .attr('transform','translate(-'+width/2+',-'+height/2+')'); var legend_channel = svg.append('g') .attr('class','legend_channel') .attr('active','no') .attr('transform','translate('+(width/2-300)+',-'+height/2+')'); var active_session = []; var active_channel = []; const tooltip = new Tooltip(); g.append(() => tooltip.node_link); ///// FORCE const simulation = d3.forceSimulation(nodes) .force("link", d3.forceLink(links).id(d => d.id)) .force("charge", d3.forceManyBody()) .force("x", d3.forceX()) .force("y", d3.forceY()); ///// FULL SCREEN function onresize () { let fs = document.webkitFullscreenElement || document.mozFullscreenElement; if (fs == svg) { svg.width = window.innerWidth; svg.height = window.innerHeight; } else { svg.width = width; svg.height = height; } legend_svg.attr('transform','translate(-'+svg.width/2+',-'+svg.width/2+')'); } window.addEventListener('resize', onresize, false); ///// ZOOM function zoomed() { g.attr("transform", d3.event.transform); } svg.call(d3.zoom() .extent([[0, 0], [width, height]]) .scaleExtent([1, 8]) .on("zoom", zoomed)); ///// GRAPH // Rendering functions var node_opacity = function(d){ if( (!show_l1) & d.group != 'recommendation' | (!show_l2) & d.group == 'recommendation' ) return 0.02; if(parseInt(d.iteration_all) < min_occ_iter_new) return 0.02; if(d.group != 'recommendation') return 1; else return 0.2; } var link_opacity = function(d){ if( (!show_l1) & d.level == 1 | (!show_l2) & d.level == 2 | (!show_l34) & d.level > 2 ) return 0.002; if(parseInt(d.iteration_all) < min_occ_iter_new) return 0.002; if(d.level == 1) return 1; else if (d.level == 2) return 0.2; else return 0; } //////////////////////////////////////////////////////////////////////////////// // GRAPH //////////////////////////////////////////////////////////////////////////////// const link = g.append("g") .selectAll("line") .data(links) .join("line") .attr("type", d => d.type) .attr("level", d => d.level) .attr("target", d => d.target.id) .attr("source", d => d.source.id) .attr("channel_source", d => d.source.channel) .attr("channel_target", d => d.target.channel) .attr("channel_source_id", d => d.source.channel_id) .attr("channel_target_id", d => d.target.channel_id) .attr("session_source", d => d.source.group) .attr("session_target", d => d.target.group) .attr("session_all", d => d.session_all) .attr("session_direct", d => d.session_direct) .attr("iteration_all", d => d.iteration_all) .attr("stroke", '#000') .attr('stroke-linecap', 'round') .attr("stroke-opacity", d => link_opacity(d)) .attr("stroke-width", d => link_width(d.level)); // Nodes const node = g.append("g") .selectAll("circle") .data(nodes) .join("a") .attr("href", d => 'https://www.youtube.com/watch?v=' + d.id) .attr('target', '_blank') .append("circle") .attr("id", d => d.id) .attr("class", 'video_node') .attr("channel", d => d.channel) .attr("channel_id", d => d.channel_id) .attr("session", d => d.session) .attr("session_all", d => d.session_all) .attr("session_direct", d => d.session_direct) .attr("session_from_reco", d => d.session_from_reco) .attr("group", d => d.group) .attr("iteration_all", d => d.iteration_all) .attr("stroke", "#fff") .attr("stroke-width", 0.5) .attr("r", d => r_scale(d3.max([10^6, +d.views^0.5]))) .attr("fill", d => color_scale(d.session)) .attr("opacity", d => node_opacity(d)) .call(drag(simulation)); ///////////////////////////////////////////////////////////////////////////////////// ///// UX FUNCTIONS ///////////////////////////////////////////////////////////////////////////////////// const undo_highlight = function(){ d3.selectAll('.session_legend').attr('opacity',1); d3.selectAll('.legend_selector_channel').attr('opacity',0); d3.selectAll('circle').attr('opacity', d => node_opacity(d)); d3.selectAll('line').attr('stroke-opacity', d => link_opacity(d)); } const hide_all = function(){ d3.selectAll('.session_legend').attr('opacity',0.5); d3.selectAll('.legend_selector_channel').attr('opacity',0.5); d3.selectAll('circle').attr('opacity',0.02); d3.selectAll('line').attr('stroke-opacity',0.001); } const hide_all_light = function(){ d3.selectAll('.session_legend').attr('opacity',0.5); d3.selectAll('.legend_selector_channel').attr('opacity',0.5); d3.selectAll('circle').attr('opacity',0.08); d3.selectAll('line').attr('stroke-opacity',0.03); } const show_session = function(label){ d3.selectAll('text[session_legend="'+label+'"]').attr('opacity', 1); if(min_occ_iter == 0){ d3.selectAll('line[session_all*="'+label+'"]').attr("stroke-opacity", d => 0.1); // link_opacity(d)); d3.selectAll('line[session_all*="'+label+'"]').attr("stroke-opacity", d => link_opacity(d)); d3.selectAll('circle[session_direct*="'+label+'"]').attr('opacity', d => node_opacity(d)); d3.selectAll('circle[session_all*="'+label+'"]').attr('opacity', d => node_opacity(d)); d3.selectAll('circle[session_from_reco*="'+label+'"]').attr('opacity', d => node_opacity(d)); } else { for(let i=1;i<=min_occ_iter;i++){ d3.selectAll('line[session_all*="'+label+'"][iteration_all*="'+i+'"]') .attr("stroke-opacity", d => 0.1); // link_opacity(d)); d3.selectAll('line[session_all*="'+label+'"][iteration_all*="'+i+'"]') .attr("stroke-opacity", d => link_opacity(d)); d3.selectAll('circle[session_direct*="'+label+'"][iteration_all*="'+i+'"]') .attr('opacity', d => node_opacity(d)); d3.selectAll('circle[session_all*="'+label+'"][iteration_all*="'+i+'"]') .attr('opacity', d => node_opacity(d)); d3.selectAll('circle[session_from_reco*="'+label+'"][iteration_all*="'+i+'"]') .attr('opacity', d => node_opacity(d)); //await sleep(2000); } } } const show_session_direct = function(label){ d3.selectAll('text[session_legend="'+label+'"]').attr('opacity', 1); if(min_occ_iter == 0){ d3.selectAll('line[session_direct*="'+label+'"][level="1"]') .attr("stroke-opacity", d => link_opacity_nofilter(d)); d3.selectAll('circle[session_direct*="'+label+'"]').attr('opacity', d => node_opacity_nofilter(d)); } else { for(let i=1;i<=min_occ_iter;i++){ d3.selectAll('line[session_direct*="'+label+'"][level="1"][iteration_all*="'+i+'"]') .attr("stroke-opacity", d => link_opacity_nofilter(d)); d3.selectAll('circle[session_direct*="'+label+'"][iteration_all*="'+i+'"]') .attr('opacity', d => node_opacity_nofilter(d)); //await sleep(2000); } } } const show_channel = function(label){ d3.selectAll('rect[channel_legend="'+label+'"]').attr('opacity', 0); if(min_occ_iter == 0){ d3.selectAll('line[channel_source="'+label+'"]').attr("stroke-opacity", d => link_opacity(d)); d3.selectAll('line[channel_target="'+label+'"]').attr("stroke-opacity", d => link_opacity(d)); d3.selectAll('circle[channel="'+label+'"]').attr('opacity', d => node_opacity(d)); } else { for(let i=1;i<=t;i++){ d3.selectAll('line[channel_source="'+label+'"][iteration_all*="'+i+'"]') .attr("stroke-opacity", d => link_opacity(d)); d3.selectAll('line[channel_target="'+label+'"][iteration_all*="'+i+'"]') .attr("stroke-opacity", d => link_opacity(d)); d3.selectAll('circle[channel="'+label+'"][iteration_all*="'+i+'"]') .attr('opacity', d => node_opacity(d)); //await sleep(2000); } } } var reshow_session_channel = function(){ if(active_session.length == 0 & active_channel.length == 0 & min_occ_iter == 0){ d3.selectAll('.legend_session').attr('active', 'no'); d3.selectAll('.legend_channel').attr('active', 'no'); undo_highlight(); } else { hide_all(); active_session.forEach(label => show_session(label)); active_channel.forEach(label => show_channel(label)); } } ///// TOOLTIP - interaction node.on('mouseover', d => { tooltip.show(d); hide_all_light(); // Higlight basic links d3.selectAll('circle[id="'+d.id+'"]').attr('opacity', node_opacity(d)); var links_src = d3.selectAll('line[source="'+d.id+'"]'); var links_tar = d3.selectAll('line[target="'+d.id+'"]'); links_src.attr('stroke-opacity', dd => link_opacity(dd)); links_tar.attr('stroke-opacity', dd => link_opacity(dd)); links_src.each(e => { d3.selectAll('circle[id="'+e.target.id+'"]').attr('opacity', node_opacity(e.target)); }); links_tar.each(e => { d3.selectAll('circle[id="'+e.source.id+'"]').attr('opacity', node_opacity(e.source)); }); // Highlight nodes & session links var session_list = []; d.session_all.split(';').forEach(s => { // s <- s.trim(); if(session_list.indexOf(s) ==-1) { session_list.push(s); } session_list.push(s.trim()); }); session_list.forEach(label => { show_session_direct(label); d3.selectAll('text[session_legend="'+label+'"]').attr('opacity', 1); }); }); node.on('mouseout', () => { tooltip.hide(); hide_all(); reshow_session_channel(); }); ////// LEGEND SESSION // Rendering functions var toggle_legend = function(label){ var element = d3.selectAll('rect[session_legend="'+label+'"]'); if(element.attr('active') == 'yes'){ element.attr('active', 'no'); active_session.splice(active_session.indexOf(label), 1); hide_all(); reshow_session_channel(); } else { element.attr('active', 'yes'); var div = d3.selectAll('.legend_div'); if(div.attr('active')=='no'){ div.attr('active', 'yes');} active_session.push(label); reshow_session_channel(); } } // Svg elements legend_svg.selectAll("rect") .data(groups) .join("rect") .attr('class', "session_swatch") .attr("session_swatch", d => d) .attr('width', 15) .attr('height', 15) .attr('fill', d => color_scale(d)) .attr('x',20) .attr('y', (d,i) => 45+i*25); legend_svg.selectAll("text") .data(groups) .join("text") .attr('class', "session_legend") .attr("session_legend", d => d) .attr('height', 15) .attr('fill', '#000') .attr('x', 45) .attr('y', (d,i) => 59+i*25) .html(d => d); var legend_rect = legend_svg.append("g") .selectAll("rect") .data(groups) .join("rect") .attr('class','legend_selector') .attr('style', 'cursor: pointer;') .attr('width', 270) .attr('height', 15) .attr('opacity', 0) .attr('x', 10) .attr('y', (d,i) => 45+i*25) .attr('session_legend', d => d) .attr('active','no') legend_rect.on('click', d => toggle_legend(d)); legend_rect.on('mouseover', d => { if(active_session.length == 0 & active_channel.length == 0 & min_occ == 1 & min_occ_link == 1) hide_all(); show_session_direct(d); }); legend_rect.on('mouseout', d => { reshow_session_channel(); }); legend_svg.append("text") .html('Sessions:') .attr('font-weight','bold') .attr('x', 25) .attr('y', 30); //////////////////////////////////////////////////////////////////////// ////// LEGEND CHANNEL //////////////////////////////////////////////////////////////////////// // Rendering functions var toggle_legend_channel = function(label){ var element = d3.selectAll('rect[channel_legend="'+label+'"]'); if(element.attr('active') == 'yes'){ element.attr('active', 'no'); d3.selectAll('rect[channel_legend="'+label+'"]').attr('opacity', 0.5); active_channel.splice(active_channel.indexOf(label), 1); hide_all(); reshow_session_channel(); } else { element.attr('active', 'yes'); var div = d3.selectAll('.legend_channel'); if(div.attr('active')=='no'){ div.attr('active', 'yes'); } active_channel.push(label); reshow_session_channel(); } } // Svg elements legend_channel.selectAll("rect") .data(channel_sorted) .join("rect") .attr('class', "channel_swatch") .attr("channel_swatch", d => d) .attr('width', 15) .attr('height', 15) .attr('fill', '#ddd') //.attr('fill', d => color_scale_channel(d)) .attr('x',20) .attr('y', (d,i) => 45+i*25); legend_channel.selectAll("text") .data(channel_sorted) .join("text") .attr('class', "channel_legend") .attr("channel_legend", d => d) .attr('height', 15) .attr('fill', '#000') .attr('x', 45) .attr('y', (d,i) => 59+i*25) .html(d => d); var legend_rect_channel = legend_channel.append("g") .selectAll("rect") .data(channel_sorted) .join("rect") .attr('class','legend_selector_channel') .attr('style', 'cursor: pointer;') .attr('width', 300) .attr('height', 18) .attr('opacity', 0) .attr('fill', '#fff') .attr('x', 10) .attr('y', (d,i) => 45+i*25) .attr('channel_legend', d => d) .attr('active','no') legend_rect_channel.on('click', d => toggle_legend_channel(d)); legend_rect_channel.on('mouseover', d => { if(active_channel.length == 0 & active_session.length == 0 & min_occ == 1 & min_occ_link == 1) hide_all(); show_channel(d); }); legend_rect_channel.on('mouseout', d => { reshow_session_channel(); }); legend_channel.append("text") .attr('x', 25) .attr('y', 30) .append('tspan') .html('Channels:') .attr('font-weight','bold') .append('tspan') .html(' (most frequent first)') .attr('font-weight','normal'); ////// FILTERS // Channel or Video var color_video = function(){ d3.select('#color_video').attr('opacity', 1); d3.select('#color_channel').attr('opacity', 0.5); d3.select('#color_none').attr('opacity', 0.5); d3.selectAll('.session_swatch').attr('fill', d => color_scale(d)); d3.selectAll('.channel_swatch').attr('fill', '#ddd'); d3.selectAll('.video_node').attr('fill', d => color_scale(d.session)); }; var color_channel = function(){ d3.select('#color_video').attr('opacity', 0.5); d3.select('#color_channel').attr('opacity', 1); d3.select('#color_none').attr('opacity', 0.5); d3.selectAll('.session_swatch').attr('fill', '#ddd'); d3.selectAll('.channel_swatch').attr('fill', d => color_scale_channel(d)); d3.selectAll('.video_node').attr('fill', d => color_scale_channel(d.channel)); }; var color_none = function(){ d3.select('#color_video').attr('opacity', 0.5); d3.select('#color_channel').attr('opacity', 0.5); d3.select('#color_none').attr('opacity', 1); d3.selectAll('.session_swatch').attr('fill', '#ddd'); d3.selectAll('.channel_swatch').attr('fill', '#ddd'); d3.selectAll('.video_node').attr('fill', '#999'); }; var filter_level_label = svg.append('g') .attr('transform','translate('+ (300 - width/2) +','+ (30 - height/2) +')') .append('text').html(`Color: `); filter_level_label.append('a').attr('style', 'cursor: pointer;').attr('id', 'color_video') .html(' Session ').on('click', () => color_video()); filter_level_label.append('a').attr('style', 'cursor: pointer;').attr('id', 'color_channel') .html(' Channel ').on('click', () => color_channel()) .attr('opacity', 0.5); filter_level_label.append('a').attr('style', 'cursor: pointer;').attr('id', 'color_none') .html(' (none) ').on('click', () => color_none()) .attr('opacity', 0.5); // Levels of links var link_l1 = function(){ show_l1 = !show_l1; if(show_l1) d3.select('#filter_l1').attr('opacity', 1); else d3.select('#filter_l1').attr('opacity', 0.5); hide_all(); reshow_session_channel(); }; var link_l2 = function(){ show_l2 = !show_l2; if(show_l2) d3.select('#filter_l2').attr('opacity', 1); else d3.select('#filter_l2').attr('opacity', 0.5); hide_all(); reshow_session_channel(); }; var link_l34 = function(){ show_l34 = !show_l34; if(show_l34) d3.select('#filter_l34').attr('opacity', 1); else d3.select('#filter_l34').attr('opacity', 0.5); hide_all(); reshow_session_channel(); }; var filter_level_label = svg.append('g') .attr('transform','translate('+ (300 - width/2) +','+ (60 - height/2) +')') .append('text').html(`Filter recommendations:`); filter_level_label.append('a').attr('style', 'cursor: pointer;').attr('id', 'filter_l1') .html(' Watched ').on('click', () => link_l1()); filter_level_label.append('a').attr('style', 'cursor: pointer;').attr('id', 'filter_l2') .html('→ Recommended ').on('click', () => link_l2()); //filter_level_label.append('a').attr('style', 'cursor: pointer;').attr('id', 'filter_l34') // .html('→ Recommended from a recommendation').on('click', () => link_l34()); // Nodes var min_occ_plus = function(){}; var min_occ_minus = function(){}; var reset_min_occ = function(){}; var filter_svg = svg.append('g').attr('transform','translate('+ (300 - width/2) +','+ (90 - height/2) +')'); var filter_node_label = filter_svg.append('text'); var min_occ_update = function(){ filter_node_label.html(`Filter nodes: Min no. of occurences ${min_occ}`); filter_node_label.append('a').attr('style', 'cursor: pointer;') .html(' ▲ More').on('click', () => min_occ_plus()); filter_node_label.append('a').attr('style', 'cursor: pointer;') .html(' ▼ Less').on('click', () => min_occ_minus()); filter_node_label.append('a').attr('style', 'cursor: pointer;') .html(' (resest)').on('click', () => reset_min_occ()); } min_occ_plus = function(){ min_occ = min_occ +1; min_occ_update(); hide_all(); reshow_session_channel(); }; min_occ_minus = function(){ min_occ = min_occ -1; min_occ_update(); hide_all(); reshow_session_channel(); }; reset_min_occ = function(){ min_occ = 1; min_occ_update(); hide_all(); reshow_session_channel(); }; min_occ_update(); // Links var min_occ_plus_link = function(){}; var min_occ_minus_link = function(){}; var reset_min_occ_link = function(){}; var filter_svg_link = svg.append('g').attr('transform','translate('+(300-width/2)+','+(120-height/2)+')'); var filter_link_label = filter_svg_link.append('text'); var min_occ_update_link = function(){ filter_link_label.html(`Filter links: Min no. of occurences ${min_occ_link}`); filter_link_label.append('a').attr('style', 'cursor: pointer;') .html(' ▲ More').on('click', () => min_occ_plus_link()); filter_link_label.append('a').attr('style', 'cursor: pointer;') .html(' ▼ Less').on('click', () => min_occ_minus_link()); filter_link_label.append('a').attr('style', 'cursor: pointer;') .html(' (resest)').on('click', () => reset_min_occ_link()); } min_occ_plus_link = function(){ min_occ_link = min_occ_link +1; min_occ_update_link(); hide_all(); reshow_session_channel(); }; min_occ_minus_link = function(){ min_occ_link = min_occ_link -1; min_occ_update_link(); hide_all(); reshow_session_channel(); }; reset_min_occ_link = function(){ min_occ_link = 1; min_occ_update_link(); hide_all(); reshow_session_channel(); }; min_occ_update_link(); // Iteration var min_occ_plus_iter = function(){}; var min_occ_minus_iter = function(){}; var reset_min_occ_iter = function(){}; var filter_svg_iter = svg.append('g').attr('transform','translate('+(300-width/2)+','+(150-height/2)+')'); var filter_iter_label = filter_svg_iter.append('text'); var min_occ_update_iter = function(){ var n_iter = min_occ_iter_new; if(n_iter == 0) { n_iter = '(All)'; } filter_iter_label.html(`Filter iterations: Show sessions until iteration ${n_iter}`); filter_iter_label.append('a').attr('style', 'cursor: pointer;') .html(' ▲ More').on('click', () => min_occ_plus_iter()); filter_iter_label.append('a').attr('style', 'cursor: pointer;') .html(' ▼ Less').on('click', () => min_occ_minus_iter()); filter_iter_label.append('a').attr('style', 'cursor: pointer;') .html(' (resest)').on('click', () => reset_min_occ_iter()); } min_occ_plus_iter = function(){ min_occ_iter_new = min_occ_iter_new +1; min_occ_update_iter(); hide_all(); reshow_session_channel(); }; min_occ_minus_iter = function(){ min_occ_iter_new = min_occ_iter_new -1; min_occ_update_iter(); hide_all(); reshow_session_channel(); }; reset_min_occ_iter = function(){ min_occ_iter_new = 0; min_occ_update_iter(); hide_all(); reshow_session_channel(); }; min_occ_update_iter(); ///// ANIMATION simulation.on("tick", () => { link .attr("x1", d => d.source.x) .attr("y1", d => d.source.y) .attr("x2", d => d.target.x) .attr("y2", d => d.target.y); node .attr("cx", d => d.x) .attr("cy", d => d.y); }); invalidation.then(function () { simulation.stop(); window.removeEventListener('resize', onresize); }); ///// DONE return svg.node(); }36:T1704,mini_chart = { var w = 200; var h = 60; var margin_left = 80; var margin_histo = 20; const svg = d3.create("svg").attr("viewBox", [0, 0, 2*w+margin_left+margin_histo, h]); //// Circle Legende svg.append("g").attr('transform','translate(0,10) scale(0.5)').call(legend_circle); //// VIDEO var x = d3.scaleLinear().domain(d3.extent(nodes_n_occ)).nice().range([margin_left, w]); var bins = d3.histogram().domain(x.domain()).thresholds(x.ticks(40))(nodes_n_occ); var y = d3.scaleLinear().domain([0, d3.max(bins, d => d.length)]).nice().range([h-10, 10]); var xAxis = g => g.attr("transform", `translate(0,${h-10})`) .call(d3.axisBottom(x).ticks(20).tickSizeOuter(0).tickSize(3)) .call(g => g.selectAll(".tick text").attr("font-size", 3.5).attr("dy", 0)) .call(g => g.selectAll(".tick line").attr("stroke-width", 0.2)) .call(g => g.select(".domain").attr("stroke-width", 0.2)) .call(g => g.append("text") .attr("x", w/2+margin_left/2+10).attr("y", 10) .attr("fill", "currentColor").attr("font-size", 5).attr("text-anchor", "end") .text('No. occurrence')); var yAxis = g => g.attr("transform", `translate(${margin_left},0)`) .call(d3.axisLeft(y).ticks(4).tickSize(2)) .call(g => g.select(".domain").remove()) .call(g => g.selectAll(".tick text").attr("font-size", 4).attr("dx", 2)) .call(g => g.selectAll(".tick line").attr("stroke-width", 0.2)) .call(g => g.select(".tick:last-of-type text").clone() .attr("x", 0).attr("y", -6) .attr("text-anchor", "start").attr("font-weight", "bold") .text('Videos having this no. occurrence')); svg.append("g").attr("fill", '#666') .selectAll("rect").data(bins).join("rect") .attr("x", d => x(d.x0) + 1) .attr("y", d => y(d.length)) .attr("width", d => Math.max(0, x(d.x1) - x(d.x0) - 1)) .attr("height", d => y(0) - y(d.length)); svg.append("g").call(xAxis); svg.append("g").call(yAxis); //// LINKS var x_link = d3.scaleLinear().domain(d3.extent(links_n_occ)).nice().range([margin_left, w]); var bins_link = d3.histogram().domain(x_link.domain()).thresholds(x_link.ticks(20))(links_n_occ); var y_link = d3.scaleLinear().domain([0, d3.max(bins_link, d => d.length)]).nice().range([h-10, 10]); var xAxis_link = g => g.attr("transform", `translate(${w-margin_left+margin_histo},${h-10})`) .call(d3.axisBottom(x_link).ticks(20).tickSizeOuter(0).tickSize(2)) .call(g => g.selectAll(".tick text").attr("font-size", 3.5).attr("dy", 0)) .call(g => g.selectAll(".tick line").attr("stroke-width", 0.2)) .call(g => g.select(".domain").attr("stroke-width", 0.2)) .call(g => g.append("text") .attr("x", w/2+margin_left/2+10).attr("y", 10) .attr("fill", "currentColor").attr("font-size", 5).attr("text-anchor", "end") .text('No. occurrence')); var yAxis_link = g => g.attr("transform", `translate(${margin_histo+w},0)`) .call(d3.axisLeft(y_link).ticks(4).tickSize(2)) .call(g => g.select(".domain").remove()) .call(g => g.selectAll(".tick text").attr("font-size", 4).attr("dx", 2)) .call(g => g.selectAll(".tick line").attr("stroke-width", 0.2)) .call(g => g.select(".tick:last-of-type text").clone() .attr("x", 0).attr("y", -6) .attr("text-anchor", "start").attr("font-weight", "bold") .text('Links having this no. occurrence')); svg.append("g").attr("fill", '#666') .selectAll("rect").data(bins_link).join("rect") .attr("x", d => x_link(d.x0) + 1 + w - margin_left + margin_histo) .attr("y", d => y_link(d.length)) .attr("width", d => Math.max(0, x_link(d.x1) - x_link(d.x0) - 1)) .attr("height", d => y_link(0) - y_link(d.length)); svg.append("g").call(yAxis_link); svg.append("g").call(xAxis_link); //// CHANNELS var x_channel = d3.scaleLinear().domain(d3.extent(channel_n_occ)).nice().range([margin_left, w+200]) var bins_channel = d3.histogram().domain(x_channel.domain()).thresholds(x_channel.ticks(60))(channel_n_occ) var y_channel = d3.scaleLinear().domain([0, d3.max(bins_channel, d => d.length)]).nice().range([h-10,10]) var xAxis_channel = g => g.attr("transform", `translate(${2*w-margin_left-margin_histo-15},${h-10})`) .call(d3.axisBottom(x_channel).ticks(60).tickSizeOuter(0).tickSize(2)) .call(g => g.selectAll(".tick text").attr("font-size", 3.5).attr("dy", 0)) .call(g => g.selectAll(".tick line").attr("stroke-width", 0.2)) .call(g => g.select(".domain").attr("stroke-width", 0.2)) .call(g => g.append("text") .attr("x", w/2+margin_left/2+10).attr("y", 10) .attr("fill", "currentColor").attr("font-size", 5).attr("text-anchor", "end") .text('No. occurrence')); var yAxis_channel = g => g.attr("transform", `translate(${2*w-margin_histo-15},0)`) .call(d3.axisLeft(y_channel).ticks(4)) .call(g => g.select(".domain").remove()) .call(g => g.selectAll(".tick text").attr("font-size", 4).attr("dx", 2)) .call(g => g.selectAll(".tick line").attr("stroke-width", 0.2)) .call(g => g.select(".tick:last-of-type text").clone() .attr("x", 0).attr("y", -6) .attr("text-anchor", "start").attr("font-weight", "bold") .text('Channels having this no. occurrence')); svg.append("g").attr("fill", '#666').attr("transform", `translate(-15,0)`) .selectAll("rect").data(bins_channel).join("rect") .attr("x", d => x_channel(d.x0) + 1 + 2*w - margin_left - margin_histo) .attr("y", d => y_channel(d.length)) .attr("width", d => Math.max(1, x_channel(d.x1) - x_channel(d.x0) - 1)) .attr("height", d => y_channel(0) - y_channel(d.length)); svg.append("g").call(xAxis_channel); svg.append("g").call(yAxis_channel); return svg.node(); } //.thresholds([...Array(+d3.format(".0f")(d3.extent(nodes_n_occ)[1]+1) ).keys()]); 37:T1064,class Tooltip { constructor() { this._tooltip_width = 250; this._bg = svg``; this._session = svg``; this._img_ratio = 1.7; this._image = svg``; this._title_words = []; this._title_l1 = svg``; this._title_l2 = svg``; this._descr_words = []; this._descr_l1 = svg``; this._descr_l2 = svg``; this._descr_l3 = svg``; this.node = svg``; this.node_link = svg`${this.node}`; } show(d) { this.node_link.setAttribute("href", 'https://www.youtube.com/watch?v=' + d.id); var max_box = this._tooltip_width; this._session.textContent = d.session_all; this._image.setAttribute("href", 'https://i.ytimg.com/vi/' + d.id + '/hqdefault.jpg'); this._title_words = d.title.split(" "); this._title_l1.textContent = ' '; this._title_l2.textContent = ' '; for (var i=0; i max_box){ max_box = this._title_l1.getBBox().width; } } else if (this._title_l2.getBBox().width < max_box-40){ this._title_l2.textContent += this._title_words[i] + ' '; if(this._title_l2.getBBox().width > max_box){ max_box = this._title_l2.getBBox().width; } } else { this._title_l2.textContent += '...'; if(this._title_l2.getBBox().width > max_box){ max_box = this._title_l2.getBBox().width; } break; } } this.descr_words = d.description.split(" "); this._descr_l1.textContent = ' '; this._descr_l2.textContent = ' '; this._descr_l3.textContent = ' '; for (var i=0; i max_box){ max_box = this._descr_l1.getBBox().width; } } else if (this._descr_l2.getBBox().width < max_box-40){ this._descr_l2.textContent += this.descr_words[i] + ' '; if(this._descr_l2.getBBox().width > max_box){ max_box = this._descr_l2.getBBox().width; } } else if (this._descr_l3.getBBox().width < max_box-40){ this._descr_l3.textContent += this.descr_words[i] + ' '; if(this._descr_l3.getBBox().width > max_box){ max_box = this._descr_l3.getBBox().width; } } else { this._descr_l3.textContent += '...'; if(this._descr_l3.getBBox().width > max_box){ max_box = this._descr_l3.getBBox().width; } break; } } this._bg.setAttribute("width", max_box+10 ); this._bg.setAttribute("height", 140+this._tooltip_width/this._img_ratio ); this.node.setAttribute("transform", `translate(${d.x+r_scale(d3.max([10^6, +d.views^0.5]))+5},${d.y-r_scale(d3.max([10^6, +d.views^0.5]))})`); this.node.removeAttribute("display"); } hide() { this.node.setAttribute("display", "none"); } }38:[{"id":"0fbc4eff7b7f43f032b52226524ca4b23e62f1b996c0af30c5be50ff3b9254e7d91baa88b03b1eafe48bb2b59f0ec1a98c3cd021ee29da663db69f00173b3bbe","href":"https://static.observableusercontent.com/files/0fbc4eff7b7f43f032b52226524ca4b23e62f1b996c0af30c5be50ff3b9254e7d91baa88b03b1eafe48bb2b59f0ec1a98c3cd021ee29da663db69f00173b3bbe","name":"results_v2_L2_pureSelenium_09_06_2020_exper.json","type":"application/json","lastModified":1591730785056.578,"size":825864}] 32:{"id":"e3f29a3fc45f9df4","slug":"v2-5-youtube-recommendations-climate-crisis-work-in-progre","title":"[V2.5] Youtube Recommendations & Climate Crisis (work in progress)","thumbnail":"82f8b6b6599cc0a784ff2f718cff7b9ca1c18380aaa37b6a37321102e5857d0b","access_level":"public","create_time":"$D2020-06-09T10:58:20.005Z","update_time":"$D2020-06-10T10:55:08.128Z","view_time":null,"first_public_time":"$D2020-06-09T19:32:26.626Z","fork":{"id":"a605278f97959204","slug":"v2-youtube-recommendations-climate-crisis-work-in-progres","title":"Youtube Recommendations & Climate Crisis (without user tracking)","owner":{"id":"5d9b00b2cdcd8028","login":"emma-ba","name":""}},"stars":2,"starred":false,"owner":{"type":"team","id":"a4b828e53b07705a","login":"tianbreznik","name":"Tian Breznik","avatar_url":"https://avatars-next.observableusercontent.com/a6bdbf4a0d21b6d05ab5774cc4a6bc4178664060566f9f14408245071eb18769","tier":"starter_2024"},"version":2589,"publish_version":2589,"role":null,"authored":false,"authors":[{"type":"individual","id":"b6d8a808a65ee7b3","login":"tianbreznik","name":"Tian Breznik","avatar_url":"https://avatars-next.observableusercontent.com/a6bdbf4a0d21b6d05ab5774cc4a6bc4178664060566f9f14408245071eb18769","tier":null}],"members":[{"type":"individual","id":"b6d8a808a65ee7b3","login":"tianbreznik","name":"Tian Breznik","avatar_url":"https://avatars-next.observableusercontent.com/a6bdbf4a0d21b6d05ab5774cc4a6bc4178664060566f9f14408245071eb18769","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"title":"[V2.5] Youtube Recommendations & Climate Crisis (work in progress)","cells":[{"id":1,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":406,"value":"{ \n function fullScreen(element) {\n if(element.requestFullScreen) {\n element.requestFullScreen();\n } else if(element.webkitRequestFullScreen ) {\n element.webkitRequestFullScreen();\n } else if(element.mozRequestFullScreen) {\n element.mozRequestFullScreen();\n }\n }\n \n let button = html``;\n button.onclick = function () { fullScreen(chart) };\n return button;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"$35","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2542,"value":"viewof t = Scrubber(d3.ticks(0, 200, 200), {\n autoplay: false,\n loop: false,\n initial: 50,\n format: x => `t = ${x.toFixed(0)}`\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1732,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2308,"value":"md`**Replace data below:**`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2566,"value":"function sleep(ms) {\n return new Promise(resolve => setTimeout(resolve, ms));\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":3,"value":"data = FileAttachment(\"results_v2_L2_pureSelenium_09_06_2020_exper.json\").json();","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2280,"value":"md`**Side note**: The formating script [3] can use Rvest to search for connections between recommendations in different user sessions, beyond what users have seen in their session. This makes data collection less rigorous (e.g., Rvest does anot allow Youtube to track the search history, and Rvest results can be very different from Selenium's). The formatting script outputs a file that does not contain these extra connections (results_v2_L2_compact.json) and the search for connection between recommendatinos is disabled by default.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":435,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":772,"value":"legend_circle = legendCircle()\n .tickValues([100, 1000000, 5000000]) \n .tickFormat((d, i) => i >= 1 ? d3.format(\"~s\")(d) + \" views\" : d + \" view\")\n .scale(\n d3.scaleSqrt()\n .domain([0, d3.max(data.nodes, d => d3.max([10^6, +d.views]))])\n .range([5, 90])\n )\n .tickSize(20);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1746,"value":"nodes_n_occ = {\n var temp = [];\n data.nodes.forEach(d => temp.push(d.n_occ));\n return temp;\n} ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1756,"value":"links_n_occ = {\n var temp = [];\n data.links.forEach(d => temp.push(d.n_occ));\n return temp;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":175,"value":"groups = {\n var grp = [];\n data.nodes.forEach( d => {\n if(grp.indexOf(d.session) == -1){\n grp.push(d.session)\n }});\n return grp.sort();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2191,"value":"channel_id = {\n var chan = [];\n data.nodes.forEach( d => {\n if(chan.indexOf(d.channel_id) == -1){\n chan.push(d.channel_id)\n }});\n return chan.sort();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2195,"value":"channel = {\n var chan = [];\n data.nodes.forEach( d => {\n if(chan.indexOf(d.channel) == -1){\n chan.push(d.channel)\n }});\n return chan.sort();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2429,"value":"count_channel = d3.nest()\n .key(function(d) { return d.channel; })\n .rollup(function(leaves) { return leaves.length; }).entries(data.nodes)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2497,"value":"count_channel.sort(function(x, y){\n return d3.descending(x.value, y.value);\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2476,"value":"channel_n_occ = {\n var chan = [];\n count_channel.forEach( d => {\n chan.push(d.value)\n });\n return chan;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2504,"value":"channel_sorted = {\n var chan = [];\n count_channel.forEach( d => {\n chan.push(d.key)\n });\n return chan;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":861,"value":"color_scale = d3.scaleOrdinal(groups, d3.schemeCategory10);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2536,"value":"color_scale_channel = d3.scaleOrdinal(channel, d3.schemeCategory10);","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":339,"value":"node_opacity_nofilter = function(d){\n if(d.group != 'recommendation') return 1;\n else return 0.2;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":287,"value":"r_scale = d3.scaleSqrt()\n .domain([0, d3.max(data.nodes, d => d3.max([10^6, +d.views]))])\n .range([5, 40]);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":366,"value":"link_opacity_nofilter = function(d){\n if(d.level == 1) return 1;\n else if (d.level == 2) return 0.2;\n else return 0.05;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":356,"value":"link_width = function(label){\n if(label == 1) return 3;\n else if (label == 2) return 1.5;\n else return 1;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":76,"value":"drag = simulation => {\n \n function dragstarted(d) {\n if (!d3.event.active) simulation.alphaTarget(0.3).restart();\n d.fx = d.x;\n d.fy = d.y;\n }\n \n function dragged(d) {\n d.fx = d3.event.x;\n d.fy = d3.event.y;\n }\n \n function dragended(d) {\n if (!d3.event.active) simulation.alphaTarget(0);\n d.fx = null;\n d.fy = null;\n }\n \n return d3.drag()\n .on(\"start\", dragstarted)\n .on(\"drag\", dragged)\n .on(\"end\", dragended);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":11,"value":"d3 = require(\"d3@5\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":176,"value":"import {legend} from \"@d3/color-legend\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":2544,"value":"import {Scrubber} from \"@mbostock/scrubber\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1441,"value":"import {View} from \"@mbostock/synchronized-views\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":773,"value":"import { legendCircle } from \"@harrystevens/circle-legend\";","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W38","stdlib":"1","titling":"auto"}} 33:"$32:body"