35:null 36:["$","title",null,{"children":"Project 3: fMRI and IEEG Raster Analysis | Jose.carmona-sanchez | Observable"}] 3c:T9a8,### Introduction and Background Neuroimaging is crucial in better understanding how the brain functions. Certain neuroimaging techniques are utilized for certain studies/diagnosis due to the specific spatial and temporal resolution that a technique offers. Spatial resolution refers to the structural information of the brain the imaging technique offers. Temporal resolution refers to information given about brain activity through out time. In this day and age there isn't a imaging technique that has both high spatial and temporal resolution. Intracranial electroencephalography or iEEG places electrodes within the cranium allowing for direct electrical brain recordings (usually in units of mV) on the surface of the brain. iEEG is characterized by having high temporal resolution but low spatial resolution. Where iEEG data is represented by a matrix that is electrodes by time. Functional magnetic resonance imaging or fMRI measures brain activity by reading blood flow throughout the brain through BOLD (blood-oxygen-level-dependent) signals. Unlike iEEG, fMRI does not have a real life unit rather the BOLD signal is a value that represents the contrast between oxygenated and de-oxygenated blood throughout the brain. fMRI is characterized by having higher spatial resolution but worse temporal resolution when compared to iEEG. The data for fMRI is represented by voxels which are similar to pixels but encoded in 3d space. Each voxel would have its own BOLD signal value where multiple voxels span the whole brain. I would like to investigate the commonalities and differences between iEEG and fMRI. ### Hypothesis The iEEG/fMRI data comes from a naturalistic study where researchers presented a short film to participants. The short film is composed of 13 different stimuli blocks where the film would either show speech stimuli (dialogue type) or music stimuli. During the film watching a select few of the participants had simultaneous iEEG and fMRI recordings. For the sake of simplicty this analysis will only be done on only one subject. Given the context I hypothesize that iEEG and fMRI activity in the temporal lobe should be more erratic in terms of frequency and amplitude when compared to other locations of the brain (since we're only looking at one subject the indvdiual has coverage in the frontal, temporal and parietal lobe). Historically speaking the temporal lobe has been associated with the processing of language and auditory stimuli. 3d:T1a2e,{ const LEFT_W = 700, RIGHT_W = 275; const fByUnit = d3.group(plotData, d => d.voxel); const iByUnit = d3.group(ieeg_plotData, d => d.voxel); const fY = d3.extent(plotData, d => d.value); const iY = d3.extent(ieeg_plotData, d => d.value); const selectors = []; // functions called with the selected electrode id let selected = 0, pinned = false; // anterior = +y (MNI) = voxel axis j; superior = +z (MNI) = voxel axis k // shifts in mm -> voxel units (2 mm voxels) const shiftVox = {j: anteriorShift / 2, k: superiorShift / 2}; const axisVal = (e, ax) => e[ax] + (shiftVox[ax] ?? 0); const header = html`

`; const fLine = html`

`, iLine = html`

`; // stimulus timeline strip, aligned with the plots (same margins / time domain) const timelineStrip = makeTimeline({width: LEFT_W, height: 44, title: "Stimulus timeline (Rest / Film: music / Film: speech)"}); const timelineLegend = html`

${ conditions.map(c => html`${c}`) }

`; function select(id) { selected = id; const e = elecVox[id]; header.textContent = `Electrode ${id}${pinned ? " (pinned – click to release)" : ""} · MNI mm (${e.x.toFixed(1)}, ${e.y.toFixed(1)}, ${e.z.toFixed(1)})`; for (const f of selectors) f(id); fLine.replaceChildren(Plot.plot({ title: `fMRI BOLD signal over time – voxel ${id}`, width: LEFT_W, height: 105, marginTop: 24, marginBottom: 20, x: {label: null, domain: timeDomain}, y: {label: "Signal", grid: true, domain: fY, ticks: 4}, marks: [Plot.lineY(fByUnit.get(id), {x: "time", y: "value", stroke: "black"}), Plot.ruleY([0])] })); iLine.replaceChildren(Plot.plot({ title: `iEEG voltage over time – channel ${id}`, width: LEFT_W, height: 105, marginTop: 24, marginBottom: 20, x: {label: null, domain: timeDomain}, y: {label: "mV", grid: true, domain: iY, ticks: 4}, marks: [Plot.lineY(iByUnit.get(id), {x: "time", y: "value", stroke: "black"}), Plot.ruleY([0])] })); } const hover = id => { if (!pinned && id !== selected) select(id); }; const click = id => { pinned = !pinned || id !== selected; select(id); }; function makeCarpet(data, {title, unitLabel, colorLabel, showX}) { const plot = Plot.plot({ title, width: LEFT_W, height: showX ? 195 : 175, marginTop: 13, marginBottom: showX ? 30 : 18, x: {label: showX ? "Time (s)" : null, domain: timeDomain}, y: {label: unitLabel}, color: {scheme: "viridis", legend: true, label: colorLabel, width: 175}, marks: [Plot.raster(data, {x: "time", y: "voxel", fill: "value", interpolate: "nearest"})] }); const svg = [...plot.querySelectorAll("svg")].pop(); const ys = plot.scale("y"), xr = plot.scale("x").range; const marker = d3.select(svg).append("line").attr("stroke", "red").attr("stroke-width", 1.5).attr("pointer-events", "none"); selectors.push(id => marker.attr("x1", xr[0]).attr("x2", xr[1]).attr("y1", ys.apply(id)).attr("y2", ys.apply(id))); svg.addEventListener("pointermove", ev => { const u = Math.round(ys.invert(d3.pointer(ev, svg)[1])); if (u >= 0 && u < elecVox.length) hover(u); }); svg.addEventListener("click", ev => { const u = Math.round(ys.invert(d3.pointer(ev, svg)[1])); if (u >= 0 && u < elecVox.length) click(u); }); return plot; } function makeContour(name, title) { const {axes: [a, b], oh, ov} = finalFits[name]; const lines = brainViews[name].flatMap((c, i) => c.map(([x, y]) => ({contour: i, x, y}))); const pts = elecVox.map(e => ({id: e.id, h: axisVal(e, a) + oh, v: axisVal(e, b) + ov})); const plot = Plot.plot({ title, width: RIGHT_W, aspectRatio: 1, marginLeft: 10, marginRight: 10, marginTop: 24, marginBottom: 10, x: {label: null, axis: null}, y: {label: null, axis: null}, marks: [ Plot.line(lines, {x: "x", y: "y", z: "contour", stroke: "gray"}), Plot.dot(pts, {x: "h", y: "v", fill: "steelblue", stroke: "white", r: 3.5}) ] }); const svg = plot.tagName === "svg" ? plot : plot.querySelector("svg"); const sx = plot.scale("x"), sy = plot.scale("y"); const px = pts.map(p => [sx.apply(p.h), sy.apply(p.v)]); const ring = d3.select(svg).append("circle").attr("r", 7).attr("fill", "none") .attr("stroke", "red").attr("stroke-width", 2.5).attr("pointer-events", "none"); selectors.push(id => ring.attr("cx", px[id][0]).attr("cy", px[id][1])); const nearest = ev => { const [mx, my] = d3.pointer(ev, svg); let best = -1, bd = 12; px.forEach(([x, y], i) => { const d = Math.hypot(x - mx, y - my); if (d < bd) { bd = d; best = i; } }); return best; }; svg.addEventListener("pointermove", ev => { const i = nearest(ev); if (i >= 0) hover(i); }); svg.addEventListener("click", ev => { const i = nearest(ev); if (i >= 0) click(i); }); return plot; } const fCarpet = makeCarpet(plotData, {title: "fMRI carpet plot – all voxels over time", unitLabel: "fMRI voxel", colorLabel: "Voxel signal", showX: false}); const iCarpet = makeCarpet(ieeg_plotData, {title: "iEEG carpet plot – all channels over time", unitLabel: "iEEG channel", colorLabel: "Value (mV)", showX: true}); const lat = makeContour("lateral", "Lateral view – electrode/voxel locations"); const sup = makeContour("superior", "Superior view – electrode/voxel locations"); select(0); display(html`

Linked fMRI and iEEG Electrode Explorer - (Subject-07)

Data for both iEEG and fMRI is sampled at around .602 sec and a spherical mask was applied to the fMRI data to isolate voxels of interest (voxels that had same coordinates as the electrodes)

${header}

${timelineStrip}${timelineLegend}${fLine}${iLine}${fCarpet}${iCarpet}

${lat}${sup}

`); }3e:T42f,### Results & Conclusions Comparing the electrodes/voxels around temporal region vs other regions you can see that in general as you go further superior and caudal (closer to upper frontal lobe region) the overall amplitude of the signals do tend to decrease; this is especially noticeable for the iEEG data. One does not see much change within frequency of signals/data but as you go further inferior and caudal (closer to downward frontal lobe region) some fMRI signals increase in frequency (this can be seen with voxel 33, 8, 9). Do note that most of the voxels that do see an increase in frequency are located within the temporal region. Looking at the results I would say that the hypothesis is confirmed. Although this does lead to many questions, one of my biggest questions is would you see this pattern of activity throughout different subjects? It would also be interesting to see data from different regions of the brain. Looking at the occipital lobe would especially be interesting due to the fact that a lot of visual information is processed in that area.3f:T6d9,const sampleTimes = [...new Set(plotData.map(d => d.time))]; const conditions = ["Rest", "Film (music)", "Film (speech)"]; const condShort = {"Rest": "Rest", "Film (music)": "Music", "Film (speech)": "Speech"}; const condColor = {"Rest": "#bdbdbd", "Film (music)": "#f28e2b", "Film (speech)": "#4e79a7"}; // run-length encode the per-sample labels into segments with start/end times (s) const tStep = sampleTimes[1] - sampleTimes[0]; const timelineSegments = []; timeline_raw.forEach((d, i) => { const condition = d[0], last = timelineSegments[timelineSegments.length - 1]; if (last && last.condition === condition) last.end = sampleTimes[i] + tStep; else timelineSegments.push({condition, start: sampleTimes[i], end: sampleTimes[i] + tStep}); }); // width/margins chosen so the x axis lines up with the carpet and line plots function makeTimeline({width = 900, height = 48, legend = false, title, subtitle} = {}) { return Plot.plot({ title, subtitle, width, height, marginTop: 4, marginBottom: 20, marginLeft: 40, marginRight: 20, x: {label: null, domain: timeDomain}, y: {axis: null}, color: {domain: conditions, range: conditions.map(c => condColor[c]), legend}, marks: [ Plot.rectX(timelineSegments, {x1: "start", x2: "end", fill: "condition", stroke: "white", title: d => `${d.condition}\n${d.start.toFixed(1)}–${d.end.toFixed(1)} s`}), Plot.text(timelineSegments, {x: d => (d.start + d.end) / 2, text: d => condShort[d.condition], frameAnchor: "middle", fontSize: 9, fill: "black", pointerEvents: "none"}) ] }); } display(timelineSegments); const timelinePlot = display(makeTimeline({title: "Experiment timeline", subtitle: "Stimulus condition over the recording", legend: true}));40:T904,function interactiveCarpet(data, {title, unitLabel, colorLabel}) { const byUnit = d3.group(data, d => d.voxel); const units = [...byUnit.keys()]; const [uMin, uMax] = d3.extent(units); const yDomain = d3.extent(data, d => d.value); const xDomain = d3.extent(data, d => d.time); const carpetPlot = Plot.plot({ title, subtitle: `Move the mouse over the carpet to plot that ${unitLabel.toLowerCase()}'s time series`, width: 900, height: 350, x: {label: "Time", domain: xDomain}, y: {label: unitLabel}, color: {scheme: "viridis", legend: true, label: colorLabel}, marks: [Plot.raster(data, {x: "time", y: "voxel", fill: "value", interpolate: "nearest"})] }); // the plot is a

(title + legend); the main chart is its last

`; const drawLine = (u) => { lineHolder.replaceChildren(Plot.plot({ title: `${unitLabel} ${u}`, subtitle: "Time series of the hovered row (y-axis fixed across rows)", width: 900, height: 220, x: {label: "Time", domain: xDomain}, y: {label: colorLabel, grid: true, domain: yDomain}, marks: [Plot.lineY(byUnit.get(u), {x: "time", y: "value", tip: true}), Plot.ruleY([0])] })); }; // highlight line marking the hovered row const marker = d3.select(svg).append("line") .attr("stroke", "red").attr("stroke-width", 1.5).attr("pointer-events", "none").style("display", "none"); const yScale = carpetPlot.scale("y"); const xRange = carpetPlot.scale("x").range; let current = null; svg.addEventListener("pointermove", (event) => { const [, py] = d3.pointer(event, svg); const u = Math.round(yScale.invert(py)); if (!byUnit.has(u)) return; marker.style("display", null).attr("x1", xRange[0]).attr("x2", xRange[1]) .attr("y1", yScale.apply(u)).attr("y2", yScale.apply(u)); if (u !== current) { current = u; drawLine(u); } }); drawLine(units[0]); return html`

${carpetPlot}${lineHolder}

`; } display(interactiveCarpet(plotData, {title: "fMRI carpet plot", unitLabel: "Voxel", colorLabel: "Voxel signal"})); display(interactiveCarpet(ieeg_plotData, {title: "iEEG carpet plot", unitLabel: "Channel", colorLabel: "Value (mV)"}));41:T59b,function plateauCentre(view, hKey, vKey) { const ok = []; for (let oh = -10; oh <= 130; oh += 0.5) for (let ov = -40; ov <= 110; ov += 0.5) { if (electrodes.every(e => brainViews[view].some(c => d3.polygonContains(c, [e[hKey] * 0.5 + oh, e[vKey] * 0.5 + ov])))) ok.push([oh, ov]); } return {oh: d3.mean(ok, d => d[0]), ov: d3.mean(ok, d => d[1]), n: ok.length}; } const latTf = plateauCentre("lateral", "y", "z"); const supTf = plateauCentre("superior", "x", "y"); display({latTf, supTf}); function electrodeView(name, tf, hKey, vKey) { const lines = brainViews[name].flatMap((c, i) => c.map(([x, y]) => ({contour: i, x, y}))); const pts = electrodes.map(e => ({...e, h: e[hKey] * 0.5 + tf.oh, v: e[vKey] * 0.5 + tf.ov})); return Plot.plot({ title: `Electrodes within the ${name} brain contour`, subtitle: `${electrodes.length} electrodes; mm scaled 0.5 to contour (2 mm voxel) units, offset fitted so all points fall inside`, width: 600, aspectRatio: 1, x: {label: "x (contour units)"}, y: {label: "y (contour units)"}, marks: [ Plot.line(lines, {x: "x", y: "y", z: "contour", stroke: "gray"}), Plot.dot(pts, {x: "h", y: "v", fill: "crimson", r: 4, tip: true, channels: {id: "id", x_mm: "x", y_mm: "y", z_mm: "z"}}) ] }); } const lateralElec = display(electrodeView("lateral", latTf, "y", "z")); const superiorElec = display(electrodeView("superior", supTf, "x", "y"));42:T4d2,const vAxes = ["i", "j", "k"]; const cRange = d3.range(40, 131, 1); function searchView(view) { const polys = brainViews[view]; const inside = (p) => polys.some(c => d3.polygonContains(c, p)); const res = []; for (const a of vAxes) for (const b of vAxes) { if (a === b) continue; // option lists: [sign, c] const optsA = [[1, 0], ...cRange.map(c => [-1, c])]; const optsB = [[1, 0], ...cRange.map(c => [-1, c])]; for (const [sa, ca] of optsA) { for (const [sb, cb] of optsB) { // quick reject on a subsample, then full count let n = 0; for (const e of elecVox) { const h = sa === 1 ? e[a] : ca - e[a]; const v = sb === 1 ? e[b] : cb - e[b]; if (inside([h, v])) n++; } if (n >= 50) res.push({view, h: sa === 1 ? a : `${ca}-${a}`, v: sb === 1 ? b : `${cb}-${b}`, inside: n}); } } } return res; } const latCands = searchView("lateral"); const supCands = searchView("superior"); display(latCands.length + " lateral, " + supCands.length + " superior candidates with >=50 inside"); display(Inputs.table(d3.sort([...latCands, ...supCands], (p, q) => d3.descending(p.inside, q.inside)).filter(d => /j|k/.test(d.h + d.v))));43:T4d0,const finalFits = { lateral: {axes: ["j", "k"], ...fitsLat[0]}, superior: {axes: ["i", "j"], ...fitsSup[0]} }; function finalView(name) { const {axes: [a, b], oh, ov} = finalFits[name]; const lines = brainViews[name].flatMap((c, i) => c.map(([x, y]) => ({contour: i, x, y}))); const pts = elecVox.map(e => ({...e, h: e[a] + oh, v: e[b] + ov})); const nIn = d3.sum(pts, p => brainViews[name].some(c => d3.polygonContains(c, [p.h, p.v]))); return Plot.plot({ title: `Electrode locations in the ${name} brain contour`, subtitle: `${nIn} of ${pts.length} electrodes inside; axes ${a}→horizontal, ${b}→vertical (voxels), offset (${oh.toFixed(1)}, ${ov.toFixed(1)}) fitted`, width: 600, aspectRatio: 1, x: {label: "horizontal (contour units)"}, y: {label: "vertical (contour units)"}, color: {scheme: "turbo", legend: true, label: "Electrode id"}, marks: [ Plot.line(lines, {x: "x", y: "y", z: "contour", stroke: "gray"}), Plot.dot(pts, {x: "h", y: "v", fill: "id", r: 4, stroke: "white", tip: true, channels: {id: "id", x_mm: "x", y_mm: "y", z_mm: "z"}}) ] }); } const lateralFinal = display(finalView("lateral")); const superiorFinal = display(finalView("superior"));44:[{"id":"749700184cc9cc13f149b676cc78d790096591cab1ed20cff3b26c33edb38337ad535f6c7c43b870d28f30a2a3f74f5cdbc2c9e8798bc7337deee3bea4334b1b","href":"https://static.observableusercontent.com/files/749700184cc9cc13f149b676cc78d790096591cab1ed20cff3b26c33edb38337ad535f6c7c43b870d28f30a2a3f74f5cdbc2c9e8798bc7337deee3bea4334b1b","name":"carpet_data.csv","type":"text/csv","lastModified":1791438492259,"size":711017},{"id":"5ad4ef600545ddab74dfa4be3d321fb467909869f6be09a89d2bb288b8ed24ed00a7ae74946b9c157547bf602b55c1c2b5fdb7bc8c646a2b9057849a713d1973","href":"https://static.observableusercontent.com/files/5ad4ef600545ddab74dfa4be3d321fb467909869f6be09a89d2bb288b8ed24ed00a7ae74946b9c157547bf602b55c1c2b5fdb7bc8c646a2b9057849a713d1973","name":"ieeg_carpet_data.csv","type":"text/csv","lastModified":1791443988699,"size":643999},{"id":"72e36010a23a86e2f99db3fda4543e420a65510cb2f44ac65a8888e0c960a85b91d9a76631da09083b5330ecfb2756a20360f98041aa5296a83c54f02465d384","href":"https://static.observableusercontent.com/files/72e36010a23a86e2f99db3fda4543e420a65510cb2f44ac65a8888e0c960a85b91d9a76631da09083b5330ecfb2756a20360f98041aa5296a83c54f02465d384","name":"brain_contours.json","type":"application/json","lastModified":1791771022524,"size":36024},{"id":"0b8c8b8220837990aa10586235d035553033e0c0e61c988ccf0a86db1f624e7a9c9eceac33194b2357e812a3669f49646485cecca1c327b7c90d40be1d3329a6","href":"https://static.observableusercontent.com/files/0b8c8b8220837990aa10586235d035553033e0c0e61c988ccf0a86db1f624e7a9c9eceac33194b2357e812a3669f49646485cecca1c327b7c90d40be1d3329a6","name":"electrode_location.csv","type":"text/csv","lastModified":1791771511963,"size":1136},{"id":"45e7d5b6a4fd3b8e09351fccc89a56aabe6b09c6120e94cdc5cdb7e26c17ddd96cc60eeb0b4b8b4a65e00660ee28d3e4f2a5ba23f084eeff99458ec590b5b189","href":"https://static.observableusercontent.com/files/45e7d5b6a4fd3b8e09351fccc89a56aabe6b09c6120e94cdc5cdb7e26c17ddd96cc60eeb0b4b8b4a65e00660ee28d3e4f2a5ba23f084eeff99458ec590b5b189","name":"timeline.csv","type":"text/csv","lastModified":1792264564074,"size":10918},{"id":"561d12aa819a02efe577090fce96c185d7b77464782f19879013342fca7c012dd8ce6872d69f3561f80149c8e56f71cd9472f6d3dde66afef2260cdc2abfad96","href":"https://static.observableusercontent.com/files/561d12aa819a02efe577090fce96c185d7b77464782f19879013342fca7c012dd8ce6872d69f3561f80149c8e56f71cd9472f6d3dde66afef2260cdc2abfad96","name":"colorblind_img.png","type":"image/png","lastModified":1792347996179,"size":441620}] 3a:{"id":"5715ca0f906c7ba0","slug":"project-3-f-mri-and-ieeg-raster-analysis","title":"Project 3: fMRI and IEEG Raster Analysis","thumbnail":"eed6bde36139417225dfae5db9fa72c2cc810eb1e8733b7cc72ea768e65da884","access_level":"public","create_time":"$D2026-10-04T05:55:29.259Z","update_time":"$D2026-10-18T18:40:10.925Z","view_time":null,"first_public_time":"$D2026-10-19T03:50:32.676Z","has_unpublished_edits":null,"fork":false,"stars":1,"starred":false,"owner":{"type":"team","id":"6e1a0b6a59a98d7d","login":"jose-carmona-sanchez","name":"Jose.carmona-sanchez","avatar_url":"https://avatars-next.observableusercontent.com/7c7c926efd11da0bb4d75ebd1a0198795094c32af03dd95ecac6eebd7c4f7fb5","tier":"starter_2024","bio":"","following":false},"version":369,"publish_version":369,"role":null,"authored":false,"authors":[{"type":"individual","id":"ef578cb74e81811e","login":"jose-carmona-sanchez","name":"Jose.carmona-sanchez","avatar_url":"https://avatars-next.observableusercontent.com/7c7c926efd11da0bb4d75ebd1a0198795094c32af03dd95ecac6eebd7c4f7fb5","tier":null,"bio":"","following":false}],"members":[{"type":"team","id":"6e1a0b6a59a98d7d","login":"jose-carmona-sanchez","name":"Jose.carmona-sanchez","avatar_url":"https://avatars-next.observableusercontent.com/7c7c926efd11da0bb4d75ebd1a0198795094c32af03dd95ecac6eebd7c4f7fb5","tier":"starter_2024","role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Project 3: fMRI and IEEG Raster Analysis","cells":[{"id":34,"value":"# Project 3: fMRI and IEEG Raster Analysis","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1,"value":"$3c","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":30,"value":"$3d","pinned":false,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":39,"value":"$3e","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":49,"value":"### Color Accessibility - Deuteranomaly\n![deuteranomaly plot](${await FileAttachment('colorblind_img.png').url()})","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":37,"value":"## Revisions","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":3,"value":"const raw = await FileAttachment(\"carpet_data.csv\").csv({ typed: true });\n\n// Reshape wide -> long for Plot's cell mark\nconst plotData = [];\nconst voxelCols = Object.keys(raw[0]).filter(k => k !== \"t\");\nfor (const row of raw) {\n for (const col of voxelCols) {\n plotData.push({\n time: row.t,\n voxel: +col.split(\"_\")[1],\n value: row[col]\n });\n }\n}","pinned":false,"hidden":false,"mode":"ts","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"const ieeg_raw = await FileAttachment(\"ieeg_carpet_data.csv\").csv({ typed: true });\n\n// Reshape wide -> long, converting volts -> millivolts\nconst ieeg_plotData = [];\nconst ieeg_voxelCols = Object.keys(ieeg_raw[0]).filter(k => k !== \"t\");\nfor (const row of ieeg_raw) {\n for (const col of ieeg_voxelCols) {\n ieeg_plotData.push({\n time: row.t,\n voxel: +col.split(\"_\")[1],\n value: row[col] * 1000 // V -> mV\n });\n }\n}","pinned":false,"hidden":false,"mode":"ts","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":35,"value":"const timeline_raw = await FileAttachment(\"timeline.csv\").csv({typed: true});","pinned":false,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":36,"value":"$3f","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":40,"value":"Having a timeline I think is important so that you can have a better understanding of what the subjects are looking at. Here I just wanted to make the timeline so that I could just add it in when everything else was ready.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":10,"value":"const zero = plotData.filter(x => x.voxel === 0);\ndisplay(zero)","pinned":true,"hidden":false,"mode":"ts","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":11,"value":"Plot.line(zero, {x: \"time\", y: \"value\",tip: true}).plot({y: {grid: true}})","pinned":true,"hidden":false,"mode":"ts","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"This was the first thing I did which is plot only one channel and see how it looked.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":5,"value":"const plot = display(Plot.plot({\n title: \"Carpet plot of voxel signal over time\",\n subtitle: \"Each row is a voxel; color shows signal value\",\n width: 900,\n height: 400,\n x: {label: \"Time\"},\n y: {label: \"Voxel\"},\n color: {scheme: \"viridis\", legend: true, label: \"Voxel Signal\"},\n marks: [\n Plot.raster(plotData, {x: \"time\", y: \"voxel\", fill: \"value\", interpolate: \"nearest\",tip:true}),\n Plot.crosshairX(plotData, {x: \"time\", y: \"voxel\",ruleStrokeWidth: 1.7, opacity: .7})\n ]\n}));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"const ieegPlot = display(Plot.plot({\n title: \"iEEG carpet plot\",\n subtitle: \"Each row is a channel; color shows signal value\",\n width: 900,\n height: 400,\n x: {label: \"Time\"},\n y: {label: \"Channel\"},\n color: {scheme: \"viridis\", legend: true, label: \"Value (mV)\"},\n marks: [\n Plot.raster(ieeg_plotData, {x: \"time\", y: \"voxel\", fill: \"value\", interpolate: \"nearest\", tip: true}),\n Plot.crosshairX(ieeg_plotData, {x: \"time\", y: \"voxel\", ruleStrokeWidth: 1.7, opacity: .7})\n ]\n}));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":42,"value":"I than went into making the carpet plots which was easy to do thanks to the plot.raster function. I also messed around with some of the parameters. I will say I really liked the crosshair interactivity but due to the interactivity in the main plot it would become obsolete.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":8,"value":"const timeDomain = d3.extent([...plotData, ...ieeg_plotData], d => d.time);\n\nfunction carpet(data, {title, yLabel, colorLabel, showX}) {\n return Plot.plot({\n title,\n width: 900,\n height: showX ? 240 : 200,\n marginBottom: showX ? 40 : 10,\n x: {label: showX ? \"Time\" : null, domain: timeDomain, axis: showX ? \"bottom\" : null},\n y: {label: yLabel},\n color: {scheme: \"viridis\", legend: true, label: colorLabel},\n marks: [\n Plot.raster(data, {x: \"time\", y: \"voxel\", fill: \"value\", interpolate: \"nearest\", tip: true}),\n Plot.crosshairX(data, {x: \"time\", y: \"voxel\", ruleStrokeWidth: 1.7, opacity: .7})\n ]\n });\n}\n\nconst combined = display(html`

\n ${carpet(plotData, {title: \"fMRI carpet plot\", yLabel: \"Voxel\", colorLabel: \"Voxel signal\", showX: false})}\n ${carpet(ieeg_plotData, {title: \"iEEG carpet plot\", yLabel: \"Channel\", colorLabel: \"Value (mV)\", showX: true})}\n

`);","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":43,"value":"Here I just messed around with the display of the carpet plot. It would be helpful to refresh myself with the html type of display which would come in use for the main plot.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":9,"value":"// z-score each modality so one shared color scale is meaningful\nfunction zscore(data, modality) {\n const m = d3.mean(data, d => d.value);\n const s = d3.deviation(data, d => d.value);\n return data.map(d => ({time: d.time, unit: d.voxel, modality, raw: d.value, z: (d.value - m) / s}));\n}\n\nconst facetData = [...zscore(plotData, \"fMRI (voxels)\"), ...zscore(ieeg_plotData, \"iEEG (channels)\")];\n\nconst facetPlot = display(Plot.plot({\n title: \"fMRI and iEEG carpet plots, faceted by modality\",\n subtitle: \"Each row is a voxel/channel; color shows z-scored signal (shared scale)\",\n width: 900,\n height: 600,\n marginLeft: 60,\n marginRight: 90,\n x: {label: \"Time\"},\n y: {label: \"Voxel / Channel\"},\n fy: {label: null},\n color: {scheme: \"viridis\", legend: true, label: \"Signal (z-score)\"},\n marks: [\n Plot.raster(facetData, {x: \"time\", y: \"unit\", fx: null, fy: \"modality\", fill: \"z\", interpolate: \"nearest\", tip: true}),\n Plot.frame()\n ]\n}));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":44,"value":"Here I wanted to see how the carpet plots would look like if I z-scored both y-axis for iEEG and fMRI data so that they would share one y-axis. But after further consideration I decided to ditch the idea due to the fact that logically it wouldn't make sense.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":12,"value":"$40","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":45,"value":"In this plot I figured out how to make a duel plot interaction. This was tough to do but thanks to the help of a little AI I was eventually able to figure it out.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":14,"value":"const brain = await FileAttachment(\"brain_contours.json\").json();\ndisplay(Array.isArray(brain) ? brain.slice(0, 2) : Object.keys(brain));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":15,"value":"const contourPoints = Object.entries(brain).flatMap(([level, contours]) =>\n contours.flatMap((pts, i) =>\n pts.map(([x, y]) => ({level: +level, contour: `${level}-${i}`, x, y}))\n )\n);\n\nconst contourPlot = display(Plot.plot({\n title: \"Brain contours\",\n subtitle: \"One line per contour, colored by level\",\n width: 700,\n aspectRatio: 1,\n x: {label: \"x\"},\n y: {label: \"y\"},\n color: {legend: true, label: \"Level\", type: \"categorical\"},\n marks: [\n Plot.line(contourPoints, {x: \"x\", y: \"y\", z: \"contour\", stroke: d => String(d.level), tip: true})\n ]\n}));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":16,"value":"const brainViews = await FileAttachment(\"brain_contours.json\").json();\nconst summarize = (o, depth = 0) =>\n Array.isArray(o) ? `Array(${o.length}) of ${o.length ? summarize(o[0], depth + 1) : \"?\"}`\n : o && typeof o === \"object\" ? `{${Object.keys(o).slice(0, 6).join(\", \")}} -> ${summarize(Object.values(o)[0], depth + 1)}`\n : typeof o;\ndisplay(Object.keys(brainViews));\ndisplay(summarize(brainViews));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":17,"value":"function viewPlot(name) {\n const pts = brainViews[name].flatMap((contour, i) =>\n contour.map(([x, y]) => ({contour: i, x, y}))\n );\n return Plot.plot({\n title: `Brain contours: ${name} view`,\n width: 600,\n aspectRatio: 1,\n x: {label: \"x\"},\n y: {label: \"y\"},\n marks: [Plot.line(pts, {x: \"x\", y: \"y\", z: \"contour\", stroke: \"currentColor\", tip: true})]\n });\n}\n\nconst lateralPlot = display(viewPlot(\"lateral\"));\nconst superiorPlot = display(viewPlot(\"superior\"));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":46,"value":"By far setting up the contour lines for the brain was the hardest part. There was a good amount of preprocessing code on python (especially to get the json polygons) that I had to get done first but I eventually got it. The first contour plot is an example of me debugging the contour lines.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":22,"value":"const affine = [[2,0,0,-98],\n [0,2,0,-134],\n [0,0,2,-72],\n [0,0,0,1]]","pinned":true,"hidden":false,"mode":"ts","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":18,"value":"const locs_data = await FileAttachment(\"electrode_location.csv\").csv({ typed: true });\ndisplay(locs_data)","pinned":true,"hidden":false,"mode":"ts","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":19,"value":"const electrodes = locs_data.map((d, i) => ({id: i, x: d[0], y: d[1], z: d[2]}));\n\n// candidate mappings from mm -> contour (voxel) coords\nconst candidates = {\n lateral: {\n \"h=(y+126)/2, v=(z+72)/2\": e => [(e.y + 126) / 2, (e.z + 72) / 2],\n \"h=(126-y)/2, v=(z+72)/2\": e => [(126 - e.y) / 2, (e.z + 72) / 2]\n },\n superior: {\n \"h=(90-x)/2, v=(y+126)/2\": e => [(90 - e.x) / 2, (e.y + 126) / 2],\n \"h=(x+90)/2, v=(y+126)/2\": e => [(e.x + 90) / 2, (e.y + 126) / 2]\n }\n};\nconst insideCount = (view, f) =>\n d3.sum(electrodes, e => brainViews[view].some(c => d3.polygonContains(c, f(e))) ? 1 : 0);\n\nconst scores = Object.entries(candidates).flatMap(([view, m]) =>\n Object.entries(m).map(([name, f]) => ({view, mapping: name, inside: insideCount(view, f), total: electrodes.length})));\ndisplay(Inputs.table(scores));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":20,"value":"function fit(view, hKey, vKey) {\n let best = [];\n for (const sh of [0.5, -0.5]) for (const sv of [0.5, -0.5]) {\n const hs = electrodes.map(e => e[hKey] * sh), vs = electrodes.map(e => e[vKey] * sv);\n for (let oh = -10; oh <= 130; oh += 1) for (let ov = -40; ov <= 110; ov += 1) {\n let n = 0;\n for (let i = 0; i < electrodes.length; i++) {\n const p = [hs[i] + oh, vs[i] + ov];\n if (brainViews[view].some(c => d3.polygonContains(c, p))) n++;\n }\n best.push({view, sh, sv, oh, ov, inside: n});\n }\n }\n return d3.sort(best, (a, b) => d3.descending(a.inside, b.inside)).slice(0, 5);\n}\nconst fitLateral = fit(\"lateral\", \"y\", \"z\");\nconst fitSuperior = fit(\"superior\", \"x\", \"y\");\ndisplay(Inputs.table([...fitLateral, ...fitSuperior]));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":21,"value":"$41","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":47,"value":"Luckily the data came with the coordinates of the electrodes so the code above is me implementing the coordinates so that they would pop up in both superior and lateral views. This part was also pretty tough because the contour lines and electrodes coordinates used a different mapping system. But again another shout out to the AI which helped me to get the points to be in the contour lines.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":31,"value":"// Convert electrode MNI mm coords -> voxel indices by inverting the affine (2 mm voxels)\nconst elecVox = electrodes.map(e => ({\n ...e,\n i: (e.x - affine[0][3]) / affine[0][0],\n j: (e.y - affine[1][3]) / affine[1][1],\n k: (e.z - affine[2][3]) / affine[2][2]\n}));\ndisplay(Inputs.table(elecVox));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":25,"value":"$42","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":26,"value":"const ext = Object.fromEntries(Object.entries(brainViews).map(([k, polys]) => {\n const pts = polys.flat();\n return [k, {n: polys.length, h: d3.extent(pts, p => p[0]), v: d3.extent(pts, p => p[1])}];\n}));\ndisplay(ext);\ndisplay(Object.fromEntries([\"i\",\"j\",\"k\"].map(a => [a, d3.extent(elecVox, e => e[a])])));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":27,"value":"function fitAffine(view, [a, b]) {\n const polys = brainViews[view];\n const res = [];\n for (const sa of [1, -1]) for (const sb of [1, -1]) {\n let best = {n: -1, pts: []};\n for (let oh = -60; oh <= 160; oh += 1) for (let ov = -60; ov <= 160; ov += 1) {\n let n = 0;\n for (const e of elecVox) if (polys.some(c => d3.polygonContains(c, [sa * e[a] + oh, sb * e[b] + ov]))) n++;\n if (n > best.n) best = {n, pts: [[oh, ov]]}; else if (n === best.n) best.pts.push([oh, ov]);\n }\n res.push({view, h: (sa < 0 ? \"-\" : \"\") + a, v: (sb < 0 ? \"-\" : \"\") + b, sa, sb, inside: best.n,\n oh: d3.mean(best.pts, d => d[0]), ov: d3.mean(best.pts, d => d[1]), plateau: best.pts.length});\n }\n return res;\n}\nconst fitsLat = fitAffine(\"lateral\", [\"j\", \"k\"]);\nconst fitsSup = fitAffine(\"superior\", [\"i\", \"j\"]);\ndisplay(Inputs.table([...fitsLat, ...fitsSup]));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":28,"value":"$43","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":29,"value":"display({\n fmriUnits: d3.extent(plotData, d => d.voxel), nF: new Set(plotData.map(d => d.voxel)).size,\n ieegUnits: d3.extent(ieeg_plotData, d => d.voxel), nI: new Set(ieeg_plotData.map(d => d.voxel)).size,\n fT: d3.extent(plotData, d => d.time), iT: d3.extent(ieeg_plotData, d => d.time),\n fCols: voxelCols.slice(0, 5), iCols: ieeg_voxelCols.slice(0, 5)\n});","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":48,"value":"Although I finally was able to get the points within the contour lines they were still not in the correct positions so the code above is me trying to figure out how to get them in the correct positions. I tried my best but ultimately settled is code to manually change the x and y positions of the code which is the sliders seen below.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":32,"value":"const anteriorShift = view(Inputs.range([-20, 40], {value: 10, step: 1, label: \"Anterior shift (mm)\"}));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":33,"value":"const superiorShift = view(Inputs.range([-20, 40], {value: 0, step: 1, label: \"Superior shift (mm)\"}));","pinned":true,"hidden":false,"mode":"js","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":50,"value":"### AI Use\nDeepseek AI was used in this project. Specifically AI was used in the duel plot interactivity initially and debugging the electrode location on the brain contour lines.\n\n\n**Specific inputs**:\n- setup duel interactivity between two plots in observable javascript where motion of cursor changes another plot\n- plotting coordinates on two different views\n- electrode coordinates outside of superior and lateral contour brain lines","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":51,"value":"### Sources:\n- Data: https://openneuro.org/datasets/ds003688/versions/1.0.7","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W44","stdlib":"2"}} 3b:"$3a:body"