41:null 42:["$","title",null,{"children":"Learning through Spike-Timing Dependent Plasticity (STDP) | Steve Oney | Observable"}] 48:T1029,## Spike-Timing Dependent Plasticity (STDP) STDP is a family of update rules for implementing Hebbian learning. Let's focus on two neurons, which we will call "pre" and "post". In STDP, if we believe that "pre" **causes** "post" to fire then we strengthen the connection between them so that "post" ends up being even more likely to fire after "pre" fires. Let's focus even more specifically on situations where "pre" and "post" fire at times ${tex`t_{pre}`} and ${tex`t_{post}`} respectively. So if "pre" fires before "post" (${tex`t_{pre} < t_{post}`}) then the connection between "pre" and "post" should be *strengthened*; it is likely that "pre" caused "post" to fire. If "pre" fires after "post" (${tex`t_{pre} \geq t_{post}`}) then the connection between "pre" and "post" should be *weakened*. Note that we are including simultaneous firing where ${tex`t_{pre} = t_{post}`} because if they fire at the same time, it is unlikely that "pre" caused "post" to fire...but whether that makes sense to do or not depends on the simulation setup (for example, if a presynaptic neuron could cause a postsynaptic neuron to fire on the same timestep in a discrete simulation). Let's define ${tex`w`} to be the weight between "pre" and "post". If ${tex`w`} is large then "post" will be more likely to fire when "pre" fires ("pre" firing caries more weight/influence on "post"). If ${tex`w`} is small then "pre" has little to no effect on "post". We could also allow ${tex`w`} to be negative so that if "pre" fires, "post" would be less likely to fire. Again, we are only focused on one firing of "pre" (at ${tex`t_{pre}`}) and "post" (at ${tex`t_{post}`}). So now, we ask the question of **how much** our weight ${tex`w`} should change based on the exact values of ${tex`t_{pre}`} and ${tex`t_{post}`}. We will only change our weight after both "pre" and "post" have fired so that we know how to change ${tex`w`}. We'll refer to the amount we change ${tex`w`} as ${tex`\Delta{}w`}. If ${tex`\Delta{}w`} is positive, we increase the strength of the connection of "pre" to "post" if ${tex`\Delta{}w`} is negative, we decrease the strength. ### Update Rules For now, let's focus on cases where "pre" fires _before_ "post" (${tex`t_{pre} < t_{post}`}). The more clearly associated "pre" and "post" are, the larger ${tex`\Delta{}w`} should be. If "post" fires *immediately* after "pre" then it's fairly likely that "pre" caused "post" to fire so ${tex`\Delta{}w`} should be larger. If "post" fires *a long time* after "pre" has fired then it's not likely that "pre" caused "post" to fire (since the influence of the spike decreased over time) so ${tex`\Delta{}w`} should be closer to ${tex`0`}. Conversely, if we focus on cases where "pre" fires _after_ "post" (${tex`t_{pre} \geq t_{post}`}) then ${tex`\Delta{}w`} will be negative. If "pre" fires a long time after "post" then ${tex`\Delta{}w`} should only be slightly negative, since we aren't necessarily sure what effect "pre" has on "post". But the closer ${tex`t_{pre}`} is to ${tex`t_{post}`} (while still occuring after), the more negative we can make ${tex`\Delta{}w`}, since we can be more sure that "pre" did **not** cause "post" to fire. The effect of "pre" on "post" decays exponentially with time, so it makes sense to make ${tex`\Delta{}w`} also depend on the exponential difference between ${tex`t_{pre}`} and ${tex`t_{post}`}: ${tex`e^{(t_{pre}-t_{post})}`}. Let's also create time constants to specify how quickly our influence decays. We'll use ${tex`\tau_{+}`} and ${tex`\tau_{-}`} as time constants to represent how quickly our ${tex`\Delta{}w`} update goes to ${tex`0`} if ${tex`t_{pre} < t_{post}`} or if ${tex`t_{pre} \geq t_{post}`} respectively. We'll also introduce two constants ${tex`A_{+}`} and ${tex`A_{-}`} to scale ${tex`\Delta{}w`} if ${tex`t_{pre} < t_{post}`} or if ${tex`t_{pre} \geq t_{post}`} respectively. So we end up the update rules: ${tex` \Delta{}w = \begin{cases} A_+e^{\frac{(t_{pre}-t_{post})}{\tau_+}}, & \text{if } t_{pre} < t_{post} \\ -A_{-}e^{\frac{-(t_{pre}-t_{post})}{\tau_{-}}}, & \text{if } t_{pre} \geq t_{post} \end{cases} `}49:T526,Let's define our `STDPWeights` class, with arguments: - `numPre`: the number of presynaptic neurons - `numPost`: the number of postsynaptic neurons - `tau_plus`: to represent ${tex`\tau_{+}`} - `tau_minus`: to represent ${tex`\tau_{-}`} It also has instance variables to track `numPre`, `numPost`, `tau_plus`, and `tau_minus`. It has instance variables `this.x` and `this.y` to represent the current values of ${tex`x`} and ${tex`y`}. Note that we will use a custom-defined [`times` function](https://observablehq.com/d/a6571460fd755600#times) that can repeat a value in an array (`times(3, 1)` returns `[1, 1, 1]`). ```JavaScript class STDPWeights { constructor(numPre, numPost, tau_plus = 0.02, tau_minus = 0.02) { this.numPre = numPre; this.numPost = numPost; this.tau_plus = tau_plus; this.tau_minus = tau_minus; this.x = times(numPre, 0); // An array with numPre zeros this.y = times(numPost, 0); // An array with numPost zeros } step(t_step) { // Loop through each value of this.x and decay for(let i = 0; i { this.x[ps_idx] += this.a_plus; }); // Increment y for spiking poast-synaptic neurons postSpikeIndices.forEach((ps_idx) => { this.y[ps_idx] -= this.a_minus; }); } ```4b:Tf8b,Now, suppose we have a pre-synaptic neuron and a post-synaptic neuron. There is some synaptic weight ${tex`w(t)`} that varies over time. We first assume that the synaptic conductance must be in the range ${tex`[g_{min}, g_{max}]`}. We can initialize it randomly within that range. We'll then scale our updates by the difference, ${tex`\alpha_g = g_{max} - g_{min}`}. - when the pre-synaptic neuron spikes, we increment every post-synaptic weight ${tex`w(t)`} by ${tex`w(t) = w(t) + \alpha_g y(t)`} - when the post-synaptic neuron spikes, we increment every pre-synaptic weight ${tex`w(t)`} by ${tex`w(t) = w(t) + \alpha_g x(t)`} So let's update our `STDPWeights` class. First, we will add two additional constructor arguments: - `g_min` to represent ${tex`g_{min}`} - `g_max` to represent ${tex`g_{max}`} And let's create a 2-D array, `this.w`, to represent the weights from each pre-synaptic neuron to each post-synaptic neuron. Our 2-D array will have `numPre` items, each item being a list of `numPost` numbers. So if we want the weight of the synapse from pre-synaptic neuron `i` to post-synaptic neuron `j`, we will access it via `this.w[i][j]`. Normally, we would initialize it randomly between `this.g_min` and `this.g_max` but it's a little easlier to visualize if we instead initialize it to `0` so we'll do that instead. ```JavaScript class STDPWeights { constructor(numPre, numPost, tau_plus = 0.02, tau_minus = 0.02, a_plus = 0.01, a_minus = 0.011, g_min = 0, g_max = 1) { this.numPre = numPre; this.numPost = numPost; this.tau_plus = tau_plus; this.tau_minus = tau_minus; this.x = times(numPre, 0); // An array with numPre zeros this.y = times(numPost, 0); // An array with numPost zeros this.a_plus = a_plus; this.a_minus = a_minus; // vvvvvvv ADDED vvvvvvv this.g_min = g_min; this.g_max = g_max; this.w = times2d(numPre, numPost, 0); // A 2-D array; numPre x numPost zeros (normally would be random) // ^^^^^^^ ADDED ^^^^^^^ } //... } ``` And then we want to update our `updateWeights` method to specify that whenever a pre-synaptic neuron fires, we should update the weights of all the post-synaptic neurons connected to it by adding that post-synapic neuron's `y` value multiplied by `this.g_max - this.g_min` (which we defined as ${tex`\alpha_g`} above). We also need additional code to ensure that the weight is between `this.g_min` and `this.g_max`. ```JavaScript const alpha_g = this.g_max - this.g_min; // Scaling factor for weight updates // Loop over the *index* of every pre-synaptic neuron that spiked preSpikeIndices.forEach((ps_idx) => { // And for every post-synaptic neuron connected to this pre-synaptic neuron, update // the connection weight by the `y` value for that post-synaptic neuron for(let i = 0; i this.g_max) { this.w[ps_idx][i] = this.g_max; } // if greater than g_max, set to g_max } }); ``` ...and conversely, when a post-synaptic neuron spikes, we update the weights of every pre-synaptic neuron connected to it: ```JavaScript // Loop over the *index* of every post-synaptic neuron that spiked postSpikeIndices.forEach((ps_idx) => { // And for every pre-synaptic neuron connected to this post-synaptic neuron, update // the connection weight by the `x` value for that pre-synaptic neuron for(let i = 0; i this.g_max) { this.w[i][ps_idx] = this.g_max; } // if greater than g_max, set to g_max } }); ```4c:Tb9b,class STDPWeights { constructor(numPre, numPost, tau_plus = 0.02, tau_minus = 0.02, a_plus = 0.01, a_minus = 0.011, g_min = 0, g_max = 1) { this.numPre = numPre; this.numPost = numPost; this.tau_plus = tau_plus; this.tau_minus = tau_minus; this.x = times(numPre, 0); // An array with numPre zeros this.y = times(numPost, 0); // An array with numPost zeros this.a_plus = a_plus; this.a_minus = a_minus; this.g_min = g_min; this.g_max = g_max; this.w = times2d(numPre, numPost, 0); // A 2-D array; numPre x numPost zeros (normally would be random) } updateWeights(preOutputs, postOutputs) { const preSpikeIndices = getIndicesGreaterThan0(preOutputs); // Indices of pre-synaptic spiking neurons const postSpikeIndices = getIndicesGreaterThan0(postOutputs); // Indices of poast-synaptic spiking neurons // Increment x for spiking pre-synaptic neurons preSpikeIndices.forEach((ps_idx) => { this.x[ps_idx] += this.a_plus; }); // Increment y for spiking poast-synaptic neurons postSpikeIndices.forEach((ps_idx) => { this.y[ps_idx] -= this.a_minus; }); const alpha_g = this.g_max - this.g_min; // Scaling factor for weight updates // Loop over the *index* of every pre-synaptic neuron that spiked preSpikeIndices.forEach((ps_idx) => { // And for every post-synaptic neuron connected to this pre-synaptic neuron, update // the connection weight by the `y` value for that post-synaptic neuron for(let i = 0; i this.g_max) { this.w[ps_idx][i] = this.g_max; } // if greater than g_max, set to g_max } }); // Loop over the *index* of every post-synaptic neuron that spiked postSpikeIndices.forEach((ps_idx) => { // And for every pre-synaptic neuron connected to this post-synaptic neuron, update // the connection weight by the `x` value for that pre-synaptic neuron for(let i = 0; i this.g_max) { this.w[i][ps_idx] = this.g_max; } // if greater than g_max, set to g_max } }); } step(t_step) { // Loop through each value of this.x and decay for(let i = 0; i s.length), ...postSpikeStrs.map(s => s.length)) const preSpikes = preSpikeStrs.map(ps => str2spikez(ps, maxLengthString * 1.2)); const postSpikes = postSpikeStrs.map(ps => str2spikez(ps, maxLengthString * 1.2)); const padding = (options.display && options.display.padding) || 5; const labelWidth = 30; const paddingLeft = padding + labelWidth + padding; const paddingBetweenSpikeView = 5; const paddingBetweenGraphs = 20; const paddingBetweenWeightView = 5; const numPreNeurons = preSpikes.length; const numPostNeurons = postSpikes.length; const perSpikeHeight = 15; const perGraphHeight = 50; const perWHeight = showW ? 100 : 0; const spikeDisplayHeight = perSpikeHeight * (preSpikes.length + postSpikes.length); const graphDisplayHeight = perGraphHeight * (preSpikes.length + postSpikes.length); const dwDisplayHeight = (perWHeight + paddingBetweenWeightView) * postSpikes.length; const height = spikeDisplayHeight + graphDisplayHeight + paddingBetweenSpikeView * (preSpikes.length + postSpikes.length) + dwDisplayHeight + 2*padding + paddingBetweenGraphs * (preSpikes.length + postSpikes.length - 1); const width = (options.display && options.display.width) || 800; const svg = DOM.svg(width + padding + paddingLeft, height + 2*padding); const allSpikes = preSpikes.concat(postSpikes); const T = d3.max(allSpikes.map(p => p.length)); const SIM_TIME = T; const weights = new STDPWeights(numPreNeurons, numPostNeurons, tau_plus2, tau_minus2, a_plus2, a_minus2, GMIN, GMAX) const x = d3.range(SIM_TIME).map(() => preSpikes.map(_ => 0)); const y = d3.range(SIM_TIME).map(() => postSpikes.map(_ => 0)); const w = d3.range(SIM_TIME).map(() => preSpikes.map(_ => postSpikes.map(_ => 0))); const dt = 10/SIM_PER_TIME; for(let t = 0; t ps[t]); const postSynapticSpikes = postSpikes.map(ps => ps[t]); weights.step(dt); weights.updateWeights(preSynapticSpikes, postSynapticSpikes); x[t+1] = clone(weights.x); y[t+1] = clone(weights.y); w[t+1] = clone2d(weights.w); } const spikeScaleX = d3.scaleLinear([0, SIM_TIME], [paddingLeft, padding+width]); for(let i = 0; i { if(spike) { const displayX = spikeScaleX(t); const locationIndicator = d3.select(svg).append("path") .attr("stroke", color) .attr("fill", "none") .attr("stroke-width", 5) .attr("d", `M${displayX},${yLoc}v${perSpikeHeight}`); const gridLineIndicator = d3.select(svg).append("path") .attr("stroke", color) .attr("fill", "none") .attr("stroke-width", 1) .attr("stroke-dasharray", "1, 5") .attr("d", `M${displayX},${yLoc}V${height}`); } }); const traceVar = isPreSynaptic ? x : y; const traceVal = traceVar.map(tr => tr[neuronIndex]); const graphScaleY = d3.scaleLinear([d3.min(traceVal), d3.max(traceVal)], [yLoc + perSpikeHeight + perGraphHeight + paddingBetweenSpikeView, yLoc + perSpikeHeight + paddingBetweenSpikeView]); const tracePath = d3.select(svg).append("path") .attr("stroke", color) .attr("fill", "none") .attr("stroke-width", 1) .attr("d", d3.line()(traceVal.map((d, t) => [spikeScaleX(t), graphScaleY(d)]))); if(isPostSynaptic && showW) { // const postWs = w.map((ws, t) => ws.map(w => w[neuronIndex])); const wScaleY = d3.scaleLinear([d3.min(w.flat(2)), d3.max(w.flat(2))], [yLoc + perSpikeHeight + perGraphHeight + perWHeight + paddingBetweenWeightView + paddingBetweenSpikeView, yLoc + perSpikeHeight + perGraphHeight + paddingBetweenWeightView + paddingBetweenSpikeView]); for(let preNeuronIndex = 0; preNeuronIndex < numPreNeurons; preNeuronIndex++) { const preColor = d3.schemeCategory10[preNeuronIndex]; const prepostw = w.map((ws, t) => ws[preNeuronIndex][neuronIndex]); const wPath = d3.select(svg).append("path") .attr("stroke", preColor) .attr("fill", "none") .attr("stroke-width", 1) .attr("d", d3.line()(prepostw.map((d, t) => [spikeScaleX(t), wScaleY(d)]))); } const y_axis = d3.axisRight(wScaleY).ticks(5); d3.select(svg).append("g") .attr("transform", `translate(${width}, 0)`) .call(y_axis); } const y_axis = d3.axisRight(graphScaleY).ticks(2); d3.select(svg).append("g") .attr("transform", `translate(${width}, 0)`) .call(y_axis); } return svg; } return getDWView; };4e:[] 46:{"id":"a6571460fd755600","slug":"nmc-stdp","title":"Learning through Spike-Timing Dependent Plasticity (STDP)","thumbnail":"e6a9181493a829e90a236ba6a46fbbcae70393e70be7b98624ad3c1ab8a25046","access_level":"public","create_time":"$D2022-10-24T15:16:04.530Z","update_time":"$D2023-02-21T21:16:38.626Z","view_time":null,"first_public_time":"$D2023-01-24T15:22:11.441Z","has_unpublished_edits":null,"fork":false,"stars":0,"starred":false,"owner":{"type":"team","id":"55b87b37da07de37","login":"soney","name":"Steve Oney","avatar_url":"https://avatars-next.observableusercontent.com/565926373d73cf99aa6425cf7c0d7ff79e41635d1a367894a5e3bce0299b18b9","tier":"starter_2024","bio":"","following":false},"version":1104,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"1d138075b46297f1","login":"soney","name":"Steve Oney","avatar_url":"https://avatars-next.observableusercontent.com/565926373d73cf99aa6425cf7c0d7ff79e41635d1a367894a5e3bce0299b18b9","tier":null,"bio":"","following":false}],"members":[{"type":"individual","id":"79581e18d086caf4","login":"ericgallo","name":"","avatar_url":null,"tier":null,"role":"editor"},{"type":"individual","id":"93113dd5fb467076","login":"sharvari-bondre","name":"Sharvari Bondre","avatar_url":"https://avatars-next.observableusercontent.com/754b967a5cf1ce71589f615fed536b94bd68509f3412154d82572c4b6aa44ead","tier":null,"role":"editor"},{"type":"individual","id":"1d138075b46297f1","login":"soney","name":"Steve Oney","avatar_url":"https://avatars-next.observableusercontent.com/565926373d73cf99aa6425cf7c0d7ff79e41635d1a367894a5e3bce0299b18b9","tier":null,"role":"editor"}],"comments":[{"id":"6f3b3867950c68ed","reply_to_id":null,"content":"Note: I want to have a simple example that illustrates STDP with something semi-useful but the closest thing seems to be the MNIST recognition example. Maybe will investigate some possible illustrative applications (maybe a simple logic XOR-like simulation?)","create_time":"$D2023-01-10T22:09:21.439Z","update_time":null,"resolved":false,"user":{"type":"individual","id":"1d138075b46297f1","login":"soney","name":"Steve Oney","avatar_url":"https://avatars-next.observableusercontent.com/565926373d73cf99aa6425cf7c0d7ff79e41635d1a367894a5e3bce0299b18b9","tier":null}}],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":{"id":"ceab76d8d8a0acbc","slug":"neuromorphic-computing","title":"Neuromorphic Tutorials: Spiking Neural Networks (SNNs) from Scratch","owner":{"id":"55b87b37da07de37","login":"soney","name":"Steve Oney"}},"type":"notebook","body":{"titling":"auto","title":"Learning through Spike-Timing Dependent Plasticity (STDP)","cells":[{"id":0,"value":"# Learning through Spike-Timing Dependent Plasticity (STDP)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":5,"value":"## Hebbian Learning\n\n[Donald Hebb](https://en.wikipedia.org/wiki/Donald_O._Hebb) was a researcher who tried to understand how synaptic connections between neurons change over time in order to enable learning. Hebb theorized that when two neurons fire at similar times, any synaptic connections between them will be strengthened. Hebbian theory can be summarized as *\"neurons that fire together wire together\"*.\n\nBut we need to get more specific; order matters. If we have neurons A and B, and a connection from A to B (A -> B) then this connection would be **strengthened if B fires immediately *after* A** because it is likely that A caused B to fire. However, **if B fires immediately *before* A then the connection will be weakened** because A did *not* cause B to fire.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":13,"value":"$48","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":60,"value":"Let's graph out the value of ${tex`\\Delta{}w`} with respect to ${tex`t_{pre}-t_{post}`}. We'll first define variables `a_plus` to represent ${tex`A_+`}, `a_minus` to represent ${tex`A_-`}, `tau_plus` to represent ${tex`\\tau_+`}, and `a_minus` to represent ${tex`\\tau_-`}.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":113,"value":"Our definition of ${tex`\\Delta{}w`} is **time-invariant** with respect to ${tex`t_{pre} - t_{post}`}, meaning that it does not depend on the exact time but only the difference between the two times (i.e., if ${tex`t_{pre} - t_{post} = 5`} then it doesn't matter what the exact value of ${tex`t_{pre}`} was). So let's graph out ${tex`\\Delta{}w`} with respect to ${tex`t_{pre} - t_{post}`}:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":108,"value":"We'll define the function `getDW` to accept two inputs that represent ${tex`t_{pre}`} and ${tex`t_{post}`} (`t_pre` and `t_post` respectively) and return ${tex`\\Delta{}w`}:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":68,"value":" function getDW(t_pre, t_post) {\n if(t_pre <= t_post) {\n return a_plus*Math.exp((t_pre - t_post)/tau_plus);\n } else {\n return -a_minus*Math.exp(-(t_pre - t_post)/tau_minus);\n } \n }","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":122,"value":"And plot it's value (note that since `getDW` is time invariant, we're just going to leave ${tex`t_{post}`} fixed at `0` and vary ${tex`t_{pre}`}):","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":70,"value":"viewof a_plus = Inputs.range([0.001, 0.03], {step: 0.001, label: \"a_plus\", value: 0.01})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":84,"value":"viewof a_minus = Inputs.range([0.001, 0.03], {step: 0.001, label: \"a_minus\", value: 0.011})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":88,"value":"viewof tau_plus = Inputs.range([5, 50], {step: 1, label: \"tau_plus\", value: 20})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":95,"value":"viewof tau_minus = Inputs.range([5, 50], {step: 1, label: \"tau_minus\", value: 20})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":63,"value":"function() {\n const [MINVAL, MAXVAL, INTERVAL] = [-100, 100, 0.1]\n const t_pre = 0;\n const results = [];\n for(let t_post = MINVAL; t_post <= MAXVAL; t_post += INTERVAL) {\n results.push({t_diff: t_pre - t_post, dw: getDW(t_pre, t_post)});\n }\n return Plot.plot({\n x: { grid: true, label: 't_pre - t_post' }, y: { grid: true },\n marks: [ Plot.line(results, {x: \"t_diff\", y: \"dw\"}) ]\n });\n}()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":137,"value":"# Implementing STDP\n\nLet's actually implement STDP in code. We are going to define a class `STDPWeights`. When working with large numbers of spikes, it would not be computationally efficient to call a function like `getDW` with every combination of spikes. Instead, we can do something that is equivalent but much more efficient by tracking two variables: ${tex`x(t)`} to represent the presynaptic neuron and ${tex`y(t)`} to represent the postsynaptic neuron. ${tex`x(t)`} and ${tex`y(t)`} decay according to the time constants ${tex`\\tau_{+}`} and ${tex`\\tau_{-}`} respectively:\n\n- ${tex`x'(t) = -\\frac{1}{\\tau_{+}}x(t)`}\n- ${tex`y'(t) = -\\frac{1}{\\tau_{-}}x(t)`}\n\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":995,"value":"$49","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":993,"value":"$4a","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":188,"value":"Let's see what this looks like for some example spike trains:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":620,"value":"Let's look at how a pre-synaptic neuron looks. Every time it spikes, ${tex`x(t)`} increments:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":203,"value":"getDWView([` x x x x x`], [], false)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":637,"value":"Conversely, when a post-synaptic neuron spikes, ${tex`y(t)`} decrements:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":628,"value":"getDWView([], [` xxx x x`], false)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":639,"value":"$4b","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1082,"value":"So the resulting code is:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":844,"value":"$4c","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1084,"value":"Let's see what this looks like:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":657,"value":"getDWView([`x x x x x x`], [` xx x x`])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":758,"value":"If we have multiple pre-synaptic neurons, the post synaptic neuron will have multiple weights (each weight associated with a given pre-synaptic neuron):","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":665,"value":"getDWView([`x x x x x x`, ` x x xxx x`], [` xx x x`])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":771,"value":"We can also have any number of post-synaptic neurons. Try modifying the spikes and see how the weights change below:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":781,"value":"viewof a_plus2 = Inputs.range([0.001, 0.03], {step: 0.001, label: \"a_plus\", value: 0.01})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":783,"value":"viewof a_minus2 = Inputs.range([0.001, 0.03], {step: 0.001, label: \"a_minus\", value: 0.011})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":785,"value":"viewof tau_plus2 = Inputs.range([5, 50], {step: 1, label: \"tau_plus\", value: 20})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":790,"value":"viewof tau_minus2 = Inputs.range([5, 50], {step: 1, label: \"tau_minus\", value: 20})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":773,"value":"getDWView(\n // Pre-synaptic spikes\n [\n \"x x x x x x\",\n \" x x xxx x \",\n ],\n // Post-synaptic spikes\n [\n \" xx x x \",\n \" x x x \",\n ])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1057,"value":"Note that there are many small variants on how STDP is implemented, including only updating on pre- or post-synaptic spikes or having multiple updates for each post-synaptic spike. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":828,"value":"# Summary\n\n- In Hebbian learning, we update synaptic weights to ensure that neurons whose firings are correlated and causal build a stronger synapctic connection\n- STDP is an implementation of Hebbian learning with update rules (when a pre-synaptic neuron fires at time ${tex`t_{pre}`} and a post-synaptic neuron fires at time ${tex`t_{post}`}):\n - ${tex`\n\\Delta{}w =\n\\begin{cases}\n A_+e^{\\frac{(t_{pre}-t_{post})}{\\tau_+}}, & \\text{if } t_{pre} < t_{post} \\\\\n -A_{-}e^{\\frac{-(t_{pre}-t_{post})}{\\tau_{-}}}, & \\text{if } t_{pre} \\geq t_{post}\n\\end{cases}\n`}\n - ${tex`A_+`} and ${tex`A_-`} specify how much to increment correlated and anti-correlated firing, respectively\n - ${tex`\\tau_+`} and ${tex`\\tau_-`} are time constants that specify how quickly our update rule for ${tex`\\Delta{}w`} should go to ${tex`0`}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":826,"value":"---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":841,"value":"# References\n\n- The Neuromatch Academy has [a helpful overview of STDP](https://compneuro.neuromatch.io/tutorials/W2D3_BiologicalNeuronModels/student/W2D3_Tutorial4.html) that helped guide some of the notation and visualizations in this notebook\n- The Neuronal Dynamics book has [a more detailed overview of Hebbian learning and STDP](https://neuronaldynamics.epfl.ch/online/Ch19.S2.html)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":874,"value":"$4d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":848,"value":"function times(n, funcVal) {\n const result = [];\n const func = (typeof funcVal === 'function') ? funcVal : (() => funcVal);\n\n for(let i = 0; i funcVal);\n\n for(let i = 0; i index).filter((index) => arr[index] > 0);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W4e","stdlib":"1"}} 47:"$46:body"