2a:null 2b:["$","title",null,{"children":"Connecting with Synapses and Post-Synaptic Potentials (PSPs) | Steve Oney | Observable"}] 30:T516,# Connecting with Synapses and Post-Synaptic Potentials (PSPs) So far, we've created neurons but we haven't actually connected them with each other. We're going to now add a model of *synapses* that connect neurons together. We use the term "pre-synaptic" to refer to the nueron that is "sending" the spike and "post-synapctic" to refer to the neuron that is receiving the spike. Many pre-synaptic neurons might connect to a single post-synapctic neuron. Post-synaptic neurons can also connect back to pre-synaptic neurons, forming loops. ## Synaptic Models Thusfar, we have been modeling spikes as binary "on" or "off" signals. However, there are some problems with this. First, it is not a realistic representation of how neurons transmit data. Action potentials are not binary; they instead consist of a continuous rise and lowering of potential energy. But biological realism is not the only problem with binary spike outputs. The second problem is that in binary representations of spikes, there is no "memory"---the history of a neuron's firing does not tell us anything about its current state. So when we connect neurons, rather than sending data between neurons with binary spikes, we will pass these binary spikes through a **synaptic model** that moderates communication between neurons.31:T5fa,## Decaying Synaptic Model A common synaptic model is one of simple exponential decay. We will call our synaptic model function ${tex`g(t)`}. Our synaptic model behaves very similar to [our LIF model](https://observablehq.com/@soney/modeling-lif-neurons), except that rather than the output being binary (spike or no spike), it will be the potential itself (a number). Our synaptic potential will start at ${tex`g(0) = 0`}. Whenever there is a spike, ${tex`g(t)`} will instantaneously increment by the spike's potential. Otherwise, it will have a rate of change ${tex`g'(t)`}: ${tex`g'(t) = -\frac{1}{\tau_s}g(t)`} Where ${tex`\tau_s`} is a constant that determines how quickly the potential decays back to ${tex`0`}. This is very similar to our definition of ${tex`v'(t)`} for LIF neurons, with ${tex`\tau_{s}`} instead of ${tex`\tau_{rc}`}. And just like we did for our LIF neurons, we can approximate its value using [Euler's method](https://en.wikipedia.org/wiki/Euler_method): ${tex`g(t + \Delta{}t) \approx g(t) + g'(t)\Delta{}t`}. In our case, we will assume that there can only be a maximum of one spike per neuron per timestep and we get: - ${tex`g[0] = 0`} - ${tex`g[t+1] = g[t] (1 - \frac{T_{step}}{\tau_s}) + \frac{T_{step}}{\tau_s}\delta_s[t]`} Where ${tex`\delta_s[t]`} represents the potential of any incoming spikes (or ${tex`0`} if there are no incoming spikes) and ${tex`T_{step}`} represents the size of our time steps. If we plot this out with a spike at time `0`, we can see that it decays exponentially:32:T491,{ const [minT, maxT] = [0, 0.1]; const tau_s = 1e-2; const gTruthTStep = 1e-3; const spikeTimes = [0]; const gTruthData = []; let gTruthG = 0; for(let t = minT; t<=maxT; t+=gTruthTStep) { const spikeAmplitude = spikeTimes.includes(t) ? 1/gTruthTStep : 0; // 0 if we don't spike. 1/Tstep if we do if(t === 0) { gTruthG = 100; } else { gTruthG = Math.exp(- gTruthTStep / tau_s) * gTruthG; } gTruthData.push({t, gTruthG}); } const data = []; let firstOrderG = 0; let analyticalG = 0; for(let t = minT; t<=maxT; t+=tstep) { if(t === 0) { analyticalG = firstOrderG = 100; } else { analyticalG = Math.exp(- tstep / tau_s) * analyticalG; firstOrderG = firstOrderG * (1 - tstep / tau_s); } data.push({t, firstOrderG, analyticalG}); } return Plot.plot({ y: { label: 'g' }, marks: [ Plot.line(data, {x: "t", y: "firstOrderG", stroke: "#999"}), Plot.line(data, {x: "t", y: "analyticalG", stroke: "red"}), Plot.line(gTruthData, {x: "t", y: "gTruthG", stroke: "#555", strokeDasharray: "5 5"}), ], }); }33:T89d,## (Optional) State Space Modeling We were able to compute our synapse's response analytically but this might be more difficult with more complex synapse models. For those, it might be easuer to model it using a state-space representation. ### Linear Time Invariance (LTI) If we had a more complex synapse model, analysis does not become too much more difficult as long as we assume our synapse model ${tex`g(t)`} is a [linear time-invariant (LTI) system](https://en.wikipedia.org/wiki/Linear_time-invariant_system). This means that: 1. **Time invariance**: If we shift our spike time by any amount of time ${tex`T`}, the synapse potential would be exactly the same but also shifted by ${tex`T`}. Mathematically, this means that if our input signal is ${tex`x(t)`} then if ${tex`x(t)`} produces response ${tex`g(t)`} then ${tex`x(t + T)`} produces response ${tex`g(t + T)`}. 2. **Linearity**: Our response scales linearly with the input. What this means is that if we have multiple inputs ${tex`x_1(t)`} and ${tex`x_2(t)`} with respective responses ${tex`g_1(t)`} and ${tex`g_2(t)`}, then if we add the inputs ${tex`x(t) = x_1(t) + x_2(t)`} then the resulting response is the sum of the outputs: ${tex`g(t) = g_1(t) + g_2(t)`}. With LTI systems, we can compute the output of any arbitrary input signal *just by knowing its "impulse response" function* ${tex`h(t)`}. The **impulse response** function ${tex`h(t)`} represents the potential energy over time if our neuron fired *exactly once* at ${tex`t=0`}. It should be ${tex`0`} for ${tex`t <0`} (if it hasn't fired, there is no energy) and can be any shape for ${tex`t \ge 0`}. We call ${tex`h(t)`} our **post-synaptic potential** function. In our case, we are using: ${tex` h(t) = \begin{cases} 0, & \text{if } t < 0\\ \frac{1}{\tau_s}e^{-\frac{t}{\tau_{s}}}, & \text{if } t \geq 0 \end{cases} `} With our impulse response ${tex`h(t)`} and any input signal ${tex`u(t)`} we can compute our synapse's output (what we called ${tex`g(t)`}) with a [**convolution**](https://en.wikipedia.org/wiki/Convolution), represented as ${tex`u(t) \ast h(t)`}: ${tex`g(t) = u(t) \ast h(t) = (u \ast h)(t) = \int_{-\infty}^{\infty} u(\tau)h(t - \tau),d\tau`}34:T8c1,### State Space Computation In systems analysis, it is common to analyze system dynamics by translating it to a "state space" representation that summarizes the dynamics and output given an input: ${tex`x'(t) = Ax(t) + Bu(t) \newline g(t) = Cx(t) + Du(t)`} Where ${tex`x(t)`} represents the state of the system, ${tex`x'(t)`} represents the rate of change of ${tex`x(t)`}, ${tex`u(t)`} represents the input, and ${tex`g(t)`} represents the output. Since we are assuming that we are working in an LTI, the output of our neuron, ${tex`y(t)`} is the input, ${tex`u(t)`} convolved with the impulse response function ${tex`h(t)`}: ${tex`g(t) = (u \ast h)(t)`} We can use the [Laplace transform](https://en.wikipedia.org/wiki/Laplace_transform) to help transform our functions into the frequency domain and simply do multiplication: ${tex`G(s) = U(s)H(s)`} We can first take the Laplace transform of ${tex`h(t)`} to transfer it into frequency space: ${tex` \mathscr{L}\{h(t)\} = H(s) = \frac{G(s)}{U(s)} = \int_{t=0}^{\infty}h(t)e^{-st}dt `} So plugging in ${tex`h(t) = \frac{1}{\tau_s}e^{-\frac{t}{\tau_s}}`}, we get ${tex` H(s) = \frac{G(s)}{U(s)} = \int_{t=0}^{\infty}\frac{1}{\tau_s}e^{-\frac{t}{\tau_s}}e^{-st}dt `} If we solve this, we get: ${tex` H(s) = \frac{1}{s\tau_s + 1} = \frac{G(s)}{U(s)} `} In our case, there is effectively no difference between ${tex`x(t)`} and ${tex`g(t)`}, as our "state"---the potential---is also the output. So let's set ${tex`x(t) = g(t)`}. Recalling our state space equations, we are saying that ${tex`C = 1`} and ${tex`D = 0`}. ${tex`x'(t) = Ax(t) + Bu(t) \newline g(t) = Cx(t) + Du(t) = x(t)`} then, substituting ${tex`X(s)`} for ${tex`G(s)`} in the above equation, we get: ${tex` H(s) = \frac{1}{s\tau_s + 1} = \frac{X(s)}{U(s)} \newline U(s) = s\tau_sX(s) + X(s) \newline sX(s) = -\frac{1}{\tau_s}X(s) + \frac{1}{\tau_s}U(s) `} Taking the inverse Laplace transform (${tex`\mathscr{L}^{-1}`}), we get: ${tex` \mathscr{L}^{-1}\{sX(s)\} = \mathscr{L}^{-1}\{-\frac{1}{\tau_s}X(s) + \frac{1}{\tau_s}U(s)\} \newline x'(t) = -\frac{1}{\tau_s}x(t) + \frac{1}{\tau_s}u(t) `} So in other words, we have ${tex`A = -\frac{1}{\tau_s}`}, ${tex`B = \frac{1}{\tau_s}`}, ${tex`C = 1`}, ${tex`D = 0`}35:T75c,### Transforming State Space to Discrete Time However, our state space representation is not yet complete because it is for **continuous time** whereas we are working in discrete time. So we need to [discretize our state space model](https://en.wikipedia.org/wiki/Discretization#Discretization_of_linear_state_space_models). In other words, we need to find coefficients ${tex`A_d`}, ${tex`B_d`}, ${tex`C_d`}, ${tex`D_d`}: ${tex` x[t+1] = A_dx[t] + B_du[t] \newline g[t] = C_dx[t] + D_du[t]`} We can find these based on our prior coefficients ${tex`A = -\frac{1}{\tau_s}`}, ${tex`B = \frac{1}{\tau_s}`}, ${tex`C = 1`}, ${tex`D = 0`}. Because the second equation is just extracting the relevant output (${tex`y`}) from our state (${tex`x`}). So ${tex`C_d = C = 1`} and ${tex`D_d = D = 0`}. Now, we need to find ${tex`A_d`} and ${tex`B_d`}. We will assume that each time step has size ${tex`T_{step}`} and that within each time step, the input ${tex`u`} is constant. We can use [a process for transforming the values for A, B, C, and D to discrete space](https://en.wikipedia.org/wiki/Discretization#Discretization_of_linear_state_space_models) which would result in: ${tex`A_d = e^{AT_{step}} \newline A_d = e^{-\frac{T_{step}}{\tau_s}} \newline`} and ${tex`\newline{}B_d = A^{-1}(A_d - I)B\newline B_d = -\tau_s(e^{-\frac{T_{step}}{\tau_s}}-1)\frac{1}{\tau_s} \newline B_d = 1 - e^{-\frac{T_{step}}{\tau_s}}\newline`} Noting that ${tex`g[t] = x[t]`} and re-naming our input to be consistent ${tex`u[t] = \delta_s[t]`} This gives us: ${tex`\newline g[t+1] = A_dg[t] + B_d\delta_s[t] \newline g[t+1] = g[t]e^{-\frac{T_{step}}{\tau_s}} + (1 - e^{-\frac{T_{step}}{\tau_s}})\delta_s[t] \newline g[t+1] = \delta_s[t] + e^{-\frac{T_{step}}{\tau_s}}(g[t] - \delta_s[t]) \newline `} And we can see that our space state representation yields the same result as our analytical solution.36:[] 2e:{"id":"3c638d3aaf9207b4","slug":"nmc-synapses","title":"Connecting with Synapses and Post-Synaptic Potentials (PSPs)","thumbnail":"99ccb17ad5d37ca57083d0f5fc6510c029547789b18c626810a2471ff738bd6c","access_level":"public","create_time":"$D2022-10-22T03:50:54.905Z","update_time":"$D2023-02-22T04:35:58.838Z","view_time":null,"first_public_time":"$D2023-01-24T15:21:58.484Z","fork":false,"stars":0,"starred":false,"owner":{"type":"team","id":"55b87b37da07de37","login":"soney","name":"Steve Oney","avatar_url":"https://avatars-next.observableusercontent.com/565926373d73cf99aa6425cf7c0d7ff79e41635d1a367894a5e3bce0299b18b9","tier":"starter_2024"},"version":694,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"1d138075b46297f1","login":"soney","name":"Steve Oney","avatar_url":"https://avatars-next.observableusercontent.com/565926373d73cf99aa6425cf7c0d7ff79e41635d1a367894a5e3bce0299b18b9","tier":null}],"members":[{"type":"individual","id":"1c09ce4b7505d91c","login":"adanielescu","name":"Andreea Danielescu","avatar_url":"https://avatars-next.observableusercontent.com/5cd8a89ba2fda44883bc5dd98ba3f3c6cb6297d77c21b9ef0c0919c5ef608a83","tier":null,"role":"editor"},{"type":"individual","id":"79581e18d086caf4","login":"ericgallo","name":"","avatar_url":null,"tier":null,"role":"editor"},{"type":"individual","id":"93113dd5fb467076","login":"sharvari-bondre","name":"Sharvari Bondre","avatar_url":"https://avatars-next.observableusercontent.com/754b967a5cf1ce71589f615fed536b94bd68509f3412154d82572c4b6aa44ead","tier":null,"role":"editor"},{"type":"individual","id":"1d138075b46297f1","login":"soney","name":"Steve Oney","avatar_url":"https://avatars-next.observableusercontent.com/565926373d73cf99aa6425cf7c0d7ff79e41635d1a367894a5e3bce0299b18b9","tier":null,"role":"editor"}],"comments":[{"id":"d9f152c75e33ac39","reply_to_id":null,"content":"I feel like the words here were insufficient to explain the difference between the synapse and LIF models. In the other word, why do we need a synapse model? Why could this function solve the two problems in binary signals?","create_time":"$D2023-05-07T20:34:52.556Z","update_time":null,"resolved":false,"user":{"type":"individual","id":"a2f9f861fa76becb","login":"a2f9f861fa76becb","name":"Ziqiao Xi","avatar_url":"https://avatars-next.observableusercontent.com/64861e404980b0ca88d5d40b5860ed28e9be4296c77b75d590ccb4e9e6535319","tier":null}},{"id":"6324a3970781d81a","reply_to_id":null,"content":"Why can't binary representations of spikes store \"memory\"? In Computer Science, aren't codes later transformed into binary representations? Could you explain more on the difference between continuous signals and discontinuous signals, and why do they differ from each other when it comes to \"memory\".","create_time":"$D2023-06-10T21:50:39.723Z","update_time":null,"resolved":false,"user":{"type":"individual","id":"4132f3acf2c585f1","login":"feiwu","name":"Fei Wu","avatar_url":"https://avatars-next.observableusercontent.com/786fbb8fd38ae9919783bb4368af88fefa862d32ba0a267d664efe2981392be7","tier":null}},{"id":"cd2b8ffff9d34feb","reply_to_id":"6324a3970781d81a","content":"In spiking neural networks (SNNs), neurons are not usually represented as binary units. This is because SNNs seek to emulate the temporal dynamics and spike-based communication found in biological neural networks. Unlike traditional artificial neural networks (ANNs), where neurons are commonly depicted as continuous activation values or binary values of 0 or 1, SNNs rely on the precise timing of individual spikes or their frequency for their operation.\n\nAs for codes being transformed into binary representations - a single bit is incapable of storing information other than its state (0 or 1) and needs complex encoding schemes, interpretation, and higher-level structures to store and convey meaningful data. Bits alone do not possess inherent meaning or context.","create_time":"$D2023-07-18T17:22:23.893Z","update_time":null,"resolved":false,"user":{"type":"individual","id":"93113dd5fb467076","login":"sharvari-bondre","name":"Sharvari Bondre","avatar_url":"https://avatars-next.observableusercontent.com/754b967a5cf1ce71589f615fed536b94bd68509f3412154d82572c4b6aa44ead","tier":null}}],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":0,"collection":{"id":"ceab76d8d8a0acbc","slug":"neuromorphic-computing","title":"Neuromorphic Tutorials: Spiking Neural Networks (SNNs) from Scratch","owner":{"id":"55b87b37da07de37","login":"soney","name":"Steve Oney"}},"type":"notebook","body":{"title":"Connecting with Synapses and Post-Synaptic Potentials (PSPs)","cells":[{"id":0,"value":"$30","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":61,"value":"$31","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":450,"value":"viewof tau_s0 = Inputs.range([0.002, 0.2], {value: 0.01, step: 0.001, label: \"tau_s\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":74,"value":"{\n const [minT, maxT] = [-0.01, 0.1];\n const Tstep = 1e-3;\n\n const spikeTimes = [0];\n const data = [];\n let g = 0;\n for(let t = minT; t<=maxT; t+=Tstep) {\n const spikeAmplitude = spikeTimes.includes(t) ? 1/Tstep : 0; // 0 if we don't spike. 1/Tstep if we do\n g = g * (1 - Tstep / tau_s0) + spikeAmplitude * Tstep / tau_s0; // update g\n data.push({t, g});\n }\n const plot = Plot.line(data, {x: \"t\", y: \"g\"}).plot();\n return plot\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":652,"value":"You might also notice the scale shifting as we modify ${tex`\\tau_s`}. That is because our synapse is delivering the same total \"potential\" to the postsynaptic neuron regardless of the value of ${tex`\\tau_s`}. However, for larger values of ${tex`\\tau_s`}, it is spreading that potential out further over time.\n\nTo put it another way, if we plotted the area under our curve (which represents the total potential delivered to the post-synaptic neuron), our sum will stay approximately the same. Below, we take the sum of the area of bars and our sum should always come out to ${tex`\\approx 1`}:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":668,"value":"viewof tau_sx = Inputs.range([0.02, 0.2], {value: 0.05, step: 0.001, label: \"tau_s\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":675,"value":"viewof tstepx = Inputs.range([0.001, 0.08], {value: 0.01, step: 0.001, label: \"T_step\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":649,"value":"{\n const [minT, maxT] = [-0.01, 1];\n const Tstep = tstepx;\n\n const data = [];\n let g = 0;\n let areaUnderCurve = 0;\n for(let t = minT; t<=maxT; t+=Tstep) {\n \n const spikeAmplitude = t === minT ? 1/Tstep : 0; // 0 if we don't spike. 1/Tstep if we do\n g = g * (1 - Tstep / tau_sx) + spikeAmplitude * Tstep / tau_sx; // update g\n areaUnderCurve += g * Tstep;\n data.push({t, g});\n }\n const plot = Plot.plot({\n y: {\n label: 'hz'\n },\n x: {\n axis: null,\n },\n marks: [\n Plot.line(data, {x: \"t\", y: \"g\", \"stroke\": \"#AAA\"}),\n Plot.barY(data, {x: \"t\", y: \"g\"}),\n ]\n });\n return html`

${plot.outerHTML}

Area under curve: ${areaUnderCurve}

`;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":87,"value":"## Modeling LIFs with PSPs\n\nNow, let's connect our LIF neurons from before with these decaying synaptic models. We are going to create a function `synapse()` that accepts two arguments: (1) a LIF neuron and (2) `tau_s`, a value for ${tex`\\tau_s`} (default: `0.01`) and returns a new object that has:\n\n- `output` --- a number representing the output potential of the synapse\n- `step(t, t_step)` --- a function that advances our neuron and synaptic model by one time step\n\nAt every step, we will update our synapse output using the formula from above:\n\n```JavaScript\nthis.output = (this.output + neuron.output) * (1 - t_step / tau_s);\n```","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":89,"value":"class LIF {\n constructor(tau_rc=0.2, v_init=0, tau_ref=0.002, v_th=1){\n this.tau_rc = tau_rc;\n this.tau_ref = tau_ref;\n this.v = v_init;\n this.v_th = v_th;\n \n this.output = 0;\n this.refractory_time = 0;\n }\n\n step(I, t_step) {\n this.refractory_time -= t_step; // Subtract the amount of time thhat passed from our refractory time\n \n if(this.refractory_time < 0) { // If we aren't in our refractory period, update the voltage\n this.v = this.v * (1 - t_step/this.tau_rc) + (I * t_step / this.tau_rc);\n }\n\n if(this.v >= this.v_th) { // Fired\n this.refractory_time = this.tau_ref; // Set refractory time\n\n this.output = 1 / t_step; // Fire\n this.v = 0; // Reset potential\n } else {\n this.output = 0; // Don't fire\n }\n return this.output;\n }\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":92,"value":"function psp(neuron, tau_s=1e-2) {\n return {\n output: 0,\n step: function (t, t_step) {\n const neuron_output = neuron.step(t, t_step);\n this.output = this.output * (1 - t_step / tau_s) + neuron_output * t_step / tau_s;\n return this.output;\n }\n };\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":101,"value":"function plotPotential(postSynaptic, inp, runtime=2) {\n const TIME_STEP = 1e-3;\n const NUM_STEPS = runtime / TIME_STEP; // Simulate for runtime seconds\n const v_history = [ {t: 0, out: postSynaptic} ];\n\n for(let i = 0; i < NUM_STEPS; i++) {\n v_history.push({t: (i+1)*TIME_STEP, out: postSynaptic.step(inp, TIME_STEP) });\n }\n const plot = Plot.line(v_history, {x: \"t\", y: \"out\"}).plot();\n return plot;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":437,"value":"viewof tau_s1 = Inputs.range([0.002, 0.2], {value: 0.01, step: 0.001, label: \"tau_s\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":107,"value":"plotPotential(psp(new LIF(0.02, 0, 0.2), tau_s1), 2)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":126,"value":"If you try increasing ${tex`\\tau_s`}, you can see that we fire before our PSP has fully decayed, which raises the resulting potential slightly. Let's increase ${tex`\\tau_s`} and zoom out:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":128,"value":"plotPotential(psp(new LIF(0.02, 0, 0.2), 1), 20, 10)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":139,"value":"Now, we can see that our output reaches and \"hovers\" around a steady state for a constant input.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":372,"value":"# Summary\n\n- Neurons communicate through a synapse model\n- Most synapse models \"convert\" binary spikes into smoother potentials that decay over time\n- We use a synapse model whose rate of change ${tex`g'(t) = -\\frac{1}{\\tau_s}g(t)`} and increments every time there is a spike\n - ${tex`\\tau_s`} determines the rate of decay back to ${tex`0`}\n- We can model our synapse in discrete time steps using:\n - ${tex`g[0] = 0`}\n - ${tex`g[t+1] = g[t] (1 - \\frac{T_{step}}{\\tau_s}) + \\frac{T_{step}}{\\tau_s}\\delta_s[t]`}\n - ${tex`\\delta_s(t)`} is the energy coming from incoming spikes (${tex`0`} if there are no spikes at time ${tex`t`})","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":687,"value":"---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":376,"value":"## (Optional) Analytical Model for Synapse\n\nAs was the case for LIFs, our update function ${tex`g[t+1] = g[t] (1 - \\frac{T_{step}}{\\tau_s}) + \\frac{T_{step}}{\\tau_s}\\delta_s[t]`} is an *estimate*. We can get more precise in our estimate if we do [the same analytical modeling that we did for our LIF neurons](https://observablehq.com/@soney/modeling-lif-neurons?collection=@soney/neuromorphic-computing#cell-87). Rather than repeating the analysis here, we will write our solution as:\n\n${tex`g[t + 1] = \\delta_s[t] + e^{-\\frac{T_{step}}{\\tau_{s}}}(g[t] - \\delta_s[t])`}\n\nand we could update our `step()` function accordingly:\n\n```JavaScript\nthis.output = neuron.output + Math.exp(-t_step/tau_s) * (this.output - neuron.output);\n```\n\nIf we plot this out (in red) compared to our first order estimate (in grey) you can see that they match almost perfectly:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":416,"value":"{\n const [minT, maxT] = [-0.01, 0.1];\n const tau_s = 1e-2;\n const Tstep = 1e-3;\n\n const spikeTimes = [0];\n const data = [];\n let firstOrderG = 0;\n let analyticalG = 0;\n for(let t = minT; t<=maxT; t+=Tstep) {\n const spikeAmplitude = spikeTimes.includes(t) ? 1/Tstep : 0; // 0 if we don't spike. 1/Tstep if we do\n firstOrderG = firstOrderG * (1 - Tstep / tau_s) + spikeAmplitude * Tstep/tau_s;\n analyticalG = spikeAmplitude + Math.exp(- Tstep / tau_s) * (analyticalG - spikeAmplitude);\n data.push({t, firstOrderG, analyticalG});\n }\n return Plot.plot({\n y: {\n label: 'g'\n },\n marks: [\n Plot.line(data, {x: \"t\", y: \"firstOrderG\", stroke: \"#999\"}),\n Plot.line(data, {x: \"t\", y: \"analyticalG\", stroke: \"red\"}),\n ],\n });\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":504,"value":"...but more importantly, we see that if we increase our time step size (${tex`T_{step}`}, our analytical solution stays more faithful to the accurate solution (the dashed line below):","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":480,"value":"viewof tstep = Inputs.range([0.001, 0.015], {value: 0.008, step: 0.001, label: \"Tstep\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":478,"value":"$32","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":570,"value":"Although our analytical solution is more accurate, there is a computational cost to performing `Math.exp` instead of simple addition and multiplication...however, we can actually address this by pre-computing the value ${tex`e^{-\\frac{T_{step}}{\\tau_{s}}}`}, assigning it to a variable (let's say `a`) and assigning:\n\n```JavaScript\nthis.output = neuron.output + a * (this.output - neuron.output);\n```","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":414,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":148,"value":"$34","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":217,"value":"$35","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":520,"value":"## Resources\n\n- The Youtube channel 3Blue1Brown has a useful [video describing convolutions](https://youtu.be/KuXjwB4LzSA)\n- The same channel has a useful [introduction to the Fourier Transform](https://youtu.be/spUNpyF58BY) (which is a good starting point for understanding the Laplace transform)\n- The use of state space modeling was inspired by [Nengo's synapse implementation](https://github.com/nengo/nengo/blob/dc0419fbe571374d0a55a7f67309dfcb254a2e88/nengo/synapses.py), which uses state space modeling for synapses","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"}],"files":"$W36","stdlib":"1","titling":"auto"}} 2f:"$2e:body"