Published
Edited
Jun 23, 2022
2 stars
Latent variable model
collection of binary random variables coupled through soft constraints
undirected graph
Distributed representation
Learn soft constraints between variables
eg neighboring pixels in images probably have similar values
Boltzmann machine
Defines a probability distribution
Probability of any joint distribution is log-linear in a happiness function H
${tex`
p(x) = \dfrac{1}{Z}\exp({H(x)})
`}
***
${tex`
Z = \displaystyle\sum_{x}\exp({H(x)})\\
\text{partition function}
`}
***
${tex`
H(x) = \displaystyle\sum_{i\not=j}w_{ij}x_{i}x_{j}+\sum_{i}b_{i}x_{i}\\
\text{happiness function, negation of energy}
`}
viewof Z = Inputs.range([0, 200], {
step: 0.01,
value: 172.42,
label: md`*Z*`
})
boltzmannCollection = {
return {
nodes: {
x1: {
group: 1,
bias: x1,
links: [
{ target: "x2", value: w12 },
{ target: "x3", value: w13 }
]
},
x2: {
group: 1,
bias: x2,
links: [
{ target: "x3", value: w23 },
{ target: "x2", value: 1 }
]
},
x3: { group: 1, bias: x3, links: [] }
},
links: [
{ source: "x1", target: "x2", value: w12 },
{ source: "x1", target: "x3", value: w13 },
{ source: "x2", target: "x3", value: w23 },
{ source: "x2", target: null, value: 1 }
]
};
}
// wijxixj
// w12x1x2
Object.keys(boltzmannCollection.nodes).map((i) =>
boltzmannCollection.nodes[i].links.map((j) => j.value)
)
R - https://en.m.wikipedia.org/wiki/Real_coordinate_space
tex`p(x)\\
x\sim p(x)~{\text{instance of our model}}\\
x\in\R^n
`
"The Boltzmann Machine defines a probability distribution over the set of possible visible binary vectors V and hidden binary vectors H. The intended analogy is that V is an observation, say the pixels on the retina, and H is the joint activity of all the neurons inside the brain. We’ll also denote the concatenation of V and H by X, so X=(V,H). The Boltzmann Machine defines a probability distribution over the configurations X=(V, H) by the equation
${tex`
P(X) = \cfrac{\exp({X^\top}WX/2)}{\sum_{X^\prime}\exp({X^{\prime\top}}WX^\prime/2)}
`}
So different choices of the matrix ${tex`W`} yield different distributions ${tex`P(X)`}
Purpose-built for displays of data
Observable is your go-to platform for exploring data and creating expressive data visualizations. Use reactive JavaScript notebooks for prototyping and a collaborative canvas for visual data exploration and dashboard creation.
