2d:null 2e:["$","title",null,{"children":"Neural Cellular Automata | KarlieDoesNotExist | Observable"}] 33:T6a4,html`

A very very brief introduction to

Growing Neural Cellular Automata

CA + Neural Network

NCA is fundamentally a cellular automaton combined with neural networks or more specifically, machine learning. With the machine learning process, the CA obtains the ability to generate a given pattern.

=> Design a CA
=> Target pattern
=> Use them to train the model
=> Get the model (the update rules for the ca)
=> Apply the rules back to the CA

Modeling: design a CA

How? - code

Cell states:

The CA runs on a regular 2D grid of 16-dimensional vectors => a 3D array of shape [height, width, 16].

Each cell state is a vector of 16 values:

The first 3 channels represent the cell color (RGB) in the range of [0, 1].
Alpha channel:
cells having alpha > 0.1 and their neighbors => “living”
others => "dead"
cells with alpha > 0.1 can be thought of as “mature”,
while their neighbors with alpha < 0.1 are “growing”, and can become mature if their alpha passes the 0.1 threshold. Hidden channels

Target pattern:

The target pattern is an image with its background transparent.
i.e. it has an alpha value equal to 1.0 for foreground pixels, and 0.0 for background.

`34:[{"id":"ff2cc2d7b6d0e75cd8dbf7d03d450fb43a63e22c6d312b053819a6b30390970844b8643eab0483debaf1a31a89fc18e9ae8fb2738ea15c24b894e541493485c5","href":"https://static.observableusercontent.com/files/ff2cc2d7b6d0e75cd8dbf7d03d450fb43a63e22c6d312b053819a6b30390970844b8643eab0483debaf1a31a89fc18e9ae8fb2738ea15c24b894e541493485c5","name":"alive2.svg","type":"image/svg+xml","lastModified":1635248986157.592,"size":26335},{"id":"1521e9e3bf259894f1fdaf75645f2276dacb467cf0962b681e9d429fd57904c7f6b232880cf5eb1bd6fd89a35d5c2c4a3f68a72424968b53b0cb4c2187abee09","href":"https://static.observableusercontent.com/files/1521e9e3bf259894f1fdaf75645f2276dacb467cf0962b681e9d429fd57904c7f6b232880cf5eb1bd6fd89a35d5c2c4a3f68a72424968b53b0cb4c2187abee09","name":"training.svg","type":"image/svg+xml","lastModified":1635251554809.973,"size":822974},{"id":"821d151f712212e3269bb8b41cbe165887bf740a16fd57de664737f9ae39731b861231f7035b775640c0aedd1a5383f122b2573b5ec5355023d62fe4c7d4ea95","href":"https://static.observableusercontent.com/files/821d151f712212e3269bb8b41cbe165887bf740a16fd57de664737f9ae39731b861231f7035b775640c0aedd1a5383f122b2573b5ec5355023d62fe4c7d4ea95","name":"model.svg","type":"image/svg+xml","lastModified":1635249008695.517,"size":183513}] 31:{"id":"741d282053ee857c","slug":"neural-cellular-automata","title":"Neural Cellular Automata","thumbnail":"7d3a9697106ef312d25d2cb730acfe5a4c6b7615ed6d695ad36ad19f44bf1226","access_level":"public","create_time":"$D2021-10-26T08:16:13.764Z","update_time":"$D2022-10-02T18:24:46.189Z","view_time":null,"first_public_time":"$D2021-10-26T12:32:53.758Z","has_unpublished_edits":null,"fork":false,"stars":1,"starred":false,"owner":{"type":"team","id":"b34c988451399e98","login":"wenrazhao2","name":"KarlieDoesNotExist","avatar_url":"https://avatars-next.observableusercontent.com/64edd542e55cff350a7503e5050267fb1768e8cc5052184b8a8cb49162d172f3","tier":"starter_2024"},"version":685,"publish_version":685,"role":null,"authored":false,"authors":[{"type":"individual","id":"302ba7635bb9b25d","login":"wenrazhao2","name":"KarlieDoesNotExist","avatar_url":"https://avatars-next.observableusercontent.com/64edd542e55cff350a7503e5050267fb1768e8cc5052184b8a8cb49162d172f3","tier":null}],"members":[{"type":"individual","id":"302ba7635bb9b25d","login":"wenrazhao2","name":"KarlieDoesNotExist","avatar_url":"https://avatars-next.observableusercontent.com/64edd542e55cff350a7503e5050267fb1768e8cc5052184b8a8cb49162d172f3","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":{"id":"a2245910e29a736f","slug":"generative-art","title":"Generative Art","owner":{"id":"b34c988451399e98","login":"wenrazhao2","name":"KarlieDoesNotExist"}},"type":"notebook","body":{"title":"Neural Cellular Automata","cells":[{"id":0,"value":"# Neural Cellular Automata","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":3,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":480,"value":"md`\n~~~js\nconst imageData = tf.tidy(() => {\n const rgba = state.slice([0, 0, 0, 0], [-1, -1, -1, 4]);\n const a = state.slice([0, 0, 0, 3], [-1, -1, -1, 1]);\n const img = tf.tensor(1.0).sub(a).add(rgba).mul(255); //tf => tensorflow library\n const rgbaBytes = new Uint8ClampedArray(img.dataSync()); \n //Uint8ClampedArray is similar to an Array where each item is an 8 bit unsigned integer\n \n return new ImageData(rgbaBytes, w, h);\n});\n\nctx.putImageData(imageData, 0, 0);\n//CanvasRenderingContext2D.putImageData() API\n\n~~~\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":105,"value":"md`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":181,"value":"md`\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":241,"value":"md`\n

\nEach cell looks at 3 things:

\n1) Cell identity: its own state (16 channels)
\n2 & 3) Look at its neighbours by doing convolutions with a sobel filter

\n\nCellular Automaton rule
\nconvolution operation + other per-cell operation => complex update rule
\nPerception
\nUpdate rule
\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":441,"value":"md`\n\n~~~Python\ndef update(perception_vector):\nx = dense(perception_vector, output_len=128)\nx = relu(x)\nds = dense(x, output_len=16, weights_init=0.0)\nreturn ds\n\n~~~\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":452,"value":"html`\n

\n\nStochastic cell update
\nLiving cell masking
\n
\n\nSobel filter is a fixed filter that do 3 x 3 convolution with each cell
\nThis is modelled after real cells looking at chemical gradients in their neighbourhoods \n

\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":413,"value":"md`
\n## Sobel filter\n![sobel](https://www.projectrhea.org/rhea/images/thumb/e/e1/Kernelconvolution_ex.png/750px-Kernelconvolution_ex.png)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":503,"value":"html`\n
\n

Generate - Stablize - Recover\n\n\n\n\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":608,"value":"md`

\n# CA + Chinese Characters\n### [New CharActers project](https://karliezhao.github.io/NewCharActers/)\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":528,"value":"html`\n
\n

A very very brief introduction to

Growing Neural Cellular Automata

CA + Neural Network

Modeling: design a CA

Cell states:

Target pattern:

Other useful links

\nGrowing Cellular Automata original paper
\nA nice Youtube video explaining NCA
\nThe Top 309 Cellular Automata Open Source Projects on Github
\nSelf-classifying MNIST Digits
\nDifferentiable-morphogenesis (infections and morphing from image to image)\n

A very very brief introduction to

Growing Neural Cellular Automata

CA + Neural Network

Modeling: design a CA

Cell states:

Target pattern:

Other useful links

\nGrowing Cellular Automata original paper\nA nice Youtube video explaining NCA\nThe Top 309 Cellular Automata Open Source Projects on Github\nSelf-classifying MNIST Digits \nDifferentiable-morphogenesis (infections and morphing from image to image)\n

\nGrowing Cellular Automata original paper
\nA nice Youtube video explaining NCA
\nThe Top 309 Cellular Automata Open Source Projects on Github
\nSelf-classifying MNIST Digits
\nDifferentiable-morphogenesis (infections and morphing from image to image)\n