Neural Cellular Automata: From Cells to Pixels

The NCA operates on a coarse lattice of cells (in this example vertices of a mesh). Center: A sampling point \(\Point\) (red dot) inside a triangle primitive, whose vertices correspond to NCA cells \(\State_i,\,\State_j,\,\State_k\). The local coordinate \(u(\Point)\) expresses the point’s position inside the primitive, while the locally averaged cell state \(\bar{\State}(\Point)\) is obtained by interpolating the surrounding cell states. Right: The Local Pattern Producing Network (LPPN), A shared lightweight MLP, receives \((\bar{\State}(\Point), u(\Point))\) as input and outputs the target properties, such as color and surface normal, at point \(\Point\). The NCA and the LPPN are trained jointly and end-to-end.

Play with the interactive visualization below to see coarse NCA cell states and the output the LPPN generates.