Your browser is no longer supported
Outdated browsers can expose your computer to security risks. We recommend you switch to one of our supported browsers, such as Microsoft Edge, Google Chrome or Mozilla Firefox.
Image caption: The 16 lines of JAX code behind Particle Lenia *(1).
Interview | Alexander Mordvintsev in conversation with Sofia Gkinko about the links interweaving his art and research.
Sofia Gkinko: Your scientific and artistic practice are deeply interconnected; did they come about the same time?
Alexander Mordvintsev: Generally, I say that my artistic career started from my research at Google, but actually I did some artist projects even earlier. Back in 2009, I was very interested in Demoscene, a computer art subculture focused on producing demos, which is still ongoing. It started with amateur hackers who were pirating software, and it developed into its own artistic genre. I decided to go to one of the biggest demoparties in Helsinki, so I hacked a little demo. Typically, those teams consist of artists, programmers, coders, and musicians but I didn’t have a team, I was alone; I just took some classical Vivaldi music, got some fluid simulation effects, and I created The Flow (*2). I didn’t have high expectations, but my demo, along with other 15 to 20 entries, was selected by the jury for display and it was shown on a huge screen in a stadium with maybe 5000 people; this felt like a kind of success for me and it was indeed my first public appearance as a generative artist.
Clip from "The Flow", 2009 watch it via link here
SG: DeepDream is – until today – your most known creation and at the very heart of it lies the freedom you gave to the machine: you allowed the computer to dream a little. Could you tell us a bit more about it?
AM: I am not the first to come up with the idea that neural networks have their input, their output, and parameters that work like software. The architecture is hardware, and the parameters are software. We know how to tune these weights to make the outputs agree with what we expect them to be, but you can also pass this signal to the input itself. You fix the rates and say, instead of changing the parameters, how do I change the input so that the output of the network still agrees with what I want to see? It was about asking the computer to show whatever it sees there, instead of asking for something specific. Those nets have layers, which are basically like measuring applications or convolutions. Instead of saying I want this output, the idea was to say that there are some numbers in between. I have no idea of what those numbers mean to you. I don’t care. I just want those numbers to be high. So, there is some pattern, there is some image; I used clouds because I enjoy looking at them and finding shapes. Whatever traits of objects you see there, I want you to enhance it, I want you to paint on top of those clouds to show what you see there.
Artwork: Alexander Mordvintsev, "Deep Dreams of the Strange Places", 2019 mint on elementum, link here
SG: In a nutshell: “Dear machine, paint for me"! *(3)
AM: Yes, that was it! Without limiting the machine’s imagination; just saying to it: whatever you are trained to recognize, whatever, you see there, I want more. *(4)
Image below: "DeepDream Cloud Watching" Retrieved from the original DeepDream code notebook
SG: In generative art, the artist defines the process and the machine completes the work of art; the output is a sort of machine and artist collaboration. How do you relate to the machine? Is the code your medium, your language?
AM: My artefact has always been the code. Code is my medium, and the visual side effects that it creates are its products. Code is the artwork. When it comes to making code out of characters and using language constructs, code is the medium, but I consider and treat the result as an artwork. I tend to focus on details that no one cares about, because probably no one reads it, and no one cares about how it’s written. I always try to make it very concise; I rewrite everything – four, five times – and I always revisit. Let me give you an example: DeepDream was approximately 30 lines of code, but this is 16; an it is probably the best code I’ve written in my life.
Images below: “Tricks” and “Octaves” on Producing Dreams Retrieved from the original DeepDream code notebook
SG: Is it a new research project of yours? Can we have a sneak peek?
AM: It is indeed a new research project and base for future artworks. It’s about artificial and mathematical life; a system with very simple, local rules and at the same time with very complex behaviour, controlled only by a few parameters. I can interact with it to some extent, and find some creatures, which then I extract and isolate in a kind of zoo; like a collection of interesting things that I found. Some of them have this very biological like life appearance and feeling, while others resemble tiny molecular machines. In a few lines, this code describes the complex behaviour of all those aspects, and that’s why I’m proud of it *(5).
Image: Particle Lenia
SG: During the writing process, you try to find the “words” that in a few lines describe “complex stories” – which is undeniably a poetic approach. Wasteland was the first work of yours that immediately caught my attention as it visually captures the echo of Eliot’s poem of the same title. Are they in any way correlated?
AM: The name Wasteland was given to this work by my wife – Iana Sam – and was indeed inspired by T.S. Eliot’s poem. A few years ago, we had a car trip to England, during which we visited the exhibition dedicated to Eliot in Margate: "On Margate Sands. I can connect Nothing with nothing". I used to call the technique I was using back then "AI Origami", as it’s based on "folding" image space many times to produce sharp edges and smooth gradients. (*6)
Artwork: Alexander Mordvintsev, "Wasteland", 2019 mint on elementum, link here
SG: In the diptych artist-machine, what is the role of the viewer? Does your work connect with the audience? Does it allow interaction and participation?
AM: In my scientific work, I always want to make things look beautiful aesthetically; I hate it when they look boring. I prefer, for instance, the publication format that is built on interaction with the reader; things that people could really try, touch, play with. I always want the audience to use and build on top of what I’m doing, and to develop all those ideas further; that’s why DeepDream and the code were published. Also, with this sneak peek at my new project, I want to invite people on a kind of a hunt: if you see something interesting, you can capture it, store it, save it, share it on social media. Speaking of interaction with the audience, Hexells *(7) is another project of mine, based on Neural Cellular Automata, which was trained to build textures upon interactive touch, and it explores the relation between parts and the whole. Everything I do cannot live without direct involvement.
Image below: Alexander Mordvintsev's Hexells self organizing patterns
SG: You call yourself a “mad scientist”. What are the questions that guide your research and how do they communicate with your artistic practice?
AM: My whole research is about self-organisation. The primary focus of my current work is Neural Cellular Automata, in three words: programmable artificial life *(8). It’s about setting ambitious goals in areas in which I have very little expertise, and hoping to bring expertise from another domain; that’s why I call myself a mad scientist. What is life, for me, is a question of aesthetics. From that perspective, the question that guides my research is: Does it look like life? Is it aesthetically similar to life? Does it have this ever changing, character of being, flexible, adapting, transforming, recreating, self-assembling? That's where my artistic feeling guides my research. One day I hope to be able to construct some experience that would maybe exist in minds and computers, as a creature.
SG: What led you to the NFTs and what is your opinion about the crypto ecosystem?
AM: First of all, it was elementum that reached me even before NFTs. Before you become an NFT platform, you contacted me, and I was happy to contribute with my work. Prior to Ethereum’s Merge to Proof of Stake, I was, quite often, rejecting offers, but now, I don’t have personal reasons to not do NFTs. Though I have complex feelings about the ecosystem: Many artists did some impressive work, and many benefited from that, but at the same time, the whole crypto landscape is such an attraction for scams and all sorts of ugly activities; and that makes me sceptical. When it comes to the viewpoint that the NFT is an artwork, it seems to me that there is an emotional connection with the object. It’s more like an artist’s signature, a postcard, a hand-signed postcard.
Artwork below: Alexander Mordvintsev, "Art Machine", 2017 sold-out on elementum, view here
SG: Could you tell us a few words about the behind the scenes of your latest series AutumnLife?
AM: It was earlier this year that my wife got very serious about gathering mushrooms. She always talks with me about mushrooms, she goes and checks the nearest forests a few times a week; sometimes we go together, and with our children. From a research perspective, there are some very impressive characteristics of them. Sometimes they are parasitic, other times they exist in symbiotic relationship with trees, they exchange nutrients, and they build very complex networks: namely wood wide web or internet of the forest. Then, trees that are connected to this internet, survive better, and they may better protect themselves from insects or other threats. My current work is about building complex systems, whose complexity is attributed to the accumulation of many simple things together, which results in a whole thing that is more than a sum of those simple things. AutumnLife is a tribute to the beauty of life and autumn forest, and especially to this mushroom kingdom, that is not super visible, and the scale of its influence on life is still yet to be fully understood.
Artwork by Alexander Mordvintsev "Autumn Wave", 2022 mint on elementum, link here
ABOUT THE ARTIST:
Alexander Mordvintsev (b. 1985) is a researcher and artist, exploring the artistic capabilities of AI; with a focus on machine learning, computer graphics and vision, and emergent phenomena. Mordvintsev works at Google Research in Zurich on Deep Neural Network visualisation, interpretation and understanding. In 2015, in order to understand how neural networks work and what each layer has learned, he developed a code example for visualising Neural Networks: DeepDream; since its creation, the algorithm has been influencing and inspiring researchers and artists alike, and Mordvintsev keeps interweaving art and research.
Image below: Alexander Mordvintsev at the Google Research offices in Zurich, November 2022. Courtesy of elementum
Notes
*(*1) Particle Lenia – Mordvintsev’s most recent research project – is an Artificial Lifeform that consists of point particles that simultaneously generate a potential energy field, and try to minimize their own local energy. Just a few parameters create a huge variety of different “creatures”. During the interview, Mordvintsev refers to these 16 lines of code as the best ones that he has ever written. The title of the interview “When Code Becomes Art, Machines Can DeepDream” pays tribute to this reference.
(*2) Video recording of The Flow demo from 2009 (link here) and pouet page (demoscene database) link here.
(*3) “Liebe Maschine, male mir” (“Dear machine, paint for me”) was a group exhibition that took place, from October to December 2022, at the pop-up elementum.art NFT Gallery in Zurich. Co-curated by Georg Bak and Kate Vass, the show covered a broad spectrum of algorithmic art with works by Frieder Nake, Manfred P. Kage, Herbert W. Franke, Hein Gravenhorst, Alexander Mordvintsev, Ganbrood, and Espen Kluge.
(*4) The detailed story and the “dream” behind the DeepDream can be followed here: Steven Levy, “Inside Deep Dreams: How Google Made Its Computers Go Crazy”, Wired, December 2015 (link here). Additionally, for a deeper understanding of DeepDream and its “hidden layers” read: Arthur I. Miller, “DeepDream: How Alexander Mordvintsev Excavated the Computer’s Hidden Layers” in The Artist in the Machine: The World of AI-Powered Creativity, Cambridge, MA: MIT Press, 2019 (link here).
(*5) Morvintsev’s most recent research work on “Particle Lenia and the energy-based formulation” link here and YouTube link here. To explore the interactive demo visit link here; the relevant thread on Twitter can be followed here.
(*6) The method is examined thoroughly in: A. Mordvintsev, N. Pezzotti N, L. Schubert, C. Olah, “Differentiable image parameterizations”, Distill, July 2018 (link here).
(*7) Hexells is a Self-Organising System of cells, and was trained to build textures using neighbour communication only. It exposes the relation between the life of an individual cell, and the cell collective as a whole, which allows interaction link here.
(8) For an overview of DeepDream, Neural Network visualisation, and Mordvintsev’s research focus on Self-Organizing Systems and Neural Cellular Automata, link here.