People scatter heaps of data to the wind, knowingly and unknowingly, but only a few outside of tech institutions truly understand how data is being used and the simulations that it feeds. Mainly because ‘Computer Simulation’ is a tricky concept, perceived as one of two: either a practical engineering tool such as wind simulators and economic models; or as a copying tool, an agent of digital fakeness – special effects in movies, Google’s Earth and the blue-skyed images made to sell apartments before they are built. Yet to understand computer simulations means nowadays to be literate – to understand, even if intuitively, tectonic shifts in our societies. The utilitarian qualities of these simulations are obsessively discussed, yet more must be said about the ways in which these simulations, and the computers that enable them, are shaping our minds and the worlds that are reflected in them. For that to happen and for us to gain a real idea of these new tools, we need to first ask what makes them so unique. One way to answer it is by tracing the main places where simulations resided, all the way to architecture, the first tool to monumentally combine culture, engineering and power.

Archi-tool

Built information was the ultimate tool in it’s days of glory.[1] Buildings served bodies, directed ceremonies and were encasements for intricate thoughts. Entering a circular stone dolmen, you could feel the presence of death and with a predicted ray of sun – the promise of a new harvest, a stepwell would offer you water, protection and connection to your ancestors. Yet the efficiency of building in mediating thoughts is limited. “At last they made books”[2], as Victor Hugo recaps. Texts, and finally heavily distributed books, are much more efficient as tools for storing, sorting and circulating thought. In fact, books are archi-tools, unique because they can hold detailed instructions for how to build any other tool. Time passed and dwellers of civilisations had to always adapt to new circumstances, but after the availability of books never did they need to master a new archi-tool, a new major breakthrough for implanting a simulation in the minds of people they will never meet.

Calculus and math presaged the coming of a new archi-tool. Numbers, like words, elusively operate in the space between describing the world and making it, though the two approach these tasks from the opposing aspects of language: the ‘open’ prose and the ‘strict’ math.[3] The notion is familiar: on the one hand we have the language of art, evoking emotions and a deep personal interpretation; and on the other hand the language of logic, equal to all things and allowing increasingly accurate simulations of the world. As the latter kept on proving its effectivity, ideas that combine universality with programmability multiplied and were crystallized by Alan Turing in 1937, who argued that any algorithmic process could be computed by a single universal, programmable machine. With this Turing Machine, finally the idea of a new archi-tool was established, a tool that can not only hold descriptions, but can also read those descriptions and make new tools all by itself. As Ed Finn puts it in his new book: “The algorithm is an idea that puts structures of symbolic logic into motion”[4]. With the rise of the computer, thought was no longer externalized but had life of its own, complex simulations could now happen outside of man’s head.

This new archi-tool destabilizes civilized societies in ways and frequencies like nothing before. With the computer, we’re no longer the only ones pushing symbols around while having physical and social abilities. Suggesting that machines are rapidly gaining spatial and social sensibilities might sound futuristic, but actually the forces behind this transposition are two very old concepts (in Silicon Valley years): artificial intelligence and spatial computing. Artificial intelligence now mostly consists of Machine Learning[5]; spatial computing is an umbrella term for everything used to collect information from the real world and then communicate it back to the environment around us. Combining the intuitive experience achieved by spatial computing, with the machines who increasingly get better at ‘making sense’ of the world, and we get computers that dissolve into the world around us.

Funes the Memorious

The myth of computer simulations is of efficiency, and when it comes to daily tasks most people expect computer programs to be doing things like bookkeeping or the pixelpushing that photoshop used to be. But actually computer programs now are mostly working in the background, unnoticingly collecting data, adding it to data sets, learning from those sets and spreading to new areas of human life which can be, in turn, computed.

In his short story ‘Funes the Memorious’ Borges describes Ireneo Funes, a young man that following a blow to the head can remember every detail of every event. He can learn all the languages and give each number a name. The story’s tragedy is that Funes can consequently no longer use generalisations, the way a normal person would, in order to make sense of the world. He would register the same dog in two different situations as two different creatures, seeing his own face in the mirror would always take him by surprise. Computers, just like Funes, are “not very good at thinking”. To think, Borges reminds us, is not to accurately store and recall information, but “to ignore (or forget) differences, to generalize, to abstract.”[6] This is the promise of Machine Learning.

Trying to summarize machine learning in a paragraph is shameless but here’s an attempt to touch two central concepts. The first relates to the algorithms named deep neural networks. Constantly tweaking and training deep neural networks, passing big amounts of data through their layers, each layer recognizing a pattern in the layer before it (e.g. first layer ‘lines’, second layer ‘shapes’, third layer ‘teeth’) you can reduce an input (e.g. a picture file, each pixel is one dimension of information), to a question (such as ‘is there a man in the picture?’ (or ‘is this a gay man?’ in a new ethically controversial research)). The second concept, is that like with all useful statistical methods you can actually reverse the direction of the function, and for instance tell the reversed neural network ‘a picture of a gay man’ and it will imagine one for you. The whole story becomes somewhat hard to digest when you realize that as input source you can use real monkey neurons and visualise the faces that the monkey have seen, and that ideal human concepts can be used in different ways as if they were mathematical vectors: subtract the output ‘man’ from a picture of Donald Trump and you’ll get an image not far from that of Hillary Clinton, add the concept ‘downtown LA’ to an image of Venice and you will get concrete cathedrals. Ireneo Funes finally found salvation.

Design Space and Spatial Computers

So how did these recent breakthroughs in Machine Learning come to be? Actually by a number of engineering achievements: cloud type server services able to host and process unimaginable amounts of data, connected to a diverse and exponentially growing horde of devices, through some very fast internet. The horde of devices part is the one to dwell on. In order to have huge amounts of data for your machines to feed on, it helps having cameras, microphones and other sensors sticking out of every hole.

The iconic form of a computer involves a screen and a keyboard, though from its early days of conception it was clear that a computer can actually take any shape. The ideal computer was already described in the 70s as being invisible. Any form of noticeable interface is just a distraction. We are, of course, not far from this desire – starting with the good old GPS, to newer forms of spatial tracking, and the omnipresent voice control (moisture sensors in smart greenhouses, gyroscopes and compasses in moving computers – the types of sensors are trying to surpass the types of information). The dream of Smart Dust – computers so small they can float in the air and befriend microbes – seems almost as an overkill.

Communicating information back to people with an invisible interface is a bit more tricky. Amazon’s Alexa solves this by talking back to you and having a delivery person appear at your door. The other ways to turn the computer boundaryless is to make people use virtual and augmented reality devices or to simply let objects communicate between themselves, while doing things on their own. Google’s server farm recently regulated itself to reduce 40% of its energy consumption bill. It’s something to get used to.

Once technology is invisible and absolutely hard to understand (even for programmers, that’s why this same Google gave up and is now asking its neural networks to create neural networks), things just start to seem like magic. Designed objects no longer need to be represented in distinct instances – 3d models are starting to be presented as a latent design space. You can choose if you want your chair to be more of an arm chair or more of a stool (Caitlin Mueller from MIT and others are working on making tools that help architects explore those endless spaces more intuitively). Ian Cheng, an artist with the appropriate background of cognitive science, names his works simulations. Whether in virtual reality or on a screen, seeing his video games endlessly play themselves as a complex system of characters and sceneries is an experience nothing like watching a movie or a play. VR can also help achieve more practical results where other architectural mediums are failing. In one of their research projects IAAC’s Areti Markopoulou and her team have managed to convince hard working families in India that they need to take care of a garden, no less, in order to be happier – all by simulating the new housing block, and letting residents experience it in VR. Following the virtual tour the opinions changed dramatically from absolute disapproval to absolute approval (the project was sacked – in the non-virtual reality you still have developers and politicians). The magic of contemporary computer simulations is bringing back the old practice of trying out architecture before building it. Not in movies or images, but as an interactive experience similar to the ones used during the renaissance, where new design ideas were tried out as sets for royal celebrations and were planted in people’s imaginations. The space around us is now the interface.

In conclusion—there’s a global computer, its artificial intelligence runs in the clouds, its spatial interface occupies every corner and pocket. We’ve come full circle with externalized imagination, all the way back to its architectural origin. Architectural imagination again becomes the prevailing type of imagination used by people to understand new and complex ideas. We must now always ask who controls these imagined spaces? How is the future shaped by this? Where do architects stand in this new situation? People are being mediated, viewed and influenced in all sorts of architectural ways: through movement and access, using space and visibility and even just by disappearing behind the scenes. And so, it is not so daring to suggest that architects should and would take a major role in the curation (curating and curing) of this new world, where human simulations and computer simulations are intertwined. Computer oriented professions will identify more as architects (already noticeable with ‘geomatics engineers’, ‘game level designers’, ‘3D experience designer’ and even ‘software architects’), architects should be open to this crossing of lines, whilst simultaneously drawing the computer simulation into their scope of influence and discussion—mainly because it’s already there.

[1] In The Work of Art in the Age of Mechanical Reproduction Walter Benjamin mused: “Architecture has never been idle. Its history is more ancient than that of any other art, and its claim to being a living force has significance in every attempt to comprehend the relationship of the masses to art.” (2007): Illuminations, translated by Harry Zohn. New York: Schocken Books. p 240.
[2] Victor Hugo: The Hunchback of Notre Dame. Book V, chapter II: http://www.literaturepage.com/read/hunchbackofnotredame-175.html
[3] Bialik, H. N. (1975): Revealment and Concealment in Language, in Modern Hebrew Literature, edited by Robert Alter. New York: Behrman House.
[4] Finn, Ed. (2017): What Algorithms Want: Imagination in the Age of Computing. MIT Press, Kindle Locations 866-867.
[5] A broad set of techniques used to automatically analyze and create big amounts of information.
[6] Borges, Jorge Luis. Funes, his Memory: https://faculty.washington.edu/timea/art360/funes.pdf p 1.37

Yonathan Stein is an architect and product designer based in Berlin.