Contact

Digital surfaces: Mixing physical textures with digital art

Interaction Design, Projection Mapping, Interactive Installations, JavaScript

Aug 2021

My Role

Concepting, Prototype, Coding

Time

3 weeks

Digital surfaces is an installation that brings textures you see all around barcelona to a digital space

We came up with a concept, Prototyped and pushed it as far as we could.

What do we do?

We started by brainstorming and deciding on the topic we want to tackle. There were a lot of ideas but in the end we thought that since we all were in Barcelona at the time, it would be interesting to do something that all of us can relate to.

Another thing we thought of was that we wanted people to use senses to interact with the installation. It would be cool if it would be tactile, had different interesting textures etc.

And finally, we wanted to get out of the digital space at least for a bit. The idea of mixing analog and digital would turn this project to something real.

These were our three pillars: being local, using sense of touch and a transition of digital to analog.

Brainstorming session

Our brainstorm session

So, we started concepting…

Barcelona is a city full of art. There are so many textures, patterns and ornaments that keep appearing all over the city. That was actually the main inspiration for us.

When we started thinking of the actual examples, we realized that there's a couple that are most popular. Our list included bricks from the area of Vila Olímpica, marble from a lot of interiors or plazas, mosaics from Gaudi's heritage and tiles that cover the streets of Barcelona. So, we got the local aspect of the installation.

Remember mixing analog and digital? We thought it would be sick to have a big surface, maybe a whole wall, covered with the textures and let people do something with them.

3 attributes of an installation
Types of textures for the installation
Sketch of an installation

Next step was to clarify the interaction. We went outside to actually find these textures and found out that all of them had very nice and distinct tactile characteristics. When we think of an interaction for an installation, we think of some touchscreen, button or something electronic. But what if your controls would be the physical textures? That's how we came up with an idea of swiping on let's say a piece of marble and that would trigger an animation or some other effect.

Our concept was that only a part of the wall in the center would be covered with physical tiles, the rest would be empty. However, when you'd touch a tile and swipe on it, your movement would continue from physical tiles to digital ones projected on the wall in a form of particles. Basically, you'd be able to shoot out digital texture from a physical one.

if we had money...

If we didn't have to think about a budget, we'd get a big exhibition space with a lot of walls. Every wall would have its own texture and it's own pattern. Chunks of physical textures would be scattered around the walls. Around every chunk we'd have special frames that would detect touch gestures on the surface and basically turn the wall into a touchscreen.

How cool would it be to just walk by the walls, swipe your hand on a piece of mosaic and see how your motion continues on huge walls around you? That would be so, if we had the budget for it.

Wall installation 3d concept renderWall installation 3d concept render

3d Renders of our concept. Cinema 4d

but we didn't, So...

Yeah, the hardware for this was out of our budget and we didn't have that big of a space, so we had to come up with something different to prove our concept.

Regarding the hardware, the solution was simple - instead of getting a special frame, we just decided to use an ipad and surround it by actual tiles. Using an ipad actually breaks one of our ideas - mixing analog and digital. But that was the easiest and the fastest way to prove our interaction model with the resources we had. So, we had to compromise.

We mounted all of that to a white board so we could carry it around campus and show to other students. Then we also decided to put on some music to match the visuals and went on to bother people with our installation.

For our mvp we decided to choose only one texture and to use just one wall to do the projection mapping on. We went with Passeig de Gracia tiles as our texture because they have a very geometric form, which is easier to work with. At the same time the tiles still look very beautiful with cool floral pattern and also associate with Barcelona.

How to place tiles around an Ipad

The physical tiles leave big gaps, which don't look good.

Fill in gaps aroubnd Ipad with tiles

Filling in the gaps with paper tiles.

Using an Ipad as an interface for the installation

Instead of a touch frame we used an Ipad.

The process of assembling everything and mounting on a whiteboard.

talking TEch

The code for this installation was done by me in JavaScript using SpaceBrew library to get the gestures from the ipad and connect them to the visuals on the wall.

When the gestures on the ipad are recognized, a vector of the movement is being sent to the server and a particle is being shot out of the center trigerring tiles to scale up depending on the proximity to the particle. All of that was drawn to a JavaScript canvas and displayed on a webpage.

For the projection mapping I used another library called MapTastic to map the webpage exactly to the whiteboard.

In the end, all of the hardware included an Ipad, a laptop and a simple projector.

Wall installation 3d concept renderWall installation 3d concept render

Our proof of concept for the installation.

The ipad is surrounded by a mix of real tiles and photocopies.

Feedback

One of the main concerns for us before launching the installation was that it would be just boring. We thought that it wouldn't be enough to keep people entertained.

And then the day of presentations comes. Imagine how happy we were when people stood there for solid 4-5 minutes just swiping left and right on the ipad. The fact that your movement goes beyond the screen of an ipad was truly fascinating. And all of that  was done by a very simple gesture.

So, we were quite satisfied with the outcome. We proved our concept, came to a conclusion that installations don't have to be crazy expensive and complicated and got a lot of inspiration for future projects.

P.S. Huge thanks to my teammates one more time, they helped bring this project to life: Amir, Masha and David.

Aug 22, 2022

Next Project

solovey cinema rebranding. what if it hasn't been demolished

Read