This post features a project by Tim de Wilde which uses Trill Craft to sense touches on a plant’s leaves and branches. The signal are then used to control Ableton and generate interactive visuals which are projected down on to the plant. Over to Tim to tell us all about this project.

Plastasia: natural music

This project uses a Bela Mini in combination with a Trill Craft touch sensor. I wanted to use the capacitive sensor to register when a plant was being touched, and to feed this sensor data into Ableton and use it to trigger sounds. Pure Data was used to convert the sensor data into MIDI values and send it over USB to my laptop running Ableton. In addition to this I created interactive visuals to be projected down on the plant. I worked with p5.js for this, using the GUI functionality in the Bela IDE to generate the visuals.

A top down view of the plant with capacitive sensors attached.

To sense touches on different parts of the plant I simply attached five jumper wires to the leaves and branches using conductive tape. The tape also has conductive adhesive and ensured that there was a good connection between Trill Craft and the plant. The other end of the jumper was connected to one of the pins of the Trill Craft, which was then in turn connected to the Bela Mini via I2C.

Bridging from Bela to Ableton

Left: the Trill Craft connected to the Bela Mini. Right: the conductive tape connecting the jumper to the plant's leaves.

The Pure Data patch I created reads the data from Trill Craft and uses it to generate MIDI notes when the readings pass over a certain threshold. There are 3 leaves and 2 branches which each trigger a short MIDI-note 60 on a specific MIDI channel (1 to 5) when touched. As the branches and leaves have different surface areas and amounts of water in them, this results in different ranges of capacitance readings from the Trill Craft sensor. I compensated for the different ranges by offsetting the readings in the Pure Data patch.

Block diagram of the whole system.

Inside Ableton I then selected BELA as a MIDI device and inserted 5 MIDI tracks and routed each of the preselected MIDI channels to those tracks. Bela shows up as a plug-and-play MIDI device so configuring this was simple. I then set up a VST (Xfer’s Serum) on the tracks and gave them all a different tonal sound. The versatility of MIDI in Ableton meant that I could change the sound of the instrument at the touch of a button.

Top view with interactive projection in action.

Alongside the MIDI note conversion, the sensor data in the Pure Data patch is also fed into the sketch.js file for triggering visuals in the Bela GUI. The visuals consist of randomised circles which change in size when the plant is touched. Finally I hooked up a projector to the laptop that displays a fullscreen image of the Bela GUI, aimed down on the Plant.

You can find all the design files for this project here.

About Tim de Wilde

Tim de Wilde.

Tim de Wilde is a sound artist based in the Netherlands. He studied at ArtEZ University of Arts, MediaMusic (until 2020), focussing on audio engineering and sound design as well as music production and recording. Over the course of those years and the years following his attention got drawn to making his own instruments and gear. Now he works as a freelancer in audio production and sound art, with his own company, TIMProductions. He started doing research, playing with Pure Data and Bela and building projects with them just two years ago, and they have been his main tools ever since.