Category Archives: practice based research

No Fixed Terrain 0.1

Last week I was lucky enough to attend the Explorations on Sound and New Media Art Conference 2026 hosted by the School of Arts, Universidade Católica Portuguesa, Porto, Portugal. The theme was ‘on territories: speculative practices, transgressing and belonging’.

I presented a paper entitled ‘No Fixed Terrain’ describing a prototype art work of the same name, building on previous research exploring real-time landscapes within a performed audiovisual context.

“No Fixed Terrain is a live interactive installation for two participants and a real-time generative landscape system, developed in TouchDesigner and driven by MIDI controller input. Building on an existing single-performer landscape generator, the work extends the system so that two participants — who need no musical training — each shape a distinct terrain layer, and its accompanying sound, through direct interaction with a MIDI controller. A third, non-human layer runs beneath both: historic environmental data plays back on its own fixed clock, indifferent to either performer’s input. This layer periodically reshapes the available ground for both terrains, compressing, flooding, or redistributing the space the participants are contesting.”

I received confirmation of acceptance in mid-September, giving me just a few weeks to develop the work, test it, and write the paper in time for the conference in early November. Inevitably, given the timescale, the resulting outcome is effectively a first stage prototype.

From a technical perspective, No Fixed Terrain uses Particle Operators (POPs) in TouchDesigner to render a landscape derived from procedural noise. Climate data was retrieved from ERA5 monthly dataset, available from Copernicus, the Earth Observation component of the European Union’s Space programme.  25 years of climate data for the UK region of East Anglia were extracted for the years 2000 to 2024.

The data does not reconstruct East Anglia. Instead, it establishes conditions within a speculative landscape.

The sound design was conceived as a musical environment rather than a literal sonification of the climate data. Bass, arpeggio and percussion form three core musical layers, with the climate data determining the material selected within each layer. These establish an underlying musical structure that remains responsive to changing environmental conditions.

Each participant is given control over one part of the generative system. Their actions alter both the landscape and the sound, so visual and musical changes take place simultaneously. Neither participant has complete control over the resulting composition, and both are operating within environmental and musical conditions established by the climate data. Importantly, the participants are not simply responding to those conditions. They can manipulate the three core musical layers associated with temperature, precipitation and wind, and can also introduce three additional sounds into the composition. So there are different levels of agency within the system: the climate data establishes conditions, while the participants intervene within, modify and add to those conditions.

Formative user testing was undertaken to explore how participants understood and interacted with the work. Responses suggested that engagement increased as participants began to discover relationships between their actions, the terrain and the sound. One participant described the interaction as being “almost as a puzzle”, while another described the experience as becoming increasingly engaging and hypnotic.

Ultimately, No Fixed Terrain explores what happens when environmental data moves from the background into the interaction itself. Participants can manipulate the landscape and its sound, but never have complete control over the conditions within which that interaction takes place.

I hope to develop this work further based on conference and testing feedback – watch this space!

Embodiment Conference

I recently presented a paper exploring embodied interaction with typography at ‘The Body and Embodiment in the Arts and Arts-based Research’ conference organised by the London Arts-based Research Centre (LABRC), Camden.

It was a great opportunity to comment on a number of influential works and reflect upon my own practice in this area. A diverse range of researchers and artists presented outcomes from a variety of disciplines with ’embodiment’ as a connecting theme. As the theme was widely interpreted, the work discussed ranged from land art to poetry to literary review to interactive VR experience and much more.

The diverse yet engaging nature of the conference, as well as the low key charm of the Camden town house setting, gave the event a creative community atmosphere with a certain level of intimacy and authenticity. Thank you LABRC.

My paper received generally positive feedback and elicited a number of interesting conversations and connections. You can check out my slides below.

360° Audiovisual Performance at Art Tech Play March 2026

On March 18th 2026 I performed an audiovisual work which, although unnamed and effectively still work in progress, is the product of research and development over the previous 6 months or so. The performance took place within the 10-metre diameter, 4-metre high, circular LED screen at Norwich University of the Arts’ Immersive Visualisation and Simulation Lab, as part of the Collusion-led Art Tech Play programme.

Inspired by the classic quote from the Bhagavad Gita, Chapter 11, Verse 32, variously translated.

“I am Time, the mighty cause of world destruction, Who has come forth to annihilate the worlds.”

Translated by Winthrop Sergeant (1984)

The work explores themes of destruction, natural resilience, and human responsibility (or lack of it) and uses TouchDesigner to create visual outcomes that respond in real time to performed electronic music, using both instrument MIDI data and audio data as inputs. Although I have been experimenting with audiovisual performance for some time, this work is somewhat new territory for me as the content is largely videographic.

It is easier than ever to collect libraries of videos, many offered free by individual videographers. Some are highly composed, others more naturalistic. Increased ownership of high quality video cameras and the proliferation of drones are both leading to greater numbers of ‘citizen videographers’ producing compelling video content, for example https://www.pexels.com/.

The principal mechanic of the work is a noise grid used to mix different elements of video that change according to a weighting of the following factors:

  • Random/Noise
  • Musical time (i.e. changing every 2 bars)
  • Response to specific instrument data (e.g. a kick drum)

The graphic below demonstrates the relationship between the noise grid and resulting video collage.

Content is organised into ‘scenes’ which map directly to scene buttons on the Roland MC-707 hardware used to drive the sound. Each scene has a number of parameters including:

  • A folder of video files to feed into the grid
  • A possible overlay layer to show on top of the grid
  • Parameters that relate to the randomisation of video grid divisions (i.e. a minimum and maximum number of columns and rows)

This graphic below demonstartes the relationship between the hardware scene buttons and the scene data stored in TouchDesigner.

For some scenes, a monochrome overlay is shown on top of the video grid.

Incidental performative effects are mapped to audio parameters. For example, modulating the release of the snare envelope (how long or short the drum sounds) results in the video grid cells rotating to the left or right.

A classic perfomance effect within many electronic music genres is ‘beat repeat’ where a small sample of audio is repeated in time with the music. By using MIDI information to trigger audio beat repeats, a visual equivalent can also be triggered. In the case of a 16th note (semiquaver) beat repeat, a visual equivalent builds from left to right every semiquaver. Other repeat intervals are represented similarly, e.g., quaver, triplet etc.

A master audio sweep filter, another common electronic music performance effect, is aligned with post visual filters i.e. final stages of the visual composition process. A black and white edge detection image is used for audio lowpass, high-brightness ‘blow out’ used for audio highpass.

The work was adapted for the 360 screen relatively quickly, so it was very much a test. However, I did have chance to perform the piece twice, each time for around 15-20 minutes which gave me a good opportunity to test out the performance mappings.

Although I don’t have a decent video capture of the event, fellow artist @oculardelusion kindly posted a couple of videos on Instagram – https://www.instagram.com/p/DWFMEk8AqM9/

Feedback from the night was generally positive although I didn’t get chance to talk to that many people in depth as I was busy performing and also co-organising the night! One interesting comment I received was that it was notable and possibly unusual for an av performer to be looking at the large screen rather than the computer screen during a performance. In fact the computer screen had nothing on it! But more importantly, this felt like the right thing to do in a large scale improvised performance situation.

The research leads me to consider a couple of themes for potential future exploration.

Theme #1: Mapping and Translation Strategies

How do different MIDI-to-visual mapping approaches affect both performer agency and audience perception?  E.g., exploration of one-to-one mappings versus many-to-many relationships, examining when literal translation (note-to-shape) versus abstracted relationships work better, potentially developing a framework for “meaningful” versus “arbitrary” mappings based on practical investigation.

Theme #2: Collage Semantics & Meaning Making

What happens when found footage collage meets electronica’s functional dance context? E.g., whether narrative fragments emerge from triggering choices, investigate how juxtaposition of source materials creates meaning (or deliberately avoids it), as well as examining the politics of source material selection – whose images, from what contexts, and how does recontextualization operate?

TEI’25

I’ve recently returned from the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI’25) hosted by Université Bordeaux. This was my first experience of TEI and it was really insightful and encouraging to meet and hear from international researchers and creative practitioners at the forefront of tangible interaction in its many forms. This annual conference is essentially an international community of talented makers keen to constructively critique one another’s research and practice.

I was very happy to show an installation ‘Suitcase Stories’ which was included as part of the conference Arts and Perfomance track, exhibiting at Capc Contemporary Art Museum Bordeaux. I proposed Suitcase Stories in response to the Art and Performance call which asked artists to propose an art work fitting into a suitcase, somewhat inspired by La Boîte-en-Valise [Box in a Suitcase] by Marcel Duchamp, also responding to the conference theme of sustainability.

A number of themes I had previously been considering fed into the ideation process. Firstly, the 17 United Nations Sustainable Development Goals, many of which are concerned with improving social and personal factors, not just developing infrastructure, although these aspects are evidently interrelated. Secondly, lived experience and personal testimony of climate change – I personally find first hand accounts of subjects such as flooding and drought very moving. Thirdly, bodily interaction with typography as a way to develop a near-physical relationship with narrative.

This led me to develop the following proposal:

The Suitcase Stories installation facilitates embodied interaction with typographic representations of firsthand accounts of climate change. The work aspires to create a powerful and visceral connection between the participant and individuals affected directly by environmental changes, thus developing greater awareness and propensity towards positive action. Participants interact with powerful and often emotive narratives through upper body movement and gesture, discovering and playing with words and phrases. The entire artwork is integrated within a vintage suitcase, adding aesthetic and thematic dimensions to the user experience. Through this installation, the author sets out to test whether interactive typography, controlled through body tracking, can offer a unique platform for disadvantaged voices to be heard in a powerful and immersive way.  

I developed the following lo-fi mock-ups to support the proposal.

Although I had already created bodily-interactive typography some years ago (e.g. Journey Words), this was created using now defunct Quartz Composer patches and recently I have been keen to push the limits of TouchDesigner. So, with just a weekend to create a prototype from scratch in time for the application deadline, I managed to develop the following demo in TouchDesigner.

My proposal was thankfully accepted and I started work in earnest on the installation following the Christmas break. My first task was to find suitable first-hand accounts of climate change that I could legitimately use. After some discourse with organisations such as the Climate and Migration Coalition, it became apparent that several first-hand accounts published by organisations such as Oxfam are already widely shared, with due reference, across media platforms. In the end I re-used and referenced the following accounts:

1 Extract from the account of Marcos Choque, 67-year-old inhabitant of Khapi, in the Bolivian Andes. Oxfam International. 2009. Bolivia: Climate change, poverty and adaptation. Oxfam International, La Paz, Bolivia. 
2 Extract from the account of Sefya Funge, 38-year-old farmer in Adamii Tulluu-Jido Kombolcha, Ethiopia. Oxfam International. 2010. The Rain Doesn’t Come on Time Anymore – Poverty, Vulnerability, and Climate Variability in Ethiopia. Oxfam International, Oxford, UK. 
3 Extract from the account of Tulsi Khara, 70-year-old inhabitant of the Sundarbans, in the Ganges delta, India. Quoted in: WWF. 2005. Climate Witness: Tulsi Khara, India.  WWF. 2005. Climate Witness: Tulsi Khara, India. Retrieved from: https://wwf.panda.org/es/?22345/Climate-Witness-Tulsi-Khara-India 
4 Extract from the account of Rosalie Ticala, 33-year-old typhoon survivor of Baganga, Mindanao Island, Philippines. Thomson Reuters Foundation. 2013. Typhoon survivor Rosalie tells her story in Philippines. Retrieved from: https://news.trust.org/item/20130214164700-xrwmo 
5 Extract from the account of Fatay and Zulaikar, husband and wife of a pastoralist family in Badin district, Sindh, Pakistan. Quoted in: Climate & Development Knowledge Network. 2012. Disaster-proofing your village before the floods – the case of Sindh, Pakistan. Retrieved from: https://cdkn.org/story/disaster-proofing-your-village-before-the-floods-%25e2%2580%2593-the-case-of-sindh-pakistan 
6 Extract from the account of Assoumane Mahamadou Issifou, NGO employee in Agadez, Nigeria. Gavi,  Quoted in: The Vaccine Alliance. 2023. Ten eyewitness reports from the frontline of climate change and health. Retrieved from: https://www.gavi.org/vaccineswork/ten-eyewitness-reports-frontline-climate-change-and-health 
7 Extract from the account of Parbati Ghat, 58-year-old community health volunteer in Melauli, Baitadi, Nepal.  Quoted in: BBC Media Action, Living Climate Change. 2024. Mosquitoes in the mountains, Nepal. Retrieved from: https://www.bbc.co.uk/mediaaction/our-work/climate-change-resilience/living-climate-change/nepal-malaria 
8 Extract from the account of Lomilio Ewoi, pastoralist in Kenya. Quoted in: BBC Media Action, Living Climate Change. 2024. The flood that took everything, Kenya. Retrieved from: https://www.bbc.co.uk/mediaaction/our-work/climate-change-resilience/living-climate-change/kenya-mental-health 

The TouchDesigner development process was quite time-consuming although I managed to achieve an outcome by resolving challenges in key stages. My original demo essentially mixed a Kinetic Typography tutorial by bileam tschepe (elekktronaut) with a 3D Grid with Depth Camera tutorial by Motus Art. This resulted in a grid of letters extruded using depth information provided by a RealSense D345 camera. However, I wanted to only show letters extruded by the body and not those on the background grid, but keeping the left to right flow of text, as shown in the diagram below.

I managed to do this by using a threshold filter to isolate image pixels above a certain greyscale level and therefore considered to be part of the body, and then used this information to reorganise the text indices relating to each grid cell so that text only appeared on cells extruded by the body.

With artistic works featuring typography, there is often a tension between aesthetics and legibility. I wanted participants to enjoy playing with the text, forming ‘word sculptures’ with their bodies, but ultimately they needed to be able to read the text. To this end, I developed a phrase cycling function that sequentially picked out individual phrases when the participant stopped moving and after a short delay, showed them as pure text overlayed along the bottom of the screen. Although it perhaps seemed slightly obvious to do so, I wanted to ensure 100% legibility of these important stories.

The fabrication side of the project came together quite serendipitously, I found a medium-sized, yet robust, vintage suitcase in an antique bazaar which just happened to be a perfect fit for an old monitor. I used photography mounting components to hold the lid of the suitcase open and support the weight of the monitor, then created a mount for the RealSense camera in the suitcase lid. After several failed attempts trying to get the RealSense camera working with my MacBook (official RealSense support for Mac is discontinued by Intel), I managed to borrow a gaming PC laptop and house that in the body of the suitcase. The last additions were a fan to cool the laptop down and some suitable material to stretch across the interior of the suitcase (which by now had several holes cut into it). You can see the results in the images below.

I had some really useful and insightful feedback from the conference attendees and also a number of productive conversations about using technology to explore human narratives – essentially Digital Humanities. I’m taking a break from this project for a short while to reflect upon where it might go next. In the meantime, if you’re interested in reading the accompanying ACM short paper, you can find it here: https://dl.acm.org/doi/pdf/10.1145/3689050.3707677

And here is a screen recording demonstrating the phrase selection functionality as it was presented at TEI’25.

Research Presentation at Beyond 21

On October the 20th, 2021, I presented my preliminary research findings from The Multitude project at Beyond 21, ‘a conference where thinkers, makers, investors and researchers across the creative industries come together to explore the relationship between creative research and business innovation’. The event was physically held in Belfast with many elements presented online.

Here is the research poster that I submitted (select to enlarge):

And here is the presentation that I delivered:

Initial testing of The Multitude

The Earth Elemental Scene

Myself and producers Collusion recently tested The Multitude at Cambridge Junction in preparation for exhibition in late June.

Cambridge Junction

The installation requires a technically demanding set-up with multiple screens, speakers and programmable lights.

Routing leads
Checking positioning

We had a couple of technical glitches, as to be expected, but managed to do a few run throughs with testers in the late afternoon.

Our very first testers!
More testers
These two should know what they’re doing (artist and producer)

Generally, the installation worked well, but there are some tweaks to be made, as to be expected.

  • Some of the voices need to be re-scripted slightly and re-recorded
  • The sound design generally works although needs to be completed
  • The aesthetics need developing more for some scenes
  • Object placement needs to be improved to make the most out of the 3 screen set up
  • The core interactions require minor tweaking
  • Secondary interactions need to be added to give players something to do during the character introductions

There is actually quite a long list of minor tasks that need to be completed in the coming weeks, the above are just the highlights based on user feedback.

A side panel from the Air Elemental scene, one of the more visually resolved scenes.

Next steps: more development and testing!

Introducing Vision Mixers

I am pleased to be working with Cambridge-based art tech producers, Collusion, on a new interactive installation to be exhibited in Cambridge in late 2020. The venue will hopefully be Sook Cambridge.

Sook is an ‘adaptive retail space’ that can be easily customised to suit typical retail media requirements and booked by the hour. They have also experimented with hosting more artsy events. Sook at the Grafton Centre, Cambridge, features a number of grouped screens and addressible lighting. It’s the perfect environment for immersive audio-visual content, but naturally there are issues of interoperability to be addressed when installing a bespoke system that features realtime interaction.

The original project elevator pitch was “a social distancing compatible, two-person installation that makes use of whole-body interaction to explore inter-personal energies and connections within a 360° audio-visual environment”.

Early mock-up.

Since acceptance of the initial proposal, this project has already come a long way with writer, artist and multi-talented creative person Anna Brownsted joining to help develop narrative elements. Anna, myself plus Rachel Drury and Rich Hall from Collusion, all took part in a socially-distanced residency at Cambridge Junction in early August in order to get things moving.

There is a significant creative concept, currently in development, that will drive all narrative components and frame the interactions. I’ll write some more about this in due course, but for now will focus on documenting the residency and discussing key intentions for the project. I should add that this project is part of my ongoing research inquiry into gestural and embodied interaction within public-facing art. Non-technical areas of knowledge that I seek to develop are:

  • the nature of agency within public space and how this is mediated through performative interaction
  • the mechanics of co-interaction, especially the compete-callaborate axis
  • non-contact interaction design, especially in the context of the current Covid-19 crisis
  • the use of narrative to frame interactive experience

The project involves a relatively complex technical set-up with a single Kinect and host computer driving multiple screens and eventually surround audio and lighting, via DMX. The Kinect (v2) runs through Unity and allows pretty robust skeleton tracking, therefore providing the foundation for full-body interactive gestures. I will cover technical aspects more fully at a later date, but of course, expect there to be much learning in this area.

Here are a few photos and a video that illustrate the prototypes that were developed and tested during the residency.

Simple visualisation of users and movement of hands.
When the hands move faster, fireballs are produced!
Lots of fireballs produced by lots of movement.
Testing a user duplication routine.
Picking up ‘leaves’, testing in Sook.

The video shows the fireball mechanic being tested at Sook.

The residency was massively informative in terms of validating the core interactions and kick-starting the project, especially by forming the team. It was a refreshing novelty to work in the same room as others after lockdown, even if we were santising left, right and centre! We also established some key technical parameters. It’s a really exciting, although slightly daunting project, especially as it runs in parallel with my full-time teaching responsibilities! More updates soon.

Being Human 2019 – Evaluation

The session began with an insightful discussion about museum display item descriptions in general and whether they are read. One opinion was that they are good to have on-hand but often may not actually be read in full. However, the group seemed to agree that it is generally better to have more detail than less.

This reminds me of the classic inverted pyramid (‘Inverted pyramid (journalism)’, 2019) writing technique where the reader delves into as much detail as they desire, assuming that the reach of the pyramid matches the reader’s expectation. Following this structure, the body of writing answers all the key questions in brief within the opening sentence(s) before moving on to incidental/secondary detail that may only be of interest to a particular audience.

The workshop proper began by asking participants to examine key texts and then devise keywords to act as labels for these. In the first pass completed as a group, participants came up with quite different keywords for the same text, perhaps unsurprisingly. A keyword in this context almost certainly denotes personal insight or individual significance to a greater extent than being constructed as a signpost for someone else. To construct signposting keywords for general usage, there would need to be a more systematic approach.

Boudica I

The story of the Queen of the Iceni has fascinated every age from Roman times to the modern day.

She was very tall and severe. Her gaze was penetrating and her voice was harsh. She grew long red hair that fell to her hips and wore a large golden torc and a vast patterned cloak with a thick plaid fastened over it.

Dio Cassio

A Roman writer

An example of a key text.

Participants continued to work through texts individually and associate keywords with chosen samples. They then chose one particular keyword and spent 20 mins developing a ‘creative outcome’ using paper, pre-printed typographic elements, pens, patterned paper etc. Some thoughtful and attractive versions of keywords were produced. This gave rise to a conversation about the typography of a keyword as an ‘affordance’ (Norman, 2013), although the actual term affordance wasn’t used directly, i.e. as a signifier of the nature of the respective content. This is an interesting area of discussion, as originally I had conceived the project as being a kind of test of textual interaction, and to see how much interest could be developed by just working with text as a user interface concept. The text-only constraint was conceived before I started working with the museum as a collaborative partner and was based on the premise of exploring texts that exist in their own right, not relating to a physical object as the museum texts generally are.

In any case, text must be rendered using a character set which effectively establishes ‘typographic voice’, a widely known graphic design principle. So, choice of font will always have some bearing on perceptions of keyword affordance and of course, some fonts are highly embellished with ornamentation and/or feature iconography to the point of being at least as pictorial as they are typographic.

There was some discussion about the information hierarchy relating to museum displays and one suggestion was that it would be great to move from one item to another ‘like using the internet’ which I interpret as the idea of there being an organic-like information structure behind a given keyword. This structure might be more shallow and wide than deep and constrained. This idea chimes with my original concept of being able to traverse an information hierarchy by using a keyword alone. But this scenario requires information design of far greater scope than the current prototype.

Although the workshop participants were quite interested in the LEAP controller, it seemed that this was more of a secondary attraction to the discussion and practical investigation of keywords and associated texts. I did manage to get a number of passers-by to try out the prototype and altogether I gleaned some useful insights. One woman managed to really get the hang of the interaction scheme. Initially the controller didn’t pick up her hands and I suggested that she either roll her sleeves up or take off her coat, which she did, showing good commitment! (I have noticed in previous situations that even slightly overhanging sleeves can affect the LEAP controller’s ability to recognise hand position.) It took her a couple of minutes, but then she was navigating, selecting and closing content quite fluidly. She commented that it was a bit like ‘learning to drive’ and that once you got the hang of it, ‘the muscle memory’ took over. She was the success story of the session and her words resonated with me.

Other participants struggled to lesser or greater extent to master control. A younger child (my son) had a go, but also struggled somewhat and this may be to do with the fact that smaller hands do not seem to register well with the LEAP.

In general, the collaborating partners agreed the Being Human 2019 project was a positive experience that helped to develop a new creative partnership and conduct preliminary investigation into the design of gestural interactive experience with textual elements of the museum collection. Although only one of the two planned workshops actually took place, valuable insights were gained:

Consideration of place –  the nature of a workshop is affected by the physical space it takes place in, particularly if this an art gallery.

The relationship between text and object – where text is derived from an object, especially one that is close by, text accompanied by visual reference to the object makes more sense. Conversely, it makes less sense when the visual reference is missing.

The power of preliminary discussion – there was profound and informative  discussion about experience design and female gaze within the museum at the beginning of the workshop.

The challenge of establishing meaningful workshop outcomes –  the hybrid format of the workshop, physical and virtual, worked to a certain extent but could be developed to guide participants towards achieving a more tangible outcome(s) potentially over a longer period of time.

The inherent challenges of gestural interaction  – some participants found the interaction tricky. It may be that gestural interaction is more suited to a gamified experience where there is a more distinct and fun ‘pay-off’ as a consequence of ‘mastering the moves’. In general, it probably needs to be made simpler for a public-facing experience.

The likely benefit of a longer collaborative cycle – with new technology, the first point of contact is effectively an introduction. Subsequent sessions would be beneficial for participants to have more meaningful input into experience design, including potentially testing work in progress with members of the public.

References

‘Inverted pyramid (journalism)’ (2019) Wikipedia. Available at https://en.wikipedia.org/wiki/Inverted_pyramid_(journalism) (accessed 24/11/19)

Norman, Don (2013) The Design of Everyday Things. Revised and expanded. Basic Books.

Being Human 2019 – First Iteration

After conversations with a number of potential partners, I ended up submitting a proposal to the Being Human Festival 2019 in collaboration with Norfolk Castle Museum and Art Gallery. The idea is to develop an experimental interactive experience, using the LEAP Motion Controller (LMC), that can be used to explore hitherto overlooked texts in the museum archive. ‘Touching the Past’ will initially be run as a closed workshop and then at a follow-up event at Norwich Castle during the week commencing the 18th of November.

“’Touching the Past’ is a series of workshops that encourages young people to explore overlooked stories in Norfolk Museums’ collection through digital experience and gestural interaction.”

Promo Image for ‘Touching the Past’

The picture is purely for promotional purposes as is often the case with events where the promotion takes place before the work has actually been made.

From a production perspective – I chose Unity over Unreal Engine as a Leap-friendly development platform. This is partly due to some prior familiarity with Unity, which I used to make Play Table, and a greater working knowledge of programming with C# over C++. It was relatively easy to get the LEAP sensor up and running with Unity although there are a few gotchas, like LEAP currently requiring an older version of Unity to work. Most of the LEAP examples in the latest version of the Unity Orion library are geared towards VR with a head-mounted sensor. Some adaption is required to work with a table-mounted sensor.

The visual metaphor I have chosen for this early work is that of post-it notes. The idea being that the user selects a post-it note containing a single word which then reveals a more detailed item of text. The detailed view can then be reverted back to its original brief form and another keyword chosen for investigation, by rotating through all available post-it notes.

The interaction metaphor is not fixed completely but I am currently experimenting with the following control scheme:

Extended hand used to rotate through post-its
Extended finger used to select a post-it
Fist used to close an opened detail view

These are a couple of screen shots of the work as it currently stands.

Selecting a post-it
A detail item revealed