A Musical Introduction to The Nexus – Prosperity4All

A Musical Introduction to The Nexus – Prosperity4All


♫ Music playing in a Gamelan style ♫ The Nexus is an integration technology that we’ve created at the IDRC as part of the Prosperity4All project, working with partners all over the world. Prosperity4All is a European Commission funded project that is creating infrastructure and tools to make building the next generation of assistive technologies easier. The Nexus enables us to connect software written in different technologies, potentially running on different devices. We can connect software that was written without knowledge of one another and we can use them to create new user experiences. So we have a number of different devices here with different user interfaces. This computer here is where the sound is coming out of. So this is running the sound synthesizer that’s connected up to the speaker we’ve been playing. And this is the computer that’s running the Nexus itself. And then each of these devices are connected wirelessly talking to the Nexus. The most straightforward mapping in terms of the interfaces that we have is the piano keyboard, so the traditional piano keyboard. Whenever you press one of the keys it sends a MIDI note number. It sends a message from the tablet wirelessly to the laptop running the Nexus saying somebody just pushed note number 70. And then the Nexus relays that information on to the synthesizer and then the synthesizer plays that note. So that’s fairly straightforward. What’s a little bit more involved is the one that we were looking at with the zone controller where we’re playing notes based on moving the cursor, moving the pointer that we have. And that one doesn’t know anything about notes, it’s not based on MIDI note numbers. It just communicates the fact that a particular pad number, a particular square was activated. So it sends a number, like this one might be 0 and then this one might be 6, 7, 8. It sends that pad number to the Nexus and then inside the Nexus we have some configuration mapping. Which is mapping from, in this particular instance, the Pad number to a MIDI note number. And then that MIDI note number is being sent to the synthesizer. We have the head tracker that is running inside a system called AsTeRICS which is a Java-based assistive
technology creation kit. And we’ve created a connector component that runs inside AsTeRICS that allows AsTeRICS to interact with the Nexus. So what’s happening in the case with playing the zone controller is AsTeRICS is running on here. It’s managing the head tracker, the face tracker that’s using the webcam to do the face tracking. It’s then relaying that position information to the Nexus. The Nexus is then relaying the position information back to the zone controller. And then the zone controller is sending the pad number. This kind of approach is going to be really useful as we have more and more sources of data. So as we connect up more and more sensors such as wearable technology or we’re connecting more and more devices to the internet. So we have lots and lots of data that can be really useful and then a system like this allows us to gather, gather that data, and create our own interactions, create our own presentations. And work with all of the data that
we’re increasingly sensing and increasingly connecting up
to computers and connecting up to the internet. ♫ Music playing in a Gamelan style ♫

Leave a Reply

Your email address will not be published. Required fields are marked *