Inkluderende Digitalt Laboratorium for Eksperimentell Kunst (IDLE) 2022-24
Inkluderende Digitalt Laboratorium for Eksperimentell Kunst (IDLE)
Introduction
Inkluderende Digitalt Laboratorium for Eksperimentell Kunst (IDLE) is an innovative artistic and participatory project based on a digitally updated art venue space, Studio 207, in Bergen. The goal is to unlock the room’s different audiovisual devices, remotely accessible via the Internet, enabling everyone to interact with them to create new artistic experiences even not being physically there.
The venue’s audiovisual devices are controlled remotely through a virtual gallery. Guest artists and audiences can manipulate lights, videos, and sound equipment to create different atmospheres and performances in the venue. Through the manipulation of the virtual interfaces (the gallery), inputs are transformed by the Internet of Things system to alter physically the space. The public designs spatial audiovisual experiences for those In Real Life at the venue and simultaneously in the virtual gallery!
IDLE intends to offer a creative virtual meeting point for school kids, youngsters, people with reduced mobility who wants to interact with the physical world, and all of those art-curious lovers that want to look for new physical-virtual new experiences. The project explores new collaborations and forms of interaction between different art and cultural forms.
2022
IDLE is a multi-disciplinary three years project from 2022-2024. First-year development, 2022, has focused on the research and prototyping of the user interaction system. Tasks have included forming the team, coordinating meetings, researching the 3 main fields of the project: IoT (Internet Of Things) devices, Sound and Video control interfaces, and Virtual Reality user interface; testing software and hardware solutions within the free/libre technologies, development of a prototype, IDLE version 0.0, a possible model to follow, a Bergen development team residency’s, and a final presentation of the results.
The IDLE version 0.0, have been premiered at the 20th Piksel anniversary: PIKSELXX AI AI AI festival for Kunst og Fri Teknologi. The artists and developers of the project traveled to Bergen to work together in an art residency, creating the first sound and visual, physical, and virtual experience. The presentation was held on Thursday, Nov 17th – 22-23h.at Studio 207 and the @Piksel Cyber Salon with a massive public attendance.
2023
As with any research project, we found different technical challenges over the year that lead to a re-schedule of the initial project plan. To absorb the delay we have overlapped the second-year production with the final tasks of the first year. The new schedule will facilitate the developments to achieve IDLE version 1.0 and the dissemination plan to collaborators of the second year.
2023 is the year focused on knitting the mediators network among the voluntary organizations, the professional sector at the healthcare facilities, the public workers in different municipalities, and any other inclusive association that works with groups that need special facilitation: children and young people with long stays at healthcare facilities, school kids, people with reduced mobility, etc. The 2023 goal is to transfer the technological competencies on how to use IDLE through meetings and “hands-on” workshops.
2024
Still, there is a challenge to overcome this year. The initial project is planned over 3 years, including Development, Dissemination to mediators, and Public events with final users. The Arts Council Norway has only provided funding for 2 years (2022-23).
2024 is a crucial period that will determine the success or failure of the entire process. Throughout this last year, a series of public events will show the “Art Experiences” in Norway and internationally created by the audiences and specially curated guest artists invited to participate.
In 2023 we will apply again with this project to achieve the necessary funding to finish the IDLE project as intended and planned.
IDLE is a project initiated by Piksel in 2022, in collaboration with CNDSD, Malitzin Cortés and Iván Abreu, APO33, Jenny Pickett, Julien Ottavi, Romain Papion, and Martin E. Koch.
Development
2022 System Development
Piksel proposed to develop the project over a three year period, from 2022 to 2024. This report is about explaining what has been done in the first project’s year.
In 2022 IDLE project goals were to develop the different components of the system and to connect them through a virtual gallery:
- To create the virtual gallery
- To implement the IoT system at Studio 207.
- To implement sound streaming tools to make online concerts with remote musicians and/or users.
- To connect all the systems.
The virtual gallery
The virtual gallery has taken the physical space as architectural model to enhance the understanding of the project and the identification of the venue. The system, based on innovative previously done projects, presents Studio 207 on a web-based internet VR platform.
The IoT system
The IoT system has been programmed using only free and open-source technologies and every developing step is published in a local wiki. This open documentation can be used to facilitate other venues to integrate IoT technologies in their spaces.
GIASO, Sound multi-user interface
The sound user interface is based on the server software developed by Apo33, Great International Audio Streaming Orchestra, GIASO, to create a place for networked performance.
The “Great International Audio Streaming Orchestra” uses a bi-directional multiplex platform to perform and mix different audio sources (streaming). In the time of the performance, the streams (transmission) are re-composed in the broadcast space through a spatialization based on a free and multi-stream internet transmission system. GIASO creates a new form of orchestral composition where composers become virtual entities that emerge from a community of nodes – audio explorers and networked performers.
Web-dev Interface
Through a browser we connect the physical devices, lights, sound, and screens at the Studio, to the VR environment, in such a way that people can interact and create new environments in real time.
Residency
The artists and developers have been in Bergen a week on a working lab residency to fine tuning the first IDLE version. The team is originally from France, Mexico, Germany, Norway and Spain. This residency has allowed to set up the systems we have been working during 2022, connect them and test it.
Premiere at Studio 207. IDLE Versión 0.0.
On the right we can see the Studio 207 were the audience could enjoy the AV performance. The audio and visual electronic artists are located on the next office, to show clearly to the public they are controlling the Studio 207 venue. The performance can be seen at the Studio but also the audience could visit the remote work from the artists. A big monitor was showing the same performance also in the virtual gallery, the Piksel Cyber Salon.
Practical goals for 2022
- To develop a virtual gallery Studio 207 where the people can interact.
- To develop a sound interface where musicians or public can interact and send the final sound stream to the Studio 207 sound system.
- To integrate the IoT system in the Studio 207, including robotic cameras, led lights, sound equipment and video equipment connected.
- To make a website to use at Studio 207 to control the IoT system in situ.
- To publish the project findings to the general public and other cultural organizations.
- To disseminate the project.
Done tasks
- To study the possibilities actual software and hardware based on free/libre technologies.
- To test them, to test the limitations and find out how to implement a solution easy to use.
- To put in common the different knowledge of the team members to achieve the final goals.
- To meet all over the year to coordinate the different teams: The IoT team, created by Martin E. Koch, Gisle Frøysland and Maite Cajaraville, the sound development system with Jenny Pickett, Julien Ottavi and Román Papión, and the virtual environment team Iván Abreu and Malitzin Cortés.
- To make a week residency inviting all team members in Bergen to be able to actually work in real life and in te real physicial space.
- To write down reports, documentation and graphical information about the system and how to implement it.
- To create a dissemination platform where we can share that knowledge https://wiki.piksel.org/index.php/Piksel_meets_IoT
Deliveries
- A VR gallery in Hubs by Mozilla
- A sound streaming tools for multiuser musicians
- A server to host all the applications
- An IoT system controlled by code
- A wiki with documentation
Software tested
VR environment
Mozilla Hubs A- frame three.js
IoT Servers
Domoticz https://www.domoticz.com/ Openhab https://www.openhab.org/ WebThings https://webthings.io/
Lights and protocols
Led lights individually addressable Led spotlights (Philips system – Philips HUE) Zigbee MIDI Robotic Cameras
Sound
Icestream server modified
Video
Kodi video player over Raspberry pi
Conclucions
– A wide research has been done on software and hardware related to each field. Therefore, we are in a better position to decide how to develop the final solution.
– We encountered several technical barriers that led us to change the software and hardware tools we had in mind. On the other side, to confirm our guess, we opened a communication channel with the Mozilla Hubs developers team. They confirmed several of our concerns, meaning we have got some awareness of the project and a better vision of how to get our goals.
– The project is very complex. We knew that. We calculate we have done approx a 75% of the planned project, which we believe is a good result.
– The team has met and decided how to work and is highly engaged in the process and the project.
– The activities in Bergen have been very successful, in public attendance and teamwork.
– To develop a project like this, the residency format works very well. It is highly recommended and effective.
Next steps
In the first 6 months of 2023 we expect to have a new IDLE version 1.0, easy to use for external audiences. In parallel, we plan to start the second year activities.
The second year, once the system is functioning, we want to develop a dissemination plan together with voluntary organizations, the professional sector at the healthcare facilities, the public workers in different municipalities, and any other inclusive association who works with groups that need special facilitation: children and young people with long stays at healthcare facilities, school kids, people with reduced mobility, etc. Adding to that we want to invite national and international artists as well as the civil society to participate in the common creation. A series of workshops will be hosted at Piksel to transfer the competences from the Piksel team to collaborators and mediators.
The third year we will execute the dissemination plan in Norway and internationally through a series of public events where the different “Art Experiences” created by the public will be shown along the year. Specially curated guests artists will be invited to participate.