CREATE Methods

For this phase, we used the Playable Prototype and EXPO Plaque from the CREATE toolkit. 

We dedicated significant effort to developing a fully functional AR prototype that visitors could experience directly. Moving beyond our earlier mockups, we refined the interactions, enhanced visual elements, and optimized performance to create a polished, playable version of PicARsso that could be effectively showcased at the EXPO. 

EXPO Preparation

Our setup consisted of an easel with a canvas, a chair and a table with all of the stuff that made up our EXPO Plaque.

For the expo setup we printed photos of all of the paintings from Human Nature Exhibiton, that were used in our game (9 in total) and laid them out on the table. We also 3D printed some objects related to our game that we put on the table - a painting palette and mini easel, for which we've bought a mini canvas.

We also have designed a logo for the game.

We also made a PicARsso webpage (https://kornimate.github.io/PicARsso/), on which people could read about our game concept, see a gameplay video, see all of the artworks, visit the itch.io page, visit the leaderboard and take the evaluation survey. To the left of the easel, we had one screen with this website opened for people to dive into our game and/or fill out the evaluation survey and we had one screen on the table showing the leaderboard live.

But we also hung up two QR codes (one for the game page and other one for the leaderboard) on our stand, so that people can easily access the info on their phones at all times.

We arranged all of these elements to create an inviting space that communicated the artistic theme of our project. The physical objects served as tangible references to the virtual painting experience within our AR game, helping visitors make connections between the physical and digital aspects of artistic creation.  Our setup was designed to facilitate a smooth flow of visitors through the experience: first understanding the cultural and artistic context through the printed materials, then experiencing the AR drawing (aligned with our chosen EU value of freedom by giving people the option to creatively express themselves), followed by the guessing phase, and finally seeing how their experience compared with others via the leaderboard.

Here is how our setup looked like at EXPO:

Technical Description

PicARsso was built using Unity as the development platform and deployed on the Meta Quest 3 headset, leveraging AR capabilities to create an immersive drawing experience. The drawing was made possible by easily integrated Logitech MX Ink pen, with which the users could virtually draw on the physical canvas. The integration of the pen to the AR environment was done by a custom input system, primarily handled by the MXInkStylusHandler and StylusInputs scripts. The MXInkStylusHandler manages the stylus model, tracks user interactions and maps them to virtual drawing actions. Input values were parsed through the StylusInputs to enable responsive, real-time drawing on the physical canvas. One of the coolest parts of this project was seeing how the pen could actually become an extension of your hand in the game world.

We implemented a basic color selection system allowing players to cycle through different predefined colors using the controller buttons. Originally, we had planned to implement a more sophisticated color-picking system with an RGB color wheel that would allow players to select any color they desired. However, this proved technically challenging to integrate within our timeframe. 

For user interaction with the paintings and guessing components, we created a straightforward controller-based interface. Players could use the left controller buttons (Y for scrolling through paintings to choose from or to guess, and X for confirming their selection). This simple but effective control scheme made navigation intuitive even for users with limited AR experience.

One significant technical challenge we encountered was implementing drawing state management. We worked on creating an eraser functionality that would allow users to correct mistakes by overdrawing already drawn lines. We also attempted to add a feature to remove the last drawn line by pressing specific buttons on the controller. Both features presented technical difficulties related to how we were storing and rendering the drawn lines, particularly with managing the trail renderer components and their associated data structures. Though we made some progress, we couldn't fully implement these features within our development timeframe, marking them as potential improvements for future iterations.

Drawing itself was implemented through a trigger-based system - whenever the tip of the pen contacted a surface, it activated the drawing function, creating a persistent trail in the AR environment. This required precise calibration of the pen tip's collision detection to ensure drawing felt natural and responsive.

The project also presented challenges in maintaining spatial consistency between the virtual and physical environments, especially when transitioning between the drawing and guessing phases. After the drawing phase, we tried to recenter the camera correctly for users to ensure the virtual environment aligned properly for them. This helped maintain a consistent spatial reference for both players, ensuring that the second player could view the drawings in their intended orientation and position.

Evaluation and insights

To gather comprehensive feedback on PicARsso, we conducted a user evaluation during EXPO. The evaluation consisted of a structured questionnaire addressing various aspects of the experience, including overall satisfaction, usability, engagement, and the impact of audio on drawing experience (our research problem). The survey included both quantitative ratings (on a 1-10 scale) and qualitative open-ended questions to capture nuanced feedback.

The overall reception of PicARsso was highly positive. All participants had a generally positive experience, with an overwhelming 87.5% rating it as excellent (9-10).

Regarding ease of play, participants similarly reported high satisfaction. These results suggest that our interface design and mechanics were intuitive for most users, with all participants finding the game relatively easy to play.

That was also confirmed by a lot of the qualitative answers we received at the end!

The engagement level of PicARsso was also rated very highly. When comparing PicARsso to traditional drawing methods, the results were particularly positive. These scores validate our concept of enhancing the drawing experience through AR technology with all of users finding it more engaging than traditional drawing.

Our evaluation also focused on the impact of audio environments on the drawing experience, which was the core research question of our project. We investigated whether different audio environments would influence users' drawing behavior, creativity, and overall artistic expression. The results we got from that very pretty interesting and quite unexpected - because they were all over the place.

Before the evaluation we thought that the results would be quite high - initially our main idea was that if you draw something and have a relaxing sound playing vs a stressful sound, they would have a positive and negative effect on your experience. But it turns out that isn't exactly m the case! The distribution shows that audio's influence was highly subjective, with some users being significantly affected while others experienced minimal to medium impact.

The open-ended responses provided deeper insights into how audio influenced the drawing experience.

One participant provided strong positive feedback, noting that "The audio inspired me to be more creative and add parts to the canvas that were not originally there", which validates our initial hypothesis that audio environments can enhance creative expression in AR drawing. 

However, another participant had the opposite experience, stating "I was more confused than inspired," suggesting that for some users, the audio may have been distracting rather than enhancing. Other participants acknowledged the concept but reported limited personal impact: "I like the idea, but the audio had not much affect" and "It didn't make much difference."

Several participants mentioned the challenging acoustic conditions at the EXPO: "Since there was a lot of background noise the sound was not really noticeable. Maybe in a more quiet environment" and "They help me, but it was a bit noisy in the room." These comments suggest that the effectiveness of our audio environments was likely diminished by the exhibition setting.

Overall, the evaluation results regarding audio environments reveal a couple of insights. The most striking finding is the substantial variation in how participants responded to audio. This suggests that audio's influence on artistic expression is highly individualized, with some people being naturally more responsive to auditory stimuli than others. While not very impactful, the fact that some participants rated audio's influence on creativity at the highest possible level (10/10) provides validation for our concept. We found that while audio environments can enhance the creative experience for some users, the effect varies widely between individuals and may be context-dependent.

Apart from that, when asked if there was anything missing or any suggestions, participants provided valuable feedback for future development about:

• Controls - "The control mechanism with x and y is a bit weird" - this highlights the need to refine our button-based navigation system for selecting paintings and options,
• Colours - "Drawing with different colours would have been really cool. also to be able to erase the stuff again" - this directly supports our plans to implement a more sophisticated colour selection system and eraser functionality, features we had attempted to develop but couldn't complete within our timeframe,
• Game progression - "More levels, overall it is good" - suggesting that expanding the game with additional challenges or complexity could enhance replayability,
• Game clarity - "It was not clear on what basis the opponent was supposed to guess!" - which highlights a need for more explicit instructions for the game.