The Dissertation I wrote for my final year of my Master of Engineering Computer Science at the University of York.

At York my final year project was focused on Human-Computer Interaction, supervised by Paul Cairns. The project consisted of a literature review, a experimental study, a presentation, and a 55 page report.

The aim of the project was to research into how audio affects immersion in games by creating a game that can be played in multiple modalities (audio-only, video-only and both) in order to perform a study on how immersion is impacted by these different modalities. As part of the project, I wrote a literature review for the project report which covered the state of research into this field at the time (including both examples of games that use audio as an important modality and formal literature).

Game Prototypes

I needed to produce a game that would work equally well audio-only and video-only to use for the study. I made two prototypes, and carried the latter one on for the study itself.

  • Bat Pong
  • Bounce

Bat Pong - (Unreal C++)

The idea behind Bat Pong was to create a pong-like game, adjusted so it can be played audio-only by making the bats and balls emit a constant sound with a pitch that varied based on the position of the object in the vertical-axis.

I decided to use Unreal C++ to prototype this to both gain experience with Unreal and because I could get low-level access to the audio buffer to send it generated sound data. I learnt a lot during creation of this prototype, having not used Unreal much before and running into many initial stumbling blocks.

We decided not to pursue this game concept further due to the difficulty in audio-only play. In order to improve audio-only play, the video-only play would be severely hampered by a slow-moving ball or large paddle size, making the game too easy.

Bounce - (Unity C#)

A rhythm game based on the Rhythm Heaven series of games. The concept is that the player must enter their input in-time with the music, as the ball reaches the centre. Part of the reason for re-making this stage rather than use it directly is to both simplify the visuals to reduce antecendent factors, as well as to allow for retrieving precise timing information.

I decided to make this in Unity C# as I was more familiar with Unity which allowed faster iteration times.

The difficult part of this game was in ensuring timing stayed consistent — the game needs to stay in time with the music. I used the dsp time that is provided by Unity, an integer that ticks up with the time of the audio driver. Using this value and a coroutine, actions could be reliably performed in-time with the music (as long as the beats per minute value was set correctly).

In order to orchestrate all the timings for when balls should appear and bounce etc. I decided to used a data-driven approach (something I hadn’t done before). I used the ByteCode programming pattern from the book “Game Programming Patterns” to facilitate this. The whole game is run off a single CSV file that can be easily swapped out for different songs/ball timings.

This system was robust enough that I quickly prototyped a third “Simon Says” type game using the exact same codebase. It also allowed me to easily adjust and change timings without recompiling/changing code.

The music is from the stage of “Rhythm Heaven: Fever” from which this game was based.

Report & Presentation

I received a mark of 82% for my dissertation/project, which I was immensely pleased with.

The full report and the presentation I gave are linked below: