Jarbles is a game about reconstructing musical pieces through touch interaction with tangible 3D objects. Developed for the iPad and iPhone, the game is a PLAYPR research project created at the Centre for Technoculture, Art and Games (TAG) at Concordia University.
Released on the App Store May 2014. Presented at GRAND 2013. With Jason Camlot (producer, designer), Ian Arawjo (programming, art), FX Dupas (sound), Taylan Ulger (art), and puzzles by Mathieu Lavoie (sound designer) and his students.
For this project, I built a 3D rigid body physics engine in Obj-C from scratch, developed mappings between waveform data and vertices, and designed the line-art style — including the font, which is hard-coded as an array of normalized coordinates:
Notably, Jarbles has no art assets. All art was hard-coded into the game.
In Jarbles, the player must merge jacks to form marbles, and then order the marbles to form a completed musical piece. Each jack has an associated sound which plays when it is picked up.
At puzzle completion, the player is graded on their listening skills. If they were successful, a point is awarded, which the player can spend on unlocking harder puzzles.
Something I learned: Sometimes software projects go on too long. They reach a point where the energy put into them derives lesser and lesser returns. Jarbles was a game that never had a clear concept beyond pairing sounds. We wanted to keep the second phase of the game — where the player orders sounds — but never understood what form it should take. In retrospect, this stifled the project. In the time it had taken us to make Jarbles, we could have finished and released the game with pairing-only — less than ideal for us, but players wouldn’t care — and jumped onto a new project that explored ordering sounds. Moreover, because we were never clear on the second phase of the game, we went through several haphazard iterations. These iterations cluttered the game’s code, confusing and twisting its logic, turning the game’s development environment into a morass. The 3D engine, developed with clarity of mind, was the only guiding beacon; the rest, the game logic, was beyond repair. And finally, the hardcoded menu — how do you make a menu out of 3D line art? — looks cool, but probably took longer than it should have.
The main thing I learned, or rather got past, is to use game engines like Unity wherever possible. Sometimes a project resists the boundaries of engines — in the PoetryLab project, we needed low-level control over audio, which is something Unity (being an interpreted language) does not provide. But if the project can conceivably be realized in a game engine, use that engine.
Ironically, however, I would not trade in the experience of Jarbles for anything: it taught me more than many projects I have done. The experience of coding a 3D engine is something I will never forget, and provides much-needed confidence to tackle harder projects.