Flashers

The first demo element I’ll reveal: the Flashers. I’ll be hooking up some Arduino-based, WiFi-enabled embedded systems.  There will be a custom Beat Saber mod listening out for when certain events happen – namely, when blocks are hit – and letting the Flashers know when that happens.

The Flashers, of course, will flash accordingly. 🙂

How it works

In fact, this part has progressed quite well already. The networking code on both the PC and Arudino side is complete. I’ll outline how it works, hopefully editing in a diagram later:

1. The Arudino connects to WiFi (it’ll be configured by some means I hope to cover in another post)

2. The mod has a background thread which regularly sends a multicast UDP packet. UDP, because they’re fire-and-forget, and typically have a lower latency. Multicast, because you don’t need to know who you’re talking to: it’s anyone who is listening that will receive them.

3. The Arudinos are of course listening: they respond to the origin of the packets to let the mod know that they are.

4. The mod has also hooked in to “Scene Change” events to understand when the player is in the main game loop – playing a level. When this happens, it hooks its claws into various game objects such that it is notified whenever a block is hit/cut.

5. Whenever a block is indeed hit/cut, it sends a UDP packet to each of the Flasher devices it knows to be listening

6. To understand when a Flasher is no longer listening, it just waits until that device hasn’t responded to a few of the multicast packets in a row

And that’s it. Plug-and-play, quick and easy transfer of in-game events to the real world! It works, and the latency is great: like, just 3-5ms!

(I’ll edit in an early video of this when I remember. If you’re reading this, oops)

Problems

I did face some issues on this journey.

1. Latency was terrible – at first, I experienced really variable latency on the Arudino side. I’d hooked up a PowerShell script to send UDP packets, but they were arriving in batches, or occasionally very late.

This turned out to be because the board defaulted to “low-power” WiFi mode. Despite the documentation stating the exact opposite, of course. Switching to regular / high-power mode, things worked very well indeed.

2. Multicast performance wasn’t the best, either – I figured just using multicast packets to broadcast a block hit event would do the trick. However, something somewhere in the chain – on the PC, router or Arduino’s network stack – was introducing additional latency.

3. VR itself has massive audio latency for observers – the audio and video to the headset is, of course, a huge priority. But not as much care has gone into ensuring things sync up for “mirrored” devices, i.e. the monitor/TV that those watching the person in VR are doing see. This is important because of course the person in VR won’t see the Flashers – they’re for the watchers! For this, there are apparently solutions like VoiceMeeter.