Create an Arduino and Unity3D Interactive Experience with No Latency!

Arduino is an incredibly powerful and flexible electronics platform that enables makers of all skill levels to create amazing projects. By connecting sensors and other inputs to an Arduino, you can build interactive systems that respond to the physical world. One of the most exciting applications is connecting an Arduino to the Unity3D game engine, allowing you to control games and 3D experiences with custom hardware.

However, if you‘ve ever tried to hook up an Arduino to Unity, you‘ve likely run into one major problem – latency. Getting sensor data from the Arduino to Unity in a timely manner is a huge challenge. The built-in serial communication is serviceable for simple projects, but the lag makes it unsuitable for anything requiring real-time interactivity.

Fortunately, there is now an open source solution that completely eliminates this latency problem. WRMHL is a library that provides lightning-fast, low-latency communication between Arduino and Unity. With wrmhl, you can create responsive interactive experiences without any noticeable delay.

How wrmhl Works

Under-the-hood, wrmhl takes advantage of a more optimized serial communication protocol than the standard Arduino Serial class. It implements a lightweight, minimal messaging format that packs data efficiently and transmits it at very high baud rates.

Wrmhl uses separate threads for reading and writing serial data, which allows it to process input and output simultaneously without blocking. This multi-threaded architecture is key to its performance.

On the Arduino side, wrmhl includes a simple API for sending data packets. It can transmit single values or arrays with only a few lines of code. The included examples demonstrate how to send data from analog and digital inputs.

In Unity, wrmhl provides a C# library that makes it easy to access the data streams from the Arduino. It automatically handles the serial port connection and parsing of incoming data. From there you can use standard Unity methods to work with the received values in your project.

Getting Started with wrmhl

To use wrmhl in your Arduino-Unity projects, first download the latest release from the GitHub repo. You‘ll get two folders – one containing the Arduino code, and one with the Unity assets.

Open the Arduino IDE and load one of the example sketches, such as AnalogRead.ino. Upload it to your board. Then open your Unity project and import the wrmhl Unity package.

Look at the wrmhl demo scenes for how to use the library. The basic steps are:

1. Create an instance of the wrmhlSerial class, specifying which serial port to use
2. In the Update loop, call wrmhlSerial.ReadLine() to get the latest data packet
3. Parse the CSV string into an array of floats
4. Use the received values to drive animations, 3D transformations, game logic, etc.

The wrmhl protocol supports a variety of data types and formats which are documented on the GitHub wiki. You can customize the Arduino output to pack data in the most efficient way for your application.

Example wrmhl Projects

So what can you actually build with wrmhl? Here are a few examples to get your creative juices flowing:

Touchless 3D Tracking

Attach ultrasonic distance sensors to the Arduino and use them to triangulate the position of your hand in 3D space. Send the XYZ coordinates to Unity and use them to control a 3D object or character. Imagine being able to reach out and grab virtual items without any handheld controllers! See this video for a demo of touchless 3D tracking with Arduino.

Brain-Computer Interfaces

Hook up an Arduino-compatible EEG headset like the OpenBCI and stream your brain waves to Unity. Plot your alpha, beta, and theta rhythms in real-time and experiment with brain-controlled UI. Or take it a step further and use machine learning to map mental commands to in-game actions. The Cortex project is building an open source brain-computer interface with Unity and wrmhl.

Sensor-Driven Environments

Fill a room with environmental sensors connected to an Arduino. Measure the ambient temperature, humidity, light level, sound, and more. Feed that data to a Unity app that generates immersive audio and visuals based on the sensor readings. Transport yourself to a relaxing beach or a terrifying dungeon that reflect your immediate surroundings.

Multiplayer Smartphone Controllers

Build smartphone-controlled multiplayer games by leveraging the Arduino Bluetooth library. Connect a bunch of phones to the Arduino over Bluetooth serial and have them send control data that gets passed on to Unity over wrmhl. Create a local multiplayer party game using just an Arduino and the smartphones everyone already has in their pockets.

The sky is the limit when it comes to physical-digital interactive projects. Wrmhl makes it dead simple to get data from Arduino to Unity where you can create any 3D experience imaginable.

Advanced Tips and Tricks

While wrmhl works great out-of-the-box, there are a few things you can do to optimize its performance and customize it for your needs:

Implement Your Own Protocol

Wrmhl includes a default protocol implementation, but it‘s also easy to create your own. Just edit the wrmhlThread_Lines.cs file in the Unity assets folder. You can define your own delimiter characters, add support for other data types, or even pack the data more compactly with a binary protocol. Wrmhl makes no assumptions about the data format, so you have complete flexibility to implement a protocol optimized for your specific application.

Speed Up the Baud Rate

The Arduino examples default to a baud rate of 9600, which is reliable but somewhat slow. If you want to push wrmhl to the limit, try increasing the baud rate. Many Arduino-compatible boards support speeds up to 115200 baud or even higher. Just make sure to change the baud rate setting on both the Arduino and Unity sides.

Filter Noisy Data

Depending on your sensors and environment, the data coming from the Arduino may be noisy or jittery. To smooth things out, you can apply some simple filters on either the Arduino or Unity side. The moving average is an easy and efficient filter to implement. Just keep a buffer of the last N sensor values and take the average each frame. This works well for cleaning up data from continuous sensors like potentiometers or infrared distance sensors.

Minimize Garbage Collection

If you‘re sending data at a high rate, the overhead of parsing the incoming CSV strings and allocating new arrays can start to add up. To avoid excessive memory allocation and garbage collection in Unity, use a pre-allocated buffer instead of creating new arrays each frame. Even better, use the unsafe code features in C# to manually parse the string data without any heap allocations.

With these tips you can push the performance of wrmhl even further and build incredibly responsive interactive applications.

Get Involved

Wrmhl is an open source project and welcomes contributions from the community. If you find a bug, have an idea for an improvement, or want to implement a new feature, feel free to open an issue or submit a pull request on GitHub. Let‘s work together to build the best Arduino-Unity communication library possible!

I‘m excited to see what you create with wrmhl. Having a reliable, low-latency, open source solution for connecting Arduino and Unity removes one of the biggest barriers to creating immersive physical-digital experiences. I believe this technology can be used to build not only entertaining games and interactive art pieces, but also powerful tools for education, scientific visualization, virtual prototyping, accessibility, and more.

If you build something cool with wrmhl, please let me know! You can reach me on Twitter at @maximecoutte or via email at [email protected]. I‘d love to share your projects with the world.

Now go forth and bridge the gap between the physical and digital with wrmhl!