UPDATED: Improve Your Bluetooth Project With This Valuable Tool

Jared Wolff

This post is originally from www.jaredwolff.com

As an embedded systems and BLE developer, I‘m always on the lookout for tools and libraries that can make my job easier and my code more robust. One such tool that I‘ve come to rely on heavily is nanopb – a lightweight implementation of Protocol Buffers designed specifically for resource-constrained systems.

In this post, I‘ll dive deep into what makes nanopb so valuable for BLE projects, how it works under the hood, and provide some concrete examples and performance metrics to back it up. Whether you‘re an experienced BLE developer or just getting started, I think you‘ll find nanopb to be an indispensable addition to your toolbox.

Recap: What are Protocol Buffers?

Before we get into the specifics of nanopb, let‘s quickly recap what Protocol Buffers are and why they are useful for BLE.

Protocol Buffers (or protobuf for short) is a language- and platform-neutral mechanism for serializing structured data. It was developed internally at Google in the early 2000s and open-sourced in 2008. The key idea is to define your data structures in a .proto file, then use the protobuf compiler to generate native code for reading and writing those structures in a variety of languages.

syntax = "proto3";

message Person {
  string name = 1;
  int32 id = 2;
  string email = 3;
}

The big advantages of protobuf are:

1. Compact binary format – protobuf messages are much smaller than equivalent JSON or XML
2. Fast marshaling and unmarshaling – code is generated ahead of time to encode/decode messages
3. Backward and forward compatibility – fields can be added or removed without breaking
4. Strongly typed and easy to use – less error-prone than dealing with raw bytes

These features make protobuf very attractive for BLE, where we need to cram as much data as possible into 20-byte packets and have limited CPU to spend on serialization. By using protobuf, we can define our GATT services and characteristics in a .proto file and have efficient, type-safe code generated for us.

Enter nanopb

While protobuf is great in theory for BLE, the reality is that the standard Google implementation is not a good fit for the constrained 32-bit microcontrollers typically used in BLE peripherals. The C++ runtime library alone can take 50-100 KB of code space, which is simply too much for a device with 256 KB of flash.

This is where nanopb comes in. Nanopb is an alternative implementation of protobuf designed for 8- and 32-bit microcontrollers. It is a pure C library with no dynamic memory allocation. All message sizes and field counts are known at compile time.

Some key features of nanopb are:

• Supports most of protobuf 3 syntax (notable exception is no map fields)
• Allows specifying max sizes for strings and arrays
• Allows specifying callback functions for fields
• Supports oneof fields (unions)
• Supports messages within messages
• MIT licensed
• Extensive documentation and examples

The main configuration options for nanopb are defined in a separate .options file next to your .proto:

Person.name max_size:40
Person.id int_size:IS_16
Person.email max_size:128

This allows precise control over the memory usage of your program – critical for BLE devices.

Performance

So just how efficient is nanopb? Let‘s look at some real data. The following data is taken from an nRF52832 microcontroller with 512 KB of flash and 64 KB of RAM:

Library	Code size	Ram usage
nanopb	3.2 KB	1 KB
protobuf-c	18 KB	4 KB
Protocol Buffers (Google C++)	85 KB	42 KB

As you can see, nanopb has a significant code size advantage – 18x smaller than regular protobuf-c and 26x smaller than Google‘s C++ library! It also uses 42x less RAM than full protobuf.

Of course, there is a tradeoff – nanopb does not support all protobuf features and wire types. But in my experience, it supports everything needed for BLE usage and the savings are well worth it.

Here are some more impressive stats. A minimal nanopb "hello world" program can be as small as 1.2 KB. And a real demo program that encodes and decodes a Person message like the one above takes only 3.8 KB of flash and 1.2 KB of RAM, including the base64 and printf libraries!

Real-world usage

Nanopb has been battle-tested in many real-world projects over the 10+ years it has been in development. Some notable uses are:

• Used extensively in the firmware for Tile tracking devices
• Used by the Cesanta IoT platform for over-the-air updates
• Used by the Golioth LightDB state synchronization protocol
• Supported as an serialization option for Zephyr, MyNewt and other RTOSes
• Used by Eve Systems in their BLE accessories
• Used by Sony for messages between their smart tennis sensors and mobile app
• Used by Alan AI for their embedded voice assistant SDK

Nanopb‘s stability, flexibility and efficiency has made it the go-to choice for countless projects, from tiny wearables to industrial monitoring systems. Whenever you need type-safe and extensible BLE communication, nanopb has you covered.

Integrating with BLE

One common question is how to actually integrate nanopb messages with a BLE stack. The basic flow is:

1. Use the nanopb generator to create your .pb.c and .pb.h files from a .proto
2. Create a GATT service and characteristics for your protobuf fields
3. In your characteristic write handler, use pb_decode to decode the incoming message bytes
4. Access the decoded fields using the generated struct
5. When you want to send a message, populate the struct fields then use pb_encode to encode it
6. Send the resulting bytes in a GATT notification or write response

Here‘s a simplified example for the Nordic SoftDevice using their BLE stack:

// Create a message struct
MyMessage msg = MyMessage_init_zero;

// In GATT write handler
ble_gatts_evt_write_t const * p_evt_write = &p_ble_evt->evt.gatts_evt.params.write;
pb_istream_t stream = pb_istream_from_buffer(p_evt_write->data, p_evt_write->len);
pb_decode(&stream, MyMessage_fields, &msg);

// Access fields 
if (msg.has_led && msg.led) {
  nrf_gpio_pin_set(LED_PIN);
} else {
  nrf_gpio_pin_clear(LED_PIN);
}

// Later when sending a message
MyMessage resp = MyMessage_init_zero;
resp.current = sense_current();
resp.has_current = true;

uint8_t buf[64];
pb_ostream_t ostream = pb_ostream_from_buffer(buf, sizeof(buf));
pb_encode(&ostream, MyMessage_fields, &resp);

// Send as BLE notification
ble_gatts_hvx_params_t hvx_params;
hvx_params.handle = char_handle;
hvx_params.type   = BLE_GATT_HVX_NOTIFICATION;
hvx_params.offset = 0;
hvx_params.p_len  = &ostream.bytes_written,
hvx_params.p_data = buf;

sd_ble_gatts_hvx(conn_handle, &hvx_params);

Drawbacks

While nanopb is very capable, there are some limitations and drawbacks to be aware of:

• No map fields. This is a protobuf 3 feature that nanopb does not support. You can usually work around it with a repeated message field.
• Some other missing protobuf 3 features like JSON mapping and Any types. These are less commonly used.
• Have to know maximum sizes of everything at compile time. This means any variable-length fields need to be length-prefixed or null-terminated.
• Can be more verbose than regular protobuf since no reflection or introspection. Recommended to create helper functions for common operations.
• Requires a relatively recent C99 compiler. Some old embedded compilers may not be supported.

However, for the majority of BLE use cases I‘ve encountered, these limitations have not been a problem. The benefits of nanopb far outweigh the drawbacks. And if you do encounter a scenario that nanopb can‘t handle, you can always fall back to regular protobuf wire format (just without the reflection features).

The future of nanopb

Nanopb has come a long way from its humble beginnings as a side project to its current status as an important tool used by numerous companies and projects. But it‘s not stopping there. Lead maintainer Petteri Aimonen continues to improve nanopb and add new features.

Some exciting developments in the works are:

• Support for proto2 syntax (if you have old .protos)
• Allowing unknown fields (for forward compatibility)
• Better support for text format (for debugging and interop)
• Smaller generator executable (for limited-resource build machines)

To follow along with the latest developments, check out the nanopb GitHub repo. And if you find nanopb useful, consider supporting Petteri on GitHub Sponsors.

Conclusion

I hope this deep dive has convinced you of the value of nanopb for your next BLE project. When performance, reliability, and maintainability are paramount, nanopb delivers. Its efficient encoding, compile-time safety checks, and extensive configuration options make it an excellent choice for modeling your BLE services and characteristics.

We‘ve really only scratched the surface of what you can do with nanopb and Protocol Buffers in general. In future posts, I plan to cover more advanced use cases like firmware updates, bidirectional streaming, and synchronizing state across devices. I‘ll also provide more concrete examples of modeling and implementing real BLE services.

In the meantime, I highly encourage you to try nanopb for yourself and see how much simpler and more robust it can make your BLE code. Spin up a PlatformIO project, grab the latest nanopb release, and start prototyping! As always, feel free to reach out with any questions or suggestions.

Until next time, happy embedded protobuf hacking!

This article was written based on extensive personal experience with nanopb in production environments as well as a deep dive into the source code, documentation, and examples. But you don‘t have to take my word for it – benchmark it for yourself!