by Victor Lai
In this article, I discuss some of the problems I’ve encountered in producing and analyzing device logs over the years, on single digit number of devices up to hundreds of thousands, and concepts I’ve applied to manage that complexity. My background is in Android development, so some of the examples may skew towards much more powerful hardware than simple microcontrollers.
Embedded engineers need something as easy to use as
printf, usable within interrupts, small enough for today’s MCU’s, and have minimal performance impact when running. The Trice technique tries to fill this gap. It is the result of a long-year dissatisfaction and several attempts to find a loophole to make embedded programming more fun and effective.
In this post, we lay the foundation for how an organization should instrument their embedded firmware to measure performance, stability, and the overall “health” of each device and an entire fleet of devices. We also compare and contrast the various approaches projects generally take to surface these metrics and I’ll discuss why I believe heartbeat metrics are the best method for driving product decisions.
Having a logger display execution information on the terminal is pretty common for firmware developers. What’s less common is having an instant stack trace when the program crashes.