AI on a Microcontroller Isn't a Novelty Anymore

A few years ago, running machine learning directly on a microcontroller was mostly a research demo. The chips were too slow, memory was measured in kilobytes, and "real" AI meant a round trip to the cloud. That's no longer true. Modern MCUs like the ESP32 and the STM32 family now run quantized neural networks fast enough for real-time keyword spotting, vibration analysis, and simple computer vision, all without ever touching the network.

For a lot of the connected products we build at Sentrivo, that shift changes the whole architecture conversation. Instead of asking "how do we get sensor data to the cloud fast enough to act on it," the question becomes "does this decision need the cloud at all?" Increasingly, the answer is no.

Why Run Inference at the Edge

Edge inference isn't just a cost-saving trick. It solves problems that a cloud round trip fundamentally can't:

• 01Latency — a motor safety cutoff or a fall-detection alert can't wait on network round-trip time, and it certainly can't wait on a dropped connection.
• 02Bandwidth — streaming raw sensor or camera data from hundreds of devices gets expensive fast; running inference locally and only sending the result cuts that cost dramatically.
• 03Privacy — keeping raw audio or video on-device, and only transmitting a classification, is often the difference between a product customers trust and one they don't.
• 04Reliability offline — industrial and field deployments don't always have connectivity. A device that only works with a live network connection isn't finished.

Picking the Right Silicon

"Just use an ESP32" is common advice, and it's often right, but not always. We typically narrow the choice down along a few axes: how much compute the model actually needs, how much power budget the product has, and how much of the workload is genuinely on-device inference versus connectivity and orchestration.

• 01ESP32 — the default choice for products that need Wi-Fi or Bluetooth alongside modest inference workloads, like keyword spotting or simple sensor fusion. Good tooling, cheap, and well supported by TensorFlow Lite Micro.
• 02STM32 — a better fit when the product is deterministic, safety-relevant, or power-constrained, and connectivity is handled by a separate radio module. STM32's CMSIS-NN and X-CUBE-AI tooling make it a strong platform for classic sensor-driven ML.
• 03Raspberry Pi / NVIDIA Jetson — when the workload is real computer vision, not just classification of a small sensor window, an MCU isn't enough. That's when we move up to a Linux-capable board with a real accelerator.

Shrinking the Model to Fit

The model that trains cleanly on a GPU almost never fits, as-is, on a device with 320KB of RAM. Getting there usually takes a few deliberate steps: quantizing weights to int8, pruning the architecture down to what the task actually needs, and validating that accuracy holds up after each cut, not just estimating it. Frameworks like TensorFlow Lite Micro, Edge Impulse, and CMSIS-NN handle a lot of the heavy lifting, but they don't replace testing on the real chip, with the real sensor, in the real enclosure.

This is where we see the most avoidable mistakes: teams that optimize a model against a validation set that looks nothing like the noise, lighting, or vibration profile the device will actually see in the field.

The hardest part of edge AI usually isn't training the model. It's making that model survive contact with real hardware, real power budgets, and real customers.

What Actually Breaks in Production

The failure modes that matter almost never show up in a lab demo. Thermal throttling changes inference timing after a device has been running for an hour in direct sun. Memory fragmentation from a poorly managed heap causes a crash after days of uptime, not minutes. A watchdog timer that isn't tuned to the model's worst-case inference time resets the device mid-cycle. And an OTA update process that wasn't designed for the model's memory footprint bricks a percentage of the fleet on rollout.

None of these are exotic problems, but they're the reason "it worked on my dev board" and "it works reliably across 5,000 units in the field" are two very different engineering bars. We design for the second one from day one.

How We Approach an Edge AI Project

When a client brings us a computer vision, keyword spotting, or anomaly detection idea for an embedded product, we work through it in roughly this order:

• 01Define the constraint budget first — before anyone touches a model, we pin down the power, memory, latency, and cost envelope the hardware actually has to live within.
• 02Prototype on target hardware, not a laptop — a model that looks great in a Jupyter notebook can behave very differently once quantized and running on the real MCU.
• 03Optimize for the failure mode, not just accuracy — a missed defect and a false alarm rarely cost the same amount, so the model gets tuned around the cost that actually matters to the business.
• 04Instrument before shipping — we pipe device telemetry back so we can see how the model actually behaves in the field, not just how it performed in the lab.