The Architecture That Works for 10 Devices Rarely Works for 10,000

Most connected products start the same way: a handful of prototype devices talking directly to a backend API, storing readings straight into a database, and a small dashboard to look at the results. That architecture is genuinely the right choice for a pilot. It's fast to build, easy to debug, and gets a product in front of real users quickly.

The problem is that very few teams revisit that architecture before it has to carry real production load, and by then the device firmware, the mobile app, and the backend are all coupled to assumptions that don't scale. Designing for scale from the start doesn't mean over-engineering a pilot. It means knowing which decisions are cheap to change later and which ones aren't.

Four Layers Every IoT System Has

Whether it's explicit in the architecture diagram or not, every IoT product has four layers, and treating them as genuinely separate concerns is what makes a system scale cleanly:

• 01Device layer — the firmware running on the hardware itself, responsible for sensing, local logic, and talking to the network.
• 02Connectivity layer — the protocol and transport that moves data between devices and the backend, whether that's cellular, Wi-Fi, LoRa, or Bluetooth to a gateway.
• 03Ingestion & backend layer — where device messages land, get validated, get a device's current "shadow" state updated, and get routed to storage or downstream processing.
• 04Application layer — the dashboards, mobile apps, and business logic that people actually interact with, which should never talk to devices directly.

Choosing a Protocol

Protocol choice gets debated more than it deserves, but it does matter once a fleet grows. The three options we reach for most often each solve a different problem:

• 01MQTT — the default for most connected products. Lightweight, built around a publish/subscribe model, and well suited to devices that need to maintain a persistent connection and receive commands, not just send data.
• 02HTTP/REST — still a reasonable choice for devices that wake up, send a batch of readings, and go back to sleep, especially in cellular-constrained or intermittently connected deployments.
• 03CoAP — worth considering for very low-power, constrained devices on lossy networks, where MQTT's persistent connection model is too heavy.

Provisioning and OTA at Fleet Scale

Onboarding device #10 by hand is fine. Onboarding device #10,000 by hand isn't a process, it's a liability. Scalable provisioning means every device gets a unique cryptographic identity at manufacturing time, not a shared credential baked into firmware. It means certificate rotation is automated, not a support ticket. And it means firmware updates roll out in stages, to a small percentage of the fleet first, with automatic rollback if error rates spike, rather than pushing to every device at once and hoping.

This is one of the areas where the cost of skipping it compounds quietly. A shared device credential is invisible technical debt right up until it's a security incident.

Security Is a Lifecycle, Not a Checkbox

Encrypting the connection between a device and the backend is necessary but not sufficient. Real device security spans the full lifecycle: attested boot so a device can prove it's running genuine firmware, encrypted transport for every message, least-privilege access so a compromised device can only publish to its own topics, and a patching process that can actually reach the fleet when a vulnerability is found.

We treat this as core architecture, not a hardening pass at the end of the project, because retrofitting device identity and secure provisioning onto a fleet that's already in the field is dramatically more expensive than designing it in from the start.

A prototype only has to work once, on your bench. A product has to keep working on ten thousand benches you'll never see.

Signals Your Architecture Won't Scale

A few patterns reliably predict trouble before a fleet grows past a few hundred devices:

• 01Devices connect directly to the application database — any change to the schema now requires coordinating a firmware release, which should never be true.
• 02No device shadow or last-known-state layer — without it, every dashboard query has to reach out to the device live, which doesn't scale and doesn't work offline.
• 03Firmware updates are manual, per device — this caps how large a fleet can realistically grow before updates become the team's full-time job.
• 04Outages are discovered from customer complaints — not from device telemetry or health monitoring, which means the team is always reacting instead of catching issues early.