Edge AITutorialIoT

On-Device ML Control: Streaming Model Inputs from React Native to Raspberry Pi

rreactnative

2026-01-25

11 min read

Technical guide: stream inputs, receive on-device ML outputs, and push signed model updates from React Native to Raspberry Pi (MQTT, WebSocket, fallbacks).

Ship low-latency on-device ML using React Native + Raspberry Pi: a practical guide

Hook: You need fast, reliable edge inference from a React Native app — but shipping a production data pipeline to a Raspberry Pi cluster introduces latency, compatibility, and update headaches. This guide walks you through proven patterns for streaming model inputs, receiving outputs, and rolling out model updates to Pi from a TypeScript React Native app in 2026.

Why this matters now (2026 context)

Edge-first architectures are mainstream in 2026. The Raspberry Pi 5 with community hardware like the AI HAT+2 (late-2025 hardware momentum) makes local inference affordable and performant. Meanwhile, React Native teams demand predictable latency, secure OTA model delivery, and clear fallback strategies when the network fails. This guide translates those requirements into concrete protocols, payload formats, and code you can run today.

Executive summary — what you'll get

Protocol decision matrix: MQTT vs WebSocket vs HTTP and when to pick each
Payload best-practices for images, audio, and telemetry (JSON vs binary, chunking, checksums)
RN TypeScript examples for streaming inputs over WebSocket and MQTT (incl. Expo guidance)
Raspberry Pi side: lightweight Python services for WebSocket and MQTT, and an ONNX/TFLite inference example
Model update/OTA flow: signing, delivery, atomic swap, rollback
Latency optimization and real-world fallback strategies (local inference, sampling, QoS)

Pick a protocol: MQTT vs WebSocket vs HTTP (quick decision)

Pick by requirements:

MQTT — best when you need reliable pub/sub, low-power clients, retained messages, and simple scaling across many Pis. Use MQTT when many devices send telemetry or you need robust reconnection semantics. Great for IoT-style edge fleets.
WebSocket (WSS) — best for low-latency RPC-style inference (request/response) between a single app and a Pi. Use binary frames for images/audio. Easier for direct browser / RN client without a broker in the middle.
HTTP(s) — simplest to implement but higher latency per request (new TCP/TLS handshake unless keep-alive). Use HTTP for larger file uploads, fallback polling, or model artifact downloads.

Recommended patterns

For single app -> single Pi: WebSocket (WSS) with binary frames.
For many clients and fleet management: MQTT over TLS with a broker (Mosquitto, EMQX) on the Pi or cloud.
For model delivery and large artifacts: HTTPS with signed downloads (store artifacts on Pi or on a private S3 bucket).

Payload formats and streaming strategies

Payload choice is the single biggest lever on latency and reliability.

Binary vs JSON

Use binary frames for images/audio to avoid base64 overhead (~33% larger) and JSON parsing cost.
Use JSON for small telemetry or control messages (e.g., inference metadata, start/stop signals).
Consider Protocol Buffers (protobuf) for compact, typed binary messages across language boundaries — great for stable schemas.

Chunking large frames

Split >1MB frames into chunks. Include a header with sequence number, total chunks, and an MD5/SHA256. This prevents head-of-line blocking and simplifies retransmit logic:

{
  "type": "chunk",
  "id": "img-2026-01-01-12-00-00",
  "seq": 3,
  "total": 10,
  "checksum": "sha256:...",
  "payload": ""
}

Example payload guideline for an image

Headers: id, timestamp, device_id, model_version, dtype (e.g., jpeg/gray), width/height, checksum
Binary body: raw JPEG/PNG bytes or preprocessed float/int8 tensor bytes
Response: small JSON with inference id, labels, confidences, elapsed_ms

React Native (TypeScript) — WebSocket streaming example

Below is a simple RN client using WebSocket to stream a jpeg and receive predictions. Works in bare RN and Expo with EAS build; for managed Expo you must use Expo's web socket APIs which are available but heavier to adapt for binary.

// src/edgeClient/wsClient.ts
import { Buffer } from 'buffer';

export type InferenceResult = {
  id: string;
  labels: { name: string; score: number }[];
  elapsed_ms: number;
};

export class WSClient {
  private ws: WebSocket | null = null;
  constructor(private url: string) {}

  connect() {
    this.ws = new WebSocket(this.url);
    this.ws.binaryType = 'arraybuffer';
    this.ws.onopen = () => console.log('WS open');
    this.ws.onmessage = (ev) => this.onMessage(ev.data);
    this.ws.onclose = () => console.log('WS closed');
    this.ws.onerror = (e) => console.error('WS error', e);
  }

  async sendImage(id: string, jpegBytes: ArrayBuffer) {
    if (!this.ws || this.ws.readyState !== WebSocket.OPEN) throw new Error('WS not open');
    // header + binary payload: small JSON header then raw bytes
    const header = JSON.stringify({ type: 'image', id, timestamp: Date.now() });
    const headerBuf = new TextEncoder().encode(header);
    const headerLen = Uint32Array.of(headerBuf.length).buffer;
    // wire: [4 bytes header length][header][jpeg bytes]
    const out = new Uint8Array(4 + headerBuf.length + jpegBytes.byteLength);
    out.set(new Uint8Array(headerLen), 0);
    out.set(new Uint8Array(headerBuf), 4);
    out.set(new Uint8Array(jpegBytes), 4 + headerBuf.length);
    this.ws.send(out.buffer);
  }

  private onMessage(data: any) {
    try {
      const str = typeof data === 'string' ? data : new TextDecoder().decode(new Uint8Array(data));
      const msg: InferenceResult = JSON.parse(str);
      console.log('Inference', msg);
    } catch (e) {
      console.error('Failed to parse message', e);
    }
  }

  close() { this.ws?.close(); }
}

Expo notes

Managed Expo supports WebSocket in JS. For binary streaming ensure you use arraybuffer handling via polyfills if needed.
If you need native TFLite inference on-device as a fallback, you will need EAS Build and a config plugin to include native modules — this cannot run in pure managed without prebuild.

React Native — MQTT example (TypeScript)

MQTT is ideal when your architecture uses a broker. The client can be a lightweight WebSocket-based MQTT to avoid native TCP requirements.

// src/edgeClient/mqttClient.ts
import mqtt from 'mqtt';

export class MQTTClient {
  private client: mqtt.MqttClient;
  constructor(private url: string, private clientId = 'rn-client-' + Date.now()) {
    this.client = mqtt.connect(url, { clientId, protocol: 'wss' });
    this.client.on('connect', () => console.log('MQTT connected'));
    this.client.on('message', (topic, payload) => console.log('msg', topic, payload.toString()));
  }

  publishImageTopic(topic: string, id: string, jpegBytes: ArrayBuffer) {
    const buf = Buffer.from(jpegBytes);
    // Use QoS 1 for delivery guarantee; keep payload binary
    this.client.publish(topic, buf, { qos: 1 }, (err) => { if (err) console.error(err); });
  }

  subscribe(topic: string) { this.client.subscribe(topic, { qos: 1 }); }
  end() { this.client.end(); }
}

Pi side: WebSocket server + ONNX/TFLite inference example (Python)

Minimal asyncio WebSocket server that reads our custom header + image, runs ONNX/TFLite inference, and returns JSON.

# pi_server/ws_infer.py
import asyncio
import websockets
import struct
from PIL import Image
import io
import time

# Example: using onnxruntime for ONNX models
import onnxruntime as ort

model = ort.InferenceSession('model.onnx')

async def handler(websocket, path):
    async for message in websocket:
        # message is bytes: [4B headerLen][headerJson][imageBytes]
        buf = message
        header_len = struct.unpack('I', buf[:4])[0]
        header = buf[4:4+header_len]
        img_bytes = buf[4+header_len:]
        meta = header.decode('utf-8')
        # preprocess for your model
        img = Image.open(io.BytesIO(img_bytes)).resize((224,224)).convert('RGB')
        arr = (np.array(img).astype('float32') / 255.0)[None,...]
        # run ONNX
        start = time.time()
        outputs = model.run(None, {'input': arr})
        elapsed = (time.time()-start)*1000
        result = {'id': '...', 'labels': [{'name':'cat','score':0.9}], 'elapsed_ms': elapsed}
        await websocket.send(json.dumps(result))

async def main():
    async with websockets.serve(handler, '0.0.0.0', 8765):
        await asyncio.Future()

if __name__ == '__main__':
    import numpy as np, json
    asyncio.run(main())

Optimizations on the Pi

Use ONNX Runtime with NNAPI/ARM acceleration where available or TFLite with the Edge TPU when using AI HAT+2 — consider portable hardware and field kits (see portable edge kits).
Quantize models to int8 to reduce latency and memory (2025-26 tooling improved quant-aware training pipelines).
Batch small sequential requests if latency budget permits.
Pin Python processes with gunicorn + uvicorn for concurrency if you use HTTP, or use asyncio for WebSocket.

Model updates and delivery (OTA) — secure, atomic, auditable)

Model updates are one of the most frequent sources of production incidents. Use signed artifacts, versioning, and an atomic swap approach:

Build model artifact (onnx/tflite) and compute SHA256 signature. Sign the checksum with your deployment key.
Publish a small JSON manifest via MQTT retained topic (e.g., models/manifest) or HTTP endpoint. Manifest: {version, url, checksum, signature, min_runtime_version}.
Pi periodically polls manifest or subscribes to MQTT retained manifest. When a new version appears, it downloads via HTTPS (S3 or your server), verifies checksum and signature, then writes to a new path (e.g., /models/model-v2.onnx.tmp).
Run a validation inference with a smoke input. If OK, atomically rename tmp to active and update a symlink. Keep previous version for rollback.
Report success/failure via MQTT telemetry and only switch traffic after smoke validation and health checks pass.

Security: Use TLS everywhere. For MQTT, use MQTT over TLS and client certs for fleet Pis. For manifests, enforce signature verification (Ed25519 or RSA) to prevent MITM updates — review threat models and hardening guidance from security playbooks like autonomous agent & agentic AI security analyses.

Fallback strategies and resilience

Design for network unreliability. Here are practical fallbacks ordered by complexity and effectiveness:

1) Retry + exponential backoff + persistent queue (client)

Queue unsent inputs in local storage (SQLite / MMKV). Retry with exponential backoff and jitter.
Cap queue size and drop oldest or lowest-priority items when full.

2) Client-side degradation

When latency increases or Pi unreachable, downsample images (e.g., to 224x224), reduce frame-rate, or switch to sending only keypoints/feature vectors instead of full images.

3) Local on-device inference fallback

When Pi is unreachable, run a compact model on-device. Implementation notes:

For iOS/Android use native TFLite or ONNX Runtime Mobile. In React Native you'll need native modules (or community plugins). For Expo, plan EAS/prebuild.
Keep the on-device model tiny (quantized, <5MB) for memory and cold-start speed — and consider desktop/agent security and user permissions models discussed in desktop agent security guides.

4) Downgrade to cloud processing

If both Pi and local fallbacks fail, route inference to a cloud endpoint if budget allows. Implement strict rate limits to control cost.

Latency tuning checklist

Measure at every segment: capture (app), preprocess (app), transport, queue, inference, postprocess, return. Log elapsed_ms.
Use binary streaming to minimize serialization cost.
Enable TCP_NODELAY for small packets on persistent sockets.
Reduce model input size; use feature extraction on client where possible.
Enable hardware acceleration on Pi (AI HAT+2/Edge TPU or GPU when available).
Use QoS=1 for MQTT when you need reliability but avoid QoS=2 (extra overhead) unless required.
Batch multiple small inferences when throughput > latency requirement.

Observability: the non-negotiable

Ship telemetry from both app and Pi — for guidance on monitoring, metrics, and alerts see monitoring & observability playbooks:

Request/response latencies and success rates
Model version used and inference times per model
Network conditions and retries
Queue depth and drop counts

Use lightweight time-series backends (Prometheus + Grafana on the Pi or push metrics to a cloud monitoring service). Correlate traces with IDs so the RN app can display when a Pi used which model version for a given inference.

Practical checklist to go from zero to production

Prototype with WebSocket: RN client & Python WebSocket server on Pi.
Measure baseline latency for your input type (image/audio). Optimize preprocessing and model size.
Decide on MQTT if you need pub/sub and fleet management. Stand up Mosquitto/EMQX with TLS and websockets enabled for RN clients.
Implement manifest-based OTA for models with signed artifacts & smoke tests on Pi — consider building CI/CD that mirrors production OTA flows (see CI/CD for model delivery patterns).
Implement client queue + local quantized fallback model via native module for resilience.
Implement observability and alerts for inference failures and model mismatches.

Code & tooling recommendations (2026)

React Native (0.72+) + TypeScript. Use EAS Build for any native module needs in Expo-managed projects.
MQTT: Mosquitto or EMQX on Pi. For RN use websocket MQTT (mqtt.js) with secure wss:// endpoints.
WebSocket libraries: built-in WebSocket for RN JS. For servers use Python websockets or uvicorn with websockets support.
Inference runtimes: ONNX Runtime 1.x (mobile builds), TensorFlow Lite with Edge TPU support, PyTorch Mobile for ARM where applicable.
Model format: ONNX for cross-framework portability; TFLite for many mobile accelerators (Edge TPU).

2026 trend: expect Pi-accelerator ecosystems (Edge TPUs, AI HAT+2) to standardize faster, making small, secure OTA model updates and quantized models the dominant path for edge ML deployment.

Troubleshooting common issues

WebSocket stalls

Ensure ws.binaryType='arraybuffer'. Check proxy/firewall that might buffer WebSocket frames. Use keep-alive pings.

MQTT connectivity problems

Enable websockets on Mosquitto for RN clients. Verify TLS certs and that the client uses correct host:port and wss scheme.

Model mismatch (app expects different output)

Embed model schema in manifest and return model_version in every inference response. Client-side must validate and support mapping layers for versioned changes.

Actionable next steps

Start small: build a WebSocket proof-of-concept that sends a downsized image, receives a label, and logs latency. Then add MQTT for fleet control and finally implement secure OTA manifest-based model updates. Use the TypeScript snippets here as your starting point and instrument every step for latency and error rates.

Call to action

If you want a production-ready starter kit for React Native & Raspberry Pi edge inference — including WebSocket and MQTT clients, Pi server templates, signed OTA workflow, and native module fallbacks — download our curated starter on reactnative.store or contact our engineering team for a hands-on audit of your pipeline.

reactnative

Contributor

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.