On-board AI inference for UAVs: edge processing without connectivity

A UAV without a datalink is not blind if its inference pipeline runs on-board. Ground-side AI processing is the common architecture for cost reasons, but it creates a hard dependency: the moment the link drops, detection stops. In contested environments where electronic warfare assets routinely degrade or sever datalinks, that dependency is operationally unacceptable. On-board AI inference moves the compute onto the aircraft, processing frames from the EO or IR sensor directly on a payload-mounted accelerator, storing annotated detections locally, and delivering a structured detection log to the C2 system the moment connectivity is restored. This article covers the hardware platforms, model compression techniques, detection pipeline design, power and thermal constraints, and C2 integration patterns that make on-device object detection and tracking viable on small tactical UAVs.

Why inference must run on-board the UAV

The operational argument for on-board inference is straightforward: any mission that requires detection continuity through a link outage cannot depend on ground-side processing. A UAV streaming raw video to a ground station requires a sustained high-bandwidth link - typically 2–10 Mbps for compressed HD video at acceptable latency. Achieving that bandwidth over a 10–30 km tactical range in an environment with active jamming is not guaranteed. When the video stream drops, the ground operator loses not just situational awareness of the UAV's position but every detection the sensor would have generated during the outage. For an intelligence, surveillance, and reconnaissance (ISR) mission spanning hours, a 20-minute link gap is not an acceptable gap in the detection record.

On-board inference resolves this by inverting the data flow. Instead of streaming raw sensor data to a ground processing node, the UAV streams only inference outputs: detection records that are orders of magnitude smaller than raw video frames. A detection record - class label, confidence score, bounding-box coordinates, GPS position, and timestamp - occupies a few hundred bytes. A full-resolution video frame at 10 Mbps occupies roughly 1.25 MB per second. When a low-bandwidth backup link is available (a short-burst data radio, a MANET mesh at reduced bitrate), detection records transit where raw video cannot. And when no link is available at all, detection records accumulate on local storage until the link recovers.

There is also a latency argument. Network round-trip time from a UAV at altitude to a ground station and back can range from 50 ms to several hundred milliseconds depending on link quality and routing. For time-sensitive targets - a vehicle exiting cover, a personnel group dispersing - that latency may exceed the window for useful cueing. On-board inference produces a detection result within the frame processing time of the local accelerator, typically 30–100 ms from frame capture to annotated output, with no network hop in the critical path.

Hardware platforms: NVIDIA Jetson Orin vs Hailo-8 vs Intel Movidius for UAV payloads

Three silicon families dominate production UAV payload deployments. Each represents a different point on the TOPS-per-watt curve, and the right choice is determined primarily by the platform's power budget and the complexity of the inference task. A detailed comparison of these options is also covered in our article on edge AI hardware for defense.

The NVIDIA Jetson Orin Nano delivers 40 TOPS at 7–15 W depending on the power mode configured in the NV power management framework. It runs a full Linux stack, supports CUDA and TensorRT natively, and accepts any model exported via the ONNX interchange format - making it the lowest-friction option for teams whose training pipeline already targets TensorRT. The Orin Nano's form factor (69.6 mm x 45 mm) fits payload bays on 5–15 kg class UAVs, but its peak thermal dissipation requires either active cooling or careful integration of heat spreaders against the airframe skin. The Jetson Orin NX, the next tier up at up to 100 TOPS and 15–25 W, is appropriate when the mission requires running multiple concurrent models (detection plus classification plus re-identification) on the same inference node.

The Hailo-8 M.2 module achieves 26 TOPS at under 5 W, delivered through a highly efficient dataflow architecture that tiles the network graph across an array of processor clusters rather than executing it sequentially on a GPU core. The trade-off is toolchain specificity: models must be compiled through the Hailo Dataflow Compiler, which generates a Hailo Execution Format (HEF) binary. The compiler handles INT4 and INT8 quantization internally and produces highly optimized binaries, but it requires that the model topology be representable in the compiler's supported operator set. Standard YOLOv5, YOLOv8, and RT-DETR architectures are in the Hailo Model Zoo and compile without modification. The Hailo-8's power envelope - 5 W peak from an M.2 slot - makes it the natural choice for sub-5 kg multi-rotor platforms with tight payload power budgets.

The Intel Movidius Myriad X (OpenVINO target) delivers 4 TOPS at approximately 1–2 W. That figure is modest compared to Hailo and Jetson, but the Myriad X's integration density - it is available in USB stick and M.2 form factors that attach to any Linux host via standard interfaces - makes it the simplest option for very small fixed-wing or tube-launched UAVs where a dedicated carrier board cannot be accommodated. OpenVINO's model optimizer handles the INT8 quantization and graph optimization path, and the tool chain is well-documented. For deployments where a YOLOv8n running at 5–10 fps is sufficient - low-speed ISR over fixed infrastructure, for example - the Myriad X is a viable option at significantly lower SWaP-C cost than Hailo or Jetson.

Model compression: quantization, pruning, and knowledge distillation for embedded deployment

A full-precision YOLOv8m model has approximately 25 million parameters and occupies 50 MB in FP32 storage. On a Hailo-8 with 8 MB of on-chip SRAM, that model cannot run directly - it must be compressed to a size the accelerator's memory hierarchy can accommodate without excessive DRAM bandwidth pressure. Three compression techniques are used in combination for production UAV deployments, and their interaction is covered in depth in our article on ONNX and TensorRT model optimization for tactical edge deployment.

Post-training quantization (PTQ) converts FP32 weights and activations to INT8 by calibrating scale factors using a representative dataset. On hardware with INT8 tensor execution units - Jetson Orin, Hailo-8, Myriad X - INT8 inference delivers 2–4x throughput improvement over FP32 at equivalent model capacity. Accuracy loss is typically 0.5–2 mAP on aerial object detection benchmarks when the calibration dataset matches the deployment domain. The calibration dataset should include samples from the actual sensor and altitude at which the model will be deployed; calibrating on publicly available aerial datasets and deploying on a different sensor creates a domain mismatch that can degrade quantized accuracy more than the raw numbers suggest.

Structured pruning removes entire convolutional filters (output channels) whose L1 norm falls below a threshold, producing a topologically smaller model that benefits from both reduced parameter count and reduced memory bandwidth. A 30% structured pruning of YOLOv8s removes roughly 30% of filters across the backbone, reducing inference latency by approximately 25% on Jetson hardware and improving the INT8 accuracy floor after requantization (since the remaining filters are the most activated ones). The pruning threshold is tuned iteratively: prune, fine-tune for 10–20 epochs on the training set, measure accuracy on the validation set, and repeat until the target latency or memory footprint is achieved without exceeding the acceptable mAP budget.

Knowledge distillation addresses the case where pruning the full model cannot reach the target parameter count without unacceptable accuracy loss. A compact student model - YOLOv8n, for example - is trained to reproduce the logit outputs of a larger teacher (YOLOv8m or YOLOv8l) on the training data. The student learns to mimic the teacher's confidence distribution across classes, not just its hard label assignments, which conveys information about class ambiguity and feature similarity that hard-label training does not. Student models trained via distillation consistently outperform identically sized models trained from scratch, typically by 1–3 mAP on aerial detection benchmarks. The technique is particularly effective when the training dataset is small - a common situation in defense deployments where labeled aerial imagery of specific target classes is scarce.

Target detection pipelines at the edge: YOLO variants and frame-rate vs accuracy trade-offs

The YOLO family dominates production edge detection deployments on UAV payloads for two reasons: the architecture is single-stage (no region proposal network), which keeps inference latency predictable and bounded, and the model zoo is extensive - pre-trained weights at multiple scales (n, s, m, l, x) let teams select the capacity tier that fits their hardware budget. YOLOv8 is the current production baseline for most defense integrators, though RT-DETR (a transformer-based single-stage detector) is gaining traction for use cases where detection at small object scales - personnel at 500 m altitude, for example - is critical and the inference hardware can absorb the higher compute cost.

Frame rate and accuracy trade against each other through three variables that the pipeline engineer controls: model scale, input resolution, and confidence threshold. Reducing input resolution from 640x640 to 320x320 cuts inference compute by approximately 4x and doubles the achievable frame rate on fixed hardware, but it reduces the effective detection range for small targets proportionally. For vehicle detection at 100–200 m altitude, 320x320 is generally sufficient. For personnel detection at the same altitude, 640x640 is the practical minimum. Reducing the confidence threshold from 0.5 to 0.35 recovers detections of partially occluded or distant targets but increases the false positive rate, which raises the annotation burden on the downstream fusion system. The correct threshold is mission-specific and should be tuned against ground truth data from the deployment sensor and altitude.

Multi-stage pipelines - a lightweight detector to identify regions of interest followed by a higher-capacity classifier to refine the label - can improve accuracy at a given compute budget by concentrating resources on candidate regions. A YOLOv8n running at 30 fps identifies bounding boxes around vehicle-sized objects; a MobileNetV3 classifier running at 10 fps processes only the cropped regions to distinguish wheeled from tracked and light from heavy. This cascade architecture is well-suited to the Hailo-8's dataflow model, where two networks can be compiled into a single HEF and executed as a pipelined graph rather than sequentially, hiding the latency of the second stage behind the frame interval of the first.

Power budget constraints and thermal management on small UAVs

The power budget for a UAV payload is not fixed by the aircraft's total energy capacity but by the payload bay's current allocation from the power distribution unit (PDU) and by the flight management system's thermal model of the payload bay. A 5 kg multi-rotor with a 100 Wh battery and a 30-minute flight time has an average power draw of approximately 200 W from all systems combined. The propulsion system consumes 80–90% of that budget under typical hover conditions, leaving 20–40 W for avionics, payload, and sensors. A payload that draws 15 W leaves only 5–25 W for everything else. In practice, payload power allocations on sub-5 kg platforms are often set at 5–8 W by the aircraft manufacturer, and exceeding them degrades endurance measurably.

Thermal management on small UAVs is harder than on ground vehicles or fixed infrastructure because the payload bay is sealed, small, and may be constructed from thermally insulating composite materials. The UAV airframe provides some cooling through skin conduction and airflow over the exterior surface, but this benefit varies with flight speed and payload bay geometry. The standard approach is to mount the inference compute board on a thermal interface pad against the thickest aluminum structural member of the payload chassis, which serves as a heat spreader to the airframe. For the Jetson Orin Nano at 10 W steady-state, a 2 mm aluminum plate bonded to the fuselage underside with thermal paste can maintain die temperature below 75°C at 40°C ambient during level flight. For the Hailo-8 at 5 W, a simple heatsink attached to the module's M.2 connector is typically sufficient.

Power consumption also responds to software configuration. The Jetson platform's nvpmodel utility exposes power mode presets that cap CPU clock, GPU clock, and memory bandwidth. Setting the Orin Nano to a 7 W power mode reduces inference throughput by approximately 35% compared to the 15 W maximum mode but cuts heat dissipation almost in half, which may be the correct trade when the payload bay thermal envelope is the binding constraint rather than inference latency. A configuration that runs the aircraft's full mission profile without thermal throttling is more reliable than one that achieves peak inference performance for 15 minutes before triggering the chip's automatic thermal protection and dropping frame rate unpredictably.

C2 integration via MANET on reconnect

The detection log accumulated during a link outage has limited value unless it can be delivered to the command and control system quickly and completely when connectivity is restored. MANET (Mobile Ad-hoc Network) radios - used in tactical UAV operations to provide mesh connectivity between the aircraft, ground vehicles, and dismounted operators - are the primary transport for both real-time detection streaming and post-outage sync. When the UAV re-enters mesh coverage, the MANET node on-board re-establishes routing within seconds, and the sync agent begins transmitting buffered detection records.

The sync agent architecture should be designed around two operating modes. In connected mode, detection records are transmitted in near-real-time as they are generated by the inference pipeline, serialized as CoT events and published to TAK Server via the MANET mesh. In disconnected mode, records accumulate in a local SQLite database with a "synced" flag. On reconnect, the agent queries for unsynced records ordered by timestamp and transmits them in chronological order, rate-limited to avoid saturating the MANET radio's bandwidth with burst traffic at the expense of other network participants. After each record is acknowledged by the server, it is marked synced. The database provides crash recovery: if the UAV lands or the inference service restarts mid-sync, the next sync cycle resumes from the last unsynced record rather than retransmitting the full log.

CoT event construction for UAV AI detections follows the standard CoT schema but carries payload-specific extensions in the detail block. The detection class and confidence score populate custom CoT detail sub-elements. The UAV's GPS position at frame capture time is used as the CoT point, and where the inference pipeline can compute a ground-projected detection position from the sensor's field of view, altitude above ground, and gimbal angle, that projected coordinate is used instead - placing the detection marker on the map at the target's actual ground position rather than the aircraft's position. TAK Server delivers these CoT events to all connected ATAK clients, where they appear as classified markers with detection provenance and confidence metadata visible in the event detail view.

Procurement: SWaP-C and MIL-STD environmental ratings

Procuring an on-board AI inference payload for a tactical UAV program requires evaluating hardware against three non-negotiable axes: SWaP-C (Size, Weight, Power, and Cost), environmental ratings, and supply chain maturity. SWaP-C drives the platform trade described above - TOPS per watt, physical dimensions, and unit cost all constrain which silicon is viable. Environmental ratings determine whether the hardware survives the deployment conditions it will encounter. Cost drives the platform's expendability doctrine: a payload intended for a recoverable ISR UAV has a different cost tolerance than one installed on a one-way loitering system.

MIL-STD-810 defines the environmental stress tests relevant to UAV payloads: temperature cycling (method 501/502), vibration (method 514), humidity (method 507), altitude (method 500), and shock (method 516). Commercial-off-the-shelf (COTS) inference modules such as the Jetson Orin Nano are rated for 0–80°C operating temperature on the commercial variant and -25 to 80°C on the industrial variant. UAV payload bays in cold climates or high-altitude operations may expose the compute board to temperatures below -25°C at startup. Where this is a concern, a thermal pre-conditioning circuit - a resistive heater controlled by a thermostat that warms the bay before main payload power is applied - is a simpler and more reliable solution than selecting a wider-temperature-range processor that may not exist at the required capability tier.

Supply chain maturity matters disproportionately in defense programs because long production runs require consistent component availability over multi-year periods. The Jetson Orin module lifecycle is published by NVIDIA with a 10-year production commitment for industrial-grade variants, which satisfies most defense program lifecycle requirements. The Hailo-8 is a newer entrant with a shorter production history, and procurement offices evaluating it for high-volume programs should require contractual supply continuity commitments. For platforms where the inference chip will be embedded in a custom carrier board designed to program specifications, the carrier board design should include footprint provisions for at least one alternative accelerator module so that a mid-program chip lifecycle change does not require a carrier board respin.