RK3566 可運行 YOLOv8 INT8，但只限於特定條件

RK3566 Can Run YOLOv8 INT8 — But Only Within These Limits

Running YOLOv8 INT8 on an RK3566 is not just a model conversion task. It is a system-level alignment problem. This article walks through how quantization, operator compatibility, and detection head design define whether real-time inference is actually possible on the RK3566 — and under what strict conditions.

Why YOLOv8 INT8 Can't Just Run on RK3566

The real issue with combining RK3566 and YOLOv8

In many edge AI projects, the RK3566 is treated as "a cost-effective platform that can also handle some AI tasks."

Its positioning isn't aggressive: controllable power consumption, full peripheral support, and limited but non-zero compute power. That means one thing — it was not designed for complex models with compute headroom.

YOLOv8 Detection sits in a delicate spot. It is no longer a "lightweight, run-anywhere" model, but it is still far from server-level detection models. In theory, it lands in the "just about doable" category for the RK3566. In practice, this assumption often fails during deployment.

Many projects run smoothly at the model stage: ONNX exports fine, PC-side inference is normal, and the structure doesn't look too complex. But once converted to RKNN and deployed on-device, performance drops — unstable FPS, high CPU usage, and rapid system resource exhaustion.

This isn't one parameter being off. It's deeper: the RK3566 is not a platform that brute-forces inference. Usability depends on having a clean execution path.

Why floating-point inference has little value on RK3566

Running YOLOv8 Detection in FP16 or FP32 on the RK3566 usually leads to a predictable result: the model runs, but runs poorly.

This isn't an "implementation issue" of RKNN or the NPU — it's a design-logic issue of the platform. On the RK3566, the NPU is not a fully independent compute unit. If there are unsupported operators in the model, execution falls back to the CPU, and detection models often include such unsupported operations.

With floating point, the problems worsen:

Low NPU coverage for FP ops
Constant data movement between CPU and NPU
Fragmented inference with high scheduling overhead

Result: single-digit FPS with near-maxed system load. In this state, optimizing FP16 further is meaningless. It isn't under-tuned; it's the wrong execution path for this hardware.

INT8 isn't a "bonus" — it's the entry point

Switching to INT8 reveals the RK3566's true nature. INT8 isn't just about precision — it unlocks the most stable, fully supported execution path on the chip.

In INT8 mode:

Operator mapping success rate rises
Ops can stay in the NPU for longer
The CPU acts more like a scheduler than a compute unit

Now YOLOv8 Detection starts "really running on the NPU," not just "partially using the NPU."

But INT8 isn't zero-cost. Detection models are sensitive to quantization, especially in the Head. Poor quantization leads to missed detections or box jitter. So the real question isn't "should we use INT8" — it's: how far can YOLOv8 Detection go on the RK3566 with INT8, and what are the boundaries?

YOLOv8 Detection's structure determines RK3566 compatibility

YOLOv8 Detection isn't structurally complex, but its complexity concentrates in subtle areas.

The Backbone is usually fine. As long as channel counts and input sizes aren't extreme, the RK3566's NPU handles it stably. The real issues appear in the later stages:

Irregular scale changes during feature fusion
Unpredictable Concat and Upsample combinations
Unnecessary tensor ops in the Detection Head

These are legal in ONNX but can prevent the RKNN compiler from statically fixing the compute graph. Most failures aren't due to model size — they're structural elements the compiler can't resolve. If the graph can't be fully static, NPU advantages disappear quickly.

Structure first — a prerequisite for INT8 success

On the RK3566, if structure doesn't serve the execution path, quantization only helps partially. Repeatedly validated practices include:

Fixed input size is more important than flexibility
Dynamic shapes offer less benefit than cost here
Simpler Detection Heads yield more stable INT8 results

These aren't flashy choices, but they aim at one thing: keep the entire inference inside the NPU without interruption. If the path breaks, even aggressive quantization can't fix overall performance.

How YOLOv8 INT8 Actually Runs on RK3566

What happens between model and device

Placing a YOLOv8 Detection model onto the RK3566 isn't determined by the export — it's determined by the in-between steps that are often oversimplified. From PyTorch to on-device execution, three major changes occur:

The graph gets compressed into a statically analyzable form.
Data precision maps from float to fixed point.
The execution path splits between NPU and CPU.

Any "gray area" here nearly guarantees performance issues. INT8 adds clarity to this path, and the RK3566's INT8 support goes beyond compute — it affects compilation, scheduling, and caching.

INT8 quantization isn't a one-click step

Many first-time RKNN INT8 users think it's just a switch: enable INT8, feed a few images, done.

It's more like a filtering process. Calibration data doesn't "train" the model — it constrains value ranges. Detection models are highly sensitive to feature distribution shifts, especially with large object-size variation.

If calibration data poorly matches real scenes, expect:

Small-object confidence drops
Box jitter across frames
Higher false positives in complex backgrounds

These problems usually occur in the Detection Head, not the Backbone.

The Detection Head is the make-or-break point for INT8

On the RK3566, bottlenecks rarely lie in the Backbone. The real gap appears downstream. The Detection Head has:

Frequent resolution shifts
Wide numerical ranges
Extreme precision demands

That makes it the most prone to quantization distortion. Even if Backbone and Neck quantize well, aggressive Head quantization can degrade detection. That's why many RK3566 failures stem not from "large models," but from Head structures misaligned with quantization.

Execution continuity beats theoretical compute

Once on-device, another critical issue surfaces: can operators stay on the NPU continuously?

Ideally, input goes into the NPU, multiple layers execute without switching, and output returns to the CPU. But if an op can't run on the NPU, control switches to the CPU. On the RK3566, with limited compute and bandwidth, this cost is high.

INT8 greatly increases the odds of uninterrupted NPU execution. That's why INT8 often yields better-than-linear performance gains on the same model.

Quantization failures don't mean the model is bad

Often, models that work fine on PC fail post-quantization on the RK3566. This doesn't mean model choice was wrong. More likely:

Inadequate calibration data coverage
Input size mismatch
Detection Head too complex

In such cases, simplifying structure — reducing branches or compressing channels — is more effective than tweaking quantization parameters.

Real-World Limits of YOLOv8 INT8 on RK3566

Test conditions and constraints

To avoid misleading results, all tests follow the same setup:

Hardware: RK3566 (NPU enabled)
Model: YOLOv8 Detection (no pruning)
Input size: 640×640, fixed
Inference mode: single-frame, Batch=1
Post-processing: on CPU, not on NPU

The goal is to evaluate a "reusable in production" YOLOv8 Detection on the RK3566, ignoring extreme tuning and special trims.

FPS: INT8 vs FP16 isn't linearly different

The raw numbers tell the story:

Model Precision	Inference FPS (640×640)	Stability
FP16	3 – 5 FPS	Unstable, high CPU load
INT8	12 – 18 FPS	Stable, sustained

This isn't just "INT8 is faster" — it's a different execution mode. FP16 triggers frequent CPU fallback. INT8 allows sustained NPU-only execution. Only in INT8 mode does YOLOv8 Detection become close to real-time on the RK3566.

Accuracy loss: localized, not global

FPS boost is just the start. Detection accuracy defines usability. INT8 quantization doesn't degrade uniformly — it shows structural patterns:

Scene Type	Accuracy Change
Medium / Large objects	Mostly stable
Simple background	Almost unaffected
Small / dense objects	Much more sensitive
Complex textures	More false positives

INT8 doesn't "weaken everything" — it amplifies pre-existing weak points. If the Detection Head and calibration data match your real use case, losses are usually acceptable.

There is no "sweet spot" between FPS and accuracy

A common RK3566 myth: find a "perfect FPS with minimal accuracy drop."

Reality: you either choose INT8 and accept structured accuracy changes, or use FP16 and sacrifice real-time capability. It's not a quantization flaw — it's a hardware limit. As model complexity nears the RK3566's ceiling, you can't have both high FPS and full accuracy.

Turning test data into engineering judgments

Compressing the findings:

The usable FPS lower bound is ~10 FPS on the RK3566.
Below that, load spikes and stability plummets.
INT8 is a must — but not sufficient on its own.
Model structure and Head design define quantized accuracy.

When it's worth using, and when to switch plans

Suitable

Single-class or few-class detection
Medium and larger object sizes
FPS target of 10–15
Prioritize response speed over peak accuracy

Not suitable

Many small objects
High precision required
Complex post-processing
Expecting PC-level accuracy on the RK3566

In the unsuitable cases, pushing the RK3566 further yields little. You have to change model size, hardware, or task design.

YOLOv8 Detection feasibility on RK3566 (INT8)

Dimension	Acceptable Range	Typical Issue When Exceeded	Takeaway
Precision type	INT8	<5 FPS in FP16/FP32	INT8 is essential
Input size	≤640×640	Larger → nonlinear FPS drop	Fixed input preferred
Real FPS	12–18 FPS	<10 FPS → system overload	10 FPS = lower bound
NPU utilization	High (continuous)	Frequent CPU fallback	Path continuity > GFLOPs
Backbone	Light to medium	Rarely a problem	Acceptable
Detection Head	Simpler = better	Box jitter / missed detections	Decides success or failure
Small-object density	Low to medium	High → misdetects increase	Not an ideal use case
Calibration data	Scene-aligned	Misaligned → accuracy loss	Critical for INT8
Long runtime	Stable in INT8	FP16 fluctuates	INT8 is sustainable

Final judgment

If only one takeaway matters, it's this: the RK3566 can run YOLOv8 Detection — if you accept INT8 and understand its limits.

It's not a failure platform, nor "AI-ready by default." When model, structure, and expectations align, the RK3566 delivers stable, predictable results. Push beyond its limits, and both performance and accuracy collapse.

Quick decision guide

Your Need	RK3566 + YOLOv8 INT8 Recommended?
Few-class detection	Yes
Medium object sizes	Yes
Real-time (≥10 FPS)	Yes
Many small objects	No
High-precision localization	No
Complex post-processing	No

Key takeaways

INT8 is the starting point for YOLOv8 on the RK3566.
The FPS gain comes from an execution-path shift, not a compute boost.
Accuracy loss centers on the Detection Head and select cases.
Once platform boundaries are clear, decisions become simpler.

FAQ

Q: What is the optimal inference mode for YOLOv8 on the RK3566?
A: INT8 quantization is the only viable mode. It ensures maximum NPU utilization, minimizes CPU fallback, and enables 12–18 FPS, compared to 3–5 FPS in FP16.

Q: Why is the RK3566 not suited for floating-point inference?
A: Floating-point ops are only partially supported by the RK3566 NPU. Unsupported ops get routed to the CPU, leading to fragmented execution and low performance.

Q: Where does INT8 quantization most affect YOLOv8 accuracy?
A: In the Detection Head, due to frequent resolution changes and high precision needs. It's the most fragile area post-quantization.

Q: How can I make sure my YOLOv8 model survives INT8 quantization?
A: Use calibration data that mirrors deployment scenarios, fix input resolution, and simplify the Detection Head architecture.

Q: Is there a performance-accuracy sweet spot for YOLOv8 on the RK3566?
A: No. You either accept INT8's structured accuracy loss or fall back to FP16 with unacceptably low FPS. It's a binary choice dictated by platform limits.