Ethos-U55 Qualification Report

FAILEDexit code 1

1 of 9 operators is not on the accelerator: L2_NORMALIZATION (Unsupported opType). The accelerator graph is split into 2 partitions.
2 of 500 decisions (0.4%) differ between GOLDEN and SCHEME (quantization).
TARGET did not execute: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached.
Limits exceeded: placement.fallback_ops.

5 declared limits: 1 pass, 1 fail, 3 without an admissible measurement.

Declared limitsVerdict evidence

Parameter	Limit	Measured	Unit	Evidence	Result
`accuracy.answer_flip_rate`Fraction of eval inputs whose decision differs between GOLDEN and TARGET.	≤ 0.0200 (2.0%)	—	fraction	TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached	NOT MEASURED
`accuracy.top1_delta`GOLDEN top-1 accuracy minus TARGET top-1 accuracy. Positive means TARGET is lower. Requires labels.	≤ 0.0100 (1.0%)	—	fraction	TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached	NOT MEASURED
`latency.p99_ms`99th percentile steady-state inference time on TARGET.	≤ 15.00	—	ms	target ethos-u55-128 does not declare latency telemetry	NOT MEASURED
`memory.flash_mb`Size of the compiled artifact as stored for deployment. MiB = 2^20 bytes.	≤ 0.5000	0.01459	MiB	host filesystem: size of artifact_int8_vela.tflite	PASS
`placement.fallback_ops`Source-graph operators the target toolchain placed on the CPU instead of the accelerator.	≤ 0	1	ops	compiler: ethos-u-vela 5.2.0; classified from the compiled artifact	FAIL

Rule: FAILED if any declared limit is measured and exceeded; otherwise INCOMPLETE if any declared limit lacks an admissible measurement; otherwise QUALIFIED. No budgets: NO_VERDICT. Limits are inclusive. Budgets file examples/demo/budgets.yaml, canonical SHA-256 b5e97b4d08db33796d0ee0534a931ad3b6eb7a269cefdfeac8901757d19bf793.

Provenance

Run id: 20260915T032617Z-ethos-u55-128-32c372d2
Created (UTC): 2026-09-15T03:26:17+00:00
Harness: gauntlet-harness 0.1.0, git: repository has no commits; uncommitted changes
Source tree: 38a45a6159a766c4948ed18b5137cc5d5cc1ce28cafffe391e56bef49ce959a6
Host: <redacted>, Darwin 25.6.0 arm64
Host CPU: Apple M2, 8 logical
Target: ethos-u55-128: Arm ethos-u55-128 (Ethos_U55_High_End_Embedded, Shared_Sram)
Target firmware: not observed: target did not execute
Hardware: adapter has never run against physical hardware
Inputs unchanged: yes: SHA-256 identical before and after the run

Files

Role	File	SHA-256	Bytes
Reference (FP32)	`examples/demo/reference.onnx`	`491aaffa3e9b7e1814c1b0effbf86c8b4198d2cc47b8e3cc89d5a70840fc01e6`	26,814
Artifact (INT8)	`examples/demo/artifact_int8.tflite`	`32c372d24760628795dae729f5298c98ef93aa3f751565fa1c0b58efda3212b0`	12,696
Compiled artifact	`runs/20260915T032617Z-ethos-u55-128-32c372d2/targets/ethos-u55-128/compile/artifact_int8_vela.tflite` compiled	`d953306bd1825e8dbfafc03a8644132303f43473e5f9b5ec85fc9035c5ce334c`	15,296
Eval set	`examples/demo/evalset.npz` 500 inputs, labels present	`6a9e56fb06bf4d88567fc9cc8a954849b0dffd194a7b2266300d2e7dc7a00d1c`	6,148,510
Budgets	`examples/demo/budgets.yaml`	`fbb08dec95b282b883020cacf83346da2a69510112f6357182de7b0a0b8500d6`	576
Comparison points	`examples/demo/points.yaml`	`170c27ac112341a1c5d967bfdfe8b7cd71f22b4f86ae71dc98ecc6879d894e94`	1,682

Measurement conditions

NOT RECORDED Measurement conditions are captured where execution happens; TARGET did not execute.

Execution settings

GOLDEN: onnxruntime 1.30.0, CPUExecutionProvider, 1 thread, sequential, graph optimizations disabled
SCHEME: ai-edge-litert 2.2.0, op resolver BUILTIN_REF, 1 thread
TARGET: not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached
Input binding: float input bound to integer input tensor as clamp(round_half_away_from_zero(x / scale) + zero_point, dtype range), using the artifact's declared input quantization parameters
Seed: 0
Bootstrap: 2000 resamples, 95% CI, percentile bootstrap, paired resampling of eval inputs
Latency trials: 50 timed, 5 warm-up discarded
Tapped inputs: first 512 inputs for intermediate tensors; all inputs for model outputs

Toolchain: ai-edge-litert 2.2.0, ethos-u-vela 5.2.0, numpy 2.5.3, onnxruntime 1.30.0, pydantic 2.13.5, python 3.12.7, pyyaml 6.0.3.

Compiler invocation and evidence files

Toolchain: compiler: ethos-u-vela 5.2.0
Command: vela examples/demo/artifact_int8.tflite --accelerator-config ethos-u55-128 --config Arm/vela.ini --system-config Ethos_U55_High_End_Embedded --memory-mode Shared_Sram --output-dir runs/20260915T032617Z-ethos-u55-128-32c372d2/targets/ethos-u55-128/compile --show-cpu-operations --verbose-performance
accelerator_config: ethos-u55-128
config_file: Arm/vela.ini
memory_mode: Shared_Sram
system_config: Ethos_U55_High_End_Embedded
vela.accelerator_configuration: Ethos_U55_128
vela.batch_size: 1
vela.core_clock: 500000000.0
vela.memory_mode: Shared_Sram
vela.system_config: Ethos_U55_High_End_Embedded

Evidence file	SHA-256
`runs/20260915T032617Z-ethos-u55-128-32c372d2/targets/ethos-u55-128/compile/vela_stdout.txt`	`b3d69d617a5465bfab09343c28f497fd3e5e2cb6338c1db0080bd9f6cd66cba2`
`runs/20260915T032617Z-ethos-u55-128-32c372d2/targets/ethos-u55-128/compile/vela_stderr.txt`	`e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`
`runs/20260915T032617Z-ethos-u55-128-32c372d2/targets/ethos-u55-128/compile/artifact_int8_per-layer.csv`	`7a7673dd6b30f47d9a13d524e831d8e2a8e6bf50cdeff78b3921dab8711a9270`
`runs/20260915T032617Z-ethos-u55-128-32c372d2/targets/ethos-u55-128/compile/artifact_int8_summary_Ethos_U55_High_End_Embedded.csv`	`ea6920b0f9aa13806d0e67d2be1b78bc17521811181a0dad9073ca999b122782`

Placement map

8accelerated1fallback to CPU0unsupported2accelerator partitions

#0CONV_2DNPU

#1MAX_POOL_2DNPU

#2CONV_2DNPU

#3MAX_POOL_2DNPU

#4CONV_2DNPU

#5MEANNPU

#6L2_NORMALIZATIONCPU

#7FULLY_CONNECTEDNPU

#8SOFTMAXNPU

Source-graph operators in execution order. Operators marked CPU run outside the accelerator; each one ends a partition. Classified by compiler: ethos-u-vela 5.2.0 from the compiled artifact.

#	Operator	Placement	Output tensor	Toolchain reason
6	L2_NORMALIZATION	FALLBACK CPU	`functional_1_1/l2norm_1/l2_normalize`	Unsupported opType

All 9 operators with evidence

#	Operator	Placement	Output tensor	Evidence
0	CONV_2D	ACCELERATED	`functional_1_1/conv1_1/Relu;functional_1_1/conv1_1/BiasAdd;functional_1_1/conv1_1/convolution;functional_1_1/conv1_1/Squeeze1`	per-layer CSV: 2 NPU operation(s) (Conv2D, Relu)
1	MAX_POOL_2D	ACCELERATED	`functional_1_1/pool1_1/MaxPool2d`	per-layer CSV: 1 NPU operation(s) (MaxPool)
2	CONV_2D	ACCELERATED	`functional_1_1/conv2_1/Relu;functional_1_1/conv2_1/BiasAdd;functional_1_1/conv2_1/convolution;functional_1_1/conv2_1/Squeeze1`	per-layer CSV: 2 NPU operation(s) (Conv2D, Relu)
3	MAX_POOL_2D	ACCELERATED	`functional_1_1/pool2_1/MaxPool2d`	per-layer CSV: 1 NPU operation(s) (MaxPool)
4	CONV_2D	ACCELERATED	`functional_1_1/conv3_1/Relu;functional_1_1/conv3_1/BiasAdd;functional_1_1/conv3_1/convolution;functional_1_1/conv3_1/Squeeze1`	per-layer CSV: 2 NPU operation(s) (Conv2D, Relu)
5	MEAN	ACCELERATED	`functional_1_1/gap_1/Mean`	per-layer CSV: 1 NPU operation(s) (Mul)
6	L2_NORMALIZATION	FALLBACK CPU	`functional_1_1/l2norm_1/l2_normalize;functional_1_1/l2norm_1/l2_normalize/Square;functional_1_1/l2norm_1/l2_normalize/Sum/reduction_indices;functional_1_1/l2norm_1/l2_normalize/Sum;functional_1_1/l2norm_1/l2_normalize/Maximum/y;functional_1_1/l2norm_1/l2_normalize/Maximum;functional_1_1/l2norm_1/l2_normalize/Rsqrt`	compiled artifact: L2_NORMALIZATION remains a CPU operator; per-layer CSV target CPU
7	FULLY_CONNECTED	ACCELERATED	`functional_1_1/fc_1/MatMul;functional_1_1/fc_1/BiasAdd`	per-layer CSV: 1 NPU operation(s) (FullyConnected)
8	SOFTMAX	ACCELERATED	`StatefulPartitionedCall_1:0`	per-layer CSV: 1 NPU operation(s) (Asr)

Toolchain counts (vela cpu operators = 1, vela npu operators = 40) are taken after its internal operator decomposition and differ from the 9 source-graph operators classified here.
UNSUPPORTED is not assigned by this adapter: Vela places every operator it cannot accelerate on the CPU, and whether the CPU runtime registers a kernel for it is set by the application's TFLite Micro op resolver, which is not observable at compile time.

Drift attributionWhere the output changes, per transformation

Quantization damage (GOLDEN→SCHEME) at the model output is RMSE 0.003327; vendor deviation (SCHEME→TARGET) was not measured: TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached.

Largest SNR decrease between consecutive comparison points in GOLDEN→SCHEME: pool2 → conv3 (−6.73 dB).

SNR at each comparison point, in data-flow order. Higher means less drift. Filled circles: GOLDEN→SCHEME. Open squares: SCHEME→TARGET. Points without a mark were not measured or not comparable (see table).

Values compare real-valued tensors (quantized tensors dequantized with their own scale and zero point). Max LSB is the largest integer-code difference where both columns share quantization parameters.
Point	Operator	GOLDEN→SCHEME: quantization damage				SCHEME→TARGET: vendor deviation
Point	Operator	Cosine	SNR dB	Max \|err\|	RMSE	Cosine	SNR dB	Max \|err\|	RMSE	Max LSB
conv1	CONV_2D	0.999971	42.33	0.274	0.00744	NOT COMPARABLE tensor does not exist in the compiled artifact (internal to a compiled partition); not comparable at this point
pool1	MAX_POOL_2D	0.999966	41.67	0.268	0.0103	NOT COMPARABLE tensor does not exist in the compiled artifact (internal to a compiled partition); not comparable at this point
conv2	CONV_2D	0.999912	37.26	0.193	0.00907	NOT COMPARABLE tensor does not exist in the compiled artifact (internal to a compiled partition); not comparable at this point
pool2	MAX_POOL_2D	0.999934	38.54	0.193	0.0114	NOT COMPARABLE tensor does not exist in the compiled artifact (internal to a compiled partition); not comparable at this point
conv3	CONV_2D	0.999674	31.81	0.140	0.00518	NOT COMPARABLE tensor does not exist in the compiled artifact (internal to a compiled partition); not comparable at this point
gap	MEAN	0.999812	34.14	0.0120	0.00116	NOT EXECUTED TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached
l2norm	L2_NORMALIZATION	0.999805	34.09	0.0295	0.00349	NOT EXECUTED TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached
logits	FULLY_CONNECTED	0.999953	40.28	0.142	0.0244	NOT COMPARABLE tensor does not exist in the compiled artifact (internal to a compiled partition); not comparable at this point
probs (output)	SOFTMAX	0.999900	36.97	0.0288	0.00333	NOT EXECUTED TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached

9 comparison points were declared in the points file.

Accuracy

Task: classification. Decision rule: argmax over the last axis of the first model output.
Column	Top-1 [95% CI]	Top-5 [95% CI]
GOLDEN	86.0% [82.8, 89.0]	100.0% [100.0, 100.0]
SCHEME	86.4% [83.4, 89.4]	100.0% [100.0, 100.0]
TARGET	NOT MEASURED TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached

Decision changes: inputs whose decision differs between the two columns.
Pair	Changed	Of	Rate [95% CI]
GOLDEN→SCHEME	2	500	0.4% [0.0, 1.0]

Accuracy deltas in percentage points: earlier column minus later column; positive means the later column is lower.
Pair	Top-1 delta [95% CI]	Top-5 delta [95% CI]
GOLDEN→SCHEME	−0.4% [−1.0, +0.0]	+0.0% [+0.0, +0.0]

Changed decisions GOLDEN→SCHEME, largest GOLDEN top-1 minus top-2 margin first (2 of 2).
Input	Label	GOLDEN class	GOLDEN score	SCHEME class	SCHEME score	GOLDEN margin
397	2	3	0.341	2	0.355	0.004
104	2	3	0.479	2	0.484	0.003

Saturation

Range-limit hits in SCHEME: 5 of 8,746,000 observed output elements, in 2 of 9 quantized operators (500 inputs).

Elements of a quantized operator output equal to the type's representable limit, counting only limits whose real value is non-zero and which are not the fused activation's clamp. Accumulator clamps inside kernels are not observable from tensor values and are not counted.
#	Operator	Output tensor	Low-limit hits	High-limit hits	Elements
5	MEAN	`functional_1_1/gap_1/Mean`	0	3	16,000
7	FULLY_CONNECTED	`functional_1_1/fc_1/MatMul`	2	0	5,000

Latency

NOT MEASURED TARGET not executed: no Ethos-U execution environment: FVP_Corstone_SSE-300_Ethos-U55 is not on PATH and no board is attached

Memory and bandwidth

Measurements. MiB = 2^20 bytes.
Quantity	Value	Unit	Source
Compiled artifact size	0.01459	MiB (15,296 bytes)	host filesystem: size of artifact_int8_vela.tflite
Peak RAM	—	MiB	NOT MEASURED target ethos-u55-128 does not declare peak_ram telemetry
On-chip SRAM working set	—	MiB	NOT MEASURED target ethos-u55-128 does not declare sram telemetry. A compile-time estimate exists (SRAM used 21.48 KiB, source: estimate) and is not admissible as a measurement

Estimates from compiler: ethos-u-vela 5.2.0. Source: estimate. Not measurements, and not admissible against declared limits.
Compile-time figure	Estimate	Unit
SRAM used	21.48EST	KiB
DRAM used	0EST	KiB
On-chip flash used	0EST	KiB
Off-chip flash used	9.328EST	KiB
Inference time	0.05368EST	ms
Inferences per second	18630EST	1/s
Neural-network MACs per inference	825684EST	MACs
NPU cycles per inference	25177EST	cycles
Total cycles per inference	26839EST	cycles
SRAM traffic per inference	87668EST	bytes
DRAM traffic per inference	0EST	bytes
Off-chip flash traffic per inference	8688EST	bytes
NPU passes after fusing	37.00EST	passes

NOT MEASURED Roofline position: requires measured external-memory traffic and bandwidth on the device. Compile-time traffic estimates do not establish whether this workload is bandwidth-bound or compute-bound.

Power and thermal

Quantity	Value	Unit	Status
Energy per inference	—	mJ	NOT MEASURED target ethos-u55-128 does not declare energy telemetry
Peak power	—	mW	NOT MEASURED target ethos-u55-128 does not declare power telemetry

Thermal at start: not recorded: TARGET did not execute
DVFS state: not recorded: TARGET did not execute
Power mode: not recorded: TARGET did not execute
Throttle onset: NOT MEASURED sustained-load runs are not implemented in this build

Regression diff

no earlier run in runs with the same budgets and target ethos-u55-128.

Limitations

GOLDEN is a reference convention (ONNX Runtime FP32, graph optimizations disabled), not ground truth; another FP32 implementation would differ slightly.
Per-layer comparison is valid only at fusion boundaries. A tensor that does not exist in a column is marked NOT_COMPARABLE; no alignment is inferred.
Saturation counts are range-limit hits on operator outputs; clamps on accumulators inside kernels are not observable and are not counted.
The ethos-u55-128 adapter has never run against physical hardware.
ethos-u55-128: compiler cycle, SRAM, flash, and traffic figures are compile-time estimates, not measurements, and are not admissible against declared limits.