Our edge AI demo worked. Then it would not start.

Three days at VISION Stuttgart, and an answer to my own test: what changed while we listened to the people at the booth, and what that means when an edge AI demo goes into operation.

Before the fair I set myself a test. Not “what did you show”, but “what changed in one day while you listened to the people at the booth”. Easy to write. Less easy to pass.

This is the answer, day by day, with numbers. It includes the morning when the booth did not start.

Before VISION I wrote about what happens to an edge AI demo when it leaves the booth: the light, the camera, the latency, the missing cloud. Those four points still stand. This is the other list. It has less theory and more cables.

What was running on the table?

Four cameras. One NXP i.MX 95 board runs the vision models. A second, smaller board (i.MX 8M Plus) with a Kinara Ara-2 accelerator card runs a 7-billion-parameter language model, so that our agent Siegfried can chat with visitors. Everything sits on the table, nothing goes to the cloud. VISION ran from 6 to 8 October at Messe Stuttgart.

Why were visitors walking past on Tuesday?

In the morning, visitors walked past two of the four cameras. Boxes and landmarks on a face make sense to engineers.

By lunch, the same face landmarks drove sunglasses and a moustache. Raise both hands for half a second and the screen shouts “HANDS UP!”, confetti falls and the lamps on the booth blink. The serious topic did not become less serious. It became something you could touch.

The numbers behind the fun, from the stats overlay: person detection about 20 frames per second on the NPU, faces 21, body pose 11. The gesture model runs on the CPU at 5 frames per second, because it was never moved to the NPU. One screenshot of the first build shows a CPU load of 88 percent. That is a snapshot, not the load across all three days.

Then came an idea we threw out within the hour: a heatmap of “who stands here longest”, green to red. It turned completely red. People at a booth stand. We replaced it with a rainbow trail: the colour runs through the rainbow every 12 seconds, the dots fade after 25.

Then a visitor said it would be cool if the booth read his name from the badge and said hello. By the end of the day it did. Offline OCR, a recogniser of 3.9 MB, about 0.6 seconds per text line on the board, a greeting three to four seconds after you hold the badge up. Visitors held their own badges up to the camera. Only the name appeared on the screen. On twelve real badge photos it read 86 to 100 percent of single frames exactly, with no wrong name. The tests grew from 50 to 73 that evening.

Why did we move the badge reader on Wednesday?

The badge reader sat on camera 3, the one mounted highest. Visitors had to hold their badge up to the lens like a ticket at a gate. On Wednesday morning it moved to camera 4, the lowest one, which delivers more pixels.

Result: it read badges that fill 11 to 22 percent of the frame width, about 1.8 seconds per attempt. At roughly a tenth of the frame width, 89 to 100 percent of test frames were read correctly.

Then we got greedy. Why not read badges on all four cameras? We tried. In 40 attempts, cameras 1 to 3 never found a line of text. They are wide-angle lenses focused on the hall, and software cannot fix focus. Worse, one search round over four full frames took 1.2 to 1.5 seconds on the board. On my laptop the same step takes 3 to 5 milliseconds. Load went up to 10.5 on six cores. Decision at 12:37: camera 4 only.

I did not measure memory bandwidth. I measured seconds. But the pixels explain it. The cameras deliver 1920 by 1280. A tile on screen is 960 by 640, about 0.6 million pixels. Our full-resolution tap was 1.4 million, and the largest one 2.5 million. Four times more pixels to read the same small letters.

Late afternoon, a new complaint: the gestures rarely fire. The hand model ran at 5 frames per second, and only every second result was used. That is 2.5 per second, and the trigger needed two fresh results plus a half-second hold. One setting changed, and it worked. We also changed the greeting to “Servus, <name>!” with three seconds of confetti, and added a small “already greeted” hint, because visitors standing in front of the camera had no idea whether it had recognised them.

One honest caveat. Our “zero wrong names” is consistency on twelve photos, not names checked by a human. A misread can happen.

Why did the chat fail on Thursday morning?

Power was the most likely cause, but we never measured the draw.

Shortly before nine, the big board was fine. The chat on screen said the model was still loading. The small board answered a ping, but the model server was not running at all. A reboot did not help.

We cut power to the small board for 20 seconds. At 09:05 the model reported ready.

Why does a reboot not help? On this board the accelerator-card slot keeps its 3.3 volts during a reset. The card keeps the state of the last session, and the vendor start-up assumes a freshly powered chip. Only a real power cut gives it a clean start. I have to be careful here: the logs of that morning no longer exist, they are wiped at every boot. That explanation comes from the later analysis, not a recording of the failure.

We built a hint for the screen and, at 10:53, I unplugged the small board myself to test it. It came back at 10:55 and hung. The “good” cold start we trusted had failed after a real power cut. We unplugged again for 30 seconds. At 11:00 it came up, and all the fans spun up hard at once. At the failed start they had stayed quiet.

Chip temperature before start was about 24 degrees both times. The small board was fed from a USB-C socket built into a power strip, 5 volts. Around 12:15 the power failed at the booth, and afterwards the board reset itself twice, each time about a minute after start-up, exactly when the model begins to load.

At 12:42 we moved it to a proper USB-C PD charger. The model was ready at 12:46 and the board did not reset again.

So the cause was most likely power. Most likely, because we never measured the draw. A damaged module remains the second, less likely explanation.

Why did the fix need a fix?

The red hint we had built said, in effect: “Model server is not responding. Unplug the i.MX 8M Plus.” I read it as a staff member would and said that someone who had not built the demo would not understand it. “Model server” means nothing to them. The text was the same whether you had already unplugged or not. There was no feedback while the board restarted, and no clear next step after repeated failures.

The replacement is a card on the table, written in plain words: all good, wait, unplug the small board for 20 seconds, and a line with a phone number to call.

What needs testing before a demo goes into operation?

Power. Treat the supply as a component, not as a socket. A reset is not a power cycle. Write down which charger feeds which board, and measure the draw under load. We did not.

Heat. A cool start is not a thermal test. Twenty-four degrees at start says nothing about two in the afternoon under load.

OCR. Accuracy in the lab and accuracy at the booth are different numbers. For us, the deciding parameter was how much of the frame the badge fills, and the lens focus, which is hardware.

Bandwidth. Count pixels per second like a budget. Four cameras at full resolution cost seconds, not milliseconds. One shared worker beat four eager ones.

People. The heatmap turned red. The hands became confetti. The badge idea came from a visitor, not from us. And the person who reboots the booth at 8:50 is not the person who built it.

One more thing. The board has no clock. On day 3 it was a day behind, so our files carried yesterday’s date.

If your demo works on the table, which cable do you plug in last, and who besides you knows which one it is?