Cámara is a robotic creature built on a commercial PTZ camera (Dahua, controlled over ONVIF) and a Raspberry Pi 5. Its most basic job is to keep a person's face centered in the image while they walk around the room and, if their back is turned, to look where their face would be.
It sounds simple, but its body was not built for this: it obeys about 7 commands per second and takes time to start and stop. Every turn of the tracking loop goes through three parts.
Capture
What Cámara sees and where it is looking right now.
Processing
Where the person is in the frame, where they are in the world and how fast they are moving.
Tracking
What speed to ask of its motors to keep up with the person without overshooting.
Capture: see what is happening now, not a moment ago
Everything Cámara sees reaches it with a delay, and every order it sends to its motors takes time too. We call the sum dead time, and it is what limits most how well it can follow someone.
Video arrives over RTSP from the PTZ substream (704×480, MJPEG). A separate thread reads it nonstop and the loop always takes the latest frame, so it never decides by looking at queued images. Motor position is read on a different channel from the one used for commands.
| format | video cost per call | video latency |
|---|---|---|
| MJPEG | +36 ms | 0.04 s |
| H.264 2 Mbps | +9 ms | 0.30–0.34 s |
| H.264 Baseline, GOP 15 | — | 0.34 s |
H.264 makes each call cheaper, but video arrives 7 times later: the loop's dead time goes from 0.25 to 0.52 s.
| kbps | command p50 | position read p50 |
|---|---|---|
| 8192 | 80 ms | 140 ms |
| 4096 · chosen | 57 ms | 95 ms |
| 2048 | 56 ms | 91 ms |
At 4096 kbps detector noise rises from σ 0.0013 to 0.0017, with 100% detection. Going lower buys nothing.
OnvifLoginCheck=false): −15% per call. In exchange, anyone on the local network can move the PTZ over ONVIF.Dead time is the number that rules
It limits how much correction the controller can take. A codec or transport is chosen by total loop latency, not by what each call costs.
Every actuator has a rate ceiling
You have to measure how many commands and reads it really absorbs. Asking for more does not speed it up: it saturates it.
Processing: where the person is and where they are going
Cámara uses two detectors. The face gives it fine framing. The body saves it when the person has their back turned or is in profile and there is no face to see. The two take turns:
Face visible: YuNet frames. No face: YOLO11n looks for the body on every frame and aims where the face was inside the box, so Cámara does not jump when switching. Following a body: YuNet runs every other turn, only to recover the face; that keeps the loop at ~22 Hz instead of ~15.
Then the person's position is converted to PTZ pan/tilt coordinates and their velocity is estimated there. Pixels do not work: the image error also changes when Cámara turns.
| model | p50 | frames | from behind |
|---|---|---|---|
| YOLO11n-256 | 33 ms | 100% | 27/27 |
| YOLO11n-320 | 48 ms | 99% | 27/27 |
| NanoDet-Plus (onnxruntime) | 45 ms | 92% | 27/27 |
| MobileNet-SSD | 54 ms | 75% | 27/27 |
| model | result |
|---|---|
| YuNet | 21–73 ms depending on processing width (384–640 px). 5 landmarks. |
| SCRFD-500M | Same coverage (80%), 12 ms more expensive. |
| Haar cascade | Loses the face every few frames. |
Detection rate matters more than precision
A light detector on every frame tracks better than a good one that looks in bursts, and combining two detectors covers the cases one alone misses.
Estimate the target in the world, not in the image
Pixel error mixes the person's motion with Cámara's own. Filtered there, the estimated velocity comes out contaminated.
Tracking: keep pace instead of chasing
Cámara does not ask the PTZ to go to a point: it asks for a speed and corrects it on the fly. That speed has two parts. The first is the person's own, so it moves along with them. The second is a correction that grows with how far they drifted from the center of the frame.
How does it predict? With a simple assumption: that in the next instant the person keeps the same speed. There is no model of how people walk and no Kalman filter. Cámara estimates the current speed and extrapolates it in a straight line 0.25 s ahead, which is how long it takes to see (0.05 s) plus how long it takes to start moving (0.20 s). That anticipation can never be larger than the error it is correcting: that way, with the person centered, noise does not move the camera on its own.
The person's speed is built from two measurements: the camera's own speed (the slope of its last positions, ~0.5 s) plus how fast the person moves within the frame. The correction is not fixed either: the gain grows with the error, so near the center it corrects gently and far from it, hard.
Command speed: ContinuousMove changes speed on the fly without braking. Its useful range is 3 to 35 °/s, and a person walking at 3 m moves at ~19 °/s. Brake early: stopping is decided by the error that will remain after 0.20 s of dead time and 0.10 s of coasting, not by the current one. Jump when the error is large: an AbsoluteMove recovers someone who ended up far away in one go.
| mode | commands | centered in | final |dx| |
|---|---|---|---|
| position | 1 | 1.3 s | 0.067 |
| pure speed | 9 | 2.7 s | 0.011 |
| chase | 7 | 1.2 s | 0.019 |
| setup | |dx| p50 | travel | p90 pause |
|---|---|---|---|
| no anticipation | 0.195 | 470° | 1.25 s |
| estimator only | 0.227 | 409° | 0.96 s |
| anticipation | 0.167 | 622° | 0.48 s |
With a still target, chase sends no commands in 30 s. With a fixed face, anticipation does not oscillate.
Keep pace instead of chasing
Commanding speed and adding the person's own gives continuity. It is paid for in travel (+32%), and that cost gets measured too.
Brake on the error that will remain
An actuator with delay and coasting overshoots if you brake on the current error. You have to discount what it will keep moving.
How we measured it
Every number in this post came from a dedicated test, run on the Pi and not on a desktop computer. Configurations are interleaved (A, B, A, B…) so light or network conditions do not favor one of them, and every comparison is repeated before we trust it.
| test | tool | stimulus | what it measures |
|---|---|---|---|
| Cost per call and codec | luf-002 plant | Still PTZ, with and without video | ms per command and read; video latency by cross-correlation |
| Substream bitrate | plant substream | fixed face | command p50 and detector noise |
| Detectors | luf-002 bench-person | 240 frames front, profile and back | p50 ms and % of frames detected |
| Full loop, A/B | luf-002 ab | printed face at 1 m (n=4); person around the room at 1–3 m (n=3, 30 s) | |dx|, commands, travel and p90 pause |
| Real use | --trace + luf-002 trace | 3 minutes in the room | Hz per detector and % of frames with a person |
Comparisons use stimuli that do not change between runs: a still printed face or a looping video of a face on an iPad. The real person comes in last, to confirm that what won with the fixed stimulus also wins in the room.
Tests that change the PTZ configuration save a backup, restore it at the end and read back every value written: the camera answers OK even to parameters that do not exist.
What carries over to other robotic creatures
Beyond this camera, these are the method lessons that came out of measuring each part.
Measure both sides of every trade-off
Almost no improvement is free. If you only measure what you gain, the decision can come out backwards.
Compare with repeatable stimuli
A fixed target or a looping video gives the same test on every run and lets you compare medians. The real person comes last, to confirm.
Verify the instrument before trusting it
A measurement harness has bugs too: an image that was never flipped upright, a setting that answers OK without being applied, a feature that shows as enabled and does nothing.