↓ Skip to main content
Side Effects
  1. Home projects/

Watch Your Six: people tracking for LED totems

··Index: 7·
Computer Vision Python Yolo Osc

Watch Your Six detects people around an LED totem installation and turns what they’re doing into events for the visuals.

Camera feeds (from USB cameras or video files) go through a YOLO11 pose model, people are tracked from frame to frame, and the program estimates the distance of every person from the camera. Raw detections are noisy, so a per-person state machine with hysteresis has to turn them into stable events:

  • enter and exit, when someone crosses a calibrated boundary line, with a separate dead-zone line on the way out so that people standing on the line don’t flicker in and out;
  • proximity of the closest person;
  • arms raised, read from the pose keypoints, with a cooldown per camera;
  • counts per camera and in total.

Those events go out as OSC messages to Resolume, which drives the LED visuals.

Watch Your Six: camera feeds, YOLO11 pose model, tracking and distance, a per-person state machine, events sent as OSC to Resolume, which drives LED visualscamerasUSB or video filesYOLO11 posekeypoints per persontrackingand distancestate machineper person, hysteresiseventsenter, exit, proximity,arms raised, countsOSC messagesResolumeLED visualsweb interface (FastAPI)live streams, event log, camera calibration with boundary and dead-zone lines
Watch Your Six: from camera feeds to OSC events that drive LED visuals

There’s a FastAPI web interface with the live camera streams and an event log, and that’s also where you add and calibrate cameras, by dragging the boundary and dead-zone lines right on the preview. The interface has a phone layout and finds the server on the network with mDNS. The 93 tests run on synthetic data, without any cameras, and production runs on Windows with CUDA.

Related