Computer vision / 2026 / Live in the browser
SignSpeed
ASL fingerspelling to text from a webcam. It reads hand geometry instead of pixels, so all 26 letters work with no training, and two seconds of calibration per letter fits it to your own hand.
01
The problem
Sign language demos usually train an image classifier on other people's hands, then break on a new signer, new lighting or a new camera. They also flicker, typing a letter every time the per-frame guess changes.
02
How I approached it
MediaPipe returns 21 landmarks per frame. Each frame is rotated into a basis built from the palm, scaled by palm width and mirrored for left hands, so position, distance, tilt and handedness drop out. Every letter is a soft score over finger curl, the in-plane angle between fingers, and where the thumb sits across and against the palm, which is what separates the closed-fist letters S, T, M, N and E. The palm side is read from the curled fingers themselves, so a wrong left or right label from the tracker cannot swap them. J and Z come from the path the fingertip draws. Calibration records the signer's own poses into a local-mean nearest-neighbour bank whose weight scales with how close the live pose is to anything recorded, and a decoder smooths in milliseconds rather than frames and types a letter only after a 320 ms steady hold.
03
The outcome
The engine was built and tuned in Python against synthetic hands with tracking noise, 15 degrees of tilt and both hands, reaching 97.8% per frame from the rules alone. It was then ported line for line to JavaScript, and a parity test holds the two within 1e-6 on 138 poses. In Chrome on Intel UHD graphics, hand tracking costs 20.9 ms a frame and recognition 0.38 ms with a full calibration bank.
Camera: this runs entirely in your browser. No video is uploaded or stored.
Related work
Hand-tracked cursed-technique visualiser: MediaPipe landmarks drive a Three.js particle engine, with an ML classifier picking the technique in real time.
Webcam AR voxel builder with a second synced screen, build in the air on one page, drive presets and watch live status from another.
A webcam focus monitor that watches your head and posture while you study, and when you keep drifting off it opens the McDonald's careers page for you.