Streaming, input
Gesture Coding
Hands-free coding sessions using MediaPipe and Monaco for live demonstrations.
Challenge
High latency or noisy triggers can break user trust. The solution focuses on local-first processing and per-desk tuning to ensure reliability.
Solution
Utilizes MediaPipe in the browser and Monaco as the buffer to map hand gestures to editor commands. The system is tuned for low-latency live streams and supports training custom landmark templates for specialized controls beyond standard labels.
Impact
Source code available on Gumroad; provides a foundation for accessibility-focused coding experiments.
Next.jsMediaPipeMonaco EditorWebRTCTypeScript
Start a similar build