01Overview
Present AI is a video analysis platform designed to help users improve their public speaking skills. By recording a presentation, the system analyzes multiple aspects of the user's performance and provides a comprehensive score and actionable feedback.
02Architecture & Tech
The frontend is built with React 18 and uses Firebase for authentication and media storage. The core processing pipeline utilizes FFmpeg running client-side via WebAssembly to extract audio and video frames. Audio is processed by the Whisper API for Speech-to-Text, analyzing filler words and clarity. Video frames are analyzed using MediaPipe for eye-contact and facial expression tracking. The final metrics are sent to the Gemini API, which acts as an AI coach to generate personalized feedback.
03Challenges
One of the biggest challenges was handling large video files in the browser without crashing the tab or requiring massive server-side computing power. Additionally, syncing the audio transcripts from Whisper with the video timestamps from MediaPipe required precise timing logic.
04Solutions
We solved the processing bottleneck by utilizing FFmpeg.wasm, allowing us to downsample and extract only the necessary audio and frames directly in the browser before sending them to the APIs. This drastically reduced bandwidth and server costs. For the timing sync, we built a custom event-loop that correlated Whisper's word-level timestamps with MediaPipe's frame index.
Role
AI Developer & Pipeline Architect
Impact
built real-time video evaluation pipeline scoring speech, confidence, eye contact, and emotion
Stack
React 18FirebaseGemini APIWhisper APIMediaPipeFFmpeg
Key Features
- ◆Filler word and speech clarity detection
- ◆Eye-contact and facial expression analysis
- ◆Gemini-powered personalized presenter feedback
- ◆Client-side media processing with FFmpeg
Next StepsLike what you see? Let's talk.