Face Recognition at Scale with OpenCV
IDGuard started as a straightforward problem: proxy attendance is common in college, and every existing solution for it is either easy to game or annoying to use. QR codes get shared. RFID cards get lent out. Manual roll call takes ten minutes of class time. I wanted to try biometric verification and see how far I could push accuracy on consumer hardware.
The short version: 99.2% verification accuracy, a FastAPI backend, Next.js dashboard, Firebase auth, and Docker tying it together. The longer version involves learning a lot about why computer vision is harder than it looks when you first open an OpenCV tutorial.
Why Face Recognition Is Harder Than It Looks
The naive approach is to use OpenCV's built-in face detection, grab the face region, and compare it to a stored image. That gets you maybe 80-85% accuracy in controlled conditions and falls apart fast in the real world. Different cameras, different lighting, students who wore glasses one day and not the next, tired morning lighting versus afternoon sun through a window. The model has no idea these are the same person.
The shift that actually moved the needle was moving from direct pixel comparison to face embeddings. Instead of comparing images, you generate a vector representation of a face using a pre-trained deep learning model, and compare those vectors. The embedding captures identity rather than appearance, which means lighting changes and small variations in angle stop mattering as much.
I used a model pretrained on a large face dataset rather than training anything from scratch. Training from scratch for a project like this would have been both unnecessary and probably worse — there is no good reason to reinvent a face recognition model when strong pretrained ones exist. The skill is in how you apply them, not in rebuilding them.
The Preprocessing Pipeline
Getting to 99.2% required more careful preprocessing than I initially expected. The pipeline runs each frame through these steps before generating an embedding:
- Face detection and bounding box extraction
- Alignment based on facial landmarks so both eyes land at consistent coordinates
- Lighting normalization to reduce the effect of variable ambient light
- Confidence threshold filtering to reject low-quality frames before they reach the verification step
The alignment step made the biggest single difference. A face tilted 15 degrees can produce an embedding that is meaningfully different from the same face at 0 degrees, even with a robust model. Normalizing orientation before embedding removed a category of false negatives that had been dragging down accuracy.
Edge Cases
The hardest cases were partial occlusions. Glasses, masks, hair across the face. Masks in particular are essentially a different face. The approach I settled on was registering multiple reference embeddings per student under different conditions rather than trying to be robust to every variation from a single image. More enrollment effort up front, more reliable verification afterward. It is a tradeoff but the right one for a real deployment context.
I also added a liveness detection step to prevent someone holding up a photo. It is not sophisticated — just checking for natural micro-movements across frames — but it blocks the obvious attack and that was enough for this use case.
The Stack
FastAPI handles the backend because it is fast to write and fast to run, and async support made managing concurrent verification requests easier. The face processing pipeline runs in its own process pool so heavy OpenCV work does not block the API. Next.js for the admin dashboard where faculty can see attendance records and manage the student database. Firebase for auth. Docker so the whole thing deploys consistently regardless of the host machine.
The most interesting architectural decision was where to do the face processing. Client-side would reduce server load but leak the model and make it easier to attack. Server-side means sending images over the network, which has latency and privacy implications. I went server-side with local processing for the camera feed and sending only the extracted face region rather than the full frame. Reasonable tradeoff for the scale this runs at.
