AI/ML7 min read

Computer Vision

Process images and video with classical OpenCV techniques and modern deep learning - covering classification, object detection, segmentation, and OCR.

Computer VisionOpenCVPyTorchYOLOImage Classification

skill: Computer Vision category: AI/ML tags: [Computer Vision, OpenCV, PyTorch, YOLO, Image Classification] readingTime: 7

Start with a pre-trained model (YOLO, CLIP, EfficientNet) before training from scratch - ImageNet pre-training provides 80%+ of domain-specific accuracy for most tasks. Only train from scratch when data is too domain-specific (medical imaging, satellite) or you need architecture control.

Workflow

Preprocess images with OpenCVConsistent preprocessing prevents the most common CV bugs - wrong channel order (RGB vs BGR), un-normalized pixel values, or mismatched input sizes.
Classify images with a pre-trained modelTransfer learning from ImageNet gives state-of-the-art accuracy on most visual tasks with minimal training data (50-500 examples per class).
Detect objects with YOLOYOLOv8/v11 runs in real-time at 30+ FPS on a GPU - the go-to for bounding box detection without building a custom detection head.
Extract text with OCRTesseract handles printed text; EasyOCR handles printed and handwritten text in 80+ languages; PaddleOCR is fastest for production.
Segment objects with SAMSegment Anything Model (SAM) segments any object in any image from a point or bounding box prompt - no training required.

Output Format

CV pipeline: preprocess → infer with pre-trained model → postprocess results.

cv_pipeline/
├── preprocess.py    # Resize, normalize, augment
├── model.py         # Load pre-trained, replace head
├── detect.py        # YOLO / detection logic
├── ocr.py           # Text extraction
└── evaluate.py      # mAP, accuracy, confusion matrix

Examples

Input

Output

Edge Cases & Gotchas

Class imbalance in detection - If background vastly outnumbers objects, the detector predicts 'background' for everything. Use focal loss (built into YOLO) or oversample minority classes.
Augmentation consistency - When augmenting (flipping, rotating) images with labels, augment the labels too. cv2.flip on image must be paired with flipping bounding box coordinates.
Inference vs. training normalization - Normalize with the same mean/std at inference as during training. Using ImageNet stats for a custom-trained model with different normalization causes poor accuracy.
Video processing frame rate - Processing every frame at 30 FPS is expensive. Sample every 5th frame for monitoring tasks; use optical flow for motion detection to skip static frames.

When to Break These Rules

Use a vision API (Google Vision, AWS Rekognition) instead of building a custom CV pipeline when the task (object detection, OCR, face detection) is generic and latency is not critical.
Use CLIP for zero-shot classification - it classifies images by text description without any labeled training data.

Reference Files

https://docs.ultralytics.com/
https://github.com/facebookresearch/segment-anything