Pillar 1 · Model Selection — family deep dive

Types of CNNs & Vision Models

Papers say "feed-forward convolutional network"; vendors say "AI vision". Both hide the decision that actually matters to you: which variant — classification, detection, segmentation, pose, anomaly — because the variant is set by what output the business needs.

Decoding the jargon: "feed-forward", "convolutional"

Two independent adjectives. Feed-forward describes the flow: data moves one way, input → layers → output, no loops, no memory between inputs. Convolutional describes the layer type: learned image filters. A CNN is simply a feed-forward network built from convolution layers — not a separate exotic family.

Feed-forward vs recurrent — the one architectural distinction worth knowing

Architect's takeaway: "feed-forward" is a property, not a product. If a vendor's "sequence model" is an RNN/LSTM, ask why it isn't a Transformer.

One backbone, different heads

The CNN variants you choose between differ by what the output head produces, not by exotic internals. A backbone (ResNet, EfficientNet, MobileNet, ConvNeXt) extracts visual features; a task-specific head turns them into a label, boxes, or masks. That's why requirements map so cleanly onto variants.

The CNN variants by output

The requirement tells you the head: "is it defective?" → classification · "where is it?" → detection · "how big exactly?" → segmentation.

The vision variants, with named models

Variant	Output	Named models	Choose it when
Image classification	One label per image	ResNet-50, EfficientNet, MobileNetV3, ConvNeXt	"What is this?" — pass/fail inspection, sorting; MobileNet-class for edge devices
Object detection — one-stage	Boxes + labels, real-time	YOLOv8 / YOLO11, SSD, RetinaNet	Live video at line speed — counting, locating defects; the factory-pipe example
Object detection — two-stage	Boxes + labels, higher accuracy	Faster R-CNN, Cascade R-CNN	Accuracy beats speed — offline analysis, small or crowded objects
Segmentation	Per-pixel masks	U-Net, Mask R-CNN, DeepLabv3+, SAM (promptable)	Exact shape or area matters — medical imaging, measuring defect size, background removal
Pose / keypoints	Skeleton joint coordinates	OpenPose, MediaPipe Pose, YOLO-pose	Ergonomics, sports analysis, gesture control, fall detection
Object tracking	Identities across frames	ByteTrack, DeepSORT (on top of a detector)	Counting unique objects/people over time, trajectories, dwell time
Face recognition	Identity embedding + match	ArcFace, FaceNet	Access control, deduplication — mind biometric-data regulation (GDPR/AI Act)
Visual anomaly detection	Anomaly score / heatmap, no defect labels needed	PatchCore, PaDiM, autoencoder approaches	Defects are rare or unknown upfront — train on "good" samples only
Vision Transformer (ViT)	Same tasks, attention-based backbone	ViT, DINOv2, Swin Transformer	Large datasets, global context; powers most hosted vision APIs — you rarely pick it explicitly
OCR pipeline	Text read from images	PaddleOCR, Tesseract, TrOCR; cloud: Azure AI Vision, Google Document AI	Documents, labels, license plates — usually a pre-built pipeline, not a model you train

Buying, not building: in practice you choose a variant and a pre-trained model, then the ML team fine-tunes it on your labeled images. The variant decision (this table) is yours; squeezing accuracy out of it is theirs.

Sizing note (ties to pillar 2): vision models are tiny next to LLMs — YOLO variants run 2–70M parameters and hit real-time frame rates on an edge GPU like a Jetson, which is why "camera + small box on the factory wall" is a realistic architecture, no data center required.