CNN Basics for Vision — Interview Q&A

Answer: Images have spatial structure; conv layers exploit local correlations and share weights—far fewer parameters and better generalization than huge FC layers on pixels.

# PyTorch: nn.Conv2d(in_c, out_c, k, padding=1)  # standard conv layer

Answer: Slides learnable filters over the input; each output location is dot product of filter with local patch—detects patterns like edges/textures at many positions.

Answer: Same filter weights used at every spatial location—if a feature is useful in one place, it can appear anywhere; drastically cuts parameters vs FC.

Answer: Each neuron sees only a small neighborhood—deeper layers indirectly see larger context via stacked convs.

Answer: Stride subsamples spatial size; same padding keeps H×W with zero border; valid shrinks without padding.

Answer: Reduces spatial resolution, adds slight translation tolerance, and lowers compute—max pool keeps strongest activations in each window.

Answer: Number of filters = number of output channels—each filter produces one feature map.

Answer: Grows with kernel sizes, strides, and stacking—after L layers network “sees” a region of that size in the input image.

Answer: Mixes channels at each spatial location—cheap way to change depth (bottleneck), add nonlinearity, or implement MLP per pixel.

Answer: FC connects all inputs to each output—no locality; used at end (or as 1×1 conv) after spatial reduction for classification.

Answer: Shift input → shifted feature maps (before pooling)—CNN respects spatial structure; pooling adds limited invariance.

Answer: First conv has 3 input channels per filter—depth matches image channels (or more for hyperspectral).

Answer: Normalize activations per channel for stable training and higher learning rates—slight regularization effect.

Answer: More common in FC heads; sometimes spatial dropout drops whole feature maps—less standard than in MLPs.

Answer: Average each channel to one value—reduces params vs large FC layers before softmax (Network in Network / ResNet style).

Answer: Cross-entropy with softmax over classes—multi-label uses sigmoid + BCE per class.

Answer: Random crop/flip, color jitter, mixup/cutmix—improves generalization and simulates viewpoint/light changes.

Answer: Initialize backbone from ImageNet pretrain, replace head, fine-tune—standard when labeled data is limited.

Answer: Conv: roughly O(H_out×W_out×C_in×C_out×k²)—depthwise separable reduces this (MobileNet).

Answer: CNN: local inductive bias and efficiency. ViT: global attention, needs more data—hybrids (ConvNeXt, Swin) blend ideas.

Answer: Won ImageNet 2012 by a large margin—showed deep CNNs + GPU + data could beat hand-crafted features, sparking the deep learning boom in vision.

Answer: 1.2M images, 1000 classes—AlexNet ~16% top-5 error vs previous ~26% with shallow methods—breakthrough result.

Answer: Five conv layers (some grouped across 2 GPUs) + max pooling + three large FC layers + softmax—deeper than prior CNNs for this task.

Answer: Faster training than saturating tanh/sigmoid; mitigates vanishing gradient in deep stacks; sparse activations.

Answer: Regularize huge FC layers by randomly zeroing neurons—reduces co-adaptation on training set.

Answer: Local response normalization—side inhibition across channels; later often replaced by batch norm; minor effect in hindsight.

Answer: Stride smaller than pool window—slightly richer downsampling vs non-overlapping; less common in newer nets.

Answer: Model split across GPUs due to memory limits—cross-GPU connections only on certain layers (engineering constraint of the time).

Answer: Random crops/flips from 256×256, PCA color jitter—reduces overfitting and increases effective data.

Answer: On order of 60M—mostly FC layers; later architectures reduce FC params with GAP.

Answer: SGD + momentum, weight decay, learning rate schedule dropping on plateaus—long schedule on two GPUs.

Answer: Large capacity vs data—addressed by dropout, aug, and weight decay; still a concern for smaller datasets when fine-tuning.

Answer: VGG uses uniform 3×3 stacks, deeper, more systematic—higher accuracy, more compute; AlexNet shallower irregular design.

Answer: ResNet adds residuals enabling much deeper nets—AlexNet depth modest by today’s standards.

Answer: Mostly for teaching/history; ResNet/EfficientNet backbones dominate transfer learning—AlexNet too weak/slow vs modern alternatives.

Answer: 224×224 crops from 256×256 resized image—standard pipeline referenced in many papers.

Answer: 1000-way softmax for ImageNet classes—cross-entropy loss during training.

Answer: For production accuracy, yes; for pedagogy and history, still the canonical “first big win” story.

Answer: Validated deep learning at scale—influenced speech, NLP later wave; proved GPUs + data + depth recipe.

Answer: MobileNet, EfficientNet achieve better accuracy/FLOPs—mobile edge rarely uses AlexNet-sized FC heads.