Deep Segmentation — Interview Q&A

Answer: Partitioning an image into regions or labeling each pixel with a category—bridges low-level pixels and high-level objects.

Answer: Each pixel gets a class label (road, sky, person) without distinguishing different instances of the same class.

Answer: Separate individual objects even within same class—each instance has its own mask and ID.

Answer: Unifies stuff (amorphous regions like sky) and things (countable objects) with non-overlapping masks covering the whole image.

Answer: Yes for binary foreground/background—simplest form; limited for complex scenes without additional cues.

Answer: Start from seeds, merge neighboring pixels similar under a criterion (intensity, texture)—sensitive to seed placement and noise.

Answer: Recursively split non-uniform regions, then merge adjacent similar regions—quadtree-style classical approach.

Answer: Cluster pixel colors (RGB or LAB); each cluster is a segment—produces patchy results without spatial smoothness unless augmented.

Answer: Mode-seeking in joint color-spatial feature space—clusters pixels to local density peaks; smooths labels but can be slow.

Answer: Treat gradient magnitude as height map; flood from markers—without markers causes oversegmentation; marker-controlled watershed is common.

Answer: Pixels as nodes, pairwise smoothness + unary data costs; find min-cut for globally good binary partition—used in GrabCut-style energy minimization.

Answer: Iterative graph-cut segmentation with Gaussian mixture models on RGB—user provides box or strokes; refines foreground/background.

Answer: Deform curve minimizing internal smoothness + external edge attraction—classic for medical boundaries; level-set extensions handle topology changes.

Answer: Too many small regions—watershed without markers; fix with merging, markers, or learned superpixels (SLIC).

Answer: Intersection over union per class or instance; mean IoU (mIoU) standard for semantic segmentation benchmarks.

Answer: Measures alignment of predicted vs GT contours—complements IoU for thin structures.

Answer: FCN replaces FC layers with convs; U-Net encoder-decoder with skip connections—dominant paradigm now with transformers emerging.

Answer: Temporal consistency, optical flow warping, or memory networks—object masks tracked across frames (VOS).

Answer: User clicks/scribbles guide model (GrabCut, deep interactive)—few-shot refinement for editing.

Answer: Classical: fast, little data, controlled scenes. Deep: cluttered natural images, need labels and compute—often hybrid for industrial + DL refinement.

Answer: Assigning a class label to every pixel (road, sky, person)—no distinction between different instances of the same class.

Answer: Classification: one label per image. Semantic segmentation: dense spatial map of labels—requires localization and context.

Answer: Replaced fully connected layers with 1×1 convolutions so arbitrary input sizes work; learnable upsampling (deconv/transposed conv) to recover resolution.

Answer: Encoder downsamples for context; decoder upsamples; skip connections fuse fine detail from shallow layers with semantic deep features—sharp boundaries.

Answer: Transposed convolution, bilinear upsample + conv, sub-pixel shuffle—each trades artifacts, parameters, and speed differently.

Answer: Mean Intersection over Union per class (then averaged): measures overlap of predicted vs ground-truth masks—standard benchmark metric.

Answer: 2|A∩B|/(|A|+|B|)—closely related to F1 for binary masks; common loss for medical segmentation when foreground is tiny.

Answer: Per-pixel cross-entropy (softmax over classes); can weight rare classes or use focal variants for hard pixels.

Answer: Ambiguous edges, thin structures disappear at low res—fixes: deep supervision, boundary-aware loss, high-res branches, or larger input crops.

Answer: Weighted CE, oversampling rare classes, focal loss, dice loss, or balanced sampling in batches.

Answer: Atrous spatial pyramid pooling—parallel dilated convs at multiple rates capture multi-scale context without losing resolution (DeepLab family).

Answer: Pyramid pooling at several scales then upsample and concatenate—rich global scene context for each pixel.

Answer: Run network on several scales / flipped inputs and average logits—boosts mIoU at inference cost.

Answer: Train from image tags, scribbles, or bounding boxes using constraints (e.g. MIL, GrabCut-style seeds)—less pixel labels needed.

Answer: Panoptic adds instance IDs for “things” while semantic handles “stuff”—semantic is a component of full scene parsing.

Answer: Historically refined CNN outputs with pairwise smoothness; less dominant now with stronger architectures but still taught in interviews.

Answer: No—both get label “person”; need instance segmentation for separate masks.

Answer: Pixel-accurate masks per image vs bounding boxes—tools like semi-auto labeling and synthetic data help.

Answer: SegFormer, Mask2Former, Segmenter—global attention and mask queries compete with CNN encoders on benchmarks.

Answer: Lightweight backbones (MobileNet), BiSeNet, Fast-SCNN—trade mIoU for FPS on edge devices.

Answer: Each object instance gets its own binary mask and class label—even two “person” pixels belong to different instances if on different people.

Answer: Semantic: one mask per class. Instance: N masks for N objects, possibly same class—handles overlap with distinct IDs.

Answer: Adds parallel mask head: small FCN on each RoI predicts K×K binary mask per class—multi-task with box + class.

Answer: RoIPool quantizes coordinates → misalignment for masks. RoIAlign uses bilinear sampling at exact float locations—critical for pixel-accurate masks.

Answer: Typically 28×28 logits upsampled to RoI size with threshold—lightweight per-region FCN.

Answer: Per-pixel sigmoid + BCE on the target class mask only (not softmax over all classes per pixel in the classic formulation).

Answer: Yes—foreground object in front of another; model must predict ordering or independent masks per instance.

Answer: Unifies semantic “stuff” and instance “things” with non-overlapping full-scene labeling—each pixel has one label + optional instance id.

Answer: One-stage: combines prototype masks with per-instance coefficients for fast instance segmentation—speed-quality tradeoff.

Answer: Define instance by grid location and scale—predict category and mask for each grid cell without anchors in the traditional sense.

Answer: Set prediction with mask head or panoptic head—queries attend to image features to produce instance masks end-to-end.

Answer: AP computed on mask IoU instead of box IoU—COCO primary metric for instance segmentation quality.

Answer: Datasets may store COCO RLE or polygons; training often rasterizes to fixed resolution masks for loss.

Answer: Things are countable instances; stuff is amorphous (grass, sky)—panoptic benchmark merges both.

Answer: High-res FPN levels, copy-paste augmentation, and specialized heads help—same challenges as object detection.

Answer: Extra per-RoI mask computation and higher memory—one-stage mask methods aim to close the gap.

Answer: Multi-scale object proposals and features so small and large instances both get good mask features.

Answer: Iteratively refine boxes and masks with cascaded stages and inter-task fusion—state-of-art on COCO era leaderboards.

Answer: Methods like PointRend adaptively sample points on uncertain boundaries for fine mask prediction—better edges.

Answer: Instance masks are most expensive—interactive tools, synthetic data, and weak supervision are active research areas.