Generative Vision Models — Interview Q&A

Answer: Neural net trained to copy input to output through a bottleneck: encoder maps x→z, decoder maps z→x̂—forces compact representation.

Answer: Maps high-dimensional input (e.g. image) to a lower-dimensional latent code z—extracts salient factors.

Answer: Maps latent z back to output space—should reconstruct structure lost only if bottleneck truly limits capacity.

Answer: Constrains information flow so the model must learn a compressed code—similar inputs map to nearby latents if the AE is well regularized.

Answer: MSE (L2) per pixel for continuous images; BCE if outputs are probabilities; perceptual losses use a pretrained net’s features.

# loss = F.mse_loss(recon, x)  # vanilla AE

Answer: Under-complete: dim(z) < dim(x)—true compression. Over-complete: dim(z) larger—needs regularization (sparse, denoising, VAE) or trivial identity.

Answer: Train on corrupted inputs (noise, masking) to reconstruct clean x—learns robust features instead of copying noise.

Answer: Penalize activations (e.g. KL on firing rates) so few units active per example—encourages meaningful distributed codes when over-complete.

Answer: VAE encodes a distribution q(z|x); sample z for decoder—adds KL to prior p(z) for a generative model with smooth latent space.

Answer: Pulls approximate posterior toward prior (often N(0,I))—balances reconstruction vs regularization; enables sampling new z ~ p(z).

Answer: Write z = μ(x) + σ(x)⊙ε with ε~N(0,1) so gradients flow through μ,σ—needed to backprop through stochastic sampling.

Answer: Train on normal data; high reconstruction error on test indicates out-of-distribution—used in defect and fraud pipelines.

Answer: Linear AE with MSE and tied weights can recover PCA subspace—deep nonlinear AE generalizes with stronger representational power.

Answer: Ideal latents align with generative factors; plain AE does not guarantee this—β-VAE and supervision help.

Answer: AE/VAE optimize likelihood-like objectives; GAN uses adversarial realism—GANs often sharper; VAEs more stable latent geometry.

Answer: Encoder stacks conv+pool/downsample; decoder uses upsample/transpose conv—standard for images.

Answer: Condition decoder on low-res input or use skip connections (U-Net style)—AE ideas plus perceptual loss improve texture.

Answer: Use encoder output as vector; nearest neighbors in latent space for similar images—may need contrastive training for metric quality.

Answer: Normalize inputs; watch for posterior collapse in VAE; use skip connections if reconstruction is blurry from pure bottleneck.

Answer: Reconstructions can be blurry (MSE averages); latent may be entangled; vanilla AE is not a sharp generative model without VAE/GAN hybrids.

Answer: Generative model with generator G(z) making samples and discriminator D(x) judging real vs fake—trained adversarially.

Answer: G minimizes log(1−D(G(z))) while D maximizes log D(x)+log(1−D(G(z)))—equivalent to Jensen–Shannon related objectives in classic formulation.

Answer: Maps noise z (latent) to data space—should match real data distribution at optimum.

# min_G max_D V(D,G) — alternate k steps on D, 1 on G

Answer: Binary classifier estimating probability “real”—provides training signal to G via gradient through D.

Answer: At optimum (ideal), p_G = p_data and D = ½ on generated samples—hard to reach in practice with finite capacity and SGD.

Answer: Oscillating dynamics, vanishing gradients when D too good, or D too weak—need balanced updates and architecture tricks.

Answer: G outputs few varieties ignoring diversity—D cannot push G to cover all modes; minibatch discrimination and unrolled GANs mitigate.

Answer: Strided convolutions, BatchNorm, no FC except input/output, ReLU in G, LeakyReLU in D—empirical recipe for stable conv GANs.

Answer: Use Wasserstein distance with Lipschitz critic (weight clip or gradient penalty)—smoother training signal than JS when distributions disjoint.

Answer: Replace sigmoid BCE with least-squares or hinge losses—often more stable gradients in practice.

Answer: Both G and D conditioned on label, text, or image—enables targeted generation (class-conditional faces, etc.).

Answer: Paired image-to-image with U-Net G and PatchGAN D—L1 + adversarial loss for aligned translation (maps→aerial).

Answer: Unpaired domains with cycle consistency L_cycle(G,F)—horse↔zebra without paired data.

Answer: Map latent through learned affine transforms per layer to control coarse-to-fine style—high-quality face generation.

Answer: IS measures classifiable diversity of G samples; FID compares Inception feature statistics to real—lower FID is better.

Answer: D loss → 0 instantly, G loss frozen, identical outputs, exploding gradients—tune LR, label smoothing, TTUR.

Answer: Constrain D’s Lipschitz constant per layer—alternative to WGAN-GP for stable critic.

Answer: Large diverse sets for photorealism; data augmentation and balancing classes help conditional GANs.

Answer: Diffusion often wins on diversity and training stability for images; GANs still valued for fast sampling and some domains.

Answer: Misinformation and consent—watermarking, detection models, and policy; same tech powers legitimate VFX and data synthesis.

Answer: Generative model that learns to reverse a gradual noising process—start from Gaussian noise and denoise into a sample.

Answer: Fixed Markov chain adding Gaussian noise over T steps until data ≈ pure noise—q(x_t|x_{t−1}) with known variances.

Answer: Learn p_θ(x_{t−1}|x_t) approximating true posterior—typically predict noise ε or x_0 with a neural net.

Answer: Simplified ε-prediction MSE: network predicts noise added at each t—equivalent to variational lower bound with reweights.

Answer: How fast variance grows with t—linear, cosine, etc.; affects training stability and sample quality.

# ε-prediction: target noise ε; pred = unet(x_t, t)

Answer: Multi-scale spatial denoising with skip connections—preserves detail while aggregating context; time t injected via embeddings.

Answer: Autoregressive in time—hundreds/thousands of steps slow; accelerators (DDIM, distillation) reduce steps.

Answer: Non-Markovian deterministic sampling sharing training objective—fewer steps with some quality tradeoff.

Answer: Use gradients from a classifier p(y|x_t) during sampling to steer generation—sharp but needs extra classifier.

Answer: Train conditional and unconditional model together; interpolate scores at sample time—no separate classifier, widely used in SD.

Answer: Run diffusion in VAE latent space (lower res)—much cheaper; decode with VAE decoder (Stable Diffusion).

Answer: CLIP text encoder, U-Net denoiser in latent space, VAE—plus schedulers and safety tooling around the stack.

Answer: Diffusion: stable training, great diversity, slower sampling. GAN: fast one-shot but trickier mode coverage.

Answer: Add temporal layers or 3D convs; causal attention across frames—data and compute heavy.

Answer: Condition on known regions by concatenating mask/channel inputs to U-Net—fill missing areas consistently.

Answer: Cross-attention from text tokens to spatial features (like transformers)—T5/CLIP embeddings as context.

Answer: Different timesteps contribute unequally to loss—reweighting (v-prediction, Min-SNR) improves quality.

Answer: Related generative path from noise to data via ODE/flows—competes with diffusion on speed and quality in recent work.

Answer: Large image-text pairs for T2I; training is GPU-heavy; inference optimizes with TensorRT, FlashAttention, distilled samplers.

Answer: FID, CLIP score for text alignment, human preference studies—no single metric captures all.