3D Vision & Depth — Interview Q&A

Answer: Reasoning about geometry of the scene—depth, shape, pose, and 3D structure—from images, video, or range sensors.

Answer: Triangulate corresponding points in two calibrated views—baseline provides parallax; disparity inversely related to depth.

Answer: Horizontal shift between conjugate pixels in rectified stereo pair—larger disparity means closer object (for standard forward stereo).

Answer: Corresponding point in second image lies on a line (epipolar line)—reduces matching from 2D search to 1D after rectification.

Answer: Uses cues (perspective, texture, learned priors) or supervised/self-supervised CNNs—scale ambiguous without extra info.

Answer: Set of 3D points (x,y,z), often with color/normal—raw output of LiDAR/stereo fusion or depth cameras.

Answer: Voxel grid discretizes 3D space—good for conv nets; mesh stores vertices+faces—compact for graphics and surface reasoning.

Answer: Projects 3D X to image x via similar triangles: x = K [R|t] X (homogeneous)—basis for calibration and triangulation.

Answer: Maps camera coordinates to pixels: focal lengths f_x,f_y and principal point c_x,c_y; may include skew in general form.

Answer: Rotation R and translation t from world to camera frame—pose of camera in scene.

Answer: Structured light or time-of-flight provides registered depth + color (Kinect, RealSense)—no stereo baseline needed but range/artifact limits.

Answer: Active ranging by laser pulses—sparse accurate 3D, widely used in autonomy; different noise profile than passive stereo.

Answer: Estimate sparse 3D points and camera poses from many images—basis of photogrammetry pipelines.

Answer: Simultaneously localize sensor and build map of environment—needs data association and loop closure.

Answer: Neural radiance field represents scene as MLP of density+color in 5D (x,y,z,θ,φ)—novel view synthesis; hot research direction.

Answer: Monocular SfM/SLAM recovers geometry up to similarity transform without metric scale—IMU or known object fixes scale.

Answer: Iterative Closest Point aligns two point clouds by minimizing distances between correspondences—registration and tracking.

Answer: Top-down grid of scene used in driving—fuses multi-view or LiDAR into 2D bird’s-eye feature maps for detection/planning.

Answer: AR overlay needs 6-DoF pose; robotics needs grasp planning collision checking—both need reliable 3D perception.

Answer: KITTI, nuScenes, ScanNet, ShapeNet—each emphasizes driving, multi-sensor, indoor scans, or CAD models respectively.

Answer: Estimating intrinsic (focal length, principal point, distortion) and often extrinsic (pose) parameters so pixel measurements map correctly to 3D rays.

Answer: Properties of the camera/lens fixed w.r.t. the sensor—encoded in K and distortion coeffs—independent of where the camera sits in the world.

Answer: Rigid transform [R|t] from world (or calibration object) frame to camera frame—changes when the camera moves.

Answer: 3×3 upper-triangular mapping normalized camera coordinates to pixels: focal lengths f_x,f_y, principal point c_x,c_y, optional skew γ.

Answer: Lens bends rays—barrel (outward) or pincushion (inward); modeled as r-dependent scaling of image radius from optical center.

Answer: Lens not perfectly parallel to sensor—modeled with extra parameters (p1,p2) shifting points tangentially; common in OpenCV 5-coeff model.

Answer: Classic polynomial radial + tangential distortion used in OpenCV calibrateCamera—may use k1–k3, p1,p2; fisheye uses different high-FOV model.

Answer: Uses multiple views of a planar calibration pattern; each view gives a homography constraining intrinsics; closed-form init then non-linear refinement minimizing reprojection error.

Answer: Corner intersections are easy to detect sub-pixel; known 3D layout on plane gives 2D–3D correspondences per image.

Answer: Distance between detected image points and projection of 3D model points with estimated parameters—lower is better; report RMS in pixels.

Answer: findChessboardCorners → calibrateCamera → get K, distCoeffs; optional stereoCalibrate for two cameras.

ret, K, dist, rvecs, tvecs = cv2.calibrateCamera(obj_pts, img_pts, image_size, None, None)

Answer: Estimate intrinsics per camera plus relative pose (R,T) between cameras and often rectify so epipolar lines align—needed for triangulation.

Answer: Very wide FOV where polynomial model breaks—OpenCV fisheye:: namespace uses different distortion and projection.

Answer: Optical axis intersection with image plane (c_x,c_y)—often near image center but not exactly; important for undistortion and 3D.

Answer: Non-orthogonal pixel axes—often assumed 0 for modern sensors; included in full K for completeness.

Answer: Usually attach to calibration board plane (Z=0 on pattern)—extrinsics are board-to-camera per capture.

Answer: Single plane gives partial constraints—need multiple orientations/distances to fix intrinsics uniquely (Zhang’s multi-view idea).

Answer: Overlay virtual objects requires accurate projection and undistortion—wrong K causes “swimming” augmentations.

Answer: Zoom/focus change, different camera, temperature extremes, or new lens—intrinsics are not universal across devices.

Answer: Joint non-linear refinement of many cameras and 3D points—structure-from-motion and SLAM extend calibration ideas to large scenes.

Answer: Using two (or more) calibrated views with known baseline to recover depth via triangulation of corresponding points.

Answer: Horizontal shift between corresponding pixels in a rectified stereo pair—larger disparity means closer surface (inverse relation to depth).

Answer: Z ≈ f × B / d (f focal length, B baseline, d disparity)—assumes rectified parallel cameras and pinhole model.

Answer: Larger B increases depth precision (more parallax) but worsens occlusions and matching in narrow scenes; small B reduces measurable disparity range.

Answer: Warp both images so epipolar lines are horizontal scanlines—reduces correspondence search to 1D and simplifies disparity.

Answer: Without rectification, match for a point lies on a line in the other image—comes from epipolar geometry of two views.

Answer: For each pixel (or patch), find best match along epipolar line using photometric cost (SAD, census, CNN features).

Answer: 3D array H×W×D of matching costs over disparity levels—winner-take-all or global optimization (SGC, belief propagation) picks disparities.

Answer: Semi-Global Matching aggregates costs along many paths with smoothness penalties—good quality/speed tradeoff in OpenCV StereoSGBM.

Answer: Pixels visible in only one view have undefined disparity—detected by consistency checks or left-right validation.

Answer: Parabolic fit around discrete minimum or phase-based methods—needed for smooth surfaces and accurate 3D.

Answer: Calibration errors, textureless regions, repetitive patterns, specular highlights, and motion if scene moves between exposures.

sgm = cv2.StereoSGBM_create(minDisparity=0, numDisparities=128, blockSize=5)
disp = sgm.compute(imgL, imgR)

Answer: BM: fixed small block, fast, blocky. SGBM: semi-global, slower, smoother—preferred when quality matters.

Answer: Single image lacks scale without priors—learned networks predict relative depth; stereo gives metric depth with calibration.

Answer: Structured light / ToF gives depth directly—no correspondence problem but range/resolution limits; stereo passive but needs texture.

Answer: Fuse many images (MVS) for dense point clouds—used in photogrammetry beyond two-camera stereo.

Answer: Wide-baseline camera pairs on vehicles for obstacle depth; often fused with radar/LiDAR and learned refinement.

Answer: Sparse accurate LiDAR anchors depth map from stereo; learning-based fusion common in autonomy stacks.

Answer: CNNs build cost volumes or regress disparity directly (e.g. PSMNet)—strong on benchmarks when enough training data.

Answer: Yes for metric depth—need K, distortion, and stereo extrinsics; rectification matrices derived from them.