Mathematics & Statistics Foundation — Q&A

Answer: The dot product of vectors \(x\) and \(w\) is \(x \cdot w = \sum_i x_i w_i\). Geometrically it relates to the cosine of the angle between vectors; in ML it appears in linear models (e.g., \(w^Tx\)), similarity measures, and in the computation inside many neural network layers.

Answer: For a matrix \(A\), an eigenvector \(v\) is a direction that is only scaled (not rotated) by \(A\), and the scale factor is the corresponding eigenvalue \(\lambda\) (i.e. \(Av = \lambda v\)). In PCA, eigenvectors of the covariance matrix give the principal directions of variance, and eigenvalues tell you how much variance each principal component explains.

Answer: It combines linear transformations. In ML, multiplying feature matrix X by weight vector w gives predictions for linear models.

Answer: Multiplication A(m×n)·B(n×p) is valid only when inner dimensions match. Shape errors often indicate pipeline bugs in feature or batch handling.

Answer: Transpose flips rows and columns. It appears in normal equations, covariance calculations, and gradient derivations (e.g., XᵀX).

Answer: Rank is the number of linearly independent rows/columns. Low rank indicates redundant features and can cause instability in regression.

Answer: For invertible A, A⁻¹ reverses the transformation A. In practice, exact inversion is often avoided for numerical stability; decompositions are preferred.

Answer: Determinant indicates volume scaling and whether a matrix is singular. det(A)=0 implies non-invertible matrix and dependent columns.

Answer: Orthogonal vectors are perpendicular (dot product zero). Orthogonal features/components reduce redundancy and simplify optimization and interpretation.

Answer: Norm is vector magnitude. L2 norm is common for distance/similarity; L1/L2 norms also appear in regularization (Lasso/Ridge).

Answer: Cosine similarity = (x·y)/(||x|| ||y||). It measures direction similarity independent of absolute scale; widely used in retrieval/embedding tasks.

Answer: Singular Value Decomposition factors A into UΣVᵀ. It is useful for dimensionality reduction, denoising, latent factor models, and numerical robustness.

Answer: PCA is variance-based; large-scale features dominate components. Standardization ensures each feature contributes comparably.

Answer: Linear algebra is the language of data representation and transformations—vectors, matrices, and decompositions power model training, optimization, and dimensionality reduction.