Harris corner detector

Idea: structure tensor

Let Ix, Iy be image gradients. Over a window, form the second-moment matrix M from summed outer products of gradients. Its eigenvalues λ₁, λ₂ describe edge strength in two orthogonal directions: both large → corner; one large → edge; both small → flat. Harris uses the response R = det(M) − k·trace(M)² (with empirical k ≈ 0.04–0.06) to avoid explicit eigenvalue decomposition.

cv2.cornerHarris

Input must be float32 grayscale. Output is a single-channel float response map; threshold and take local maxima to list corners.

import cv2
import numpy as np

gray = cv2.imread("checkerboard.png", cv2.IMREAD_GRAYSCALE)
gray_f = np.float32(gray)

block, ksz, k = 3, 3, 0.04
resp = cv2.cornerHarris(gray_f, block, ksz, k)

# Dilate to help local-max suppression in a simple way
resp_d = cv2.dilate(resp, None)
vis = np.zeros_like(gray)
vis[resp_d > 0.01 * resp_d.max()] = 255

Stricter threshold

thresh = 0.05 * resp.max()
mask = resp > thresh
# optional: keep only local maxima of `resp` on `mask` with further NMS

Sub-pixel refinement

cornerSubPix refines corner locations to sub-pixel accuracy using the local intensity pattern—useful for calibration, stitching, and metrology.

import cv2
import numpy as np

gray = cv2.imread("grid.jpg", cv2.IMREAD_GRAYSCALE)
gray_f = np.float32(gray)
resp = cv2.cornerHarris(gray_f, 3, 3, 0.04)
yxs = np.argwhere(resp > 0.01 * resp.max()).astype(np.float32)
# cornerSubPix expects shape (N, 1, 2) with (x, y) order
pts = np.zeros((len(yxs), 1, 2), dtype=np.float32)
pts[:, 0, 0] = yxs[:, 1]
pts[:, 0, 1] = yxs[:, 0]

criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 40, 0.001)
refined = cv2.cornerSubPix(gray, pts, (5, 5), (-1, -1), criteria)

Harris vs Shi-Tomasi

goodFeaturesToTrack implements the Shi-Tomasi criterion (minimum eigenvalue threshold)—often picks a cleaner set of points for tracking. Harris gives a dense response map you threshold yourself; Shi-Tomasi returns a capped list sorted by quality.

corners = cv2.goodFeaturesToTrack(
    gray, maxCorners=100, qualityLevel=0.01, minDistance=10,
    blockSize=3, useHarrisDetector=True, k=0.04)

corners_st = cv2.goodFeaturesToTrack(
    gray, maxCorners=100, qualityLevel=0.01, minDistance=10, blockSize=3)