Title: Revision / Robotics / ComputerVision
On every note of this deck, unless a note says otherwise:
title: computer-vision

Front: * Computer Vision — COMP3631
** Why Computer Vision is Hard
*** Why is computer vision difficult?
Back: the gap between pixel values (numbers) and semantic meaning is enormous; machines cannot handle it the way humans do

* Computer Vision — COMP3631
** Why Computer Vision is Hard
*** Challenges that make CV difficult::
-Illumination: same object looks entirely different under different lighting
-Viewpoint: same object looks different from different camera angles
-Scale: same object at different distances has very different apparent size
-Occlusion: objects are partially or fully hidden by other objects
-Deformation / intra-class variation: instances of the same category look different
-Background clutter: objects blend into the background texture or colour
title: computer-vision

Front: * Computer Vision — COMP3631
** Why Computer Vision is Hard
*** Why image dimensionality is a problem
Back: a 256×256 RGB image has \(\approx 200{,}000\) pixel values; these must all be processed while the useful features are a tiny fraction of that

Front: * Computer Vision — COMP3631
** Why Computer Vision is Hard
*** Why spatial structure matters
Back: image properties (texture, patterns) are consistent across spatial regions → features can be shared across positions, enabling compression

Q: * Computer Vision — COMP3631
** The Computer Vision Pipeline
*** The three stages of the CV pipeline [...]
A: preprocessing → feature extraction → classifier
title: computer-vision

Front: * Computer Vision — COMP3631
** The Computer Vision Pipeline
*** CV pipeline — what preprocessing does
Back: smoothing, resizing, normalising, contrast adjustment; prepares raw pixels for reliable feature extraction

Front: * Computer Vision — COMP3631
** The Computer Vision Pipeline
*** CV pipeline — what feature extraction does
Back: computes compact, discriminative representations (edges, keypoints, histograms, descriptors) from preprocessed pixels

Q: * Computer Vision — COMP3631
** The Computer Vision Pipeline
*** The three categories of image processing::
-Point: [...]
-Spatial: [...]
-Global: [...]
A: operate on each pixel independently; operate on a pixel's neighbourhood (filter); operate on the whole image (e.g. histogram)
title: computer-vision

Front: * Computer Vision — COMP3631
** The Computer Vision Pipeline
*** CV pipeline overview

Front: * Computer Vision — COMP3631
** Image Representation
*** Digital image definition
Back: a 2D array of discrete pixel values \(f(x,y)\) that approximates a continuous scene; \(x\), \(y\) are integer spatial coordinates

Front: * Computer Vision — COMP3631
** Image Representation
*** Sampling definition
Back: discretising the continuous spatial coordinates of a scene into a finite \(M \times N\) grid of pixel locations

Front: * Computer Vision — COMP3631
** Image Representation
*** Quantization definition
Back: discretising continuous intensity values into \(L = 2^k\) discrete levels

Front: * Computer Vision — COMP3631
** Image Representation
*** Why the image is only an approximation
Back: both spatial sampling and intensity quantization introduce discretisation errors relative to the original continuous scene

Front: * Computer Vision — COMP3631
** Image Representation
*** k-bit image — intensity levels
Back: \(L = 2^k\); 8-bit gives \(L = 256\) grey levels (\(0\) = black, \(255\) = white)

Front: * Computer Vision — COMP3631
** Image Representation
*** Image storage size
Back: \[b = M \times N \times k \text{ bits}\] for an \(M \times N\) image with \(k\) bits per pixel

Front: * Computer Vision — COMP3631
** Colour Models
*** Purpose of a colour model
Back: a coordinate system and subspace in which every colour is a single point; the choice depends on application (capture, display, printing, human perception)

Front: * Computer Vision — COMP3631
** Colour Models
*** RGB — device use case
Back: image capture (cameras) and display (monitors); additive primary colours

Front: * Computer Vision — COMP3631
** Colour Models
*** RGB — how a colour is represented
Back: a point \((R,\,G,\,B)\) in a Cartesian cube; values in \([0,1]\) or \([0,255]\) for 8-bit

Front: * Computer Vision — COMP3631
** Colour Models
*** RGB — total colours in a 24-bit image
Back: \(256^3 = 16{,}777{,}216\) distinct colours

Front: * Computer Vision — COMP3631
** Colour Models
*** Bayer filter — purpose
Back: digital cameras place only one colour filter (R, G, or B) per sensor site in a mosaic pattern; the missing two channels are interpolated from neighbours

Front: * Computer Vision — COMP3631
** Colour Models
*** Bayer filter — why twice as many green sites
Back: the human visual system is most sensitive to green wavelengths; extra green sampling improves perceived image sharpness

Front: * Computer Vision — COMP3631
** Colour Models
*** CMY — what each component subtracts
Back: C subtracts red, M subtracts green, Y subtracts blue from reflected white light (subtractive mixing)

Front: * Computer Vision — COMP3631
** Colour Models
*** CMY — conversion from RGB
Back: \[C = 1-R, \quad M = 1-G, \quad Y = 1-B\]

Front: * Computer Vision — COMP3631
** Colour Models
*** CMY — use case
Back: colour printing (ink on white paper)

Q: * Computer Vision — COMP3631
** Colour Models
*** HSI — three components::
-Hue: [...]
-Saturation: [...]
-Intensity: [...]
A: the pure colour (e.g. red, yellow, blue); degree to which pure colour is diluted by white light; achromatic brightness (measurable grey level)
title: computer-vision

Front: * Computer Vision — COMP3631
** Colour Models
*** HSI — advantage over RGB
Back: decouples intensity from colour information; ideal when illumination varies; matches the natural human description of colour

Front: * Computer Vision — COMP3631
** Colour Models
*** L*a*b* — key property
Back: perceptually uniform (equal numerical differences ≈ equal perceived differences); device-independent; used for colour measurement and calibration

Front: * Computer Vision — COMP3631
** Colour Models
*** Colour model comparison
Back: Model | Primary use | Key property
RGB | Cameras, monitors | Additive primaries; Cartesian cube
CMY/CMYK | Colour printing | Subtractive; \(C=1-R\), \(M=1-G\), \(Y=1-B\)
HSI | Segmentation under varying light | Decouples intensity from colour; human-intuitive
L*a*b* | Colour measurement, calibration | Perceptually uniform; device-independent

Front: * Computer Vision — COMP3631
** Point Operators
*** Point operator definition
Back: transforms each pixel independently via \(s = T(r)\), where \(r\) is the input and \(s\) is the output intensity

Front: * Computer Vision — COMP3631
** Point Operators
*** Thresholding — purpose and rule
Back: segment an object from background; \(s = 1\) if \(r > T\), else \(s = 0\); produces a binary image

Front: * Computer Vision — COMP3631
** Point Operators
*** Image negative — formula and use case
Back: \(s = (L-1) - r\); inverts brightness; enhances white/grey detail embedded in dark regions (e.g. mammograms)

Front: * Computer Vision — COMP3631
** Point Operators
*** Log transformation — formula and effect
Back: \[s = c \cdot \log(1 + r)\] expands the range of dark pixel values, compresses bright ones

Front: * Computer Vision — COMP3631
** Point Operators
*** Log transformation — use case
Back: display Fourier spectra or images with a very large dynamic range (many low values, a few very high ones)

Front: * Computer Vision — COMP3631
** Point Operators
*** Power-law (gamma) — formula
Back: \[s = c \cdot r^\gamma\]

Front: * Computer Vision — COMP3631
** Point Operators
*** Power-law — effect of \(\gamma < 1\)
Back: brightens image — maps a narrow dark input range to a wider output range

Front: * Computer Vision — COMP3631
** Point Operators
*** Power-law — effect of \(\gamma > 1\)
Back: darkens image — compresses dark values, expands bright values

Front: * Computer Vision — COMP3631
** Point Operators
*** Power-law vs log
Back: \(\gamma\) is tuneable; \(\gamma < 1\) brightens, \(\gamma > 1\) darkens; log can only expand dark regions

Front: * Computer Vision — COMP3631
** Point Operators
*** Power-law example

Front: * Computer Vision — COMP3631
** Point Operators
*** Piecewise linear transform — when to use
Back: when an arbitrary, user-defined contrast stretch between two specific intensity levels is needed (contrast stretching)

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Spatial filter (correlation) definition
Back: replaces each pixel with a weighted neighbourhood sum: \[g(x,y) = \sum_{s,t} w(s,t) \cdot f(x+s,\; y+t)\]

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Averaging (box) filter — mechanism
Back: uniform weight \(1/n^2\) on all \(n \times n\) neighbours; replaces each pixel with the neighbourhood mean; blurs/smooths image

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Effect of increasing averaging filter size
Back: larger neighbourhood → more pixels averaged → stronger blur → finer detail progressively lost

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Weighted averaging filter — advantage
Back: pixels closer to centre get higher weight → more natural, less blocky blur than uniform average

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Gaussian filter — \(\sigma\) role
Back: \(\sigma\) controls the radius of smoothing; larger \(\sigma\) = stronger, wider blur

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Median filter — use case
Back: removes impulse noise (salt-and-pepper) without blurring edges; non-linear (no weighted sum)

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Median vs averaging filter
Back: median preserves edges and handles impulse noise better; averaging is faster but smears edges

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Correlation vs convolution
Back: correlation slides the kernel as-is; convolution flips the kernel 180° first; for symmetric kernels (Gaussian, box blur) the results are identical

* Computer Vision — COMP3631
** Spatial Filtering
*** Border handling strategies::
-Zero padding: fill missing pixels with 0 → dark border artefact
-Replicate (extend): copy the nearest edge pixel outward
-Wrap: pixels from the opposite edge wrap around
-Crop: discard output pixels whose neighbourhood extends beyond the image
title: computer-vision

Front: * Computer Vision — COMP3631
** Spatial Filtering
*** Spatial filter comparison
Back: Filter | Type | Best for | Limitation
Box (averaging) | Linear | General smoothing; fast | Blurs edges; uniform weighting
Gaussian | Linear | Natural blur; pre-processing before edge detection | Does not remove impulse noise
Median | Non-linear | Impulse / salt-and-pepper noise | Slower; can distort fine detail

Front: * Computer Vision — COMP3631
** Image Histograms
*** Histogram definition
Back: unnormalised: \(h(r_k) = n_k\); normalised: \[p(r_k) = \frac{n_k}{MN}\] where \(n_k\) = pixels at intensity \(r_k\), \(MN\) = total pixels; \(\sum_k p(r_k) = 1\)

Front: * Computer Vision — COMP3631
** Image Histograms
*** Reading a histogram — dark image
Back: most bins concentrated at the low end of the intensity scale

Front: * Computer Vision — COMP3631
** Image Histograms
*** Reading a histogram — light image
Back: most bins concentrated at the high end of the intensity scale

Front: * Computer Vision — COMP3631
** Image Histograms
*** Reading a histogram — low contrast image
Back: bins clustered in a narrow band near the middle; washed-out grey appearance

Front: * Computer Vision — COMP3631
** Image Histograms
*** Reading a histogram — high contrast image
Back: bins spread widely across the full intensity range

Front: * Computer Vision — COMP3631
** Image Histograms
*** Histogram for image retrieval
Back: represent each image by a histogram of colour/feature frequencies; compare histograms with cosine similarity to retrieve similar images

Front: * Computer Vision — COMP3631
** Edge Detection
*** Edge definition
Back: a location of sudden brightness discontinuity (step-change) in an image

Q: * Computer Vision — COMP3631
** Edge Detection
*** Three edge models::
-Step: [...]
-Ramp: [...]
-Roof: [...]
A: immediate one-pixel transition between two brightness levels; gradual transition; realistic model of real-world edges; brightness rises then falls; models edges of thin objects
title: computer-vision

Front: * Computer Vision — COMP3631
** Edge Detection
*** Edge strength — formula
Back: \[s = \|\nabla f\| = \sqrt{\left(\frac{\partial f}{\partial x}\right)^2 + \left(\frac{\partial f}{\partial y}\right)^2}\]

Front: * Computer Vision — COMP3631
** Edge Detection
*** Edge direction — formula
Back: \[\theta = \arctan\!\left(\frac{\partial f}{\partial y},\; \frac{\partial f}{\partial x}\right)\] points perpendicular to the edge

Front: * Computer Vision — COMP3631
** Edge Detection
*** Why smooth before detecting edges
Back: first-order derivatives amplify noise; pre-smoothing (Gaussian) suppresses noise so only genuine intensity changes produce large gradients

Front: * Computer Vision — COMP3631
** Edge Detection
*** Sobel operator — how it works
Back: two 3×3 kernels estimate \(\partial f/\partial x\) and \(\partial f/\partial y\); combine gradient magnitudes; apply threshold

Front: * Computer Vision — COMP3631
** Edge Detection
*** Sobel — limitation
Back: susceptible to noise; produces thick, poorly localised edges; no multi-scale analysis

Q: * Computer Vision — COMP3631
** Edge Detection
*** Canny — three optimality criteria::
-Low error rate: [...]
-Good localisation: [...]
-Single response: [...]
A: detect real edges, miss few; detected edge close to the true edge location; one detected edge per real edge
title: computer-vision

Front: * Computer Vision — COMP3631
** Edge Detection
*** Canny pipeline
Back: Input image
         ↓
   Gaussian blur       ← suppress noise
         ↓
  Gradient (Sobel)     ← compute ∂f/∂x, ∂f/∂y
         ↓
  Non-max suppression  ← thin edges to 1 pixel wide
         ↓
  Double threshold     ← strong / weak / non-edge
         ↓
Hysteresis tracking    ← keep weak if connected to strong
         ↓
    Edge image

Front: * Computer Vision — COMP3631
** Edge Detection
*** Canny advantage over Sobel
Back: three optimality criteria give clean, thin, well-localised edges; Gaussian pre-smoothing handles noise; hysteresis avoids spurious breaks

Front: * Computer Vision — COMP3631
** Edge Detection
*** LoG (Laplacian of Gaussian) — how it works
Back: smooth with Gaussian (suppress noise), then apply Laplacian (second spatial derivative); detect edges at zero-crossings

Front: * Computer Vision — COMP3631
** Edge Detection
*** LoG as blob detector — characteristic scale
Back: LoG response peaks at \(\sigma = r/\sqrt{2}\) for a blob of radius \(r\); this \(\sigma\) is the characteristic scale of the feature

Front: * Computer Vision — COMP3631
** Edge Detection
*** DoG — relation to LoG
Back: \[\text{DoG} = G_{\sigma_1} - G_{\sigma_2} \approx \text{LoG}\] cheaper to compute; used in SIFT scale-space

Front: * Computer Vision — COMP3631
** Edge Detection
*** Edge detector comparison
Back: Detector | Mechanism | Advantage | Limitation | Use when
Sobel | 3×3 gradient; threshold | Simple, fast | Thick, noisy edges | Speed matters; clean images
Canny | Gaussian + gradient + NMS + hysteresis | Thin, well-localised; noise-robust | More computation; thresholds to tune | Best-quality edges needed
LoG | Gaussian + Laplacian; zero-crossings | Precise localisation; blob detection | Expensive; sensitive to \(\sigma\) | Blob/scale analysis
DoG | Difference of two Gaussians \(\approx\) LoG | Fast LoG approximation | Approximation only | SIFT scale-space

* Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Four desired properties of interest points::
-Repeatability: detectable under geometric and photometric changes
-Saliency: unique, distinctive descriptor per point
-Compactness: few points; efficient to extract
-Locality: covers a small region; robust to occlusion
title: computer-vision

Front: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Why corners rather than edges
Back: corners show large brightness change when a window shifts in any direction; edges change only along one axis → ambiguous for matching

Front: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Harris — structure tensor \(M\)
Back: \[M = \sum_{(x,y) \in \text{window}} \begin{pmatrix} I_x^2 & I_x I_y \\ I_x I_y & I_y^2 \end{pmatrix}\] sums outer products of gradients over a local window

Front: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Harris — corner response \(R\)
Back: \[R = \det(M) - k \cdot \mathrm{tr}(M)^2, \quad k \approx 0.04\text{–}0.06\]

Q: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Harris — how to classify a point using \(R\)::
-Corner: [...]
-Edge: [...]
-Flat region: [...]
A: \(R \gg 0\) — both eigenvalues large; \(R \ll 0\) — one eigenvalue large, one small; \(|R|\) small — both eigenvalues small
title: computer-vision

Front: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Harris — invariances
Back: invariant to image rotation; partially invariant to affine intensity change (derivatives cancel a constant offset)

Front: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Harris — NOT invariant to
Back: scale changes; large viewpoint changes; significant contrast changes

Front: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** Multi-scale Harris — why needed
Back: a corner at one scale may appear as an edge at a coarser scale; must search across scales for the correct response

Front: * Computer Vision — COMP3631
** Interest Points and Corner Detection
*** LoG as scale selector
Back: compute LoG at multiple \(\sigma\); keypoint scale = \(\sigma\) where LoG response is a local maximum in both position and scale

Front: * Computer Vision — COMP3631
** Feature Descriptors
*** SIFT pipeline
Back: Build DoG scale-space
           ↓
  Detect local extrema  (candidate keypoints)
           ↓
  Localise & filter  (remove low-contrast / edge responses)
           ↓
  Assign dominant orientation  (peak of gradient histogram)
           ↓
  Build 4×4 grid descriptor  (8 orientation bins per cell)
           ↓
  128-dim vector → normalise → clamp 0.2 → renormalise

Front: * Computer Vision — COMP3631
** Feature Descriptors
*** SIFT descriptor dimensionality
Back: \(4 \times 4\) cells \(\times\) 8 orientation bins = 128-dimensional vector

Front: * Computer Vision — COMP3631
** Feature Descriptors
*** SIFT — rotation invariance
Back: subtract the dominant orientation (histogram peak) from all gradient directions → descriptor is relative to keypoint orientation

Front: * Computer Vision — COMP3631
** Feature Descriptors
*** SIFT — illumination invariance
Back: threshold all values \(> 0.2\) at 0.2, then re-normalise; reduces sensitivity to large brightness changes

Q: * Computer Vision — COMP3631
** Feature Descriptors
*** SIFT — what it is invariant to::
-Scale: [...]
-Rotation: [...]
-Illumination: [...]
A: DoG extrema across scale-space; dominant orientation normalisation; gradient clamping and re-normalisation
title: computer-vision

* Computer Vision — COMP3631
** Feature Matching
*** TP / FP / TN / FN in matching::
-TP: match found AND it is correct
-FP: match found BUT it is incorrect (false alarm)
-FN: correct match NOT found (missed detection)
-TN: non-match correctly rejected
title: computer-vision

Front: * Computer Vision — COMP3631
** Feature Matching
*** NNDR (Nearest Neighbour Distance Ratio) — rule
Back: accept match only if \(d_1 / d_2 < \text{threshold}\); where \(d_1\) = nearest-neighbour distance, \(d_2\) = second-nearest distance; low ratio = uniquely good match

Front: * Computer Vision — COMP3631
** Feature Matching
*** NNDR — advantage
Back: automatically rejects ambiguous matches where two candidates are similarly close → selects only distinctive points

Front: * Computer Vision — COMP3631
** Feature Matching
*** RANSAC — purpose
Back: robustly estimate a model (homography, translation) from correspondences that include outliers

Front: * Computer Vision — COMP3631
** Feature Matching
*** RANSAC algorithm
Back: Repeat until best model found:
    1. Randomly sample minimum points to fit model
    2. Solve for model parameters
    3. Count inliers (distance to model < threshold)
  Return model with most inliers

Front: * Computer Vision — COMP3631
** Feature Matching
*** RANSAC — advantage
Back: robust to a high fraction of outliers; works for any parametric model

Front: * Computer Vision — COMP3631
** Feature Matching
*** RANSAC — disadvantage
Back: computation grows quickly with outlier fraction and number of model parameters

Front: * Computer Vision — COMP3631
** Feature Matching
*** ICP (Iterative Closest Point) — purpose
Back: align two dense point sets when no pre-matched pairs exist (unknown correspondence)

* Computer Vision — COMP3631
** Feature Matching
*** ICP algorithm steps::
-1: initialise transform (e.g. match centroids)
-2: assign each point in set 1 to nearest neighbour in set 2
-3: estimate transform by least squares
-4: apply transform to set 1
-5: repeat until convergence
title: computer-vision

Front: * Computer Vision — COMP3631
** Feature Matching
*** ICP vs RANSAC — when to use ICP
Back: when there are no pre-matched feature pairs (dense point clouds, model alignment); RANSAC requires sparse feature correspondences

Front: * Computer Vision — COMP3631
** Feature Matching
*** Feature matching comparison
Back: Method | Input | Purpose | Advantage | Limitation
NNDR | Descriptor pairs | Filter ambiguous matches | Removes non-distinctive matches automatically | Threshold needs tuning
RANSAC | Noisy correspondences | Robust model fitting | Handles high outlier fractions | Slow with many outliers / parameters
ICP | Two unmatched point sets | Dense alignment without prior matches | No feature matching needed | Sensitive to initialisation; local minima

Front: * Computer Vision — COMP3631
** Object Recognition
*** Why pixel-level nearest-neighbour fails
Back: slow at test time; not robust to colour, illumination, rotation, translation, scaling

Front: * Computer Vision — COMP3631
** Object Recognition
*** BoW pipeline
Back: Training images
       ↓
  Extract SIFT features
       ↓
  k-means clustering  →  visual vocabulary (K words)
       ↓
  For each image: quantise features  →  word histogram
       ↓
  Train classifier on histograms  (kNN, SVM, …)
  ── Query image ──
  Same pipeline → histogram → inverted-index search

Front: * Computer Vision — COMP3631
** Object Recognition
*** Visual vocabulary — how created
Back: k-means on a large pool of SIFT descriptors; each cluster centre = one visual word (codeword)

Front: * Computer Vision — COMP3631
** Object Recognition
*** How to assign a descriptor to a visual word
Back: nearest-neighbour search in descriptor space to find the closest cluster centre

Front: * Computer Vision — COMP3631
** Object Recognition
*** Inverted file index — purpose
Back: maps each visual word to the list of images containing it; enables fast retrieval without comparing all images

Front: * Computer Vision — COMP3631
** Object Recognition
*** tf-idf weighting
Back: downweights common words (low discriminability) and upweights rare, distinctive words \[\text{tf-idf}(i,d) = \frac{n_{id}}{n_d} \cdot \log\frac{N}{N_i}\] where \(N\) = total images, \(N_i\) = images containing word \(i\)

Front: * Computer Vision — COMP3631
** Object Recognition
*** BoW cosine similarity
Back: \[\text{sim}(d_j, q) = \frac{d_j \cdot q}{\|d_j\|\;\|q\|}\] rank images by normalised inner product of their word-frequency vectors

Front: * Computer Vision — COMP3631
** Object Recognition
*** BoW limitation
Back: discards all spatial layout — two images with the same words in different positions get identical representations

Front: * Computer Vision — COMP3631
** Object Recognition
*** How to improve BoW
Back: use tf-idf weighting; spatial pyramid matching (divide image into spatial bins and concatenate histograms); larger / better vocabulary; denser feature sampling

Front: * Computer Vision — COMP3631
** Object Recognition
*** Part-based / constellation model — core idea
Back: represent an object as \(N\) parts with learned spatial relationships; detect each part independently, then verify geometric consistency by voting

* Computer Vision — COMP3631
** Object Recognition
*** Constellation approach — training steps::
-1: extract keypoint patches from training images
-2: cluster patches to find prototype parts (k-means)
-3: record relative position of each prototype to object centre
-4: at test time, each detected part casts a vote for an object centre
title: computer-vision

Front: * Computer Vision — COMP3631
** Object Recognition
*** Part-based model advantage
Back: robust to partial occlusion; bottom-up (feature-driven); handles appearance variation across instances

Front: * Computer Vision — COMP3631
** Object Recognition
*** HOG descriptor — how computed
Back: divide image into cells; compute histogram of gradient orientations per cell; normalise across overlapping blocks; concatenate into feature vector

Front: * Computer Vision — COMP3631
** Object Recognition
*** HOG — primary use case
Back: human/pedestrian detection; captures shape and silhouette from edge orientations

Front: * Computer Vision — COMP3631
** Object Recognition
*** DPM (Deformable Part-based Model) — what it adds
Back: root HOG filter + deformable part filters, each with a deformation cost; allows flexible spatial layout of parts

Front: * Computer Vision — COMP3631
** Object Recognition
*** Object recognition approach comparison
Back: Approach | Spatial info | Advantage | Limitation | Use when
Bag of Words | None (histogram) | Fast; scalable with inverted index; tf-idf weighting | No spatial layout | Image retrieval; category classification
Part-based / constellation | Relative part positions | Robust to occlusion; handles deformation | Complex to train | Deformable objects; occlusion likely
HOG / DPM | Local cell histograms | Good shape capture; deformable parts | Hand-crafted; fixed trained categories | Pedestrian / rigid object detection

Front: * Computer Vision — COMP3631
** Object Detection
*** Sliding window detection — how it works
Back: run a classifier on every possible window position and scale; report windows with high confidence scores

Front: * Computer Vision — COMP3631
** Object Detection
*** Sliding window — limitation
Back: computationally very expensive — classifier evaluates at every position and scale; many redundant overlapping windows

Front: * Computer Vision — COMP3631
** Object Detection
*** Region proposals — improvement
Back: use objectness cues (edges, colour, texture) to propose class-agnostic candidate regions; far fewer windows to classify

Front: * Computer Vision — COMP3631
** Object Detection
*** NMS (Non-Maximum Suppression) — purpose
Back: remove duplicate bounding boxes that detect the same object

Front: * Computer Vision — COMP3631
** Object Detection
*** NMS — rule
Back: among overlapping boxes (\(\text{IoU} > \text{threshold}\)), keep the one with highest confidence; discard the rest

Front: * Computer Vision — COMP3631
** Object Detection
*** IoU definition
Back: \[\text{IoU} = \frac{\text{area of intersection}}{\text{area of union}}\] of predicted and ground-truth bounding boxes

Front: * Computer Vision — COMP3631
** Object Detection
*** IoU threshold for a true positive
Back: detection is TP if \(\text{IoU} > 0.5\) (some datasets use 0.7)

* Computer Vision — COMP3631
** Object Detection
*** How to compute Average Precision (AP)::
-1: rank all detections by confidence score
-2: assign each to ground-truth if \(\text{IoU} > 0.5\)
-3: compute precision and recall at each rank
-4: AP = area under the precision–recall curve
title: computer-vision

Front: * Computer Vision — COMP3631
** Object Detection
*** mAP definition
Back: mean AP averaged over all object classes; the standard benchmark metric for detection

Front: * Computer Vision — COMP3631
** Object Tracking
*** Tracking by initialisation — goal
Back: given one bounding box in frame 0, estimate the object's location in every subsequent frame

Front: * Computer Vision — COMP3631
** Object Tracking
*** Correlation filter — mechanism
Back: cross-correlate the object template with the search window each frame; the peak of the correlation map = new object location

Front: * Computer Vision — COMP3631
** Object Tracking
*** Correlation filter — efficiency
Back: cross-correlation = element-wise product in Fourier domain; runs at \(\approx 600\) FPS

Front: * Computer Vision — COMP3631
** Object Tracking
*** MOT pipeline
Back: Frame t
      ↓
  Detector  →  detections  {y¹, …, yᴹ}
      ↓
  Kalman predict  →  predicted positions  {x̂¹, …, x̂ᴺ}
      ↓
  Hungarian matching  →  track–detection assignments
      ↓
  Kalman update  →  refined tracks  {x¹, …, xᴺ}
      ↓
  Frame t+1

Front: * Computer Vision — COMP3631
** Object Tracking
*** Kalman filter — two outputs
Back: predicted mean location \(\hat{x}\); prediction covariance \(P\) (uncertainty of where the object will be)

Front: * Computer Vision — COMP3631
** Object Tracking
*** Kalman filter covariance — interpretation
Back: small \(P\) = confident, tight prediction; large \(P\) = wide uncertainty; directly defines the gating ellipse size

Front: * Computer Vision — COMP3631
** Object Tracking
*** Kalman filter gating
Back: only consider detections inside the 95% confidence ellipse (defined by \(P\)) when associating; reduces false associations and ID switches

Front: * Computer Vision — COMP3631
** Object Tracking
*** Hungarian algorithm — purpose
Back: solve the bipartite assignment problem — find the unique, cost-minimising matching between \(N\) predicted tracks and \(M\) detections

Front: * Computer Vision — COMP3631
** Object Tracking
*** Online vs offline tracking
Back: online: uses only current and past frames → real-time capable; offline (batch): uses all frames → better accuracy, not real-time

Front: * Computer Vision — COMP3631
** Object Tracking
*** Why tracking beyond per-frame detection
Back: detectors fail under occlusion, blur, viewpoint change; tracking provides temporal consistency and enables trajectory prediction

Front: * Computer Vision — COMP3631
** Object Tracking
*** Tracking approach comparison
Back: | Correlation filter (single object) | MOT (multi-object)
Initialisation | One bounding box | Per-frame detections
Motion model | Implicit (template correlation peak) | Kalman filter (explicit state + uncertainty)
Association | Correlation peak | Hungarian algorithm
Speed | ~600 FPS (Fourier domain) | Slower (detector + Kalman + assignment)
Best for | Single target, real-time | Multiple targets, identity preservation

Front: * Computer Vision — COMP3631
** Optical Flow
*** Optical flow definition
Back: a 2D displacement field \((u, v)\) at each pixel describing apparent motion between consecutive frames

Front: * Computer Vision — COMP3631
** Optical Flow
*** Brightness constancy assumption
Back: \[I(x,\; y,\; t) = I(x+u,\; y+v,\; t+1)\] the intensity of a pixel does not change as it moves

Front: * Computer Vision — COMP3631
** Optical Flow
*** Optical flow constraint equation
Back: linearising brightness constancy via Taylor expansion: \[I_x u + I_y v + I_t = 0\] where \(I_x, I_y\) are spatial gradients and \(I_t\) is the temporal gradient

Front: * Computer Vision — COMP3631
** Optical Flow
*** Aperture problem
Back: the flow constraint provides one equation for two unknowns \((u, v)\); only the motion component along the gradient direction is observable through a local aperture

Front: * Computer Vision — COMP3631
** Optical Flow
*** Lucas-Kanade — additional constraint
Back: spatial coherence — all pixels in a local window share the same flow vector; over-determines the system, solved by least squares

Front: * Computer Vision — COMP3631
** Optical Flow
*** Horn-Schunck — additional constraint
Back: global smoothness — neighbouring pixels should have similar flow; minimise a global energy function

Front: * Computer Vision — COMP3631
** Optical Flow
*** LK vs HS — key trade-off
Back: LK is local and fast, robust in textured regions, fails at large motions; HS gives globally smooth fields, slower, less robust where smoothness is violated

Front: * Computer Vision — COMP3631
** Optical Flow
*** Coarse-to-fine strategy — purpose
Back: handle large displacements that violate the small-motion assumption; build Gaussian pyramid, estimate flow at coarsest scale, warp and refine at each finer level

Front: * Computer Vision — COMP3631
** Optical Flow
*** Optical flow method comparison
Back: | Lucas-Kanade | Horn-Schunck
Extra constraint | Spatial coherence (same flow in window) | Global smoothness
Scope | Local (per window) | Global (whole image)
Speed | Fast | Slower (global optimisation)
Failure | Large motions; textureless regions | Sharp boundaries; non-smooth scenes
Output | Sparse / dense in textured regions | Dense, globally smooth field

Front: * Computer Vision — COMP3631
** Machine Learning Recap
*** Regression vs classification
Back: regression predicts a continuous value (e.g. bounding box coordinates); classification assigns a discrete label (e.g. cat vs dog)

Front: * Computer Vision — COMP3631
** Machine Learning Recap
*** Overfitting — definition
Back: model fits training data too closely (memorises noise); performs well on training set but poorly on unseen data

Front: * Computer Vision — COMP3631
** Machine Learning Recap
*** Overfitting — diagnostic sign
Back: training error keeps decreasing while test/validation error increases or plateaus

* Computer Vision — COMP3631
** Machine Learning Recap
*** How to reduce overfitting::
-More training data: harder to memorise noise with more examples
-Simpler model: fewer parameters → less capacity to overfit
-Regularisation: penalise large weights to constrain model complexity
-Dropout: randomly disable neurons during training
title: computer-vision

Front: * Computer Vision — COMP3631
** Neural Networks
*** Perceptron — equation
Back: \[y = f\!\left(\sum_i w_i x_i + b\right)\] where \(w_i\) are weights, \(b\) is bias, \(f\) is a non-linear activation

Front: * Computer Vision — COMP3631
** Neural Networks
*** Perceptron — limitation
Back: a single layer can separate only linearly separable classes

Front: * Computer Vision — COMP3631
** Neural Networks
*** Why multiple layers
Back: hidden layers learn hierarchical features; non-linear activations enable arbitrary decision boundaries

Front: * Computer Vision — COMP3631
** Neural Networks
*** Backpropagation — what it does
Back: uses the chain rule to compute gradients of the loss with respect to all network weights; enables gradient-descent weight updates

Front: * Computer Vision — COMP3631
** Neural Networks
*** Mini-batch gradient descent — advantage
Back: balances the noise of single-sample updates and the expense of full-batch; faster and more stable convergence than either extreme

Front: * Computer Vision — COMP3631
** Neural Networks
*** Dropout — mechanism
Back: randomly zero a fraction of neuron outputs during training; at test time all neurons active with weights scaled by keep probability

Front: * Computer Vision — COMP3631
** Neural Networks
*** Dropout — why it reduces overfitting
Back: forces the network to learn redundant, distributed representations; prevents co-adaptation of neurons on training noise

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** Why CNNs over fully-connected for images
Back: images are high-dimensional (\(200 \times 200 \times 3 = 120{,}000\) inputs); fully-connected layers don't scale; images have spatial structure CNNs exploit

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** Local connectivity — idea
Back: each neuron connects only to a small local region (receptive field) of the input, not every input; reduces number of parameters dramatically

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** Weight sharing — idea
Back: all spatial positions in a depth slice use the same filter weights; one filter slides across the entire image

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** Weight sharing — benefit
Back: reduces parameters from \(\mathcal{O}(H \times W)\) per filter to fixed \(K \times K\); enforces translation invariance

* Computer Vision — COMP3631
** Convolutional Neural Networks
*** CNN layer types::
-CONV: applies learnable filters; detects local spatial patterns
-RELU: element-wise activation \(\max(0,x)\); introduces non-linearity
-POOL: spatially downsamples; provides invariance to small translations
-FC: fully-connected layer; maps to class scores
title: computer-vision

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** Convolutional output size
Back: \[\text{output} = \left\lfloor \frac{W - K + 2P}{S} \right\rfloor + 1\] where \(W\) = input size, \(K\) = filter size, \(P\) = padding, \(S\) = stride

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** Pooling — purpose
Back: spatially downsample feature maps; provides invariance to small translations; reduces computation for subsequent layers

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** Max pooling vs average pooling
Back: max keeps the strongest activation in each region → more discriminative; average takes the mean → smoother but less sensitive to peak activations

Front: * Computer Vision — COMP3631
** Convolutional Neural Networks
*** AlexNet — significance
Back: first deep CNN to win ImageNet (2012, 18.2% top-5 error); showed that depth + ReLU + dropout + weight sharing beats hand-crafted features at scale