Classical Features: Edges, Corners, and Descriptors

The pipeline separates where to look from how to describe what was found. Each of the five stages has its own literature and its own measured result. Ten descriptors were ranked against each other by Mikolajczyk and Schmid in 2005. The matching rule most systems still use is Lowe's distance ratio, from 2004. The geometric verifier at the end is random sample consensus, from Fischler and Bolles in 1981.

That division matters for reading results. A single accuracy figure quoted at the end of the chain is a figure about all five stages at once. The published record — set out in the next section — shows that changing one stage can reverse the reported ranking of another.

Image gradients respond where intensity changes across space; they can mark object boundaries, shadows, texture, reflections, or sensor artifacts.

Derivative filters amplify noise. So smoothing and scale matter. A strong edge does not automatically identify which side belongs to an object.

The field has treated this as a measurement problem from the beginning. Random sample consensus arrived in 1981, and Fischler and Bolles justified it by the state of the front end feeding it. The method, their abstract says, “is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors.” The premise has not been retired. Everything downstream of a gradient is built to survive it.

These components solve different subproblems and should be evaluated separately. The reason that instruction exists is documented, and it is not a matter of taste.

For over a decade the standard testbed for local features was the Oxford affine covariant regions set. The HPatches authors describe it in one sentence: “It consists of image sequences of 8 scenes, each containing 6 images, and ground-truth homographies.” Oxford's Visual Geometry Group distributes exactly those eight sequences — Bikes and Trees for blur, Graffiti and Bricks for viewpoint, Bark and Boat for zoom and rotation, Cars for light, UBC for JPEG compression.

Table 1 of the HPatches paper collects what the literature concluded from that one small set, and the caption of the table is the finding. It is headed: “Contradicting conclusions reported in literature while evaluating the same descriptors on the same benchmark (Oxford [22]).” The rows are pairs published in both directions — LIOP>SIFT and SIFT>LIOP, BRISK>SIFT and SIFT>BRISK, ORB>SIFT and SIFT>ORB, ORB>BRIEF and BRIEF>ORB. Same descriptors, same eight scenes, opposite verdicts.

A ranking, then, is a property of a whole pipeline measured on a particular set of images. It is not a property of the descriptor whose name appears in the conclusion. HPatches was built in 2017 as the replacement, and it separates the two nuisance factors by construction: “In 57 scenes the main nuisance factors are photometric changes and the remaining 59 sequences show significant geometric deformations due to viewpoint change.” Each sequence is a reference image plus five targets. Detector, descriptor, matcher and verifier are listed apart below because the alternative has already been tried, in print, and produced both answers.

These methods are valuable because their invariances and losses are comparatively explicit — explicit enough to be counted and compared. Ten of them were put on the same footing in 2005: “We use 10 different descriptors”, Mikolajczyk and Schmid write, and their abstract states the outcome: “Furthermore, we observe that the ranking of the descriptors is mostly independent of the interest region detector and that the SIFT-based descriptors perform best.”

Their own GLOH shows what the design choices cost, in numbers. SIFT uses a 4×4 Cartesian grid of 8-orientation histograms: 128 dimensions. GLOH replaces that with a log-polar grid of 17 location bins — 3 radial × 8 angular, the central bin undivided — times 16 orientation bins, a 272-bin histogram. PCA then returns it to 128, using a covariance matrix estimated on 47,000 image patches.

The paper's matching example prices the difference. Hessian-Affine regions, a viewpoint change of more than 50 degrees, a fixed 400 nearest-neighbour matches: GLOH obtained the highest recall at 0.25 and complex filters the lowest at 0.06. That is 192 correct matches down to 44, or “approximately 4.4 times less correct matches for complex filters than for GLOH”. The paper's own summary of the whole comparison is flatter: “In most of the tests, GLOH obtains the best results, closely followed by SIFT.”

A transparent baseline can reveal whether a deep model is solving the intended task or merely exploiting dataset scale. For most of the period in which the best-known classical descriptor defined the field, building that baseline was also a licensing question.

SIFT was patented. US Patent 6,711,293 B1 opens its abstract this way: “A method and apparatus for identifying scale invariant features in an image and a further method and apparatus for using such scale invariant features to locate an object in an image are disclosed.” The inventor is David G. Lowe, the assignee the University of British Columbia. Priority 8 March 1999, filed 6 March 2000, granted 23 March 2004, anticipated expiration 6 March 2020. The status now reads “Expired - Lifetime”. Sixteen years.

The tooling followed within weeks. OpenCV merged pull request #17119 on 24 April 2020: 21 commits moving SIFT out of the xfeatures2d module in opencv_contrib and into the main features2d module, one of them titled “features2d(sift): patent expiration note”. The step below that says construct the representation became a default import on that date, and not before it.

Learned representations dominate many benchmark tasks. On geometry the published comparisons do not read that way, and two unrelated groups have put the finding in print.

The Image Matching Challenge benchmark scores methods on “the accuracy of the reconstructed camera pose” rather than on an intermediate proxy, and embeds dozens of algorithms “from seminal works to the cutting edge of machine learning research”. Jin and co-authors published it in the International Journal of Computer Vision in 2020. Their abstract states the result: “We show that with proper settings, classical solutions may still outperform the perceived state of the art.”

The second finding comes from a survey arguing the opposite case. A 2019 survey from NAVER LABS Europe set out to show that detectors and descriptors are better learned than designed by hand. Its numbered conclusion 10 records the reverse anyway: “recent benchmarks targeting their application in image-based reconstruction and localisation pipelines suggest that handcrafted features still perform just as well or even better than recent deep-learned features on such tasks”. It adds that those benchmarks predate methods such as SuperPoint, KSP and DOAP.

So classical features stay useful for low-data geometry, deterministic preprocessing, interpretability, and sanity checks. They can expose whether a task depends on shape, texture, alignment, or color. A strong handcrafted baseline may show that a complex model is unnecessary.

Aggregate match counts can hide a pipeline that fails for the wrong reasons. Recall of 0.25 and recall of 0.06 came from the same image pair, with the same regions and the same fixed 400 nearest-neighbour matches. Only the descriptor differed. Nothing in a headline count would have separated them.