Blocking-Evaluation

Blocking evaluation

Measure how effectively your blocking strategy reduces record comparisons without losing genuine matches. MatchForge provides evaluateBlocking() to evaluate candidate generation against labelled ground truth.

What is blocking evaluation?

Blocking reduces the number of record pairs that need to be compared. Instead of comparing every source record against every target record, it generates a smaller set of candidate pairs using selected rules such as exact matching, prefix matching, or Soundex.

Blocking in action

From comparisons to candidates

Animated overview
Before blocking
P1
P2
P3
P4
P5
P6
P7
P8
P9
All possible pairs9
After blocking
—
—
—
—
—
—
Candidate pairs3

Comparisons eliminated

66.67%

Comparisons retained

33.33%

Illustrative visualization: 9 possible pairs reduced to 3 candidate pairs. The animation demonstrates the concept, not a live execution of the matching algorithm.

However, reducing comparisons is not enough. A blocking strategy must also retain genuine matching pairs. Blocking evaluation measures this trade-off using three metrics: candidate recall, candidate precision, and reduction ratio.

Evaluation workflow

Blocking evaluation follows four steps, from generating all possible comparisons to measuring the quality of the resulting candidates.

01

All possible pairs

Every source record can be compared with every target record.

N × M comparisons

02

Blocking rules

Generate keys and group records using selected rules.

Exact · Prefix · Soundex

03

Candidate pairs

Keep only pairs that share at least one blocking key.

Unique candidate pairs

04

Evaluate quality

Compare candidates against labelled ground truth.

Recall · Precision · Reduction

Ground truth

Ground truth is a collection of labelled source-target pairs. Each pair identifies two records using their original dataset indexes and specifies whether they represent the same entity. The same format is used by matching evaluation.

typescript
const groundTruth = [
  {
    sourceIndex: 0,
    targetIndex: 0,
    isMatch: true,
  },
  {
    sourceIndex: 0,
    targetIndex: 1,
    isMatch: false,
  },
  {
    sourceIndex: 1,
    targetIndex: 2,
    isMatch: true,
  },
];
PropertyDescription
sourceIndexZero-based index of the record in the source dataset.
targetIndexZero-based index of the record in the target dataset.
isMatchTrue if the pair represents the same entity; otherwise, false.

Ground-truth indexes must refer to the original source and target dataset positions. Each pair must be labelled only once. Duplicate labelled pairs are rejected.

Blocking evaluation metrics

MatchForge calculates three metrics. Together, they show whether a blocking strategy retains genuine matches while reducing the comparison workload.

Candidate Recall

Higher is better

True Candidate Pairs / Total True Pairs

Measures how many genuine matching pairs survive candidate generation. Higher recall means fewer actual matches are discarded during blocking.

Example: 0.90 means 90% of labelled genuine matches were retained.

Candidate Precision

Higher is better

True Candidate Pairs / Labelled Candidate Pairs

Measures how many labelled candidate pairs are genuine matches. Higher precision means fewer labelled non-matches are included in the candidate set.

Example: 0.80 means 80% of labelled candidate pairs are genuine matches.

Reduction Ratio

Higher is better

1 − (Candidate Pairs / Total Possible Pairs)

Measures how much of the original comparison space blocking eliminates. Higher reduction means fewer pairs need expensive similarity comparisons.

Example: 0.90 means blocking eliminated 90% of all possible comparisons.

Understanding the trade-off

A blocking strategy with a very high reduction ratio may eliminate too many genuine matches, lowering candidate recall. A strategy with very high candidate recall may retain many unnecessary pairs, lowering reduction ratio. Evaluate all three metrics together rather than optimizing only one.

All metrics are returned as decimal values between 0 and 1. When a metric's denominator is zero, MatchForge returns 0 for that metric.

Understanding the calculations

Consider two datasets with 3 source records and 3 target records. There are 9 possible source-target pairs. Suppose blocking generates 3 unique candidates.

Possible pairs

9

Candidate pairs

3

Labelled true pairs

2

Assume the labelled data contains 2 genuine matches and 2 non-matches. Blocking retains 1 of the 2 genuine matches, and 2 of the labelled pairs appear in the candidate set.

MetricCalculationResult
Candidate Recall1 / 20.50
Candidate Precision1 / 20.50
Reduction Ratio1 − (3 / 9)0.6667

This example illustrates the metric calculations. Candidate precision uses only labelled pairs that were generated as candidates, not every generated candidate pair.

Basic usage

First, generate candidate pairs using generateCandidates(). Then pass the generated candidates and labelled ground truth to evaluateBlocking().

typescript
import {
  generateCandidates,
  evaluateBlocking,
} from "@matchforge/core";

const source = [
  { name: "Rahul Sharma" },
  { name: "Amit Patel" },
  { name: "Pavan Kapadiya" },
];

const target = [
  { fullName: "Rahul Sharma" },
  { fullName: "Rahul Verma" },
  { fullName: "Amit Patel" },
];

// Generate candidate pairs.
const candidates = generateCandidates(source, target, {
  rules: [
    {
      sourceKey: "name",
      targetKey: "fullName",
      method: "prefix",
      length: 3,
    },
  ],
});

// Labelled source-target pairs.
const groundTruth = [
  { sourceIndex: 0, targetIndex: 0, isMatch: true },
  { sourceIndex: 0, targetIndex: 1, isMatch: false },
  { sourceIndex: 1, targetIndex: 2, isMatch: true },
  { sourceIndex: 2, targetIndex: 2, isMatch: false },
];

// Evaluate candidate generation.
const evaluation = evaluateBlocking({
  candidates,
  groundTruth,
});

console.log(evaluation.metrics);
console.log(evaluation.totalCandidatePairs);

API reference

evaluateBlocking()

Evaluates a candidate-generation result against labelled ground-truth pairs.

evaluateBlocking(options: BlockingEvaluationOptions)

BlockingEvaluationOptions

OptionTypeDescription
candidatesCandidateGenerationResultOutput returned by generateCandidates().
groundTruthGroundTruthPair[]Labelled source-target pairs used for evaluation.

BlockingEvaluationResult

The function returns the calculated metrics and counts used to evaluate candidate generation.

typescript
{
  metrics: {
    candidateRecall: 0.5,
    candidatePrecision: 0.5,
    reductionRatio: 0.6667,
  },
  totalPossiblePairs: 9,
  totalCandidatePairs: 3,
  totalEvaluatedPairs: 4,
  totalLabelledCandidatePairs: 2,
  trueCandidatePairs: 1,
  totalTruePairs: 2,
}
PropertyDescription
metricsCandidate recall, candidate precision, and reduction ratio.
totalPossiblePairsTotal possible source-target record combinations.
totalCandidatePairsNumber of unique candidate pairs generated by blocking.
totalEvaluatedPairsNumber of labelled ground-truth pairs supplied.
totalLabelledCandidatePairsNumber of labelled pairs that were also generated as candidates.
trueCandidatePairsNumber of labelled genuine matches retained as candidates.
totalTruePairsTotal number of labelled genuine matching pairs.

Important considerations

Unlabelled candidate pairs are not false positives

Candidate precision is calculated using labelled candidate pairs only. Unlabelled candidate pairs are not automatically treated as non-matches. This is important when the ground truth contains only a subset of all possible pairs.

Sparse ground truth affects candidate precision

When only a small number of candidate pairs are labelled, candidate precision may not represent the precision across the complete candidate set. Use sufficiently representative labels when comparing blocking strategies.

Missing genuine matches reduce candidate recall

If a labelled genuine match is not generated as a candidate, it is not retained by blocking and contributes to a lower candidate recall.

Multiple rules can increase candidate generation

MatchForge combines multiple blocking rules using union (OR) semantics. A pair generated by more than one rule appears only once. Adding rules may improve recall but can also increase the number of candidates.

Compare configurations using all three metrics

A useful blocking configuration should retain genuine matches while reducing unnecessary comparisons. Consider candidate recall, candidate precision, and reduction ratio together.

NEXT STEP

Experiment with blocking evaluation

Configure blocking rules, inspect generated candidate pairs, and evaluate their recall, precision, and reduction ratio.

Open blocking playground