Blocking evaluation
Measure how effectively your blocking strategy reduces record comparisons without losing genuine matches. MatchForge provides evaluateBlocking() to evaluate candidate generation against labelled ground truth.
What is blocking evaluation?
Blocking reduces the number of record pairs that need to be compared. Instead of comparing every source record against every target record, it generates a smaller set of candidate pairs using selected rules such as exact matching, prefix matching, or Soundex.
Blocking in action
From comparisons to candidates
Comparisons eliminated
66.67%
Comparisons retained
33.33%
Illustrative visualization: 9 possible pairs reduced to 3 candidate pairs. The animation demonstrates the concept, not a live execution of the matching algorithm.
However, reducing comparisons is not enough. A blocking strategy must also retain genuine matching pairs. Blocking evaluation measures this trade-off using three metrics: candidate recall, candidate precision, and reduction ratio.
Evaluation workflow
Blocking evaluation follows four steps, from generating all possible comparisons to measuring the quality of the resulting candidates.
All possible pairs
Every source record can be compared with every target record.
N × M comparisons
Blocking rules
Generate keys and group records using selected rules.
Exact · Prefix · Soundex
Candidate pairs
Keep only pairs that share at least one blocking key.
Unique candidate pairs
Evaluate quality
Compare candidates against labelled ground truth.
Recall · Precision · Reduction
Ground truth
Ground truth is a collection of labelled source-target pairs. Each pair identifies two records using their original dataset indexes and specifies whether they represent the same entity. The same format is used by matching evaluation.
const groundTruth = [
{
sourceIndex: 0,
targetIndex: 0,
isMatch: true,
},
{
sourceIndex: 0,
targetIndex: 1,
isMatch: false,
},
{
sourceIndex: 1,
targetIndex: 2,
isMatch: true,
},
];| Property | Description |
|---|---|
| sourceIndex | Zero-based index of the record in the source dataset. |
| targetIndex | Zero-based index of the record in the target dataset. |
| isMatch | True if the pair represents the same entity; otherwise, false. |
Ground-truth indexes must refer to the original source and target dataset positions. Each pair must be labelled only once. Duplicate labelled pairs are rejected.
Blocking evaluation metrics
MatchForge calculates three metrics. Together, they show whether a blocking strategy retains genuine matches while reducing the comparison workload.
Candidate Recall
Higher is better
True Candidate Pairs / Total True PairsMeasures how many genuine matching pairs survive candidate generation. Higher recall means fewer actual matches are discarded during blocking.
Example: 0.90 means 90% of labelled genuine matches were retained.
Candidate Precision
Higher is better
True Candidate Pairs / Labelled Candidate PairsMeasures how many labelled candidate pairs are genuine matches. Higher precision means fewer labelled non-matches are included in the candidate set.
Example: 0.80 means 80% of labelled candidate pairs are genuine matches.
Reduction Ratio
Higher is better
1 − (Candidate Pairs / Total Possible Pairs)Measures how much of the original comparison space blocking eliminates. Higher reduction means fewer pairs need expensive similarity comparisons.
Example: 0.90 means blocking eliminated 90% of all possible comparisons.
Understanding the trade-off
A blocking strategy with a very high reduction ratio may eliminate too many genuine matches, lowering candidate recall. A strategy with very high candidate recall may retain many unnecessary pairs, lowering reduction ratio. Evaluate all three metrics together rather than optimizing only one.
All metrics are returned as decimal values between 0 and 1. When a metric's denominator is zero, MatchForge returns 0 for that metric.
Understanding the calculations
Consider two datasets with 3 source records and 3 target records. There are 9 possible source-target pairs. Suppose blocking generates 3 unique candidates.
Possible pairs
9
Candidate pairs
3
Labelled true pairs
2
Assume the labelled data contains 2 genuine matches and 2 non-matches. Blocking retains 1 of the 2 genuine matches, and 2 of the labelled pairs appear in the candidate set.
| Metric | Calculation | Result |
|---|---|---|
| Candidate Recall | 1 / 2 | 0.50 |
| Candidate Precision | 1 / 2 | 0.50 |
| Reduction Ratio | 1 − (3 / 9) | 0.6667 |
This example illustrates the metric calculations. Candidate precision uses only labelled pairs that were generated as candidates, not every generated candidate pair.
Basic usage
First, generate candidate pairs using generateCandidates(). Then pass the generated candidates and labelled ground truth to evaluateBlocking().
import {
generateCandidates,
evaluateBlocking,
} from "@matchforge/core";
const source = [
{ name: "Rahul Sharma" },
{ name: "Amit Patel" },
{ name: "Pavan Kapadiya" },
];
const target = [
{ fullName: "Rahul Sharma" },
{ fullName: "Rahul Verma" },
{ fullName: "Amit Patel" },
];
// Generate candidate pairs.
const candidates = generateCandidates(source, target, {
rules: [
{
sourceKey: "name",
targetKey: "fullName",
method: "prefix",
length: 3,
},
],
});
// Labelled source-target pairs.
const groundTruth = [
{ sourceIndex: 0, targetIndex: 0, isMatch: true },
{ sourceIndex: 0, targetIndex: 1, isMatch: false },
{ sourceIndex: 1, targetIndex: 2, isMatch: true },
{ sourceIndex: 2, targetIndex: 2, isMatch: false },
];
// Evaluate candidate generation.
const evaluation = evaluateBlocking({
candidates,
groundTruth,
});
console.log(evaluation.metrics);
console.log(evaluation.totalCandidatePairs);API reference
evaluateBlocking()
Evaluates a candidate-generation result against labelled ground-truth pairs.
evaluateBlocking(options: BlockingEvaluationOptions)
BlockingEvaluationOptions
| Option | Type | Description |
|---|---|---|
| candidates | CandidateGenerationResult | Output returned by generateCandidates(). |
| groundTruth | GroundTruthPair[] | Labelled source-target pairs used for evaluation. |
BlockingEvaluationResult
The function returns the calculated metrics and counts used to evaluate candidate generation.
{
metrics: {
candidateRecall: 0.5,
candidatePrecision: 0.5,
reductionRatio: 0.6667,
},
totalPossiblePairs: 9,
totalCandidatePairs: 3,
totalEvaluatedPairs: 4,
totalLabelledCandidatePairs: 2,
trueCandidatePairs: 1,
totalTruePairs: 2,
}| Property | Description |
|---|---|
| metrics | Candidate recall, candidate precision, and reduction ratio. |
| totalPossiblePairs | Total possible source-target record combinations. |
| totalCandidatePairs | Number of unique candidate pairs generated by blocking. |
| totalEvaluatedPairs | Number of labelled ground-truth pairs supplied. |
| totalLabelledCandidatePairs | Number of labelled pairs that were also generated as candidates. |
| trueCandidatePairs | Number of labelled genuine matches retained as candidates. |
| totalTruePairs | Total number of labelled genuine matching pairs. |
Important considerations
Unlabelled candidate pairs are not false positives
Candidate precision is calculated using labelled candidate pairs only. Unlabelled candidate pairs are not automatically treated as non-matches. This is important when the ground truth contains only a subset of all possible pairs.
Sparse ground truth affects candidate precision
When only a small number of candidate pairs are labelled, candidate precision may not represent the precision across the complete candidate set. Use sufficiently representative labels when comparing blocking strategies.
Missing genuine matches reduce candidate recall
If a labelled genuine match is not generated as a candidate, it is not retained by blocking and contributes to a lower candidate recall.
Multiple rules can increase candidate generation
MatchForge combines multiple blocking rules using union (OR) semantics. A pair generated by more than one rule appears only once. Adding rules may improve recall but can also increase the number of candidates.
Compare configurations using all three metrics
A useful blocking configuration should retain genuine matches while reducing unnecessary comparisons. Consider candidate recall, candidate precision, and reduction ratio together.
NEXT STEP
Experiment with blocking evaluation
Configure blocking rules, inspect generated candidate pairs, and evaluate their recall, precision, and reduction ratio.
Open blocking playground