Blocking & Candidate Generation
Blocking reduces the number of record comparisons by grouping records using selected fields. Instead of comparing every source record against every target record, MatchForge generates a smaller set of candidate pairs for further matching.
Why blocking?
Suppose the source dataset contains N records and the target dataset contains M records. Without blocking, exhaustive comparison requires N × M record comparisons. This becomes expensive as datasets grow.
N × M
Every source record may be compared with every target record.
Fewer pairs
Only pairs sharing at least one blocking key are retained.
Blocking is not the final matching decision. It generates candidate pairs that can subsequently be evaluated using similarity algorithms and matching thresholds.
How candidate generation works
MatchForge generates a blocking key for each record using the selected field and method. Records with matching keys are included as candidate pairs.
SOURCE DATASET
Pavan Kapadiya
Delhi
Rahul Patel
Ahmedabad
TARGET DATASET
Pavan Kapadia
Delhi
Rahul Patel
Ahmedabad
Supported blocking methods
Each rule selects a source field and a target field, then applies one of the following blocking methods.
exactExact
Records are grouped when their normalized field values are identical.
Example: "Delhi" and "delhi" with lowercase normalization
prefixPrefix
Records are grouped when the first configured number of characters match.
Example: "Pavan" and "Pavani" with length: 3
soundexSoundex
Records are grouped when their Soundex phonetic codes match.
Example: "Smith" and "Smyth"
Basic usage
Import generateCandidates() and provide source records, target records, and at least one blocking rule.
import { generateCandidates } from "@matchforge/core";
const source = [
{ name: "Pavan Kapadiya", city: "Delhi" },
{ name: "Rahul Patel", city: "Ahmedabad" },
{ name: "Amit Shah", city: "Mumbai" },
];
const target = [
{ fullName: "Pavan Kapadia", location: "Delhi" },
{ fullName: "Rahul Patel", location: "Ahmedabad" },
{ fullName: "Neha Shah", location: "Mumbai" },
];
const result = generateCandidates(source, target, {
rules: [
{
sourceKey: "city",
targetKey: "location",
method: "exact",
},
],
});
console.log(result.pairs);
console.log(result.stats);Every returned pair contains the original source and target array indexes. Use these indexes to retrieve the corresponding records from the input datasets.
Multiple blocking rules
You can configure multiple rules. MatchForge combines their candidate pairs using OR semantics: a pair is included if it matches at least one rule. Pairs generated by multiple rules are deduplicated.
const result = generateCandidates(source, target, {
rules: [
{
sourceKey: "name",
targetKey: "fullName",
method: "prefix",
length: 3,
},
{
sourceKey: "city",
targetKey: "location",
method: "exact",
},
],
normalization: {
lowercase: true,
trim: true,
normalizeWhitespace: true,
removeAccents: true,
},
});Prefix blocking defaults to a length of 3 when length is not provided. The configured length must be a positive integer. Normalization is applied before generating blocking keys.
Configuration
rulesBlockingRule[]Required. At least one rule.
sourceKeystringSource record field used for the blocking key.
targetKeystringTarget record field used for the blocking key.
method"exact" | "prefix" | "soundex"Blocking method.
lengthnumberOptional prefix length. Defaults to 3.
normalizationNormalizeOptionsOptional normalization applied before key generation.
Available normalization options include lowercase, trim, removePunctuation, removeAccents, and normalizeWhitespace.
Candidate generation statistics
The result contains the generated pairs and statistics describing the reduction in the comparison search space.
console.log(result.stats);
// {
// totalSourceRecords: 3,
// totalTargetRecords: 3,
// totalPossiblePairs: 9,
// candidatePairs: 3,
// reductionPercentage: 66.6666666667
// }totalSourceRecords — Number of records in the source dataset.
totalTargetRecords — Number of records in the target dataset.
totalPossiblePairs — Total possible comparisons: source count × target count.
candidatePairs — Number of unique candidate pairs generated.
reductionPercentage — Percentage of possible comparisons eliminated by blocking.
Use blocking with record matching
Pass blocking configuration directly to matchRecords(). MatchForge generates candidate pairs first and performs matching only for those pairs.
import { matchRecords } from "@matchforge/core";
const result = matchRecords(source, target, {
fields: [
{ sourceKey: "name", targetKey: "fullName" },
{ sourceKey: "city", targetKey: "location" },
],
method: "jaro-winkler",
threshold: 0.8,
blocking: {
rules: [
{
sourceKey: "city",
targetKey: "location",
method: "exact",
},
],
},
});Use blocking with dataset deduplication
For deduplication, use the blocking option in findDuplicates(). Each duplicate blocking rule selects a single field from the dataset.
import { findDuplicates } from "@matchforge/core";
const result = findDuplicates(records, {
fields: [
{ key: "name", weight: 0.7 },
{ key: "city", weight: 0.3 },
],
method: "jaro-winkler",
threshold: 0.8,
blocking: {
rules: [
{ key: "city", method: "exact" },
{ key: "name", method: "prefix", length: 3 },
],
},
});Important considerations
Blocking generates candidate pairs; it does not determine whether two records are a match.
Multiple rules use OR semantics. A pair matching any rule is included.
Duplicate candidate pairs are returned only once.
Missing, null, empty, and normalized-empty blocking values are skipped.
Present blocking field values must be strings. A non-string value causes an error.
A restrictive blocking strategy may exclude genuine matches. Evaluate candidate recall before relying on it.
Candidate indexes refer to the original source and target arrays.
Explore blocking interactively
Configure blocking methods, inspect candidate pairs, and see how blocking changes the number of comparisons.
Open blocking playground