MATCHFORGE DOCUMENTATION

Blocking CSV Evaluation

Evaluate blocking strategies directly from CSV datasets using MatchForge. The CSV evaluation wrapper parses your source and target datasets, generates candidate pairs using configured blocking rules, and evaluates the results against labelled ground truth.

Candidate generation

Generate candidate pairs from two CSV datasets using configurable blocking rules.

Ground-truth evaluation

Compare generated candidates against labelled source-target record pairs.

Performance metrics

Measure candidate recall, candidate precision, and reduction ratio.

01

How it works

evaluateBlockingCSV() provides a convenient way to evaluate blocking without manually parsing CSV files or generating candidate pairs.

01

Provide CSV datasets

Pass the source and target CSV strings to the function.

02

Configure blocking

Define blocking rules and provide labelled ground-truth pairs.

03

Evaluate candidates

Inspect evaluation metrics and candidate-generation statistics.

The CSV wrapper reuses MatchForge's existing candidate-generation and blocking-evaluation logic. It does not introduce a separate blocking algorithm or evaluation metric.

02

Basic usage

Import evaluateBlockingCSV() from @matchforge/core and provide both CSV strings along with the blocking configuration and ground truth.

typescript
import { evaluateBlockingCSV } from "@matchforge/core";

const sourceCSV = `name
Rahul Sharma
Amit Patel`;

const targetCSV = `fullName
Rahul Sharma
Rahul Verma`;

const result = evaluateBlockingCSV(sourceCSV, targetCSV, {
  rules: [
    {
      sourceKey: "name",
      targetKey: "fullName",
      method: "prefix",
      length: 3,
    },
  ],
  groundTruth: [
    { sourceIndex: 0, targetIndex: 0, isMatch: true },
    { sourceIndex: 0, targetIndex: 1, isMatch: false },
    { sourceIndex: 1, targetIndex: 0, isMatch: false },
    { sourceIndex: 1, targetIndex: 1, isMatch: false },
  ],
});

console.log(result.metrics);
console.log(result.totalPossiblePairs);
console.log(result.totalCandidatePairs);
console.log(result.sourceHeaders);
console.log(result.targetHeaders);
console.log(result.parseErrors);

Function arguments

sourceCSV

CSV string containing the source records.

targetCSV

CSV string containing the target records.

options

Blocking configuration, ground-truth pairs, and optional CSV parsing configurations.

03

Blocking rules

Blocking rules determine which source-target record pairs become candidates. Each rule specifies the fields to compare and the blocking method to apply.

MethodDescription
exact

Generates candidates when the selected field values match exactly after normalization.

prefix

Generates candidates using matching prefixes of the configured length.

soundex

Generates candidates using phonetic Soundex encodings.

Rule configuration

typescript
{
  rules: [
    {
      sourceKey: "name",
      targetKey: "fullName",
      method: "prefix",
      length: 3,
    },
  ],
}

sourceKey and targetKey identify the corresponding fields in the source and target records. The length property configures the prefix length for prefix-based rules.

04

Evaluation metrics

Blocking evaluation measures how effectively a blocking strategy reduces comparisons while retaining genuine matching pairs.

Candidate Recall

The proportion of all labelled true pairs retained by blocking.

Candidate Precision

The proportion of labelled candidate pairs that are genuine matches.

Reduction Ratio

The proportion of all possible comparisons eliminated by blocking.

Metric calculations

Candidate Recall

trueCandidatePairs / totalTruePairs

Candidate Precision

trueCandidatePairs / totalLabelledCandidatePairs

Reduction Ratio

1 - (totalCandidatePairs / totalPossiblePairs)

All metrics are returned as decimal values between 0 and 1. When a denominator is zero, the corresponding metric is 0.

05

Understanding the result

The function returns the blocking evaluation result together with CSV metadata from both parsed datasets.

PropertyDescription
metrics

Candidate recall, candidate precision, and reduction ratio.

totalPossiblePairs

Total possible source-target record combinations.

totalCandidatePairs

Number of candidate pairs generated by blocking.

totalEvaluatedPairs

Number of ground-truth pairs supplied.

totalLabelledCandidatePairs

Number of labelled pairs that were generated as candidates.

trueCandidatePairs

Number of labelled true matches retained as candidates.

totalTruePairs

Total number of labelled true matches.

sourceHeaders

Column headers parsed from the source CSV.

targetHeaders

Column headers parsed from the target CSV.

parseErrors

Parsing errors reported separately for source and target CSV inputs.

06

CSV parsing options

The source and target CSV inputs can use separate parsing configurations.

typescript
const result = evaluateBlockingCSV(sourceCSV, targetCSV, {
  rules: [
    {
      sourceKey: "name",
      targetKey: "fullName",
      method: "prefix",
      length: 3,
    },
  ],
  groundTruth,

  sourceParseOptions: {
    // Source CSV parsing configuration
  },

  targetParseOptions: {
    // Target CSV parsing configuration
  },
});

Parsing errors are returned through parseErrors.source and parseErrors.target. Inspect them before relying on the evaluation metrics.

07

Ground-truth indexing

Ground-truth indexes refer to parsed data records, not physical line numbers in the CSV files.

The header row is not included in record indexing.

Record indexes start at 0.

Empty lines skipped by the parser are not counted as records.

sourceIndex identifies a source record, while targetIndex identifies a target record.

Example

csv
name
Rahul Sharma
Amit Patel

Rahul Sharma has index 0, and Amit Patel has index 1. The header row is excluded.

08

Important considerations

Keep these points in mind when interpreting blocking evaluation results.

Unlabelled pairs are not negative pairs

Only explicitly labelled pairs are used for evaluation. Unlabelled candidate pairs are not automatically treated as false positives.

Candidate precision depends on label coverage

With sparse ground truth, candidate precision may not represent the precision across every generated candidate pair.

Blocking affects matching recall

A genuine pair removed during candidate generation cannot be recovered by the matching stage that follows.

Compare different blocking configurations

Use the metrics to understand the trade-off between retaining genuine pairs and reducing unnecessary comparisons.

EXPLORE THE PLAYGROUND

Experience Blocking Evaluation

Try different blocking rules, work with sample datasets, and understand how candidate generation affects evaluation metrics in the interactive Blocking Evaluation playground.