Screening engines and match resolution

Version the list pipeline

A screening decision depends on the list data used at that time. Track source, retrieval time, publication version where available, integrity checks, and activation time. A download that succeeds does not prove the file was parsed completely or deployed to every worker.

Use a staged activation: fetch, validate, compare, approve according to policy, activate, and confirm coverage. Preserve prior versions for replay and investigation. Alert on stale data, malformed records, unexpected count changes, and inconsistent worker versions. Do not silently continue forever with an old list after an update failure.

List ingestion is a production dependency. The system needs evidence that it obtained the expected release, parsed it correctly, loaded the intended records, and activated the right version. A successful download can still contain an unexpected format or an incomplete file. Compare counts and structural properties, preserve the source version, and define a controlled response when validation fails.

Screening results should identify the list and matching logic used at the decision time. Without that information, a later reviewer may replay an old payment against today’s data and mistake a changed result for an earlier system error. A current rescreen and a historical reconstruction are both useful, but they answer different questions and need separate labels.

Inside the mechanism. Treat a list update as a versioned data release. Retain source, retrieval time, integrity checks, parsed record count, effective version, and deployment status. Validate additions, removals, identifier changes, and failed parsing before activating the update. A successful download is not proof that the live matcher uses the new data. Re-screening and pending decisions need a clear relationship to the active version.

A concrete example. A list release must be retrieved, parsed, validated, activated, and used by production. Each step can fail independently. The matcher returns 296 candidates from 27,500 records. It identifies 77 of 84 known fictional identity matches and misses 7. After scoped suppressions and stale-evidence returns, review demand is 238. The example keeps identity resolution, control availability, and the final legal disposition separate.

When the assumption fails. The monitoring dashboard reports download success while the matcher still uses the old version. Record the active list version on each decision and test activation and rescreen coverage. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.

Follow a worked case3 conditions · 36 figures

A list release must be retrieved, parsed, validated, activated, and used by production. Each step can fail independently.

Versioned list ingestion: reference condition figure preview

01Reference conditionVersioned list ingestion

Versioned list ingestion: failure and stress figure preview

02Failure and stressVersioned list ingestion

Versioned list ingestion: controlled response figure preview

03Controlled responseVersioned list ingestion

Version the list pipeline Version the list pipeline — the flow Follow the sequence. Confirm the version used by workers. Fetch Retrieve the authoritative data Validate Check integrity and record changes Activate Confirm the version used by workers — Follow the sequence. Confirm the version used by workers. Chapter sources · Open image

Version the list pipeline Version the list pipeline — the distinction These concepts answer different questions. Read each definition in the context of the section. Downloaded version File reached the pipeline Active version Matching workers actually use it — These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image

Version the list pipeline List deployment Fictional teaching record. Partial activation remains. List deployment Illustrative data; not a real customer record or a prescribed policy. Fetched v102 New source data Validated v102 Parsing passed Worker group B v101 Partial activation remains A successful download is not deployment completion — Fictional teaching record. Partial activation remains. Chapter sources · Open image

Version the list pipeline Version the list pipeline — control and failure modes A successful download is not deployment completion. The branches show why alternative designs fail. Control design Verify active versions across workers. A successful download is not deployment completion. Failure mode 1 Overwrite the prior list without history. Past decisions become hard to reproduce. avoid Failure mode 2 Ignore stale-list alerts. The control may miss new restrictions. avoid Failure mode 3 Accept an empty parse as a clean list. It may be a pipeline failure. avoid — A successful download is not deployment completion. The branches show why alternative designs fail. Chapter sources · Open image

Normalize without destroying identity

Normalization can standardize spacing, case, punctuation, and script handling. Preserve original names and identifiers. Some transformations improve matching; others collapse distinct names or lose useful information.

Test transliteration, name order, aliases, organization suffixes, and non-Latin scripts using representative fixtures. Keep the normalization version with the result. A matching change can alter both missed matches and false positives. Do not assume English name structure applies everywhere. The review screen should show the original source strings alongside normalized forms so the analyst can inspect the evidence.

Inside the mechanism. Normalization can improve comparison but also destroy useful distinctions. Keep original names and scripts alongside normalized forms. Record the transformations used for punctuation, ordering, transliteration, and token handling. A toy token-overlap score can illustrate similarity but is not a production screening method. Evaluate known variants and confusing nonmatches, including the effect of missing identifiers and common names.

A concrete example. Case, spacing, word order, transliteration, and punctuation can affect matching. Normalization should improve comparison while retaining the original evidence. The matcher returns 177 candidates from 16,800 records. It identifies 43 of 47 known fictional identity matches and misses 4. After scoped suppressions and stale-evidence returns, review demand is 143. The example keeps identity resolution, control availability, and the final legal disposition separate.

When the assumption fails. Aggressive normalization removes distinctions needed to separate unrelated entities. Keep original and normalized values, document the transform, and evaluate known matches and lookalikes. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.

Follow a worked case3 conditions · 36 figures

Case, spacing, word order, transliteration, and punctuation can affect matching. Normalization should improve comparison while retaining the original evidence.

Identity-preserving normalization: reference condition figure preview

01Reference conditionIdentity-preserving normalization

Identity-preserving normalization: failure and stress figure preview

02Failure and stressIdentity-preserving normalization

Identity-preserving normalization: controlled response figure preview

03Controlled responseIdentity-preserving normalization

Normalize without destroying identity Normalize without destroying identity — the flow Follow the sequence. Use aliases and relevant identifiers. Preserve Keep original identity strings Normalize Apply documented transformations Compare Use aliases and relevant identifiers — Follow the sequence. Use aliases and relevant identifiers. Chapter sources · Open image

Normalize without destroying identity Normalize without destroying identity — the distinction These concepts answer different questions. Read each definition in the context of the section. Normalized form Representation for comparison Original form Source evidence with its original detail — These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image

Name processing

Fictional educational excerpt / Not for execution

Name processing

Illustrative data; not a real customer record or a prescribed policy.

Originalaccented source name
Retained evidence
Normalizedstandard comparison form
Search input
Versionnormalizer-v4
Reproducible transformation

Normalization can change identity meaning

Fictional teaching record. Reproducible transformation. Chapter sources · Open image

Normalize without destroying identity Normalize without destroying identity — control and failure modes Normalization can change identity meaning. The branches show why alternative designs fail. Control design Preserve originals and version transformations. Normalization can change identity meaning. Failure mode 1 Discard original script. Review loses useful evidence. avoid Failure mode 2 Assume all names have one first and last name. Name structures vary. avoid Failure mode 3 Tune only on English examples. Coverage may fail for other scripts. avoid — Normalization can change identity meaning. The branches show why alternative designs fail. Chapter sources · Open image

Use more than name similarity

A similarity score ranks candidate strings. It is not the probability that a person is sanctioned unless a separate validated model establishes that meaning. Dates, addresses, identifiers, aliases, and other relevant attributes can help resolve identity.

Define which mismatches are reliable and which fields can be incomplete. A missing birth date is not the same as a conflicting verified birth date. Record the evidence used for resolution and avoid a single unexplained false-positive button. Review decisions should be reproducible from the list record and customer evidence available then.

Inside the mechanism. Identity resolution should use the available identifying attributes and their reliability: names, dates, addresses, identifiers, and other supported facts. Missing data is not a contradiction. A conflicting date may be meaningful only if both dates are reliable and refer to the same attribute. Preserve the analyst’s reasoning and source references. The conclusion should state what was resolved and any remaining limits.

A concrete example. A name candidate becomes useful when reliable identifiers and context help establish whether it is the same subject. Missing attributes leave uncertainty. The matcher returns 236 candidates from 22,400 records. It identifies 57 of 62 known fictional identity matches and misses 5. After scoped suppressions and stale-evidence returns, review demand is 190. The example keeps identity resolution, control availability, and the final legal disposition separate.

When the assumption fails. A similarity score is treated as a complete identity conclusion. Compare reliable attributes under a defined resolution method and preserve conflicting evidence. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.

Follow a worked case3 conditions · 36 figures

A name candidate becomes useful when reliable identifiers and context help establish whether it is the same subject. Missing attributes leave uncertainty.

Attributes beyond the name: reference condition figure preview

01Reference conditionAttributes beyond the name

Attributes beyond the name: failure and stress figure preview

02Failure and stressAttributes beyond the name

Attributes beyond the name: controlled response figure preview

03Controlled responseAttributes beyond the name

Use more than name similarity Use more than name similarity — the flow Follow the sequence. Record the evidence and reason. Candidate Generate plausible matches Identifiers Compare supporting attributes Resolution Record the evidence and reason — Follow the sequence. Record the evidence and reason. Chapter sources · Open image

Use more than name similarity Use more than name similarity — the distinction These concepts answer different questions. Read each definition in the context of the section. Missing identifier No value to compare Conflicting identifier Evidence supports a different value — These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image

Use more than name similarity Match resolution Fictional teaching record. Documented resolution. Match resolution Illustrative data; not a real customer record or a prescribed policy. Name similar Candidate generated Identifier verified different number Distinguishing evidence Reason separate entity Documented resolution Name similarity alone is not a legal identity finding — Fictional teaching record. Documented resolution. Chapter sources · Open image

Use more than name similarity Use more than name similarity — control and failure modes Name similarity alone is not a legal identity finding. The branches show why alternative designs fail. Control design Resolve with identifying evidence. Name similarity alone is not a legal identity finding. Failure mode 1 Call a score a probability without validation. The scale may only rank strings. avoid Failure mode 2 Treat missing as conflicting. Absence is not disagreement. avoid Failure mode 3 Allow unexplained dismissals. The result cannot be reviewed. avoid — Name similarity alone is not a legal identity finding. The branches show why alternative designs fail. Chapter sources · Open image

Control suppression and re-screening

A resolved false match can reduce repeated work if the evidence and list context remain applicable. Suppression should be specific, documented, and reviewable. A blanket allowlist on a common name can hide a future relevant match.

Define when a prior resolution expires or must be reconsidered: changed customer data, changed list record, new ownership evidence, or another relevant trigger. Re-screen the required populations after material updates. Preserve why an alert was suppressed and what evidence supported the reuse. The system must be able to distinguish a reused valid resolution from an alert that was never evaluated.

Suppression reduces repeated work when a prior review has established a valid reason to clear a recurring match. Its scope matters. A clearance for one customer and one list entry should not silently clear every customer with a similar name. Bind the suppression to the evidence, relevant identifiers, reviewer, date, and conditions that invalidate it. Changes to the list, customer identity, or ownership can require a new assessment. The queue becomes quieter for a defensible reason rather than because yesterday’s conclusion became an unlimited exemption.

Inside the mechanism. A suppression needs a subject, matched record, reason, supporting evidence, scope, version, and expiry or review trigger. Do not turn one resolved false positive into a permanent exemption from all future screening. A list change, new identifier, or changed customer fact can invalidate the original basis. Measure both suppressed volume and stale suppression evidence so a quieter queue does not conceal reduced control coverage.

A concrete example. A previous false-match resolution can reduce repeated work within its supported scope. New list or customer facts can invalidate that conclusion. The matcher returns 335 candidates from 31,200 records. It identifies 86 of 93 known fictional identity matches and misses 7. After scoped suppressions and stale-evidence returns, review demand is 269. The example keeps identity resolution, control availability, and the final legal disposition separate.

When the assumption fails. A clearance becomes a permanent global exemption for similar names. Scope suppressions to the subject, list record, evidence, dates, and invalidation triggers. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.

Follow a worked case3 conditions · 36 figures

A previous false-match resolution can reduce repeated work within its supported scope. New list or customer facts can invalidate that conclusion.

Suppression and rescreening: reference condition figure preview

01Reference conditionSuppression and rescreening

Suppression and rescreening: failure and stress figure preview

02Failure and stressSuppression and rescreening

Suppression and rescreening: controlled response figure preview

03Controlled responseSuppression and rescreening

Control suppression and re-screening Control suppression and re-screening — the flow Follow the sequence. Trigger on relevant changes. Resolve Store the specific false-match basis Reuse Apply only within the valid scope Re-screen Trigger on relevant changes — Follow the sequence. Trigger on relevant changes. Chapter sources · Open image

Control suppression and re-screening Control suppression and re-screening — the distinction These concepts answer different questions. Read each definition in the context of the section. Specific suppression Customer-list pair with supporting evidence Blanket allowlist Broad bypass without sufficient scope — These concepts answer different questions. Read each definition in the context of the section. Chapter sources · Open image

Control suppression and re-screening Suppression record Fictional teaching record. Requires reconsideration. Suppression record Illustrative data; not a real customer record or a prescribed policy. Customer entity-72 Specific subject List record target-18 Specific candidate Trigger list identity changed Requires reconsideration New evidence can change the match — Fictional teaching record. Requires reconsideration. Chapter sources · Open image

Control suppression and re-screening Control suppression and re-screening — control and failure modes New evidence can change the match. The branches show why alternative designs fail. Control design Scope and expire reused resolutions. New evidence can change the match. Failure mode 1 Allowlist every similar name. That can hide another person. avoid Failure mode 2 Suppress without an audit record. The decision is not explainable. avoid Failure mode 3 Skip re-screening after relevant changes. The old conclusion may no longer apply. avoid — New evidence can change the match. The branches show why alternative designs fail. Chapter sources · Open image

Test failures and release controls

Screening outages need a defined product response. Mandatory controls cannot be bypassed merely to preserve conversion. A preapproved contingency might stop new releases, queue eligible work, or route to an alternate controlled process, depending on the legal and operational context.

Test timeouts, stale lists, partial updates, queue failures, and unauthorized release attempts. Keep manual release authority limited and logged. Reconcile all transactions held during an outage with their eventual disposition. A system that recovers technically but loses the held population has not recovered operationally.

Inside the mechanism. Test the complete path from input eligibility through matching and disposition to release. Include list unavailability, parser failure, stale caches, queue failure, and unknown decision outcomes. Define the permitted behavior for the affected operation under each failure. A technically available API that serves an obsolete list can be a control failure. Recovery needs evidence that the affected population was screened under the correct version.

A concrete example. The system must represent unavailable screening and unresolved candidates without turning them into false clears. Release authority needs an auditable condition. The matcher returns 223 candidates from 19,800 records. It identifies 65 of 71 known fictional identity matches and misses 6. After scoped suppressions and stale-evidence returns, review demand is 179. The example keeps identity resolution, control availability, and the final legal disposition separate.

When the assumption fails. A dependency timeout returns an empty result that the caller treats as safe. Use explicit pending states, approved failure behavior, and evidence-linked release permissions. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.

Follow a worked case3 conditions · 36 figures

The system must represent unavailable screening and unresolved candidates without turning them into false clears. Release authority needs an auditable condition.