What is PII Detection?
Also called Personal Data Detection, Sensitive Data Discovery.
PII detection is the automated identification of personally identifiable information inside text, files, or database fields. Techniques combine pattern matching with checksums for structured identifiers, named entity recognition for names and addresses, and contextual classifiers for ambiguous cases. Detection is a prerequisite for redaction, access control, retention enforcement, and breach assessment.
What counts as personal data depends on jurisdiction, and the difference is substantive. United States practice often centers on a list of direct identifiers. Under the General Data Protection Regulation, personal data is any information relating to an identified or identifiable natural person, which can include online identifiers, device identifiers, and location data that a narrow identifier list would miss.
Detection methods have different failure modes. Regular expressions with checksum validation are precise for formats such as payment card or bank account numbers, and blind to anything unformatted. Named entity models catch names, organizations, and places but produce both false positives on common words and false negatives on unusual names and non-Latin scripts. Layering the two is standard.
Quasi-identifiers complicate the picture. A date of birth, a postal code, and a gender are each innocuous alone, yet in combination they can single out an individual within a small population. Detection tuned only to direct identifiers will pass such a record as clean, which is why re-identification risk is assessed on combinations rather than on fields in isolation.
Because detection is imperfect, the threshold is a policy decision. In redaction pipelines a false negative exposes real data, so recall is usually favored over precision, accepting some over-masking. In routing or alerting pipelines an excess of false positives destroys trust in the signal. Rates should be measured on representative data, including the languages and formats actually seen in production.
Key points
- What counts as personal data varies by jurisdiction
- GDPR personal data is broader than a direct identifier list
- Pattern rules are precise, entity models catch unstructured cases
- Quasi-identifiers can re-identify someone in combination
- Threshold is a policy choice: recall for redaction, precision for alerts
In practice
A support pipeline scans inbound messages before they enter an assistant's context. Pattern rules with checksum validation catch card and IBAN numbers reliably. An entity model catches most names and street addresses, but misses transliterated names and treats a product code as a national identifier. Measured on a labeled sample of real tickets, recall on direct identifiers is high while precision on names needs a context rule to be usable.