presidio-research

This package features data-science related tasks for developing new recognizers for Presidio. It is used for the evaluation of the entire system, as well as for

What is presidio-research?

This package features data-science related tasks for developing new recognizers for Presidio. It is used for the evaluation of the entire system, as well as for evaluating specific PII recognizers or PII detection models.

Overview of presidio-research

What it is

nPresidio-research is a package that provides evaluation and data-science capabilities for Presidio and PII detection models. It also includes a fake data generator that creates synthetic sentences based on templates and fake PII.nn

Key features

n

  • Fake data generator for PII recognizers and NER modelsn
  • Data representation layer for data generation, modeling and analysisn
  • Model/Recognizer evaluation for Presidio Analyzer and custom Presidio recognizersn
  • Helper functions for results analysisn
  • Data generation from templates with tokenization, tags (IO/BIO/BILUO), and spansn
  • Dataset splitting into train/test/validation sets while avoiding leakage from repeated templatesn
  • Conversion between standardized InputSample format and CoNLL, spaCy v3, and JSONnn

Use cases

n

  • Evaluating how well Presidio detects PII, including vanilla Presidio and custom configurationsn
  • Evaluating specific PII recognizers or PII detection modelsn
  • Generating synthetic sentences to increase coverage of entity values for Named Entity Recognition modelsn
  • Splitting generated datasets into train/test/validation sets while avoiding template leakagen
  • Mapping entities when evaluating multiple models that return different entity setsnn

Getting started

nFrom PyPI:nnshnpip install presidio-evaluatornpython -m spacy download en_core_web_smnpython -m spacy download en_core_web_lgnnnFrom source:nnshnpip install uv nuv sync --extra devnuv run python -m spacy download en_core_web_smnuv run python -m spacy download en_core_web_lgnuv run pytestnnnThe README also describes six notebooks covering data generation, dataset analysis, train/test/validation splitting, evaluation, configuration, and entity mapping.nn

License and pricing

nLicense: MIT. Pricing: open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/data-privacy-stackpresidio-research.svg)](https://yunodb.com/tools/data-privacy-stackpresidio-research?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/data-privacy-stackpresidio-research?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/data-privacy-stackpresidio-research.svg" alt="Featured on YunoDB" height="20"></a>