Skip to content
piighostdocs
Technical docs
On this page

De-identify and restore a text

You have a text with confidential data, and you want to de-identify it, send it to an LLM, then restore the original values in the reply. This guide does the round-trip with the piighost core alone, no model and no optional dependency. The detector's patterns come from the piighost catalog. They are fetched every time the detector is built, which needs network access.

Install the core.

uv add piighost

Do the round-trip

A pipeline chains a detector, a linker, and an anonymizer. Only the detector is required. The linker defaults to ExactEntityLinker and the anonymizer to Anonymizer(LabelCounterPlaceholderFactory()). anonymize returns the de-identified text and the token assigned to each entity. deanonymize replays that mapping in reverse.

import asyncio

from piighost.components.detector import RegexDetector
from piighost.pipeline import AnonymizationPipeline

detector = RegexDetector.from_catalog("catalog:piighost/generic")
pipeline = AnonymizationPipeline(detector)


async def main() -> None:
    result = await pipeline.anonymize("Contact alice@example.com from 192.168.1.42.")
    print(result.text)

    restored = pipeline.deanonymize(result.text, result.tokens)
    print(restored)


asyncio.run(main())

The output should be:

Contact <<EMAIL:1>> from <<IPV4:1>>.
Contact alice@example.com from 192.168.1.42.

result.text carries <<EMAIL:1>> in place of alice@example.com. result.tokens maps each entity to its token. Pass it as-is to deanonymize to recover the original text.

Restore an LLM reply

deanonymize restores any text that carries the tokens, not only the one the pipeline produced. If the LLM answers with <<EMAIL:1>>, put the real values back with the same result.tokens mapping.

async def main() -> None:
    result = await pipeline.anonymize("Contact alice@example.com from 192.168.1.42.")

    llm_reply = "I sent the message to <<EMAIL:1>>."
    print(pipeline.deanonymize(llm_reply, result.tokens))


asyncio.run(main())

The output should be:

I sent the message to alice@example.com.

Group repeated occurrences

A value cited several times gets a single token, so the LLM keeps the thread. ExactEntityLinker groups occurrences by value and label.

from piighost.components.detector import ExactMatchDetector

detector = ExactMatchDetector({"Patrick": "PERSON", "Paris": "LOCATION"})
pipeline = AnonymizationPipeline(detector)


async def main() -> None:
    result = await pipeline.anonymize("Patrick lives in Paris. Patrick loves Paris.")
    print(result.text)


asyncio.run(main())

The output should be:

<<PERSON:1>> lives in <<LOCATION:1>>. <<PERSON:1>> loves <<LOCATION:1>>.

ExactMatchDetector detects fixed literal values. The example therefore stays reproducible without loading a model. For free text, swap it for an NER (named entity recognition) or LLM detector, see the detectors reference.

Change the token shape

LabelCounterPlaceholderFactory, the default factory, produces <<LABEL:N>>. If you want another token shape, pass the pipeline an Anonymizer built on another factory. Here, LabelHashPlaceholderFactory replaces the number with a short digest.

import asyncio

from piighost.components.anonymizer import Anonymizer
from piighost.components.detector import ExactMatchDetector
from piighost.components.placeholder import LabelHashPlaceholderFactory
from piighost.pipeline import AnonymizationPipeline

detector = ExactMatchDetector({"Patrick": "PERSON", "Marie": "PERSON"})
anonymizer = Anonymizer(LabelHashPlaceholderFactory())
pipeline = AnonymizationPipeline(detector, anonymizer=anonymizer)


async def main() -> None:
    result = await pipeline.anonymize("Patrick, Marie, Patrick.")
    print(result.text)


asyncio.run(main())

The output should be:

<<PERSON:09ef3b74>>, <<PERSON:c4912b76>>, <<PERSON:09ef3b74>>.

The digest is computed from the entity's label and rank, never from the value. Patrick therefore keeps the same token at both appearances, and Marie gets another one.

To restore the values, the factory must preserve identity, that is, give each value a distinct token. LabelCounterPlaceholderFactory does. LabelPlaceholderFactory does not, because it gives the same <<PERSON>> to two distinct people. See the placeholder factories page.

See also

Edit this page Report a problem