---
icon: lucide/gavel
tags:
  - Advanced
  - Detector
---

# Deny and allow lists

Your detector reads your company name as a person and you want that name left alone. Your internal codenames go undetected and you want them replaced every time. Both are decisions about the detection set rather than about the detector. `DetectionOverride` is the stage that imposes them, with two detectors. The deny list (`deny_list`) holds what is always masked, and its detector's hits are forced into the set. The allow list (`allow_list`) holds what is always left in clear, and its detector's hits are dropped from the set.

The stage runs right after detection, before overlap resolution and linking. Its two lists therefore trump the detector's reading, and also a corrected set coming back from a human review. See [Architecture](../architecture.md) for the full stage order.

!!! note "Prerequisites"
    `piighost` alone, `pip install piighost`. Every example below runs as is, with no model download. Section 2 and the config file pull the generic group of the [piighost catalog](https://catalog.piighost.dev), fetched every time the pipeline is built, which needs network access. The last section reads a config file, which needs the config extra, `pip install "piighost[config]"`.

!!! note "Renamed in 2.0"
    The deny list was called the whitelist before `piighost` 2.0, and the allow list the blacklist. A config that still uses the old names is refused at load time, see [Upgrading to 2.0](../community/upgrading.md#the-override-lists-are-renamed).

## 1. Keep a value in clear with an allow list

Point a detector at the value, hand it to `DetectionOverride` as the allow list, and pass the override to the pipeline. What the allow list finds leaves the detection set, so the value reaches the model in clear.

```python
import asyncio

from piighost.components.detector import ExactMatchDetector
from piighost.components.override import DetectionOverride
from piighost.pipeline import AnonymizationPipeline

detector = ExactMatchDetector({"Emma": "PERSON", "Acme": "ORG"})
allow_list = ExactMatchDetector({"Acme": "ORG"})
override = DetectionOverride(allow_list=allow_list)
pipeline = AnonymizationPipeline(detector, override=override)


async def main() -> None:
    result = await pipeline.anonymize("Emma works at Acme.")
    print(result.text)


asyncio.run(main())
```

The output should be:

```text
<<PERSON:1>> works at Acme.
```

`allow_list_strategy` decides which detections an allow list hit takes down.

- Keep `AllowListStrategy.VALUE`, the default, when the value must never be de-identified whatever the detector calls it. It clears every detection carrying the same text, whatever its case, position and label. The label you write beside the value therefore never has to match the one the primary detector emits.
- Use `AllowListStrategy.EXACT` when the label is the point, and both detectors read the value the same way. It clears a detection only when its span and its label both match the hit.
- Use `AllowListStrategy.OVERLAP` when a longer detection containing the value must go down too. It clears any detection whose span touches a span on the allow list, labels ignored.

The three modes on one text, with a detector that mislabels `Acme`{ .pii } as a person and reads `Globex Ltd`{ .pii } as one organization.

```python
import asyncio

from piighost.components.detector import ExactMatchDetector
from piighost.components.override import AllowListStrategy, DetectionOverride
from piighost.pipeline import AnonymizationPipeline


def build_pipeline(strategy: AllowListStrategy) -> AnonymizationPipeline:
    detector = ExactMatchDetector(
        {"Emma": "PERSON", "Acme": "PERSON", "Globex Ltd": "ORG"}
    )
    allow_list = ExactMatchDetector({"Acme": "ORG", "Globex": "ORG"})
    override = DetectionOverride(allow_list=allow_list, allow_list_strategy=strategy)
    return AnonymizationPipeline(detector, override=override)


async def main() -> None:
    text = "Emma works at Acme, formerly Globex Ltd."
    for strategy in AllowListStrategy:
        pipeline = build_pipeline(strategy)
        result = await pipeline.anonymize(text)
        print(strategy.value, "->", result.text)


asyncio.run(main())
```

The output should be:

```text
exact -> <<PERSON:1>> works at <<PERSON:2>>, formerly <<ORG:1>>.
value -> <<PERSON:1>> works at Acme, formerly <<ORG:1>>.
overlap -> <<PERSON:1>> works at Acme, formerly Globex Ltd.
```

`EXACT` matched neither detection, for two reasons. The allow list says `Acme`{ .pii } is an organization, where the detector says a person. And the detector's span covers `Globex Ltd`{ .pii }, where the allow list covers `Globex`{ .pii } alone. `VALUE` compares whole values, so it cleared `Acme`{ .pii } and left `Globex Ltd`{ .pii }, whose text is not the one on the allow list. `OVERLAP` cleared both, because the span on the allow list sits inside the longer detection.

!!! note "A value on the allow list does not trip the guard rail"
    A [guard rail](../reference/guard-rails.md) re-reads the output and refuses residual confidential data. The pipeline hands it the values the allow list matched in the text, so a value you deliberately left in clear is exempt. Any other leak still raises `PIIRemainingError`.

## 2. Force a missed value with a deny list

Point a detector at the pattern the primary detector misses, here a codename a regex describes exactly, and hand it over as the deny list. Its hits enter the detection set whatever the primary detector saw.

```python
import asyncio

from piighost.components.detector import RegexDetector
from piighost.components.override import DetectionOverride
from piighost.pipeline import AnonymizationPipeline

detector = RegexDetector.from_catalog("catalog:piighost/generic")
deny_list = RegexDetector({"CODENAME": r"ACME-[A-Z]+"})
override = DetectionOverride(deny_list=deny_list)
pipeline = AnonymizationPipeline(detector, override=override)


async def main() -> None:
    result = await pipeline.anonymize("Ship ACME-FALCON to alice@example.com.")
    print(result.text)


asyncio.run(main())
```

The output should be:

```text
Ship <<CODENAME:1>> to <<EMAIL:1>>.
```

A forced hit also replaces every detection it overlaps, so the deny list label wins over the primary reading. Use that to correct a label, not only to add a detection.

```python
from piighost.components.detector import ExactMatchDetector

detector = ExactMatchDetector({"Emma": "PERSON", "Acme": "PERSON"})
deny_list = ExactMatchDetector({"Acme": "ORG"})
override = DetectionOverride(deny_list=deny_list)
pipeline = AnonymizationPipeline(detector, override=override)


async def main() -> None:
    result = await pipeline.anonymize("Acme hired Emma.")
    print(result.text)


asyncio.run(main())
```

The output should be:

```text
<<ORG:1>> hired <<PERSON:1>>.
```

A forced value goes through linking and token assignment like any other detection, so the conversational pipeline stores it in memory and `deanonymize` restores it.

## 3. De-identify a value the assistant introduced

In a thread, a value the assistant wrote first stays in clear even when the deny list matches it. The model produced that value because it was useful in context, and it does not know the value is confidential. Replacing it would strip the model's world knowledge, and signal that this precise value is sensitive. `deny_list_strategy` decides who wins.

- Keep `DenyListStrategy.RESPECT_PROVENANCE`, the default, to leave an assistant-introduced value in clear. The deny list still guarantees the value is detected, and the same value introduced by the user is still tokenized.
- Use `DenyListStrategy.FORCE` to tokenize a value on the deny list whoever wrote it first.

```python
import asyncio

from piighost.components.detector import ExactMatchDetector
from piighost.components.override import DenyListStrategy, DetectionOverride
from piighost.conversation_memory import MessageRole
from piighost.pipeline import ThreadAnonymizationPipeline


def build_pipeline(strategy: DenyListStrategy) -> ThreadAnonymizationPipeline:
    detector = ExactMatchDetector({})
    deny_list = ExactMatchDetector({"Acme": "ORG"})
    override = DetectionOverride(deny_list=deny_list, deny_list_strategy=strategy)
    return ThreadAnonymizationPipeline(detector, override=override)


async def main() -> None:
    for strategy in DenyListStrategy:
        pipeline = build_pipeline(strategy)
        assistant = await pipeline.anonymize(
            "Acme rocks", thread_id="t1", role=MessageRole.ASSISTANT
        )
        user = await pipeline.anonymize("I love Acme", thread_id="t1")
        print(strategy.value, "->", assistant.text, "|", user.text)


asyncio.run(main())
```

The output should be:

```text
respect_provenance -> Acme rocks | I love Acme
force -> <<ORG:1>> rocks | I love <<ORG:1>>
```

## 4. Decide who wins when the two lists contradict

A value both lists match is a contradiction, and `conflict_strategy` names the winner.

- Keep `OverrideConflictStrategy.DENY_LIST_WINS`, the default, to de-identify the contradicted value. The allow list applies to the primary detections first, then the deny list is forced in last.
- Use `OverrideConflictStrategy.ALLOW_LIST_WINS` to keep it in clear. The deny list is forced in first, then the allow list clears the result, forced hits included.
- Use `OverrideConflictStrategy.RAISE` to refuse the contradiction. A span on the deny list overlapping one on the allow list raises `ConflictingOverrideError` before either list is applied.

```python
import asyncio

from piighost.components.detector import ExactMatchDetector
from piighost.components.override import DetectionOverride, OverrideConflictStrategy
from piighost.exceptions import ConflictingOverrideError
from piighost.pipeline import AnonymizationPipeline


def build_pipeline(strategy: OverrideConflictStrategy) -> AnonymizationPipeline:
    detector = ExactMatchDetector({"Emma": "PERSON"})
    deny_list = ExactMatchDetector({"Acme": "ORG"})
    allow_list = ExactMatchDetector({"Acme": "ORG"})
    override = DetectionOverride(
        deny_list=deny_list,
        allow_list=allow_list,
        conflict_strategy=strategy,
    )
    return AnonymizationPipeline(detector, override=override)


async def main() -> None:
    for strategy in OverrideConflictStrategy:
        pipeline = build_pipeline(strategy)
        try:
            result = await pipeline.anonymize("Emma works at Acme.")
        except ConflictingOverrideError as error:
            print(strategy.value, "->", type(error).__name__, error)
        else:
            print(strategy.value, "->", result.text)


asyncio.run(main())
```

The output should be:

```text
deny_list_wins -> <<PERSON:1>> works at <<ORG:1>>.
allow_list_wins -> <<PERSON:1>> works at Acme.
raise -> ConflictingOverrideError Overrides contradict each other on 'Acme': a span on the deny list overlaps one on the allow list.
```

`ALLOW_LIST_WINS` clears a forced hit according to the allow list strategy. With the default `VALUE`, a forced value is therefore cleared whatever label the deny list attached to it. Under `EXACT` the two lists have to agree on the label for the allow list to win.

## 5. Drive both lists from a config file

Both lists are detector configs, `[override.deny_list]` and `[override.allow_list]`, and the three strategies are keys of `[override]`. The file below forces the codename and keeps a public mailbox in clear.

```toml
[detector]
type = "regex"
catalogs = ["catalog:piighost/generic"]

[override]
allow_list_strategy = "value"

[override.deny_list]
type = "regex"
patterns = { CODENAME = 'ACME-[A-Z]+' }

[override.allow_list]
type = "exact"
values = { "public@corp.com" = "EMAIL" }
```

`load_pipeline` parses the file and builds the pipeline, both lists included.

```python
import asyncio

from piighost.config import load_pipeline

pipeline = load_pipeline("pipeline.toml")


async def main() -> None:
    result = await pipeline.anonymize(
        "Mail public@corp.com or alice@example.com about ACME-FALCON."
    )
    print(result.text)


asyncio.run(main())
```

The output should be:

```text
Mail public@corp.com or <<EMAIL:1>> about <<CODENAME:1>>.
```

For every key and every accepted value, see the [TOML configuration](../configuration/toml.md).

## See also

- [Pre-built detectors](detectors.md) for the detectors the two lists are built on.
- [Pipeline reference](../reference/pipeline.md) for the `override` parameter and the stage order.
- [Guard rails](../reference/guard-rails.md) for the output check the allow list exempts a value from.
- [TOML configuration](../configuration/toml.md) for the `[override]` keys.
- [Impose a deny list and an allow list](../../../openwiki/en/processes/impose-a-whitelist-and-blacklist.md), for the business rules of the two lists, `BR-LIST-01` to `BR-LIST-08`.