Flattening PDF Layers for Irreversible Redaction Before Disclosure

Within PII Redaction Pipelines, this is the stage where a redaction becomes real. Detection and exemption mapping decide what to withhold, but until the covered content is physically destroyed on a flattened copy, the “redacted” PDF still contains every character underneath the black boxes. This guide builds the burn-in step: removing the underlying text and image layers, rasterizing the redacted pages, stripping metadata and hidden objects, and then verifying — as an audit-logged assertion — that no covered content survives anywhere in the output.

Scenario & Compliance Stakes

The most damaging redaction failures in public-records history share one cause: a black rectangle was drawn over sensitive text, but the text itself was never removed. A PDF is a layered document — a text layer of selectable characters, an image or scan layer, and a collection of hidden objects like annotations, form fields, and metadata. A shape drawn on top of the text layer changes what a human sees on screen while leaving the underlying characters fully intact in the content stream. A requester recovers them in seconds: select-all and copy, run a text-extraction script, or simply delete the annotation. Agencies have exposed Social Security numbers, informant identities, and witness addresses exactly this way, and because a disclosed record cannot be recalled, the breach is permanent.

The legal stakes track the personal-privacy exemptions the redaction serves: 5 U.S.C. § 552(b)(6) and § 552(b)(7)© authorize withholding personal information, but that authority is meaningless if the withheld content ships inside the release copy. Irreversibility is therefore not a nice-to-have — it is what makes the withholding effective. The burn-in step consumes the coordinate-mapped spans produced upstream in PII Redaction Pipelines, including those detected in Redacting SSNs and PII from FOIA PDFs with Microsoft Presidio, and it is the last transformation before the file leaves the agency, so it fails closed on any sign that covered content survived.

Prerequisites

  • Python 3.11+ with pymupdf (imported as fitz), whose apply_redactions primitive deletes the text and image objects inside a redaction rectangle rather than layering a shape over them.
  • Coordinate-mapped redaction spans — the (page, bbox) pairs produced by the detection stage. This step does not decide what to redact; it destroys what it is told to.
  • A structured audit sink for append-only verification records, forwarded in production through Audit Logging Architecture, so the proof that nothing survived is preserved alongside the release.
  • Least-privilege execution, so the worker reads only the working copy and writes only the flattened release copy and its verification record.

Implementation

Burn-in is three deterministic stages: apply the redactions to physically remove content and rasterize each page, strip residual metadata and hidden objects, then verify the output carries no extractable text under any redaction box. The stages are ordered so verification runs last and independently — it does not trust that the earlier steps worked, it proves it.

PDF layer stack flattened to a verified single-layer copy Three input layers stacked on the left — a selectable text layer, an image or scan layer, and hidden objects including annotations and metadata — all feed a flatten and rasterize step that applies redactions to physically remove content and converts the page to pixels. That output passes to a verification step that extracts text under the redaction boxes and confirms the result is empty, producing a single-layer disclosure copy as the final emphasized output. Text layer selectable · copy-paste Image / scan layer rasterized page content Hidden objects annotations · metadata Flatten + rasterize physically remove content Verify: text under redaction boxes = empty Single-layer disclosure copy

1. Apply redactions and rasterize to remove the underlying layers

The core mistake this step exists to prevent is a redaction that only covers. add_redact_annot marks a rectangle, and apply_redactions then deletes every text and image object intersecting it from the page’s content stream. Rasterizing the page afterward — rendering it to a pixmap and rebuilding the page from those pixels — guarantees that no vector text survives beneath the fill, because the flattened page has no text layer at all.

python 
import json
import logging
from datetime import datetime, timezone

import fitz   # pymupdf

# Structured JSON audit logging: irreversibility is a compliance claim, so the
# proof that content was destroyed must be recorded, not assumed.
# 5 U.S.C. § 552(b)(6)/(b)(7)(C): personal-privacy exemptions are only effective
# if the withheld content never ships inside the release copy.
logging.basicConfig(
    format='{"ts":"%(asctime)s","level":"%(levelname)s","msg":"%(message)s"}',
    level=logging.INFO,
)
logger = logging.getLogger("pdf_flatten")


def burn_in_page(page: fitz.Page, boxes: list, dpi: int = 200) -> fitz.Pixmap:
    """Physically remove covered content, then rasterize to a flat pixel layer."""
    for bbox in boxes:
        rect = fitz.Rect(*bbox)
        # add_redact_annot + apply_redactions DELETES the underlying text/image
        # objects — unlike a drawn rectangle, which only hides them.
        page.add_redact_annot(rect, fill=(0, 0, 0))
    # apply_redactions() rewrites the content stream without the covered objects.
    page.apply_redactions()
    # Rasterize: the resulting page has no vector text layer to recover.
    pix = page.get_pixmap(dpi=dpi)
    logger.info("page_flattened %s",
                json.dumps({"boxes": len(boxes), "dpi": dpi}))
    return pix

2. Rebuild the document and strip metadata and hidden objects

A page can be perfectly rasterized while the same name persists in the document’s metadata, an embedded attachment, or a leftover form field. Building the release copy from the flattened pixmaps — a fresh document containing only images — discards the original object tree, and explicitly clearing metadata closes the last channel through which a withheld term could leak.

python 
def build_release_copy(source_path: str, redaction_map: dict,
                       out_path: str) -> str:
    """Return a new single-layer PDF built from flattened, metadata-stripped pages."""
    src = fitz.open(source_path)
    out = fitz.open()
    try:
        for page_no in range(src.page_count):
            boxes = redaction_map.get(page_no, [])
            pix = burn_in_page(src[page_no], boxes)
            # Insert the flattened image as the entire new page: no text layer,
            # no annotations, no form fields carry over from the source.
            new_page = out.new_page(width=pix.width, height=pix.height)
            new_page.insert_image(new_page.rect, pixmap=pix)
        # Wipe document metadata — author, title, and producer fields routinely
        # carry names that must not survive into a disclosure copy.
        out.set_metadata({})
        out.del_xml_metadata()
        out.save(out_path, garbage=4, deflate=True)
        logger.info("release_built %s",
                    json.dumps({"pages": out.page_count, "out": out_path}))
        return out_path
    finally:
        src.close()
        out.close()

3. Verify no covered content survives

Verification is the assertion that makes irreversibility a fact rather than a hope. Re-open the release copy, extract text within each redaction rectangle, and confirm it is empty; extract the full-document text and confirm no withheld term appears anywhere. A failure here must block the release, because a copy that passes visual inspection but yields text under a box is the exact breach the whole pipeline exists to prevent.

python 
def verify_irreversible(out_path: str, redaction_map: dict,
                        withheld_terms: list) -> bool:
    """Assert no text survives under any box and no withheld term survives anywhere."""
    doc = fitz.open(out_path)
    try:
        for page_no, boxes in redaction_map.items():
            page = doc[page_no]
            for bbox in boxes:
                # get_text clipped to the redaction rect must return nothing:
                # a non-empty result means content survived the burn-in.
                residual = page.get_text("text", clip=fitz.Rect(*bbox)).strip()
                if residual:
                    logger.error("verify_failed page=%s reason=text_under_box", page_no)
                    return False
        full_text = "".join(doc[p].get_text("text") for p in range(doc.page_count))
        for term in withheld_terms:
            if term and term in full_text:
                # Metadata/attachment/vector-text leak: fail closed, do not ship.
                logger.error("verify_failed reason=term_survived")
                return False
        logger.info("verify_passed %s",
                    json.dumps({"pages": doc.page_count, "ts":
                               datetime.now(timezone.utc).isoformat()}))
        return True
    finally:
        doc.close()

Expected Output & Verification

A clean run emits a flatten line per page, a build line, and a single pass record:

text 
{"ts":"2026-07-12 11:04:19","level":"INFO","msg":"page_flattened {\"boxes\": 4, \"dpi\": 200}"}
{"ts":"2026-07-12 11:04:20","level":"INFO","msg":"release_built {\"pages\": 12, \"out\": \"release_2024-CR-01892.pdf\"}"}
{"ts":"2026-07-12 11:04:21","level":"INFO","msg":"verify_passed {\"pages\": 12}"}

Assert irreversibility directly in a test so a regression that reintroduces a recoverable layer fails CI rather than a live disclosure:

python 
def test_no_text_survives_under_redaction_box():
    src = fitz.open()
    page = src.new_page()
    page.insert_text((72, 72), "SSN 123-45-6789", fontsize=12)
    src.save("/tmp/_pii_src.pdf")
    src.close()
    redaction_map = {0: [[70, 60, 220, 84]]}
    build_release_copy("/tmp/_pii_src.pdf", redaction_map, "/tmp/_pii_out.pdf")
    # The withheld string must not be recoverable anywhere in the output.
    assert verify_irreversible("/tmp/_pii_out.pdf", redaction_map,
                              ["123-45-6789"]) is True

Common Pitfalls

  1. Copy-paste recovery from a covered-only redaction. The classic breach: a rectangle is drawn but apply_redactions is never called, so the text underneath survives and copy-paste reveals it. Diagnosis: get_text clipped to a redaction box returns the “hidden” content. Fix: never treat an annotation as a redaction — always call apply_redactions and rasterize, and let the verification step prove the box is empty before release.
  2. Residual metadata. The page is flattened but the document’s author, title, or producer fields still carry a name, or an embedded attachment retains the original text. Diagnosis: a withheld term survives the full-document text scan even though every box verifies empty. Fix: rebuild the release from flattened images, call set_metadata({}) and del_xml_metadata(), and include the full-text term scan in verification rather than checking only under the boxes.
  3. Vector text under an image layer. A scanned page can still carry an invisible OCR text layer positioned beneath the image, so rasterizing the visible image without removing that layer leaves recoverable characters. Diagnosis: text extraction returns content that is not visible on the rendered page. Fix: build the output purely from pixmaps so no source text layer carries over, and verify with a full get_text pass that the release yields only the text you intend — which for a fully rasterized copy is none.

FAQ

Why rasterize the page instead of just deleting the text objects?

Because deletion alone can leave recoverable traces, and rasterization removes the entire class of risk. apply_redactions removes the objects inside each box, but a document can still carry vector text, invisible OCR layers, or overlapping objects elsewhere on the page. Rebuilding the page from a flattened pixmap produces a page that has no text layer at all, so there is nothing under any box to recover. Rasterization trades selectable text and file size for a guarantee of irreversibility, which is the correct trade for a disclosure copy.

How do we know the redaction is actually irreversible?

By proving it, not assuming it. After building the release copy, the verification step extracts text clipped to every redaction rectangle and asserts each is empty, then scans the full document text — including what would surface from metadata or a stray layer — for every withheld term. Only when both checks pass does the copy become releasable, and the pass is recorded as an audit event. A copy that yields text under a box or a surviving withheld term is blocked, because that is the exact breach the pipeline exists to prevent.

Does flattening remove metadata and hidden objects too?

Not by itself — you have to strip them explicitly. Rasterizing pages handles the visible content, but document metadata, embedded attachments, and form fields live outside the page pixels and can retain a withheld name. Building the release from flattened images discards the original object tree, and clearing metadata with set_metadata and del_xml_metadata closes the remaining channels. The full-document term scan in verification is what confirms nothing leaked through metadata or a hidden object.

← Back to all public records automation topics