Classifying FOIA Privacy Exemptions (b)(6) and (b)(7)© in Python

Within FOIA Request Taxonomy Design, the two personal-privacy exemptions — Exemption 6 and Exemption 7© — are the pair most often confused at the review stage, and confusing them produces both wrongful disclosures and unlawful over-redaction. This guide builds a rule-based classifier in Python that distinguishes the two, flags candidate records for a human reviewer, and audit-logs every exemption decision with its statutory rationale. It sits inside the broader Core Architecture & Compliance Mapping model and hands its flags to the PII Redaction Pipelines stage only after a reviewer has applied the balancing test.

The Statutory Distinction

The Freedom of Information Act protects personal privacy through two separate provisions, and their texts differ in a way that is easy to gloss over and consequential to get wrong.

Exemption 6, at 5 U.S.C. § 552(b)(6), covers “personnel and medical files and similar files the disclosure of which would constitute a clearly unwarranted invasion of personal privacy.” The reach of the exemption is broad — courts read “similar files” to cover any government record containing information that applies to a particular individual — but its threshold is demanding. Disclosure must be a clearly unwarranted invasion, and the word “clearly” tips the balance toward disclosure when the privacy and public interests are close.

Exemption 7©, at 5 U.S.C. § 552(b)(7)©, covers “records or information compiled for law enforcement purposes” but only “to the extent that the production of such [records] could reasonably be expected to constitute an unwarranted invasion of personal privacy.” Two things change. First, the record must have a law-enforcement provenance to qualify at all. Second, and critically, the threshold is lower: 7© drops the word “clearly” and asks only whether disclosure could reasonably be expected to constitute an unwarranted invasion. For the same piece of personal information, 7© is the more protective screen — it withholds on a weaker showing of privacy harm than 6 requires.

Both exemptions turn on a balancing test that weighs the individual’s privacy interest against the public interest in disclosure — and that balance is a legal judgment, not a mechanical output. The public-interest side of the scale is itself narrowly defined: the interest that counts is the public’s interest in knowing what its government is up to, not a requester’s private curiosity about a named individual. A record can identify a person and still be disclosable if releasing it would meaningfully illuminate agency conduct, and it can be withheld even when the requester has a compelling personal reason if disclosure reveals nothing about how the government operates. That asymmetry is exactly why the decision cannot be reduced to a lookup table.

The engineering goal, therefore, is narrow and deliberate: a classifier that flags which exemption a reviewer should consider, surfaces the correct statutory threshold, and records the decision — never one that withholds on its own authority. A rule engine can reliably identify that a record names an individual and where it came from; it cannot weigh a privacy interest against a public one. Encoding that boundary into the software — flags on one side, human judgment on the other — is what keeps an automated intake pipeline defensible on administrative appeal.

Architecture Overview

The classifier is a decision tree over three properties of a record: whether it identifies an individual at all, whether it was compiled for a law-enforcement purpose, and whether it is a personnel, medical, or similar file. Law-enforcement provenance is tested first, because when it is present the lower 7© threshold is the more protective screen for the same personal information. Every path ends at a human reviewer who applies the balancing test.

Distinguishing FOIA Exemption 6 from Exemption 7(C) A record proposed for withholding passes a gate testing whether it was compiled for a law-enforcement purpose. If yes, it routes to the Exemption 7(C) balancing test, whose threshold is whether disclosure could reasonably be expected to constitute an unwarranted invasion of personal privacy, a lower bar. If no, a personnel, medical, or similar file routes to the Exemption 6 balancing test, whose threshold is a clearly unwarranted invasion of personal privacy. Both branches converge on a named reviewer who applies the balancing test and records the rationale; the classifier only flags candidates. Record proposed for withholding Compiled for a law-enforcement purpose? no yes Exemption 6 personnel & medical files Balancing test: would disclosure be a clearly unwarranted invasion of privacy? Exemption 7(C) law-enforcement records Balancing test: could reasonably be expected to constitute an unwarranted invasion? (lower bar than b6) REVIEWER applies the balancing test and records the rationale the classifier only flags candidates — it never auto-withholds

The validated taxonomy record that reaches this stage already carries a record class and subject domain, so the classifier reasons over structured inputs rather than raw text. The output it produces is a candidate flag and a statutory threshold, both of which are handed forward for a human to resolve. Keeping the decision tree small and explicit is deliberate: three booleans and two ordered tests are auditable in a way that an opaque scoring model is not, and every branch maps directly to a sentence of statutory text a reviewer can be pointed to. When a record satisfies neither exemption’s predicate, the classifier says so plainly rather than guessing, so a reviewer’s attention is spent only where the statute actually creates a privacy question.

Prerequisites

  • Python 3.11+ for StrEnum-style enums, dataclasses, and datetime.timezone.utc.
  • Standard library only — enum, dataclasses, json, logging, datetime. Keeping the privacy path free of third-party dependencies limits the supply-chain surface on code that decides what the public sees.
  • Structured inputs, not free text. The classifier consumes a small context object describing the record; deriving identifies_individual and compiled_for_law_enforcement upstream is where entity extraction and provenance metadata belong, out of scope here.
  • An append-only audit sink so every exemption decision and its rationale are preserved for administrative appeal, consistent with the FOIA Request Taxonomy Design contract that classification is always logged.

Step-by-Step Implementation

1. Model the exemptions and the record context

Each exemption is a closed enum value, and the record context carries exactly the three booleans the decision tree needs. A Flag value makes explicit that the classifier output is a candidate signal, never a disposition.

python 
from __future__ import annotations

import json
import logging
from dataclasses import dataclass
from datetime import datetime, timezone
from enum import Enum


# Structured JSON audit logging (NIST SP 800-53 AU-3: content of audit records).
class _JsonFormatter(logging.Formatter):
    def format(self, record: logging.LogRecord) -> str:
        line = {
            "ts": datetime.now(timezone.utc).isoformat(),
            "level": record.levelname,
            "event": record.getMessage(),
        }
        for key in ("record_id", "exemption", "disposition", "rationale"):
            if hasattr(record, key):
                line[key] = getattr(record, key)
        return json.dumps(line, sort_keys=True)


logger = logging.getLogger("privacy_exemption_classifier")
logger.setLevel(logging.INFO)
_handler = logging.StreamHandler()
_handler.setFormatter(_JsonFormatter())
logger.addHandler(_handler)


class Exemption(str, Enum):
    B6 = "b6"          # 5 U.S.C. 552(b)(6): personnel, medical, and similar files
    B7C = "b7c"        # 5 U.S.C. 552(b)(7)(C): law-enforcement records, privacy
    NONE = "none"


class Flag(str, Enum):
    CANDIDATE = "candidate"     # a rule-based signal only, NOT a decision
    NO_SIGNAL = "no_signal"


@dataclass
class DocumentContext:
    record_id: str
    is_similar_personal_file: bool     # personnel / medical / "similar file"
    compiled_for_law_enforcement: bool
    identifies_individual: bool

2. Flag the candidate exemption — never withhold

The classifier returns a Candidate carrying the proposed exemption, the statutory threshold the reviewer must apply, and a short rationale. It withholds nothing. Law-enforcement provenance is checked before the “similar file” test so the lower 7© bar screens the same personal information first.

python 
@dataclass
class Candidate:
    record_id: str
    exemption: Exemption
    flag: Flag
    threshold: str          # the statutory standard the reviewer must apply
    rationale: str


def flag_privacy_exemption(doc: DocumentContext) -> Candidate:
    """Rule-based screen that FLAGS a candidate exemption. It never withholds.

    The distinction that matters:
      * (b)(6) reaches personnel, medical, and "similar files" and asks whether
        disclosure would be a *clearly unwarranted* invasion of personal privacy.
      * (b)(7)(C) reaches records compiled for law-enforcement purposes and asks
        whether disclosure *could reasonably be expected to* constitute an
        unwarranted invasion — a lower bar than (b)(6).
    """
    try:
        if not doc.identifies_individual:
            return Candidate(doc.record_id, Exemption.NONE, Flag.NO_SIGNAL,
                             threshold="n/a",
                             rationale="No individual is identifiable in the record.")

        # Law-enforcement provenance routes to 7(C) first: its lower threshold
        # is the more protective screen for the same personal information.
        if doc.compiled_for_law_enforcement:
            return Candidate(
                doc.record_id, Exemption.B7C, Flag.CANDIDATE,
                threshold="could reasonably be expected to constitute an "
                          "unwarranted invasion of personal privacy",
                rationale="Record compiled for law-enforcement purposes and "
                          "identifies an individual; apply 7(C) balancing.")

        if doc.is_similar_personal_file:
            return Candidate(
                doc.record_id, Exemption.B6, Flag.CANDIDATE,
                threshold="clearly unwarranted invasion of personal privacy",
                rationale="Personnel, medical, or similar file identifying an "
                          "individual; apply (b)(6) balancing.")

        return Candidate(doc.record_id, Exemption.NONE, Flag.NO_SIGNAL,
                         threshold="n/a",
                         rationale="Identifies an individual but is neither a "
                                   "similar file nor a law-enforcement record.")
    except Exception as exc:  # fail safe: flag for human review, never release
        logger.error("classifier_error", extra={"record_id": doc.record_id,
                                                 "rationale": repr(exc)})
        return Candidate(doc.record_id, Exemption.NONE, Flag.CANDIDATE,
                         threshold="manual review",
                         rationale=f"Classifier error, routed to reviewer: {exc!r}")

Expected behaviour: a record compiled for law enforcement that identifies an individual returns a B7C candidate even when it is also a personnel file, because the lower threshold governs; a personnel file with no law-enforcement provenance returns a B6 candidate; a record identifying no individual returns NO_SIGNAL. On any internal error the function fails safe by flagging for a human rather than releasing.

3. Record the reviewer’s balancing decision

The balancing test is a human judgment. This function persists the reviewer’s outcome and rationale, audit-logging both. The classifier proposes; a named reviewer disposes.

python 
def record_reviewer_decision(candidate: Candidate, *, withhold: bool,
                             reviewer: str, balancing_note: str) -> dict:
    """Persist the human balancing decision. The classifier proposes; a named
    reviewer disposes. Both the outcome and its rationale are audit-logged
    (NIST SP 800-53 AU-2 events and AU-3 record content).
    """
    disposition = "withheld" if withhold else "released"
    decision = {
        "record_id": candidate.record_id,
        "exemption": candidate.exemption.value,
        "statutory_threshold": candidate.threshold,
        "disposition": disposition,
        "reviewer": reviewer,
        "balancing_note": balancing_note,
    }
    logger.info("privacy_exemption_decision",
                extra={"record_id": candidate.record_id,
                       "exemption": candidate.exemption.value,
                       "disposition": disposition,
                       "rationale": balancing_note})
    return decision

Expected output: a single JSON audit line such as {"disposition": "withheld", "event": "privacy_exemption_decision", "exemption": "b7c", "rationale": "...", "record_id": "R-4", ...}, tying the withholding to a named reviewer and the exact statutory threshold they applied — the record an appeals officer will expect to see.

Validation & Verification

Assert the routing logic and the human-in-the-loop guarantee directly:

python 
def test_law_enforcement_record_routes_to_7c():
    doc = DocumentContext("R-1", is_similar_personal_file=True,
                          compiled_for_law_enforcement=True,
                          identifies_individual=True)
    c = flag_privacy_exemption(doc)
    # Law-enforcement provenance wins: 7(C)'s lower bar screens the same PII.
    assert c.exemption is Exemption.B7C and c.flag is Flag.CANDIDATE


def test_personnel_file_routes_to_b6():
    doc = DocumentContext("R-2", is_similar_personal_file=True,
                          compiled_for_law_enforcement=False,
                          identifies_individual=True)
    assert flag_privacy_exemption(doc).exemption is Exemption.B6


def test_no_individual_is_not_flagged():
    doc = DocumentContext("R-3", is_similar_personal_file=True,
                          compiled_for_law_enforcement=False,
                          identifies_individual=False)
    c = flag_privacy_exemption(doc)
    assert c.flag is Flag.NO_SIGNAL and c.exemption is Exemption.NONE


def test_reviewer_makes_the_final_call():
    doc = DocumentContext("R-4", is_similar_personal_file=False,
                          compiled_for_law_enforcement=True,
                          identifies_individual=True)
    c = flag_privacy_exemption(doc)
    decision = record_reviewer_decision(
        c, withhold=True, reviewer="foia.officer@agency.gov",
        balancing_note="Third-party privacy interest outweighs public interest.")
    assert decision["disposition"] == "withheld"
    assert decision["exemption"] == "b7c"

The verification that matters most to an auditor is the human-in-the-loop invariant: flag_privacy_exemption produces only a Candidate, and no code path lets that candidate become a withholding without a record_reviewer_decision call carrying a named reviewer. Capture the audit stream in a test and confirm every withheld disposition has a matching reviewer and balancing note.

Troubleshooting & Edge Cases

  • Treating the classifier as authoritative. The most dangerous failure is wiring flag_privacy_exemption straight into redaction so a candidate flag auto-withholds. Diagnosis: withholdings in the audit log with no reviewer field. Fix: make record_reviewer_decision the only path to a disposition and reject any redaction request lacking a reviewer and balancing note.
  • Missing the 7© threshold nuance. Applying Exemption 6’s “clearly unwarranted” standard to a law-enforcement record under-protects privacy, because 7© withholds on the lower “could reasonably be expected to” showing. Diagnosis: released law-enforcement records whose privacy interest would have failed only the stricter test. Fix: route law-enforcement provenance to 7© first, as the classifier does, and surface the applicable threshold string to the reviewer so they apply the correct bar.
  • Over-redaction. Reflexively withholding every record that names a person inverts FOIA’s disclosure presumption and invites an appeal. Diagnosis: a withholding rate near 100% on flagged records. Fix: require the balancing note to articulate the specific privacy interest, and treat NO_SIGNAL and released decisions as normal outcomes rather than failures.
  • Law-enforcement provenance that is assumed rather than established. Tagging a record 7© because it merely mentions an investigation stretches the “compiled for law enforcement purposes” predicate. Fix: derive compiled_for_law_enforcement from authoritative provenance metadata upstream, not from keyword matching in the body.
  • Segregability ignored. Withholding an entire document because part of it is exempt violates FOIA’s duty to release reasonably segregable non-exempt portions. Fix: treat the exemption flag as applying to specific fields or passages that the PII Redaction Pipelines stage redacts, releasing the remainder.

Compliance Verification Checklist

FAQ

Why does the classifier check law-enforcement provenance before the personnel-file test?

Because when a record was compiled for a law-enforcement purpose, Exemption 7© applies and its threshold is lower than Exemption 6’s. 7© asks only whether disclosure “could reasonably be expected to” constitute an unwarranted invasion, while 6 requires a “clearly unwarranted” invasion. For the same personal information, routing to 7© first means the more protective standard screens it. A law-enforcement personnel file that might survive the stricter 6 balancing can still be withheld under 7©, so testing provenance first avoids under-protecting privacy.

Can the classifier withhold a record automatically?

No, and that constraint is deliberate. The balancing test that both exemptions require weighs an individual’s privacy interest against the public interest in disclosure, which is a legal judgment a rule engine cannot make. flag_privacy_exemption returns only a candidate flag and the statutory threshold; a withholding exists only after a named reviewer records a decision with a balancing note. Wiring the flag directly into redaction would turn a screening aid into an unlawful automated denial.

What audit trail does an exemption decision leave for an appeal?

Each decision emits one structured JSON line recording the record ID, the exemption applied, the disposition, the named reviewer, and the balancing note explaining why the privacy interest outweighed the public interest. On administrative appeal, that line ties the withholding to a specific statutory threshold and a specific person’s reasoning, which is exactly what an appeals officer needs to evaluate whether the balance was struck correctly.

← Back to all public records automation topics