Data Privacy in AI Systems
GDPR, differential privacy, and practical techniques for privacy-preserving ML
Privacy Risks in ML Systems
ML models are not just consumers of private data — they can become leakers of it. Three categories of attack can extract private information from trained models:
Membership inference attacks
Given a data record, determine whether it was in the training set. Successful attacks have been demonstrated against language models (can you identify whether this person's medical record was used to train this clinical note generator?) and recommendation systems.
Model inversion attacks
Reconstruct training samples from model parameters or outputs. Large language models can reproduce verbatim text from training data, including personally identifiable information, emails, and medical records.
Attribute inference
Given a partially known record and access to model predictions, infer unknown sensitive attributes (HIV status, sexual orientation, income) with higher accuracy than chance.
Differential Privacy in Practice
Differential privacy adds calibrated noise to model training to provide mathematical privacy guarantees. TensorFlow Privacy implements DP-SGD (differentially private stochastic gradient descent).
import tensorflow as tf
from tensorflow_privacy.privacy.optimizers.dp_optimizer_keras import DPKerasSGDOptimizer
# Privacy parameters
l2_norm_clip = 1.0 # Max gradient norm per sample
noise_multiplier = 1.1 # Noise scale (higher = more private, less accurate)
batch_size = 256
num_microbatches = 256 # Should equal batch_size for per-sample gradients
learning_rate = 0.01
# Replace standard optimizer with DP optimizer
optimizer = DPKerasSGDOptimizer(
l2_norm_clip=l2_norm_clip,
noise_multiplier=noise_multiplier,
num_microbatches=num_microbatches,
learning_rate=learning_rate,
)
model.compile(optimizer=optimizer, loss="sparse_categorical_crossentropy")
model.fit(X_train, y_train, epochs=10, batch_size=batch_size)
# Compute the privacy budget spent (epsilon)
from tensorflow_privacy.privacy.analysis import compute_dp_sgd_privacy
epsilon, _ = compute_dp_sgd_privacy(
n=len(X_train), batch_size=batch_size,
noise_multiplier=noise_multiplier, epochs=10, delta=1e-5
)
print(f"Privacy budget: ε = {epsilon:.2f} (lower is more private)")