Skip to content

feat: add RiskAssessment.load() for cross-session reuse#214

Open
frankentini wants to merge 1 commit intoconfident-ai:mainfrom
frankentini:feat/risk-assessment-load
Open

feat: add RiskAssessment.load() for cross-session reuse#214
frankentini wants to merge 1 commit intoconfident-ai:mainfrom
frankentini:feat/risk-assessment-load

Conversation

@frankentini
Copy link
Copy Markdown
Contributor

Summary

Adds a RiskAssessment.load(path) classmethod that deserializes JSON files produced by RiskAssessment.save(), enabling users to reload previous risk assessment results across Python sessions.

This addresses the core ask in #200 — save/load for iterative red-teaming workflows where you want to generate attacks once and re-run them against improved model versions.

What it does

  • RiskAssessment.load(path) — reads a saved JSON file and reconstructs a fully typed RiskAssessment instance
  • Enum restoration — builds a reverse lookup across all VulnerabilityType enum classes to map serialized string values (e.g. "gender") back to their proper enum members (e.g. BiasType.GENDER)
  • Multi-turn support — reconstructs RTTurn objects for multi-turn test cases
  • Graceful fallback — unknown/custom vulnerability type values are preserved as strings rather than raising

Usage

# Save after a red-teaming run
risk_assessment.save('./results/')

# Load in a new session
loaded = RiskAssessment.load('./results/20260402_081100.json')

# Re-inject into a RedTeamer for re-evaluation
red_teamer.test_cases = loaded.test_cases

Tests

Added comprehensive test suite covering:

  • Roundtrip save → load with enum integrity
  • Multi-turn test case reconstruction
  • Multiple vulnerability types in one assessment
  • Error handling (missing file, invalid JSON)
  • Enum reverse-lookup correctness

Closes #200

Add a classmethod to deserialize saved risk assessments from JSON,
enabling users to reload previous results and re-run attacks against
improved models without regenerating test cases.

The implementation:
- Builds a reverse enum lookup across all VulnerabilityType enums
  to restore typed vulnerability_type fields from saved string values
- Reconstructs RTTurn objects for multi-turn test cases
- Falls back gracefully for custom vulnerability types
- Includes comprehensive tests covering roundtrip save/load,
  multi-turn cases, multiple vulnerability types, and error handling

Closes confident-ai#200
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 2, 2026

@frankentini is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Save/load attack datasets for cross-session re-evaluation

1 participant