Generate OSCAL catalog release artifact for SCuBA baselines (#1058)
Generate OSCAL Baseline Catalog / generate-oscal (push) Has been cancelled
Run Link Checker for Cron Job / markdown-link-check (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Pylint (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run OPA Tests (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run Markdown Link Check (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run Python Tests (push) Has been cancelled
Run Smoke Test / configuration (push) Has been cancelled
Scorecard supply-chain security / Scorecard analysis (push) Has been cancelled
Run Smoke Test / smoke-test (push) Has been cancelled
Run Smoke Test / lhci (push) Has been cancelled

* Add OSCAL baseline generation release artifacts

* Update generate_oscal_baselines.yml

updated trigger for this branch also

* Update generate_oscal_baselines.yml

remove temporary testing of run on the branch

* Adjust OSCAL baseline workflow triggers

* Generate OSCAL catalog release artifact

Add a generator that converts all README-listed baseline Markdown files into a single OSCAL catalog JSON artifact. Include the OSCAL version in metadata and filenames, validate generated catalogs in GitHub Actions with oscal-cli, and add release workflow packaging for the catalog artifact.

Document the Markdown-to-OSCAL cross-walk and preserve baseline hierarchy so section-level prose is represented once while policy-specific content remains on each control.

Closes #1004

* Validate OSCAL catalog generation with pinned CLI

Use OSCAL-valid catalog part names while preserving Markdown labels in part titles for human-readable rendering and future reversibility.

Pin OSCAL CLI validation in the OSCAL artifact and release workflows, replace the floating container image, and document the part-name mapping assumptions.

Also de-duplicate generated control links and extend generator tests for allowed part names and unique links.

* Use available setup-java version for OSCAL workflow

* Order OSCAL parts for readability

* Order OSCAL properties for readability

* Use Node 24 artifact upload action

* Improve OSCAL catalog cross-walk readability

* Add collapsible OSCAL catalog excerpt to docs

* Address OSCAL catalog review feedback

Add catalog viewing guidance, clarify ad hoc metadata versioning, and resolve Markdown-relative links copied into generated OSCAL prose.

Validate generated catalog behavior with focused unit tests.

* Ignore OSCAL viewer link in markdown checker

Add a narrow markdown-link-check ignore pattern for viewer.oscal.io because the checker cannot validate its certificate chain.

* Verify OSCAL CLI download checksum

* Add reusable OSCAL CLI setup action

* Add OSCAL catalog validation utility
This commit is contained in:
Eugene Aronne authored and GitHub committed 2026-09-28 15:02:20 -04:00
1 parent 95039bdd34
commit f21b3c7988
14 files changed
+2063 -1

No files matched your search

@@ -0,0 +1,28 @@
name: Setup OSCAL CLI
description: Set up Java and install a checksum-verified OSCAL CLI.
inputs:
oscal-cli-version:
required: false
default: "3.2.0"
oscal-cli-sha256:
required: false
default: "e001b353ee245da6f4ff0ad930c4dfaa88083c63243ca004bd5f5bcef44e8ec8"
runs:
using: "composite"
steps:
- name: Set up Java
uses: actions/setup-java@v5
with:
distribution: temurin
java-version: "17"
- name: Install OSCAL CLI
shell: bash
run: |
OSCAL_CLI_ZIP="$RUNNER_TEMP/oscal-cli.zip"
curl --fail --location --retry 3 --output "$OSCAL_CLI_ZIP" \
"https://repo1.maven.org/maven2/dev/metaschema/oscal/oscal-cli-enhanced/${{ inputs.oscal-cli-version }}/oscal-cli-enhanced-${{ inputs.oscal-cli-version }}-oscal-cli.zip"
printf '%s %s\n' "${{ inputs.oscal-cli-sha256 }}" "$OSCAL_CLI_ZIP" | sha256sum --check --strict
unzip -q "$OSCAL_CLI_ZIP" -d "$RUNNER_TEMP/oscal-cli"
echo "$RUNNER_TEMP/oscal-cli/bin" >> "$GITHUB_PATH"
@@ -0,0 +1,48 @@
name: Generate OSCAL Baseline Catalog
on:
push:
paths:
- "scubagoggles/baselines/**/*.md"
- "scubagoggles/scuba_constants.py"
- "scubagoggles/utils/generate_oscal_baselines.py"
- "scubagoggles/utils/validate_oscal_baselines.py"
- ".github/actions/setup-oscal-cli/action.yml"
- ".github/workflows/generate_oscal_baselines.yml"
workflow_dispatch:
permissions:
contents: read
jobs:
generate-oscal:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
- name: Set up OSCAL CLI
uses: ./.github/actions/setup-oscal-cli
- name: Generate OSCAL baseline catalog
run: |
python3 scubagoggles/utils/generate_oscal_baselines.py \
--input-dir scubagoggles/baselines \
--output-dir dist/oscal-baseline-catalog
- name: Validate generated OSCAL catalog metadata
run: |
python3 scubagoggles/utils/validate_oscal_baselines.py \
--output-dir dist/oscal-baseline-catalog
- name: Validate OSCAL catalog schema
run: |
CATALOG_FILE="$(python3 scubagoggles/utils/validate_oscal_baselines.py \
--output-dir dist/oscal-baseline-catalog \
--print-catalog-path)"
oscal-cli --version
oscal-cli validate "$CATALOG_FILE"
- name: Upload generated OSCAL catalog
uses: actions/upload-artifact@v7
with:
name: oscal-baseline-catalog
path: dist/oscal-baseline-catalog/
+18
View File
@@ -52,6 +52,8 @@ jobs:
uses: actions/setup-python@v7
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Set up OSCAL CLI
uses: ./.github/actions/setup-oscal-cli
- name: Install Python dependencies
run: pip install -r requirements.txt
@@ -67,6 +69,21 @@ jobs:
[[ "$reqv" == "$sgv" ]] \
|| (echo "version mismatch - current: $sgv" && exit 1)
./scubagoggles/utils/build.sh -r "$PWD" -t "$GITHUB_REF_NAME"
- name: Generate OSCAL baseline catalog artifact
shell: bash
run: |
python3 scubagoggles/utils/generate_oscal_baselines.py \
--input-dir scubagoggles/baselines \
--output-dir dist/oscal-baseline-catalog \
--release-version "$RELEASE_VERSION"
CATALOG_FILE="$(python3 scubagoggles/utils/validate_oscal_baselines.py \
--output-dir dist/oscal-baseline-catalog \
--release-version "$RELEASE_VERSION" \
--print-catalog-path)"
oscal-cli --version
oscal-cli validate "$CATALOG_FILE"
cd dist
zip -r "scubagoggles-oscal-baseline-catalog-v${RELEASE_VERSION}.zip" oscal-baseline-catalog
- name: Get previous release tag
id: previous-release
@@ -631,4 +648,5 @@ jobs:
scubagoggles-*-py3-none-any.whl
scubagoggles-*.tar.gz
scubagoggles-*.zip
dist/scubagoggles-oscal-baseline-catalog-v${{ inputs.version }}.zip
fail_on_unmatched_files: true
+1
View File
@@ -1,6 +1,7 @@
.vscode
.venv
build
dist
scubagoggles.egg-info
.DS_Store
*.bak
+1
View File
@@ -63,6 +63,7 @@ We use a three-step process:
### Misc
- [Mappings](docs/misc/mappings.md)
- [OSCAL Catalog](docs/misc/oscal-catalog.md)
## Project License
Unless otherwise noted, this project is distributed under the Creative
+3
View File
@@ -36,6 +36,7 @@ The checklist below is used by the development team when it prepares a new relea
- [ ] Update the ScubaGoggles and SCB version in the [README.md](https://github.com/cisagov/ScubaGoggles/blob/main/README.md) badge image links.
- [ ] Update and redact the sample report using the redaction tool and manual review
- [ ] Check README for any necessary changes and documentation updates as needed
- [ ] Verify OSCAL baseline catalog generation and OSCAL CLI schema validation pass for all Markdown baselines.
- [ ] Build initial release candidate by manually triggering [`Build Draft Release`](https://github.com/cisagov/ScubaGoggles/actions/workflows/run_release.yml)
- Use the expected release name (vX.Y.Z) and release version (X.Y.Z) based on semantic versioning
- The **version** input must match `__version__` in [__init__.py](https://github.com/cisagov/ScubaGoggles/blob/main/scubagoggles/__init__.py)
@@ -69,5 +70,7 @@ The generated notes are grouped from merged pull requests between the previous p
- Click **Publish Release**
- [ ] Verify the new release is shown as latest on GitHub repository main page
- [ ] Verify that the GitHub release contains both the scubagoggles-X.Y.Z.tar.gz and scubagoggles-X.Y.Z-py3-none-any.whl files
- [ ] Verify that the GitHub release contains scubagoggles-oscal-baseline-catalog-vX.Y.Z.zip
- [ ] Verify that the OSCAL catalog zip contains scubagoggles-vX.Y.Z-oscal-catalog-oscal-A.B.C.json
- [ ] Verify the [`Publish ScubaGoggles to PyPI`](https://github.com/cisagov/ScubaGoggles/actions/workflows/publish_to_pypi.yml) workflow is triggered after publishing the GitHub release
- [ ] Verify the release is displayed on the cisagov [pypi.org](https://pypi.org/user/cisagov/) page
+8 -1
View File
@@ -8,4 +8,11 @@ SCuBA controls have been mapped to both NIST SP 800-53 and the MITRE ATT&CK fram
- See [scuba-to-nist-sp-800-53-r5-fedramp-high.csv](/scubagoggles/mappings/scuba-to-nist-sp-800-53-r5-fedramp-high.csv) for a consolidated list of these mappings.
## MITRE ATT&CK
Mappings to the MITRE ATT&CK framework are displayed within the baseline documents themselves, within the bulleted list following each policy statement.
Mappings to the MITRE ATT&CK framework are displayed within the baseline documents themselves, within the bulleted list following each policy statement.
## OSCAL Catalog
The generated [OSCAL Catalog](oscal-catalog.md) release artifact preserves the
NIST SP 800-53 mappings as policy control properties and the MITRE ATT&CK
mappings as policy control links. The catalog keeps the source baseline
hierarchy so section-level context is represented once and inherited by the
policy controls in that section.
+347
View File
@@ -0,0 +1,347 @@
# OSCAL Catalog
ScubaGoggles can generate a single OSCAL catalog JSON release artifact from the
Google Workspace Secure Configuration Baseline Markdown files in
`scubagoggles/baselines`.
The Markdown baselines remain the source of truth. The generated catalog is a
machine-readable representation of the same baseline policy content, grouped by
baseline, section, and policy.
The catalog is intended to carry enough Markdown-derived content for downstream
systems to render a complete human-readable baseline view from OSCAL without
using the source Markdown document as the only copy of baseline prose.
The generated OSCAL keeps repeated prose at the highest applicable catalog
level. Baseline overview text remains on the baseline group, section overview,
prerequisites, and common implementation guidance remain on the section group,
and policy-specific statements, details, rationale, notes, and implementation
guidance remain on the policy control. This keeps the catalog readable while
preserving enough hierarchy for downstream tools to render a complete baseline.
OSCAL constrains `part.name` values in catalogs. The generator uses OSCAL
names such as `overview`, `instruction`, `guidance`, and `statement`, then
preserves the source Markdown heading in `part.title` and a stable source-based
identifier in `part.id`.
Generated OSCAL part objects are ordered for readability as `name`, optional
`title`, `id`, and then `prose`. JSON object order does not change the OSCAL
meaning, but this order helps reviewers scan the OSCAL bucket, source Markdown
label, stable identifier, and content in sequence.
Generated SCuBA property objects are ordered as `name`, `value`, and then
`ns`, so readers see the property label and value before the namespace that
defines that label.
## Catalog Excerpt
The excerpt below shows the generated catalog structure from the top-level
`catalog` object down to the first policy in the first baseline group. It uses
the real first baseline, section, and policy identifiers from the generated
catalog. The shown prose fields are copied from the generated catalog as-is,
while sibling groups, controls, properties, links, and back-matter resources are
omitted for readability.
<details>
<summary>Show top-down OSCAL catalog excerpt</summary>
```json
{
"catalog": {
"uuid": "1a21eb2b-368d-5781-84c9-2ccadb555670",
"metadata": {
"title": "Google Workspace SCuBA Baselines - OSCAL Catalog",
"last-modified": "2026-08-18T17:33:09Z",
"version": "v1.0.0",
"oscal-version": "1.2.2"
},
"groups": [
{
"id": "gws-assuredcontrols",
"class": "scuba-baseline",
"title": "Google Workspace Assured Controls Baseline",
"props": [
{
"name": "source-baseline",
"value": "assuredcontrols.md",
"ns": "https://cisa.gov/scuba"
}
],
"links": [
{
"href": "#5774c748-0839-5ce7-bc40-cae2d2730853",
"rel": "source",
"text": "SCuBA Goggles Assured Controls baseline Markdown"
}
],
"parts": [
{
"name": "overview",
"id": "gws-assuredcontrols_overview",
"prose": "# CISA Google Workspace Secure Configuration Baseline for Assured Controls and Assured Controls Plus\n\nAssured Controls and Assured Controls Plus are paid add-ons within Google Workspace (GWS) relating to compliance and security.\nThis Secure Configuration Baseline (SCB) for Assured Controls provides specific policies to strengthen an organization's data security.\nThis baseline is intended as guidance for agencies that already have Assured Controls or Assured Controls Plus licenses.\nUsers who choose to implement this baseline should carefully consider the tradeoffs involved, including the potential security benefits, usability impacts, and possible increased fees for additional licenses.\n\nThe Cybersecurity and Infrastructure Security Agency's (CISA) Secure Cloud Business Applications (SCuBA) project, provides guidance and capabilities to secure federal civilian executive branch (FCEB) agencies' cloud business application environments and protect federal information that is created, accessed, shared, and stored in those environments.\n\nThe CISA SCuBA SCBs for GWS help secure federal information assets stored within GWS cloud business application environments through consistent, effective, and manageable security configurations. CISA created baselines tailored to the federal government's threats and risk tolerance. Organizations outside of the federal government may also find these baselines useful references to help reduce risks even if such organizations have different risk tolerances or face different threats.\n\nFor non-federal users, the information in this document is being provided \"as is\" for INFORMATIONAL PURPOSES ONLY. CISA does not endorse any commercial product or service, including any subjects of analysis. Any reference to specific commercial entities or commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply endorsement, recommendation, or favoritism by CISA. Without limiting the generality of the foregoing, some controls and settings are not available in all products. CISA has no control over vendor changes to products offerings or features. Accordingly, these SCuBA SCBs for GWS may not be applicable to the products available to you. This document does not address, ensure compliance with, or supersede any law, regulation, or other authority. Entities are responsible for complying with any recordkeeping, privacy, and other laws that may apply to the use of technology. This document is not intended to, and does not, create any right or benefit for anyone against the United States, its departments, agencies, or entities, its officers, employees, or agents, or any other person.\n\nThis baseline is based on Google documentation and addresses the following:\nGoogle Support Staff Data Access (https://github.com/cisagov/ScubaGoggles/blob/main/scubagoggles/baselines/assuredcontrols.md#1-google-support-staff-data-access)\nData Regions Advanced Settings (https://github.com/cisagov/ScubaGoggles/blob/main/scubagoggles/baselines/assuredcontrols.md#2-data-regions-advanced-settings)"
}
],
"groups": [
{
"id": "gws-assuredcontrols-section-1",
"class": "scuba-baseline-section",
"title": "1. Google Support Staff Data Access",
"parts": [
{
"name": "overview",
"id": "gws-assuredcontrols-section-1_overview",
"prose": "Google Workspace (GWS) includes several mechanisms to control how Google support staff access an organization's data.\nAccess Approvals requires Google support staff to request approval before viewing an organization's data.\nAccess can also be restricted to specific demographics, such as access by U.S.-based Google staff only.\nHowever, these features require additional licensing and are not available by default with GWS Enterprise Plus."
},
{
"name": "instruction",
"title": "Prerequisites",
"id": "gws-assuredcontrols-section-1_prerequisites",
"prose": "Access Approvals requires either the Assured Controls or the Assured Controls Plus add-ons.\nAccess Management requires the Assured Controls Plus add-on. However, customers who purchased the Assured Controls and the Assured Support add-on prior to June 17, 2024, also have access to Access Management."
}
],
"controls": [
{
"id": "gws.assuredcontrols.1.1v1",
"class": "scuba-policy",
"title": "Access Approvals SHOULD be enabled.",
"props": [
{
"name": "source-policy-id",
"value": "GWS.ASSUREDCONTROLS.1.1v1",
"ns": "https://cisa.gov/scuba"
},
{
"name": "last-modified",
"value": "November 2025",
"ns": "https://cisa.gov/scuba"
},
{
"name": "nist-sp800-53-rev5-fedramp-high-mapping",
"value": "SC-7(10)(a)",
"ns": "https://cisa.gov/scuba"
}
],
"links": [
{
"href": "https://attack.mitre.org/techniques/T1530/",
"rel": "threat-mapping",
"text": "MITRE ATT&CK T1530: Data from Cloud Storage"
}
],
"parts": [
{
"name": "statement",
"id": "gws.assuredcontrols.1.1v1_statement",
"prose": "Access Approvals SHOULD be enabled."
},
{
"name": "guidance",
"title": "Rationale",
"id": "gws.assuredcontrols.1.1v1_rationale",
"prose": "Unauthorized data access increases the risk of exposing sensitive data to untrusted entities. Unauthorized access and actions to an organization's data may be reduced by requiring approval of Google staff requests to access organizations' data."
},
{
"name": "guidance",
"title": "Implementation",
"id": "gws.assuredcontrols.1.1v1_implementation",
"prose": "1. Sign in to the Google Admin console (https://admin.google.com/) as a super admin.\n2. Select Data -\\> Compliance -\\> Access Approvals.\n3. Check the Require Google staff to request approval before viewing data necessary for support services box.\n4. Click SAVE."
}
]
}
]
}
]
}
]
}
}
```
</details>
## Viewing the Catalog
After downloading and extracting the generated OSCAL artifact, users can inspect
the catalog with an OSCAL-aware viewer such as <https://viewer.oscal.io/>.
This can help browse the catalog hierarchy and policy parts. Schema validation
is performed by the pinned OSCAL CLI in CI.
## Part Name Mapping
The generator uses the following OSCAL part-name assumptions.
| Markdown content | OSCAL `part.name` | Preserved label |
| --- | --- | --- |
| Main SHALL/SHOULD policy sentence | `statement` | Not needed; also stored in `control.title` |
| Baseline introduction | `overview` | Not needed |
| Baseline assumptions | `overview` | `title: "Assumptions"` |
| Baseline key terminology | `overview` | `title: "Key Terminology"` |
| Section overview | `overview` | Not needed |
| Section prerequisites | `instruction` | `title: "Prerequisites"` |
| Common section implementation steps | `instruction` | `title: "Implementation"` |
| Policy detail | `guidance` | `title: "Policy Detail"` |
| Rationale | `guidance` | `title: "Rationale"` |
| Policy-specific implementation steps | `guidance` | `title: "Implementation"` |
| Note | `guidance` | `title: "Note"` |
## Markdown to OSCAL Cross-Walk
The tables below summarize how source Markdown content is represented in the
generated OSCAL catalog. They are split by catalog layer so the documentation
stays readable in GitHub's Markdown view.
### Catalog Inputs
| Markdown or input | OSCAL representation |
| --- | --- |
| Baseline README list | Creates ordered top-level `catalog.groups[]`. |
| Generated release version | Stored in `catalog.metadata.version` and the generated filename for release artifacts. |
| Ad hoc generator run | Uses the ScubaGoggles package version with a development build label in `catalog.metadata.version`. |
| OSCAL version constant | Stored in `catalog.metadata.oscal-version` and the generated filename. |
The baseline list comes from `scubagoggles/baselines/README.md`.
The OSCAL version is defined by `DEFAULT_OSCAL_VERSION` in
`scubagoggles/scuba_constants.py`.
Markdown links copied into OSCAL prose are resolved to absolute GitHub URLs so
the generated catalog remains portable outside the source repository checkout.
### Baseline Groups
| Markdown source | OSCAL representation |
| --- | --- |
| Baseline title and file name | Each README-listed baseline becomes one top-level group. |
| Baseline title | Stored in `catalog.groups[].title`. |
| Source Markdown filename | Stored in a `source-baseline` property. |
| Source Markdown file | Linked from the baseline group with `rel=source`. |
| Baseline introduction | Stored as an `overview` part on the baseline group. |
| Baseline preamble sections | Stored as titled `overview` parts on the baseline group. |
Examples of baseline preamble sections include `## Assumptions` and
`## Key Terminology`.
### Section Groups
| Markdown source | OSCAL representation |
| --- | --- |
| Numbered Markdown section | Becomes a nested group under the baseline group. |
| Section heading | Stored in the section group's `title`. |
| Section overview text | Stored once as an `overview` part on the section group. |
| `### Prerequisites` | Stored once as an `instruction` part titled `Prerequisites`. |
| Common implementation instructions | Stored once as an `Implementation` instruction. |
Section-level overview, prerequisites, and common implementation guidance are
not repeated on every child control. OSCAL consumers should read them from the
containing section group.
For example, `## 1. Phishing-Resistant Multifactor Authentication` becomes a
nested section group.
### Policy Controls
| Markdown source | OSCAL representation |
| --- | --- |
| `### Policies` block | Each SCuBA policy becomes one OSCAL control. |
| Policy heading | The normalized value becomes `control.id`. |
| Original SCuBA policy ID | Preserved in a `source-policy-id` property. |
| Main policy sentence | Stored in `control.title` and a `statement` part. |
| Additional policy detail | Stored as a `guidance` part titled `Policy Detail`. |
| `_Rationale:_` | Stored as a `guidance` part titled `Rationale`. |
| `_Note:_` | Stored as a `guidance` part titled `Note`. |
| Policy-specific implementation | Stored as a `guidance` part titled `Implementation`. |
The main policy statement is stored in both `control.title` and
`control.parts[name=statement]`. The statement part is the authoritative copy for
future OSCAL-to-Markdown conversion.
### Policy Properties and Links
| Markdown source | OSCAL representation |
| --- | --- |
| Policy badges | Converted to boolean-like string properties. |
| `_Last modified:_` | Stored in a `last-modified` property. |
| FedRAMP/NIST mapping line | Stored in a FedRAMP High mapping property. |
| Multiple NIST/FedRAMP mappings | Counted in a source mapping count property. |
| MITRE ATT&CK links | Stored as `threat-mapping` links on the control. |
| Resource links | Stored in back matter and linked as references. |
| Embedded baseline images | Preserved as back-matter resources and reference links. |
Shared references are de-duplicated in back matter and linked from applicable
controls.
This includes `### Resources` links, section overview links, and embedded
source images such as `<img src="images/MFA.PNG">`.
The FedRAMP High mapping property name is
`nist-sp800-53-rev5-fedramp-high-mapping`.
## Policy Example
The example below shows a concrete mapping for
`GWS.ASSUREDCONTROLS.1.1v1` from `assuredcontrols.md`.
### Example Identity
| Field | Example |
| --- | --- |
| Baseline Markdown | `assuredcontrols.md` |
| Baseline group title | `Google Workspace Assured Controls Baseline` |
| Section Markdown | `## 1. Google Support Staff Data Access` |
| Section group title | `1. Google Support Staff Data Access` |
| Policy Markdown | `#### GWS.ASSUREDCONTROLS.1.1v1` |
| OSCAL control ID | `gws.assuredcontrols.1.1v1` |
The original policy ID is also preserved in a `source-policy-id` property with
the value `GWS.ASSUREDCONTROLS.1.1v1`.
### Example Inherited Context
| Markdown content | OSCAL representation |
| --- | --- |
| Section overview text | Stored once in the section group's `overview` part. |
| `### Prerequisites` | Stored once in an `instruction` part titled `Prerequisites`. |
In this example, the section overview describes Google support staff data access
mechanisms, and the prerequisites describe licensing assumptions.
### Example Policy Content
| Markdown content | OSCAL representation |
| --- | --- |
| `Access Approvals SHOULD be enabled.` | Stored in `control.title` and the `statement` part. |
| `_Rationale:_ Unauthorized access to data increases...` | Stored in `guidance` titled `Rationale`. |
| Policy implementation instructions | Stored in `guidance` titled `Implementation`. |
### Example Policy Metadata
| Markdown content | OSCAL representation |
| --- | --- |
| `_Last modified:_ November 2025` | `last-modified = "November 2025"` |
| FedRAMP High mapping | `SC-7(10)(a)` |
| MITRE ATT&CK links | Stored as `threat-mapping` links on the control. |
| Automated Check badge | `automated-check = "true"` |
| Log-Based Check badge | `log-based-check = "true"` |
The example MITRE ATT&CK links are `T1530`, `T1537`, and `T1589`.
## Hierarchy and Flattening
OSCAL consumers should read inherited context from the containing catalog
groups. For example, a renderer for `gws.assuredcontrols.1.1v1` should combine
the control content with the overview, prerequisites, and common implementation
parts on its parent section group.
The catalog intentionally avoids repeating section-level prose on every child
control. If a later release adds a CSV or other row-oriented artifact, that
artifact can intentionally flatten inherited section context into each row
without making the canonical OSCAL catalog repetitive.
## Human-Readable Rendering
The generator keeps the main policy statement separate from supplemental policy
detail. For example, `control.parts[name=statement]` contains the SHALL/SHOULD
statement, while `control.parts[name=guidance,title=Policy Detail]` contains
extra explanatory content that appears before the rationale in Markdown.
This structure is intended to help any downstream OSCAL consumer render a full
human-readable baseline without losing baseline preamble text, policy
statements, supplemental policy detail, rationale, implementation guidance, or
other source content represented in the Markdown baselines.
+3
View File
@@ -7,6 +7,9 @@
{
"pattern":"^https://github.com/cisagov/ScubaGoggles/blob/main*"
},
{
"pattern":"^https://viewer\\.oscal\\.io"
},
{
"pattern":"^mailto:user@company.com"
},
@@ -0,0 +1,324 @@
"""Unit tests for OSCAL baseline catalog generation."""
import importlib.util
import json
import re
from pathlib import Path
from scubagoggles.scuba_constants import DEFAULT_OSCAL_VERSION
REPO_ROOT = Path(__file__).resolve().parents[4]
GENERATOR_PATH = REPO_ROOT / "scubagoggles" / "utils" / "generate_oscal_baselines.py"
OSCAL_GROUP_PART_NAMES = {"instruction", "overview"}
OSCAL_CONTROL_PART_NAMES = {
"assessment",
"assessment-method",
"assessment-objective",
"example",
"guidance",
"objective",
"overview",
"statement",
}
def load_generator():
"""Load the OSCAL generator module by path."""
spec = importlib.util.spec_from_file_location("generate_oscal_baselines", GENERATOR_PATH)
if spec is None or spec.loader is None:
raise ImportError(f"Could not load {GENERATOR_PATH}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def source_policy_count(path):
"""Return the number of unique SCuBA policy IDs in a Markdown file."""
text = path.read_text(encoding="utf-8")
ids = re.findall(
r"^#### (GWS\.[A-Z]+(?:CONTROLS)?\.\d+\.\d+v\d+)\s*$",
text,
re.M,
)
return len(ids)
def source_mapping_count(path):
"""Return the total count of source NIST mapping entries."""
text = path.read_text(encoding="utf-8")
mapping_lines = re.findall(
r"NIST SP 800-53 Rev\. 5 FedRAMP High Baseline Mapping:_\s*([^\r\n]+)",
text,
)
return sum(len(line.split(",")) for line in mapping_lines)
def catalog_controls(catalog):
"""Yield every control from the generated catalog."""
for baseline_group in catalog["groups"]:
for section_group in baseline_group.get("groups", []):
yield from section_group.get("controls", [])
def prop_values(props, name):
"""Return all values for a property name."""
return [prop["value"] for prop in props if prop["name"] == name]
def assert_prop_order(props):
"""Assert generated properties read as name, value, then namespace."""
for prop in props:
assert list(prop) == ["name", "value", "ns"]
def iter_prose(value):
"""Yield every prose string from a generated OSCAL object."""
if isinstance(value, dict):
if "prose" in value:
yield value["prose"]
for child in value.values():
yield from iter_prose(child)
elif isinstance(value, list):
for child in value:
yield from iter_prose(child)
def part_values(parts, name, title=None):
"""Return all prose values for a part name."""
return [
part["prose"]
for part in parts
if part["name"] == name and (title is None or part.get("title") == title)
]
class GenerateOscalBaselinesTest:
"""Tests for converting Markdown baselines to one OSCAL catalog."""
def test_policy_ids_become_catalog_control_ids(self):
"""SCuBA policy IDs should become stable catalog control IDs."""
generator = load_generator()
assert (
generator.control_id_from_policy("GWS.COMMONCONTROLS.1.1v1")
== "gws.commoncontrols.1.1v1"
)
def test_catalog_filename_includes_relevant_versions(self):
"""Release catalog filenames should show source and OSCAL versions."""
generator = load_generator()
expected_default = (
f"scubagoggles-oscal-catalog-oscal-{generator.DEFAULT_OSCAL_VERSION}.json"
)
expected_release = (
f"scubagoggles-v1.0.0-oscal-catalog-oscal-"
f"{generator.DEFAULT_OSCAL_VERSION}.json"
)
assert generator.catalog_file_name() == expected_default
assert generator.catalog_file_name("1.0.0") == expected_release
assert generator.catalog_file_name("v1.0.0") == expected_release
def test_catalog_metadata_version_marks_ad_hoc_runs(self):
"""Ad hoc catalog metadata should distinguish non-release generation."""
generator = load_generator()
assert generator.catalog_metadata_version("1.0.0") == "1.0.0"
assert generator.catalog_metadata_version() == (
f"{generator.SCUBAGOGGLES_VERSION} "
f"({generator.DEVELOPMENT_VERSION_LABEL})"
)
def test_clean_text_resolves_relative_links_in_prose(self):
"""Relative Markdown links copied into OSCAL prose should be portable."""
generator = load_generator()
prose = generator.clean_text(
(
"See [Limitations](../../docs/usage/Limitations.md"
"#log-based-policy-checks) and [this section](#key-terminology)."
),
"assuredcontrols.md",
)
assert (
"Limitations "
"(https://github.com/cisagov/ScubaGoggles/blob/main/"
"docs/usage/Limitations.md#log-based-policy-checks)"
) in prose
assert (
"this section "
"(https://github.com/cisagov/ScubaGoggles/blob/main/"
"scubagoggles/baselines/assuredcontrols.md#key-terminology)"
) in prose
def test_generation_covers_readme_baselines(self, tmp_path):
"""Every README-listed baseline should be represented in one catalog."""
generator = load_generator()
input_dir = REPO_ROOT / "scubagoggles" / "baselines"
output_dir = tmp_path / "oscal-baseline-catalog"
release_version = "0.0.0-test"
expected_catalog_file = generator.catalog_file_name(release_version)
summary = generator.generate_baselines(input_dir, output_dir, release_version)
discovered = generator.parse_baselines_readme(input_dir)
catalog_path = output_dir / expected_catalog_file
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))["catalog"]
controls = list(catalog_controls(catalog))
assert generator.DEFAULT_OSCAL_VERSION == DEFAULT_OSCAL_VERSION
assert summary["output"] == expected_catalog_file
assert f"v{release_version}" in catalog_path.name
assert generator.DEFAULT_OSCAL_VERSION in catalog_path.name
assert summary["baselines"] == len(discovered)
assert summary["source_policies"] == sum(
source_policy_count(input_dir / file_name) for _, file_name in discovered
)
assert summary["source_mappings"] == sum(
source_mapping_count(input_dir / file_name) for _, file_name in discovered
)
assert (output_dir / "generation-summary.json").exists()
assert not list(output_dir.glob("*-oscal-component-definition.json"))
assert catalog["metadata"]["oscal-version"] == generator.DEFAULT_OSCAL_VERSION
assert catalog["metadata"]["version"] == release_version
assert_prop_order(catalog["metadata"]["props"])
assert len(catalog["groups"]) == len(discovered)
assert len(controls) == summary["source_policies"]
for prose in iter_prose(catalog):
assert not any(
marker in prose
for marker in ("(../../", "(../", "(./", "(images/", "(#")
)
for baseline_group in catalog["groups"]:
assert_prop_order(baseline_group["props"])
assert all(
part["name"] in OSCAL_GROUP_PART_NAMES
for part in baseline_group.get("parts", [])
)
source_baseline = prop_values(baseline_group["props"], "source-baseline")[0]
group_controls = [
control
for section_group in baseline_group.get("groups", [])
for control in section_group.get("controls", [])
]
assert len(group_controls) == source_policy_count(input_dir / source_baseline)
for section_group in baseline_group.get("groups", []):
assert_prop_order(section_group["props"])
assert all(
part["name"] in OSCAL_GROUP_PART_NAMES
for part in section_group.get("parts", [])
)
assured_controls = next(
group
for group in catalog["groups"]
if prop_values(group["props"], "source-baseline") == ["assuredcontrols.md"]
)
common_controls = next(
group
for group in catalog["groups"]
if prop_values(group["props"], "source-baseline") == ["commoncontrols.md"]
)
mfa_section = next(
section
for section in common_controls["groups"]
if prop_values(section["props"], "source-section") == ["1"]
)
assert "Assured Controls Plus add-on" in part_values(
assured_controls["parts"],
"overview",
"Assumptions",
)[0]
assert 'The key words "MUST,"' in part_values(
assured_controls["parts"],
"overview",
"Key Terminology",
)[0]
assert part_values(mfa_section["parts"], "overview")
assert "FIDO2-compliant security keys" in part_values(
mfa_section["parts"],
"instruction",
"Prerequisites",
)[0]
assert "Policy 1 Common Instructions" in part_values(
mfa_section["parts"],
"instruction",
"Implementation",
)[0]
for control in controls:
assert_prop_order(control["props"])
policy_ids = prop_values(control["props"], "source-policy-id")
mappings = prop_values(
control["props"],
"nist-sp800-53-rev5-fedramp-high-mapping",
)
assert len(policy_ids) == 1
assert control["id"] == generator.control_id_from_policy(policy_ids[0])
assert len(mappings) == 1
assert any(part["name"] == "statement" for part in control["parts"])
assert all(
part["name"] in OSCAL_CONTROL_PART_NAMES
for part in control.get("parts", [])
)
link_keys = [
(link["rel"], link["href"]) for link in control.get("links", [])
]
assert len(link_keys) == len(set(link_keys))
assert not {
"section-prerequisites",
"source-note",
"source-policy-detail",
"source-section-overview",
}.intersection(prop["name"] for prop in control["props"])
phishing_resistant_mfa = next(
control
for control in controls
if prop_values(control["props"], "source-policy-id")
== ["GWS.COMMONCONTROLS.1.1v1"]
)
assert part_values(phishing_resistant_mfa["parts"], "statement") == [
"Phishing-Resistant MFA SHALL be required for all users."
]
statement_part = next(
part
for part in phishing_resistant_mfa["parts"]
if part["name"] == "statement"
)
assert list(statement_part) == ["name", "id", "prose"]
phishing_implementation = part_values(
phishing_resistant_mfa["parts"],
"guidance",
"Implementation",
)[0]
implementation_part = next(
part
for part in phishing_resistant_mfa["parts"]
if part["name"] == "guidance"
and part.get("title") == "Implementation"
)
assert list(implementation_part) == ["name", "title", "id", "prose"]
assert "Under Authentication" in phishing_implementation
assert "Sign in to" not in phishing_implementation
assert "FIDO2 Security Key" in part_values(
phishing_resistant_mfa["parts"],
"guidance",
"Policy Detail",
)[0]
@@ -0,0 +1,86 @@
"""Unit tests for generated OSCAL baseline catalog validation."""
import importlib.util
import json
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[4]
GENERATOR_PATH = REPO_ROOT / "scubagoggles" / "utils" / "generate_oscal_baselines.py"
VALIDATOR_PATH = REPO_ROOT / "scubagoggles" / "utils" / "validate_oscal_baselines.py"
def load_module(path, name):
"""Load a module by path."""
spec = importlib.util.spec_from_file_location(name, path)
if spec is None or spec.loader is None:
raise ImportError(f"Could not load {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def test_validate_release_catalog_output(tmp_path):
"""Release catalog output should pass validation."""
generator = load_module(GENERATOR_PATH, "generate_oscal_baselines")
validator = load_module(VALIDATOR_PATH, "validate_oscal_baselines")
output_dir = tmp_path / "oscal-baseline-catalog"
generator.generate_baselines(
REPO_ROOT / "scubagoggles" / "baselines",
output_dir,
"1.0.0",
)
catalog_path = validator.validate_catalog_output(output_dir, "1.0.0")
assert catalog_path.name == validator.expected_catalog_file_name("1.0.0")
def test_validate_ad_hoc_catalog_output(tmp_path):
"""Ad hoc catalog output should pass validation."""
generator = load_module(GENERATOR_PATH, "generate_oscal_baselines")
validator = load_module(VALIDATOR_PATH, "validate_oscal_baselines")
output_dir = tmp_path / "oscal-baseline-catalog"
generator.generate_baselines(
REPO_ROOT / "scubagoggles" / "baselines",
output_dir,
)
catalog_path = validator.validate_catalog_output(output_dir)
assert catalog_path.name == validator.expected_catalog_file_name()
def test_validate_catalog_output_rejects_unexpected_filename(tmp_path):
"""Validation should fail when the summary points to the wrong filename."""
validator = load_module(VALIDATOR_PATH, "validate_oscal_baselines")
output_dir = tmp_path / "oscal-baseline-catalog"
output_dir.mkdir()
(output_dir / "unexpected.json").write_text(
json.dumps(
{
"catalog": {
"metadata": {
"oscal-version": validator.DEFAULT_OSCAL_VERSION,
"version": validator.expected_metadata_version(),
}
}
}
),
encoding="utf-8",
)
(output_dir / "generation-summary.json").write_text(
json.dumps({"output": "unexpected.json"}),
encoding="utf-8",
)
with pytest.raises(ValueError, match="Expected generated catalog"):
validator.validate_catalog_output(output_dir)
+1
View File
@@ -57,6 +57,7 @@ API_LINKS = {
OPA_VERSION = 'v1.20.1'
DEFAULT_OSCAL_VERSION = '1.2.2'
NUMBER_OF_UUID_CHARACTERS_TO_TRUNCATE_CHOICES = (
0, 13, 18, 36
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,172 @@
"""Validate generated SCuBA OSCAL baseline catalog outputs."""
# pylint: disable=wrong-import-position
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
from typing import TypeAlias
REPO_ROOT = Path(__file__).resolve().parents[2]
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
from scubagoggles import __version__ as SCUBAGOGGLES_VERSION
from scubagoggles.scuba_constants import DEFAULT_OSCAL_VERSION
JsonValue: TypeAlias = (
str | int | float | bool | None | list["JsonValue"] | dict[str, "JsonValue"]
)
JsonObject: TypeAlias = dict[str, JsonValue]
DEVELOPMENT_VERSION_LABEL = "development build"
SUMMARY_FILE = "generation-summary.json"
def version_filename_fragment(version: str) -> str:
"""Return a safe version fragment for generated filenames."""
fragment = re.sub(r"[^a-z0-9.-]+", "-", version.strip().lower()).strip("-")
if fragment.startswith("v"):
fragment = fragment[1:]
return fragment or "unversioned"
def expected_catalog_file_name(release_version: str | None = None) -> str:
"""Return the expected generated OSCAL catalog filename."""
if release_version:
source_version = version_filename_fragment(release_version)
return (
f"scubagoggles-v{source_version}-oscal-catalog-"
f"oscal-{DEFAULT_OSCAL_VERSION}.json"
)
return f"scubagoggles-oscal-catalog-oscal-{DEFAULT_OSCAL_VERSION}.json"
def expected_metadata_version(release_version: str | None = None) -> str:
"""Return the expected OSCAL metadata version."""
if release_version:
return release_version
return f"{SCUBAGOGGLES_VERSION} ({DEVELOPMENT_VERSION_LABEL})"
def read_json_object(path: Path) -> JsonObject:
"""Read a JSON file and require its top-level value to be an object."""
data = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise ValueError(f"{path} must contain a JSON object")
return data
def require_json_object(value: JsonValue | None, label: str) -> JsonObject:
"""Require a JSON value to be an object."""
if not isinstance(value, dict):
raise ValueError(f"{label} must be a JSON object")
return value
def require_string(value: JsonValue | None, label: str) -> str:
"""Require a JSON value to be a string."""
if not isinstance(value, str):
raise ValueError(f"{label} must be a string")
return value
def validate_catalog_output(
output_dir: Path,
release_version: str | None = None,
) -> Path:
"""Validate generated catalog summary and metadata, returning catalog path."""
summary_path = output_dir / SUMMARY_FILE
if not summary_path.exists():
raise FileNotFoundError(f"Missing OSCAL generation summary: {summary_path}")
summary = read_json_object(summary_path)
output_name = require_string(summary.get("output"), "summary.output")
expected_output_name = expected_catalog_file_name(release_version)
if output_name != expected_output_name:
raise ValueError(
f"Expected generated catalog {expected_output_name}, got {output_name}"
)
catalog_path = output_dir / output_name
if not catalog_path.exists():
raise FileNotFoundError(f"Missing generated OSCAL catalog: {catalog_path}")
catalog_json = read_json_object(catalog_path)
catalog = require_json_object(catalog_json.get("catalog"), "catalog")
metadata = require_json_object(catalog.get("metadata"), "catalog.metadata")
oscal_version = require_string(
metadata.get("oscal-version"),
"catalog.metadata.oscal-version",
)
if oscal_version != DEFAULT_OSCAL_VERSION:
raise ValueError(
f"Generated OSCAL catalog uses OSCAL {oscal_version}; "
f"expected {DEFAULT_OSCAL_VERSION}"
)
metadata_version = require_string(
metadata.get("version"),
"catalog.metadata.version",
)
expected_version = expected_metadata_version(release_version)
if metadata_version != expected_version:
raise ValueError(
f"Generated OSCAL catalog metadata.version is {metadata_version}; "
f"expected {expected_version}"
)
return catalog_path
def parse_args() -> argparse.Namespace:
"""Parse command-line arguments."""
parser = argparse.ArgumentParser(
description="Validate a generated SCuBA OSCAL baseline catalog."
)
parser.add_argument(
"--output-dir",
type=Path,
default=REPO_ROOT / "dist" / "oscal-baseline-catalog",
help="Directory containing generated OSCAL catalog files.",
)
parser.add_argument(
"--release-version",
default=None,
help="Release version expected in the generated OSCAL catalog.",
)
parser.add_argument(
"--print-catalog-path",
action="store_true",
help="Print only the validated catalog path.",
)
return parser.parse_args()
def main() -> None:
"""Run the validator."""
args = parse_args()
catalog_path = validate_catalog_output(args.output_dir, args.release_version)
if args.print_catalog_path:
print(catalog_path)
else:
print(f"Validated OSCAL catalog JSON: {catalog_path}")
if __name__ == "__main__":
main()