mirror of
https://github.com/cisagov/ScubaGoggles.git
synced 2026-10-16 04:16:51 +02:00
Generate OSCAL catalog release artifact for SCuBA baselines (#1058)
Generate OSCAL Baseline Catalog / generate-oscal (push) Has been cancelled
Run Link Checker for Cron Job / markdown-link-check (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Pylint (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run OPA Tests (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run Markdown Link Check (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run Python Tests (push) Has been cancelled
Run Smoke Test / configuration (push) Has been cancelled
Scorecard supply-chain security / Scorecard analysis (push) Has been cancelled
Run Smoke Test / smoke-test (push) Has been cancelled
Run Smoke Test / lhci (push) Has been cancelled
Generate OSCAL Baseline Catalog / generate-oscal (push) Has been cancelled
Run Link Checker for Cron Job / markdown-link-check (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Pylint (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run OPA Tests (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run Markdown Link Check (push) Has been cancelled
Run CI Pipeline for ScubaGoggles / Run Python Tests (push) Has been cancelled
Run Smoke Test / configuration (push) Has been cancelled
Scorecard supply-chain security / Scorecard analysis (push) Has been cancelled
Run Smoke Test / smoke-test (push) Has been cancelled
Run Smoke Test / lhci (push) Has been cancelled
* Add OSCAL baseline generation release artifacts * Update generate_oscal_baselines.yml updated trigger for this branch also * Update generate_oscal_baselines.yml remove temporary testing of run on the branch * Adjust OSCAL baseline workflow triggers * Generate OSCAL catalog release artifact Add a generator that converts all README-listed baseline Markdown files into a single OSCAL catalog JSON artifact. Include the OSCAL version in metadata and filenames, validate generated catalogs in GitHub Actions with oscal-cli, and add release workflow packaging for the catalog artifact. Document the Markdown-to-OSCAL cross-walk and preserve baseline hierarchy so section-level prose is represented once while policy-specific content remains on each control. Closes #1004 * Validate OSCAL catalog generation with pinned CLI Use OSCAL-valid catalog part names while preserving Markdown labels in part titles for human-readable rendering and future reversibility. Pin OSCAL CLI validation in the OSCAL artifact and release workflows, replace the floating container image, and document the part-name mapping assumptions. Also de-duplicate generated control links and extend generator tests for allowed part names and unique links. * Use available setup-java version for OSCAL workflow * Order OSCAL parts for readability * Order OSCAL properties for readability * Use Node 24 artifact upload action * Improve OSCAL catalog cross-walk readability * Add collapsible OSCAL catalog excerpt to docs * Address OSCAL catalog review feedback Add catalog viewing guidance, clarify ad hoc metadata versioning, and resolve Markdown-relative links copied into generated OSCAL prose. Validate generated catalog behavior with focused unit tests. * Ignore OSCAL viewer link in markdown checker Add a narrow markdown-link-check ignore pattern for viewer.oscal.io because the checker cannot validate its certificate chain. * Verify OSCAL CLI download checksum * Add reusable OSCAL CLI setup action * Add OSCAL catalog validation utility
This commit is contained in:
1 parent
95039bdd34
commit
f21b3c7988
14 files changed
+2063
-1
No files matched your search
@@ -0,0 +1,28 @@
|
||||
name: Setup OSCAL CLI
|
||||
description: Set up Java and install a checksum-verified OSCAL CLI.
|
||||
inputs:
|
||||
oscal-cli-version:
|
||||
required: false
|
||||
default: "3.2.0"
|
||||
oscal-cli-sha256:
|
||||
required: false
|
||||
default: "e001b353ee245da6f4ff0ad930c4dfaa88083c63243ca004bd5f5bcef44e8ec8"
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Set up Java
|
||||
uses: actions/setup-java@v5
|
||||
with:
|
||||
distribution: temurin
|
||||
java-version: "17"
|
||||
|
||||
- name: Install OSCAL CLI
|
||||
shell: bash
|
||||
run: |
|
||||
OSCAL_CLI_ZIP="$RUNNER_TEMP/oscal-cli.zip"
|
||||
curl --fail --location --retry 3 --output "$OSCAL_CLI_ZIP" \
|
||||
"https://repo1.maven.org/maven2/dev/metaschema/oscal/oscal-cli-enhanced/${{ inputs.oscal-cli-version }}/oscal-cli-enhanced-${{ inputs.oscal-cli-version }}-oscal-cli.zip"
|
||||
printf '%s %s\n' "${{ inputs.oscal-cli-sha256 }}" "$OSCAL_CLI_ZIP" | sha256sum --check --strict
|
||||
unzip -q "$OSCAL_CLI_ZIP" -d "$RUNNER_TEMP/oscal-cli"
|
||||
echo "$RUNNER_TEMP/oscal-cli/bin" >> "$GITHUB_PATH"
|
||||
@@ -0,0 +1,48 @@
|
||||
name: Generate OSCAL Baseline Catalog
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- "scubagoggles/baselines/**/*.md"
|
||||
- "scubagoggles/scuba_constants.py"
|
||||
- "scubagoggles/utils/generate_oscal_baselines.py"
|
||||
- "scubagoggles/utils/validate_oscal_baselines.py"
|
||||
- ".github/actions/setup-oscal-cli/action.yml"
|
||||
- ".github/workflows/generate_oscal_baselines.yml"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
generate-oscal:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Set up OSCAL CLI
|
||||
uses: ./.github/actions/setup-oscal-cli
|
||||
- name: Generate OSCAL baseline catalog
|
||||
run: |
|
||||
python3 scubagoggles/utils/generate_oscal_baselines.py \
|
||||
--input-dir scubagoggles/baselines \
|
||||
--output-dir dist/oscal-baseline-catalog
|
||||
- name: Validate generated OSCAL catalog metadata
|
||||
run: |
|
||||
python3 scubagoggles/utils/validate_oscal_baselines.py \
|
||||
--output-dir dist/oscal-baseline-catalog
|
||||
- name: Validate OSCAL catalog schema
|
||||
run: |
|
||||
CATALOG_FILE="$(python3 scubagoggles/utils/validate_oscal_baselines.py \
|
||||
--output-dir dist/oscal-baseline-catalog \
|
||||
--print-catalog-path)"
|
||||
oscal-cli --version
|
||||
oscal-cli validate "$CATALOG_FILE"
|
||||
- name: Upload generated OSCAL catalog
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: oscal-baseline-catalog
|
||||
path: dist/oscal-baseline-catalog/
|
||||
@@ -52,6 +52,8 @@ jobs:
|
||||
uses: actions/setup-python@v7
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
- name: Set up OSCAL CLI
|
||||
uses: ./.github/actions/setup-oscal-cli
|
||||
|
||||
- name: Install Python dependencies
|
||||
run: pip install -r requirements.txt
|
||||
@@ -67,6 +69,21 @@ jobs:
|
||||
[[ "$reqv" == "$sgv" ]] \
|
||||
|| (echo "version mismatch - current: $sgv" && exit 1)
|
||||
./scubagoggles/utils/build.sh -r "$PWD" -t "$GITHUB_REF_NAME"
|
||||
- name: Generate OSCAL baseline catalog artifact
|
||||
shell: bash
|
||||
run: |
|
||||
python3 scubagoggles/utils/generate_oscal_baselines.py \
|
||||
--input-dir scubagoggles/baselines \
|
||||
--output-dir dist/oscal-baseline-catalog \
|
||||
--release-version "$RELEASE_VERSION"
|
||||
CATALOG_FILE="$(python3 scubagoggles/utils/validate_oscal_baselines.py \
|
||||
--output-dir dist/oscal-baseline-catalog \
|
||||
--release-version "$RELEASE_VERSION" \
|
||||
--print-catalog-path)"
|
||||
oscal-cli --version
|
||||
oscal-cli validate "$CATALOG_FILE"
|
||||
cd dist
|
||||
zip -r "scubagoggles-oscal-baseline-catalog-v${RELEASE_VERSION}.zip" oscal-baseline-catalog
|
||||
|
||||
- name: Get previous release tag
|
||||
id: previous-release
|
||||
@@ -631,4 +648,5 @@ jobs:
|
||||
scubagoggles-*-py3-none-any.whl
|
||||
scubagoggles-*.tar.gz
|
||||
scubagoggles-*.zip
|
||||
dist/scubagoggles-oscal-baseline-catalog-v${{ inputs.version }}.zip
|
||||
fail_on_unmatched_files: true
|
||||
@@ -1,6 +1,7 @@
|
||||
.vscode
|
||||
.venv
|
||||
build
|
||||
dist
|
||||
scubagoggles.egg-info
|
||||
.DS_Store
|
||||
*.bak
|
||||
|
||||
@@ -63,6 +63,7 @@ We use a three-step process:
|
||||
|
||||
### Misc
|
||||
- [Mappings](docs/misc/mappings.md)
|
||||
- [OSCAL Catalog](docs/misc/oscal-catalog.md)
|
||||
|
||||
## Project License
|
||||
Unless otherwise noted, this project is distributed under the Creative
|
||||
|
||||
@@ -36,6 +36,7 @@ The checklist below is used by the development team when it prepares a new relea
|
||||
- [ ] Update the ScubaGoggles and SCB version in the [README.md](https://github.com/cisagov/ScubaGoggles/blob/main/README.md) badge image links.
|
||||
- [ ] Update and redact the sample report using the redaction tool and manual review
|
||||
- [ ] Check README for any necessary changes and documentation updates as needed
|
||||
- [ ] Verify OSCAL baseline catalog generation and OSCAL CLI schema validation pass for all Markdown baselines.
|
||||
- [ ] Build initial release candidate by manually triggering [`Build Draft Release`](https://github.com/cisagov/ScubaGoggles/actions/workflows/run_release.yml)
|
||||
- Use the expected release name (vX.Y.Z) and release version (X.Y.Z) based on semantic versioning
|
||||
- The **version** input must match `__version__` in [__init__.py](https://github.com/cisagov/ScubaGoggles/blob/main/scubagoggles/__init__.py)
|
||||
@@ -69,5 +70,7 @@ The generated notes are grouped from merged pull requests between the previous p
|
||||
- Click **Publish Release**
|
||||
- [ ] Verify the new release is shown as latest on GitHub repository main page
|
||||
- [ ] Verify that the GitHub release contains both the scubagoggles-X.Y.Z.tar.gz and scubagoggles-X.Y.Z-py3-none-any.whl files
|
||||
- [ ] Verify that the GitHub release contains scubagoggles-oscal-baseline-catalog-vX.Y.Z.zip
|
||||
- [ ] Verify that the OSCAL catalog zip contains scubagoggles-vX.Y.Z-oscal-catalog-oscal-A.B.C.json
|
||||
- [ ] Verify the [`Publish ScubaGoggles to PyPI`](https://github.com/cisagov/ScubaGoggles/actions/workflows/publish_to_pypi.yml) workflow is triggered after publishing the GitHub release
|
||||
- [ ] Verify the release is displayed on the cisagov [pypi.org](https://pypi.org/user/cisagov/) page
|
||||
@@ -8,4 +8,11 @@ SCuBA controls have been mapped to both NIST SP 800-53 and the MITRE ATT&CK fram
|
||||
- See [scuba-to-nist-sp-800-53-r5-fedramp-high.csv](/scubagoggles/mappings/scuba-to-nist-sp-800-53-r5-fedramp-high.csv) for a consolidated list of these mappings.
|
||||
|
||||
## MITRE ATT&CK
|
||||
Mappings to the MITRE ATT&CK framework are displayed within the baseline documents themselves, within the bulleted list following each policy statement.
|
||||
Mappings to the MITRE ATT&CK framework are displayed within the baseline documents themselves, within the bulleted list following each policy statement.
|
||||
|
||||
## OSCAL Catalog
|
||||
The generated [OSCAL Catalog](oscal-catalog.md) release artifact preserves the
|
||||
NIST SP 800-53 mappings as policy control properties and the MITRE ATT&CK
|
||||
mappings as policy control links. The catalog keeps the source baseline
|
||||
hierarchy so section-level context is represented once and inherited by the
|
||||
policy controls in that section.
|
||||
@@ -0,0 +1,347 @@
|
||||
# OSCAL Catalog
|
||||
|
||||
ScubaGoggles can generate a single OSCAL catalog JSON release artifact from the
|
||||
Google Workspace Secure Configuration Baseline Markdown files in
|
||||
`scubagoggles/baselines`.
|
||||
|
||||
The Markdown baselines remain the source of truth. The generated catalog is a
|
||||
machine-readable representation of the same baseline policy content, grouped by
|
||||
baseline, section, and policy.
|
||||
|
||||
The catalog is intended to carry enough Markdown-derived content for downstream
|
||||
systems to render a complete human-readable baseline view from OSCAL without
|
||||
using the source Markdown document as the only copy of baseline prose.
|
||||
|
||||
The generated OSCAL keeps repeated prose at the highest applicable catalog
|
||||
level. Baseline overview text remains on the baseline group, section overview,
|
||||
prerequisites, and common implementation guidance remain on the section group,
|
||||
and policy-specific statements, details, rationale, notes, and implementation
|
||||
guidance remain on the policy control. This keeps the catalog readable while
|
||||
preserving enough hierarchy for downstream tools to render a complete baseline.
|
||||
|
||||
OSCAL constrains `part.name` values in catalogs. The generator uses OSCAL
|
||||
names such as `overview`, `instruction`, `guidance`, and `statement`, then
|
||||
preserves the source Markdown heading in `part.title` and a stable source-based
|
||||
identifier in `part.id`.
|
||||
|
||||
Generated OSCAL part objects are ordered for readability as `name`, optional
|
||||
`title`, `id`, and then `prose`. JSON object order does not change the OSCAL
|
||||
meaning, but this order helps reviewers scan the OSCAL bucket, source Markdown
|
||||
label, stable identifier, and content in sequence.
|
||||
|
||||
Generated SCuBA property objects are ordered as `name`, `value`, and then
|
||||
`ns`, so readers see the property label and value before the namespace that
|
||||
defines that label.
|
||||
|
||||
## Catalog Excerpt
|
||||
|
||||
The excerpt below shows the generated catalog structure from the top-level
|
||||
`catalog` object down to the first policy in the first baseline group. It uses
|
||||
the real first baseline, section, and policy identifiers from the generated
|
||||
catalog. The shown prose fields are copied from the generated catalog as-is,
|
||||
while sibling groups, controls, properties, links, and back-matter resources are
|
||||
omitted for readability.
|
||||
|
||||
<details>
|
||||
<summary>Show top-down OSCAL catalog excerpt</summary>
|
||||
|
||||
```json
|
||||
{
|
||||
"catalog": {
|
||||
"uuid": "1a21eb2b-368d-5781-84c9-2ccadb555670",
|
||||
"metadata": {
|
||||
"title": "Google Workspace SCuBA Baselines - OSCAL Catalog",
|
||||
"last-modified": "2026-08-18T17:33:09Z",
|
||||
"version": "v1.0.0",
|
||||
"oscal-version": "1.2.2"
|
||||
},
|
||||
"groups": [
|
||||
{
|
||||
"id": "gws-assuredcontrols",
|
||||
"class": "scuba-baseline",
|
||||
"title": "Google Workspace Assured Controls Baseline",
|
||||
"props": [
|
||||
{
|
||||
"name": "source-baseline",
|
||||
"value": "assuredcontrols.md",
|
||||
"ns": "https://cisa.gov/scuba"
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
{
|
||||
"href": "#5774c748-0839-5ce7-bc40-cae2d2730853",
|
||||
"rel": "source",
|
||||
"text": "SCuBA Goggles Assured Controls baseline Markdown"
|
||||
}
|
||||
],
|
||||
"parts": [
|
||||
{
|
||||
"name": "overview",
|
||||
"id": "gws-assuredcontrols_overview",
|
||||
"prose": "# CISA Google Workspace Secure Configuration Baseline for Assured Controls and Assured Controls Plus\n\nAssured Controls and Assured Controls Plus are paid add-ons within Google Workspace (GWS) relating to compliance and security.\nThis Secure Configuration Baseline (SCB) for Assured Controls provides specific policies to strengthen an organization's data security.\nThis baseline is intended as guidance for agencies that already have Assured Controls or Assured Controls Plus licenses.\nUsers who choose to implement this baseline should carefully consider the tradeoffs involved, including the potential security benefits, usability impacts, and possible increased fees for additional licenses.\n\nThe Cybersecurity and Infrastructure Security Agency's (CISA) Secure Cloud Business Applications (SCuBA) project, provides guidance and capabilities to secure federal civilian executive branch (FCEB) agencies' cloud business application environments and protect federal information that is created, accessed, shared, and stored in those environments.\n\nThe CISA SCuBA SCBs for GWS help secure federal information assets stored within GWS cloud business application environments through consistent, effective, and manageable security configurations. CISA created baselines tailored to the federal government's threats and risk tolerance. Organizations outside of the federal government may also find these baselines useful references to help reduce risks even if such organizations have different risk tolerances or face different threats.\n\nFor non-federal users, the information in this document is being provided \"as is\" for INFORMATIONAL PURPOSES ONLY. CISA does not endorse any commercial product or service, including any subjects of analysis. Any reference to specific commercial entities or commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply endorsement, recommendation, or favoritism by CISA. Without limiting the generality of the foregoing, some controls and settings are not available in all products. CISA has no control over vendor changes to products offerings or features. Accordingly, these SCuBA SCBs for GWS may not be applicable to the products available to you. This document does not address, ensure compliance with, or supersede any law, regulation, or other authority. Entities are responsible for complying with any recordkeeping, privacy, and other laws that may apply to the use of technology. This document is not intended to, and does not, create any right or benefit for anyone against the United States, its departments, agencies, or entities, its officers, employees, or agents, or any other person.\n\nThis baseline is based on Google documentation and addresses the following:\nGoogle Support Staff Data Access (https://github.com/cisagov/ScubaGoggles/blob/main/scubagoggles/baselines/assuredcontrols.md#1-google-support-staff-data-access)\nData Regions Advanced Settings (https://github.com/cisagov/ScubaGoggles/blob/main/scubagoggles/baselines/assuredcontrols.md#2-data-regions-advanced-settings)"
|
||||
}
|
||||
],
|
||||
"groups": [
|
||||
{
|
||||
"id": "gws-assuredcontrols-section-1",
|
||||
"class": "scuba-baseline-section",
|
||||
"title": "1. Google Support Staff Data Access",
|
||||
"parts": [
|
||||
{
|
||||
"name": "overview",
|
||||
"id": "gws-assuredcontrols-section-1_overview",
|
||||
"prose": "Google Workspace (GWS) includes several mechanisms to control how Google support staff access an organization's data.\nAccess Approvals requires Google support staff to request approval before viewing an organization's data.\nAccess can also be restricted to specific demographics, such as access by U.S.-based Google staff only.\nHowever, these features require additional licensing and are not available by default with GWS Enterprise Plus."
|
||||
},
|
||||
{
|
||||
"name": "instruction",
|
||||
"title": "Prerequisites",
|
||||
"id": "gws-assuredcontrols-section-1_prerequisites",
|
||||
"prose": "Access Approvals requires either the Assured Controls or the Assured Controls Plus add-ons.\nAccess Management requires the Assured Controls Plus add-on. However, customers who purchased the Assured Controls and the Assured Support add-on prior to June 17, 2024, also have access to Access Management."
|
||||
}
|
||||
],
|
||||
"controls": [
|
||||
{
|
||||
"id": "gws.assuredcontrols.1.1v1",
|
||||
"class": "scuba-policy",
|
||||
"title": "Access Approvals SHOULD be enabled.",
|
||||
"props": [
|
||||
{
|
||||
"name": "source-policy-id",
|
||||
"value": "GWS.ASSUREDCONTROLS.1.1v1",
|
||||
"ns": "https://cisa.gov/scuba"
|
||||
},
|
||||
{
|
||||
"name": "last-modified",
|
||||
"value": "November 2025",
|
||||
"ns": "https://cisa.gov/scuba"
|
||||
},
|
||||
{
|
||||
"name": "nist-sp800-53-rev5-fedramp-high-mapping",
|
||||
"value": "SC-7(10)(a)",
|
||||
"ns": "https://cisa.gov/scuba"
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
{
|
||||
"href": "https://attack.mitre.org/techniques/T1530/",
|
||||
"rel": "threat-mapping",
|
||||
"text": "MITRE ATT&CK T1530: Data from Cloud Storage"
|
||||
}
|
||||
],
|
||||
"parts": [
|
||||
{
|
||||
"name": "statement",
|
||||
"id": "gws.assuredcontrols.1.1v1_statement",
|
||||
"prose": "Access Approvals SHOULD be enabled."
|
||||
},
|
||||
{
|
||||
"name": "guidance",
|
||||
"title": "Rationale",
|
||||
"id": "gws.assuredcontrols.1.1v1_rationale",
|
||||
"prose": "Unauthorized data access increases the risk of exposing sensitive data to untrusted entities. Unauthorized access and actions to an organization's data may be reduced by requiring approval of Google staff requests to access organizations' data."
|
||||
},
|
||||
{
|
||||
"name": "guidance",
|
||||
"title": "Implementation",
|
||||
"id": "gws.assuredcontrols.1.1v1_implementation",
|
||||
"prose": "1. Sign in to the Google Admin console (https://admin.google.com/) as a super admin.\n2. Select Data -\\> Compliance -\\> Access Approvals.\n3. Check the Require Google staff to request approval before viewing data necessary for support services box.\n4. Click SAVE."
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Viewing the Catalog
|
||||
|
||||
After downloading and extracting the generated OSCAL artifact, users can inspect
|
||||
the catalog with an OSCAL-aware viewer such as <https://viewer.oscal.io/>.
|
||||
This can help browse the catalog hierarchy and policy parts. Schema validation
|
||||
is performed by the pinned OSCAL CLI in CI.
|
||||
|
||||
## Part Name Mapping
|
||||
|
||||
The generator uses the following OSCAL part-name assumptions.
|
||||
|
||||
| Markdown content | OSCAL `part.name` | Preserved label |
|
||||
| --- | --- | --- |
|
||||
| Main SHALL/SHOULD policy sentence | `statement` | Not needed; also stored in `control.title` |
|
||||
| Baseline introduction | `overview` | Not needed |
|
||||
| Baseline assumptions | `overview` | `title: "Assumptions"` |
|
||||
| Baseline key terminology | `overview` | `title: "Key Terminology"` |
|
||||
| Section overview | `overview` | Not needed |
|
||||
| Section prerequisites | `instruction` | `title: "Prerequisites"` |
|
||||
| Common section implementation steps | `instruction` | `title: "Implementation"` |
|
||||
| Policy detail | `guidance` | `title: "Policy Detail"` |
|
||||
| Rationale | `guidance` | `title: "Rationale"` |
|
||||
| Policy-specific implementation steps | `guidance` | `title: "Implementation"` |
|
||||
| Note | `guidance` | `title: "Note"` |
|
||||
|
||||
## Markdown to OSCAL Cross-Walk
|
||||
|
||||
The tables below summarize how source Markdown content is represented in the
|
||||
generated OSCAL catalog. They are split by catalog layer so the documentation
|
||||
stays readable in GitHub's Markdown view.
|
||||
|
||||
### Catalog Inputs
|
||||
|
||||
| Markdown or input | OSCAL representation |
|
||||
| --- | --- |
|
||||
| Baseline README list | Creates ordered top-level `catalog.groups[]`. |
|
||||
| Generated release version | Stored in `catalog.metadata.version` and the generated filename for release artifacts. |
|
||||
| Ad hoc generator run | Uses the ScubaGoggles package version with a development build label in `catalog.metadata.version`. |
|
||||
| OSCAL version constant | Stored in `catalog.metadata.oscal-version` and the generated filename. |
|
||||
|
||||
The baseline list comes from `scubagoggles/baselines/README.md`.
|
||||
The OSCAL version is defined by `DEFAULT_OSCAL_VERSION` in
|
||||
`scubagoggles/scuba_constants.py`.
|
||||
Markdown links copied into OSCAL prose are resolved to absolute GitHub URLs so
|
||||
the generated catalog remains portable outside the source repository checkout.
|
||||
|
||||
### Baseline Groups
|
||||
|
||||
| Markdown source | OSCAL representation |
|
||||
| --- | --- |
|
||||
| Baseline title and file name | Each README-listed baseline becomes one top-level group. |
|
||||
| Baseline title | Stored in `catalog.groups[].title`. |
|
||||
| Source Markdown filename | Stored in a `source-baseline` property. |
|
||||
| Source Markdown file | Linked from the baseline group with `rel=source`. |
|
||||
| Baseline introduction | Stored as an `overview` part on the baseline group. |
|
||||
| Baseline preamble sections | Stored as titled `overview` parts on the baseline group. |
|
||||
|
||||
Examples of baseline preamble sections include `## Assumptions` and
|
||||
`## Key Terminology`.
|
||||
|
||||
### Section Groups
|
||||
|
||||
| Markdown source | OSCAL representation |
|
||||
| --- | --- |
|
||||
| Numbered Markdown section | Becomes a nested group under the baseline group. |
|
||||
| Section heading | Stored in the section group's `title`. |
|
||||
| Section overview text | Stored once as an `overview` part on the section group. |
|
||||
| `### Prerequisites` | Stored once as an `instruction` part titled `Prerequisites`. |
|
||||
| Common implementation instructions | Stored once as an `Implementation` instruction. |
|
||||
|
||||
Section-level overview, prerequisites, and common implementation guidance are
|
||||
not repeated on every child control. OSCAL consumers should read them from the
|
||||
containing section group.
|
||||
|
||||
For example, `## 1. Phishing-Resistant Multifactor Authentication` becomes a
|
||||
nested section group.
|
||||
|
||||
### Policy Controls
|
||||
|
||||
| Markdown source | OSCAL representation |
|
||||
| --- | --- |
|
||||
| `### Policies` block | Each SCuBA policy becomes one OSCAL control. |
|
||||
| Policy heading | The normalized value becomes `control.id`. |
|
||||
| Original SCuBA policy ID | Preserved in a `source-policy-id` property. |
|
||||
| Main policy sentence | Stored in `control.title` and a `statement` part. |
|
||||
| Additional policy detail | Stored as a `guidance` part titled `Policy Detail`. |
|
||||
| `_Rationale:_` | Stored as a `guidance` part titled `Rationale`. |
|
||||
| `_Note:_` | Stored as a `guidance` part titled `Note`. |
|
||||
| Policy-specific implementation | Stored as a `guidance` part titled `Implementation`. |
|
||||
|
||||
The main policy statement is stored in both `control.title` and
|
||||
`control.parts[name=statement]`. The statement part is the authoritative copy for
|
||||
future OSCAL-to-Markdown conversion.
|
||||
|
||||
### Policy Properties and Links
|
||||
|
||||
| Markdown source | OSCAL representation |
|
||||
| --- | --- |
|
||||
| Policy badges | Converted to boolean-like string properties. |
|
||||
| `_Last modified:_` | Stored in a `last-modified` property. |
|
||||
| FedRAMP/NIST mapping line | Stored in a FedRAMP High mapping property. |
|
||||
| Multiple NIST/FedRAMP mappings | Counted in a source mapping count property. |
|
||||
| MITRE ATT&CK links | Stored as `threat-mapping` links on the control. |
|
||||
| Resource links | Stored in back matter and linked as references. |
|
||||
| Embedded baseline images | Preserved as back-matter resources and reference links. |
|
||||
|
||||
Shared references are de-duplicated in back matter and linked from applicable
|
||||
controls.
|
||||
This includes `### Resources` links, section overview links, and embedded
|
||||
source images such as `<img src="images/MFA.PNG">`.
|
||||
|
||||
The FedRAMP High mapping property name is
|
||||
`nist-sp800-53-rev5-fedramp-high-mapping`.
|
||||
|
||||
## Policy Example
|
||||
|
||||
The example below shows a concrete mapping for
|
||||
`GWS.ASSUREDCONTROLS.1.1v1` from `assuredcontrols.md`.
|
||||
|
||||
### Example Identity
|
||||
|
||||
| Field | Example |
|
||||
| --- | --- |
|
||||
| Baseline Markdown | `assuredcontrols.md` |
|
||||
| Baseline group title | `Google Workspace Assured Controls Baseline` |
|
||||
| Section Markdown | `## 1. Google Support Staff Data Access` |
|
||||
| Section group title | `1. Google Support Staff Data Access` |
|
||||
| Policy Markdown | `#### GWS.ASSUREDCONTROLS.1.1v1` |
|
||||
| OSCAL control ID | `gws.assuredcontrols.1.1v1` |
|
||||
|
||||
The original policy ID is also preserved in a `source-policy-id` property with
|
||||
the value `GWS.ASSUREDCONTROLS.1.1v1`.
|
||||
|
||||
### Example Inherited Context
|
||||
|
||||
| Markdown content | OSCAL representation |
|
||||
| --- | --- |
|
||||
| Section overview text | Stored once in the section group's `overview` part. |
|
||||
| `### Prerequisites` | Stored once in an `instruction` part titled `Prerequisites`. |
|
||||
|
||||
In this example, the section overview describes Google support staff data access
|
||||
mechanisms, and the prerequisites describe licensing assumptions.
|
||||
|
||||
### Example Policy Content
|
||||
|
||||
| Markdown content | OSCAL representation |
|
||||
| --- | --- |
|
||||
| `Access Approvals SHOULD be enabled.` | Stored in `control.title` and the `statement` part. |
|
||||
| `_Rationale:_ Unauthorized access to data increases...` | Stored in `guidance` titled `Rationale`. |
|
||||
| Policy implementation instructions | Stored in `guidance` titled `Implementation`. |
|
||||
|
||||
### Example Policy Metadata
|
||||
|
||||
| Markdown content | OSCAL representation |
|
||||
| --- | --- |
|
||||
| `_Last modified:_ November 2025` | `last-modified = "November 2025"` |
|
||||
| FedRAMP High mapping | `SC-7(10)(a)` |
|
||||
| MITRE ATT&CK links | Stored as `threat-mapping` links on the control. |
|
||||
| Automated Check badge | `automated-check = "true"` |
|
||||
| Log-Based Check badge | `log-based-check = "true"` |
|
||||
|
||||
The example MITRE ATT&CK links are `T1530`, `T1537`, and `T1589`.
|
||||
|
||||
## Hierarchy and Flattening
|
||||
|
||||
OSCAL consumers should read inherited context from the containing catalog
|
||||
groups. For example, a renderer for `gws.assuredcontrols.1.1v1` should combine
|
||||
the control content with the overview, prerequisites, and common implementation
|
||||
parts on its parent section group.
|
||||
|
||||
The catalog intentionally avoids repeating section-level prose on every child
|
||||
control. If a later release adds a CSV or other row-oriented artifact, that
|
||||
artifact can intentionally flatten inherited section context into each row
|
||||
without making the canonical OSCAL catalog repetitive.
|
||||
|
||||
## Human-Readable Rendering
|
||||
|
||||
The generator keeps the main policy statement separate from supplemental policy
|
||||
detail. For example, `control.parts[name=statement]` contains the SHALL/SHOULD
|
||||
statement, while `control.parts[name=guidance,title=Policy Detail]` contains
|
||||
extra explanatory content that appears before the rationale in Markdown.
|
||||
|
||||
This structure is intended to help any downstream OSCAL consumer render a full
|
||||
human-readable baseline without losing baseline preamble text, policy
|
||||
statements, supplemental policy detail, rationale, implementation guidance, or
|
||||
other source content represented in the Markdown baselines.
|
||||
@@ -7,6 +7,9 @@
|
||||
{
|
||||
"pattern":"^https://github.com/cisagov/ScubaGoggles/blob/main*"
|
||||
},
|
||||
{
|
||||
"pattern":"^https://viewer\\.oscal\\.io"
|
||||
},
|
||||
{
|
||||
"pattern":"^mailto:user@company.com"
|
||||
},
|
||||
|
||||
@@ -0,0 +1,324 @@
|
||||
"""Unit tests for OSCAL baseline catalog generation."""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from scubagoggles.scuba_constants import DEFAULT_OSCAL_VERSION
|
||||
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[4]
|
||||
GENERATOR_PATH = REPO_ROOT / "scubagoggles" / "utils" / "generate_oscal_baselines.py"
|
||||
OSCAL_GROUP_PART_NAMES = {"instruction", "overview"}
|
||||
OSCAL_CONTROL_PART_NAMES = {
|
||||
"assessment",
|
||||
"assessment-method",
|
||||
"assessment-objective",
|
||||
"example",
|
||||
"guidance",
|
||||
"objective",
|
||||
"overview",
|
||||
"statement",
|
||||
}
|
||||
|
||||
|
||||
def load_generator():
|
||||
"""Load the OSCAL generator module by path."""
|
||||
|
||||
spec = importlib.util.spec_from_file_location("generate_oscal_baselines", GENERATOR_PATH)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError(f"Could not load {GENERATOR_PATH}")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def source_policy_count(path):
|
||||
"""Return the number of unique SCuBA policy IDs in a Markdown file."""
|
||||
|
||||
text = path.read_text(encoding="utf-8")
|
||||
ids = re.findall(
|
||||
r"^#### (GWS\.[A-Z]+(?:CONTROLS)?\.\d+\.\d+v\d+)\s*$",
|
||||
text,
|
||||
re.M,
|
||||
)
|
||||
return len(ids)
|
||||
|
||||
|
||||
def source_mapping_count(path):
|
||||
"""Return the total count of source NIST mapping entries."""
|
||||
|
||||
text = path.read_text(encoding="utf-8")
|
||||
mapping_lines = re.findall(
|
||||
r"NIST SP 800-53 Rev\. 5 FedRAMP High Baseline Mapping:_\s*([^\r\n]+)",
|
||||
text,
|
||||
)
|
||||
return sum(len(line.split(",")) for line in mapping_lines)
|
||||
|
||||
|
||||
def catalog_controls(catalog):
|
||||
"""Yield every control from the generated catalog."""
|
||||
|
||||
for baseline_group in catalog["groups"]:
|
||||
for section_group in baseline_group.get("groups", []):
|
||||
yield from section_group.get("controls", [])
|
||||
|
||||
|
||||
def prop_values(props, name):
|
||||
"""Return all values for a property name."""
|
||||
|
||||
return [prop["value"] for prop in props if prop["name"] == name]
|
||||
|
||||
|
||||
def assert_prop_order(props):
|
||||
"""Assert generated properties read as name, value, then namespace."""
|
||||
|
||||
for prop in props:
|
||||
assert list(prop) == ["name", "value", "ns"]
|
||||
|
||||
|
||||
def iter_prose(value):
|
||||
"""Yield every prose string from a generated OSCAL object."""
|
||||
|
||||
if isinstance(value, dict):
|
||||
if "prose" in value:
|
||||
yield value["prose"]
|
||||
for child in value.values():
|
||||
yield from iter_prose(child)
|
||||
elif isinstance(value, list):
|
||||
for child in value:
|
||||
yield from iter_prose(child)
|
||||
|
||||
|
||||
def part_values(parts, name, title=None):
|
||||
"""Return all prose values for a part name."""
|
||||
|
||||
return [
|
||||
part["prose"]
|
||||
for part in parts
|
||||
if part["name"] == name and (title is None or part.get("title") == title)
|
||||
]
|
||||
|
||||
|
||||
class GenerateOscalBaselinesTest:
|
||||
"""Tests for converting Markdown baselines to one OSCAL catalog."""
|
||||
|
||||
def test_policy_ids_become_catalog_control_ids(self):
|
||||
"""SCuBA policy IDs should become stable catalog control IDs."""
|
||||
|
||||
generator = load_generator()
|
||||
|
||||
assert (
|
||||
generator.control_id_from_policy("GWS.COMMONCONTROLS.1.1v1")
|
||||
== "gws.commoncontrols.1.1v1"
|
||||
)
|
||||
|
||||
def test_catalog_filename_includes_relevant_versions(self):
|
||||
"""Release catalog filenames should show source and OSCAL versions."""
|
||||
|
||||
generator = load_generator()
|
||||
expected_default = (
|
||||
f"scubagoggles-oscal-catalog-oscal-{generator.DEFAULT_OSCAL_VERSION}.json"
|
||||
)
|
||||
expected_release = (
|
||||
f"scubagoggles-v1.0.0-oscal-catalog-oscal-"
|
||||
f"{generator.DEFAULT_OSCAL_VERSION}.json"
|
||||
)
|
||||
|
||||
assert generator.catalog_file_name() == expected_default
|
||||
assert generator.catalog_file_name("1.0.0") == expected_release
|
||||
assert generator.catalog_file_name("v1.0.0") == expected_release
|
||||
|
||||
def test_catalog_metadata_version_marks_ad_hoc_runs(self):
|
||||
"""Ad hoc catalog metadata should distinguish non-release generation."""
|
||||
|
||||
generator = load_generator()
|
||||
|
||||
assert generator.catalog_metadata_version("1.0.0") == "1.0.0"
|
||||
assert generator.catalog_metadata_version() == (
|
||||
f"{generator.SCUBAGOGGLES_VERSION} "
|
||||
f"({generator.DEVELOPMENT_VERSION_LABEL})"
|
||||
)
|
||||
|
||||
def test_clean_text_resolves_relative_links_in_prose(self):
|
||||
"""Relative Markdown links copied into OSCAL prose should be portable."""
|
||||
|
||||
generator = load_generator()
|
||||
|
||||
prose = generator.clean_text(
|
||||
(
|
||||
"See [Limitations](../../docs/usage/Limitations.md"
|
||||
"#log-based-policy-checks) and [this section](#key-terminology)."
|
||||
),
|
||||
"assuredcontrols.md",
|
||||
)
|
||||
|
||||
assert (
|
||||
"Limitations "
|
||||
"(https://github.com/cisagov/ScubaGoggles/blob/main/"
|
||||
"docs/usage/Limitations.md#log-based-policy-checks)"
|
||||
) in prose
|
||||
assert (
|
||||
"this section "
|
||||
"(https://github.com/cisagov/ScubaGoggles/blob/main/"
|
||||
"scubagoggles/baselines/assuredcontrols.md#key-terminology)"
|
||||
) in prose
|
||||
|
||||
def test_generation_covers_readme_baselines(self, tmp_path):
|
||||
"""Every README-listed baseline should be represented in one catalog."""
|
||||
|
||||
generator = load_generator()
|
||||
input_dir = REPO_ROOT / "scubagoggles" / "baselines"
|
||||
output_dir = tmp_path / "oscal-baseline-catalog"
|
||||
release_version = "0.0.0-test"
|
||||
expected_catalog_file = generator.catalog_file_name(release_version)
|
||||
|
||||
summary = generator.generate_baselines(input_dir, output_dir, release_version)
|
||||
discovered = generator.parse_baselines_readme(input_dir)
|
||||
catalog_path = output_dir / expected_catalog_file
|
||||
catalog = json.loads(catalog_path.read_text(encoding="utf-8"))["catalog"]
|
||||
controls = list(catalog_controls(catalog))
|
||||
|
||||
assert generator.DEFAULT_OSCAL_VERSION == DEFAULT_OSCAL_VERSION
|
||||
assert summary["output"] == expected_catalog_file
|
||||
assert f"v{release_version}" in catalog_path.name
|
||||
assert generator.DEFAULT_OSCAL_VERSION in catalog_path.name
|
||||
assert summary["baselines"] == len(discovered)
|
||||
assert summary["source_policies"] == sum(
|
||||
source_policy_count(input_dir / file_name) for _, file_name in discovered
|
||||
)
|
||||
assert summary["source_mappings"] == sum(
|
||||
source_mapping_count(input_dir / file_name) for _, file_name in discovered
|
||||
)
|
||||
assert (output_dir / "generation-summary.json").exists()
|
||||
assert not list(output_dir.glob("*-oscal-component-definition.json"))
|
||||
|
||||
assert catalog["metadata"]["oscal-version"] == generator.DEFAULT_OSCAL_VERSION
|
||||
assert catalog["metadata"]["version"] == release_version
|
||||
assert_prop_order(catalog["metadata"]["props"])
|
||||
assert len(catalog["groups"]) == len(discovered)
|
||||
assert len(controls) == summary["source_policies"]
|
||||
for prose in iter_prose(catalog):
|
||||
assert not any(
|
||||
marker in prose
|
||||
for marker in ("(../../", "(../", "(./", "(images/", "(#")
|
||||
)
|
||||
|
||||
for baseline_group in catalog["groups"]:
|
||||
assert_prop_order(baseline_group["props"])
|
||||
assert all(
|
||||
part["name"] in OSCAL_GROUP_PART_NAMES
|
||||
for part in baseline_group.get("parts", [])
|
||||
)
|
||||
source_baseline = prop_values(baseline_group["props"], "source-baseline")[0]
|
||||
group_controls = [
|
||||
control
|
||||
for section_group in baseline_group.get("groups", [])
|
||||
for control in section_group.get("controls", [])
|
||||
]
|
||||
assert len(group_controls) == source_policy_count(input_dir / source_baseline)
|
||||
for section_group in baseline_group.get("groups", []):
|
||||
assert_prop_order(section_group["props"])
|
||||
assert all(
|
||||
part["name"] in OSCAL_GROUP_PART_NAMES
|
||||
for part in section_group.get("parts", [])
|
||||
)
|
||||
|
||||
assured_controls = next(
|
||||
group
|
||||
for group in catalog["groups"]
|
||||
if prop_values(group["props"], "source-baseline") == ["assuredcontrols.md"]
|
||||
)
|
||||
common_controls = next(
|
||||
group
|
||||
for group in catalog["groups"]
|
||||
if prop_values(group["props"], "source-baseline") == ["commoncontrols.md"]
|
||||
)
|
||||
mfa_section = next(
|
||||
section
|
||||
for section in common_controls["groups"]
|
||||
if prop_values(section["props"], "source-section") == ["1"]
|
||||
)
|
||||
assert "Assured Controls Plus add-on" in part_values(
|
||||
assured_controls["parts"],
|
||||
"overview",
|
||||
"Assumptions",
|
||||
)[0]
|
||||
assert 'The key words "MUST,"' in part_values(
|
||||
assured_controls["parts"],
|
||||
"overview",
|
||||
"Key Terminology",
|
||||
)[0]
|
||||
assert part_values(mfa_section["parts"], "overview")
|
||||
assert "FIDO2-compliant security keys" in part_values(
|
||||
mfa_section["parts"],
|
||||
"instruction",
|
||||
"Prerequisites",
|
||||
)[0]
|
||||
assert "Policy 1 Common Instructions" in part_values(
|
||||
mfa_section["parts"],
|
||||
"instruction",
|
||||
"Implementation",
|
||||
)[0]
|
||||
|
||||
for control in controls:
|
||||
assert_prop_order(control["props"])
|
||||
policy_ids = prop_values(control["props"], "source-policy-id")
|
||||
mappings = prop_values(
|
||||
control["props"],
|
||||
"nist-sp800-53-rev5-fedramp-high-mapping",
|
||||
)
|
||||
assert len(policy_ids) == 1
|
||||
assert control["id"] == generator.control_id_from_policy(policy_ids[0])
|
||||
assert len(mappings) == 1
|
||||
assert any(part["name"] == "statement" for part in control["parts"])
|
||||
assert all(
|
||||
part["name"] in OSCAL_CONTROL_PART_NAMES
|
||||
for part in control.get("parts", [])
|
||||
)
|
||||
link_keys = [
|
||||
(link["rel"], link["href"]) for link in control.get("links", [])
|
||||
]
|
||||
assert len(link_keys) == len(set(link_keys))
|
||||
assert not {
|
||||
"section-prerequisites",
|
||||
"source-note",
|
||||
"source-policy-detail",
|
||||
"source-section-overview",
|
||||
}.intersection(prop["name"] for prop in control["props"])
|
||||
|
||||
phishing_resistant_mfa = next(
|
||||
control
|
||||
for control in controls
|
||||
if prop_values(control["props"], "source-policy-id")
|
||||
== ["GWS.COMMONCONTROLS.1.1v1"]
|
||||
)
|
||||
assert part_values(phishing_resistant_mfa["parts"], "statement") == [
|
||||
"Phishing-Resistant MFA SHALL be required for all users."
|
||||
]
|
||||
statement_part = next(
|
||||
part
|
||||
for part in phishing_resistant_mfa["parts"]
|
||||
if part["name"] == "statement"
|
||||
)
|
||||
assert list(statement_part) == ["name", "id", "prose"]
|
||||
phishing_implementation = part_values(
|
||||
phishing_resistant_mfa["parts"],
|
||||
"guidance",
|
||||
"Implementation",
|
||||
)[0]
|
||||
implementation_part = next(
|
||||
part
|
||||
for part in phishing_resistant_mfa["parts"]
|
||||
if part["name"] == "guidance"
|
||||
and part.get("title") == "Implementation"
|
||||
)
|
||||
assert list(implementation_part) == ["name", "title", "id", "prose"]
|
||||
assert "Under Authentication" in phishing_implementation
|
||||
assert "Sign in to" not in phishing_implementation
|
||||
assert "FIDO2 Security Key" in part_values(
|
||||
phishing_resistant_mfa["parts"],
|
||||
"guidance",
|
||||
"Policy Detail",
|
||||
)[0]
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Unit tests for generated OSCAL baseline catalog validation."""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[4]
|
||||
GENERATOR_PATH = REPO_ROOT / "scubagoggles" / "utils" / "generate_oscal_baselines.py"
|
||||
VALIDATOR_PATH = REPO_ROOT / "scubagoggles" / "utils" / "validate_oscal_baselines.py"
|
||||
|
||||
|
||||
def load_module(path, name):
|
||||
"""Load a module by path."""
|
||||
|
||||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError(f"Could not load {path}")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def test_validate_release_catalog_output(tmp_path):
|
||||
"""Release catalog output should pass validation."""
|
||||
|
||||
generator = load_module(GENERATOR_PATH, "generate_oscal_baselines")
|
||||
validator = load_module(VALIDATOR_PATH, "validate_oscal_baselines")
|
||||
output_dir = tmp_path / "oscal-baseline-catalog"
|
||||
|
||||
generator.generate_baselines(
|
||||
REPO_ROOT / "scubagoggles" / "baselines",
|
||||
output_dir,
|
||||
"1.0.0",
|
||||
)
|
||||
|
||||
catalog_path = validator.validate_catalog_output(output_dir, "1.0.0")
|
||||
|
||||
assert catalog_path.name == validator.expected_catalog_file_name("1.0.0")
|
||||
|
||||
|
||||
def test_validate_ad_hoc_catalog_output(tmp_path):
|
||||
"""Ad hoc catalog output should pass validation."""
|
||||
|
||||
generator = load_module(GENERATOR_PATH, "generate_oscal_baselines")
|
||||
validator = load_module(VALIDATOR_PATH, "validate_oscal_baselines")
|
||||
output_dir = tmp_path / "oscal-baseline-catalog"
|
||||
|
||||
generator.generate_baselines(
|
||||
REPO_ROOT / "scubagoggles" / "baselines",
|
||||
output_dir,
|
||||
)
|
||||
|
||||
catalog_path = validator.validate_catalog_output(output_dir)
|
||||
|
||||
assert catalog_path.name == validator.expected_catalog_file_name()
|
||||
|
||||
|
||||
def test_validate_catalog_output_rejects_unexpected_filename(tmp_path):
|
||||
"""Validation should fail when the summary points to the wrong filename."""
|
||||
|
||||
validator = load_module(VALIDATOR_PATH, "validate_oscal_baselines")
|
||||
output_dir = tmp_path / "oscal-baseline-catalog"
|
||||
output_dir.mkdir()
|
||||
(output_dir / "unexpected.json").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"catalog": {
|
||||
"metadata": {
|
||||
"oscal-version": validator.DEFAULT_OSCAL_VERSION,
|
||||
"version": validator.expected_metadata_version(),
|
||||
}
|
||||
}
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(output_dir / "generation-summary.json").write_text(
|
||||
json.dumps({"output": "unexpected.json"}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Expected generated catalog"):
|
||||
validator.validate_catalog_output(output_dir)
|
||||
@@ -57,6 +57,7 @@ API_LINKS = {
|
||||
|
||||
|
||||
OPA_VERSION = 'v1.20.1'
|
||||
DEFAULT_OSCAL_VERSION = '1.2.2'
|
||||
|
||||
NUMBER_OF_UUID_CHARACTERS_TO_TRUNCATE_CHOICES = (
|
||||
0, 13, 18, 36
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,172 @@
|
||||
"""Validate generated SCuBA OSCAL baseline catalog outputs."""
|
||||
|
||||
# pylint: disable=wrong-import-position
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import TypeAlias
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(REPO_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from scubagoggles import __version__ as SCUBAGOGGLES_VERSION
|
||||
from scubagoggles.scuba_constants import DEFAULT_OSCAL_VERSION
|
||||
|
||||
|
||||
JsonValue: TypeAlias = (
|
||||
str | int | float | bool | None | list["JsonValue"] | dict[str, "JsonValue"]
|
||||
)
|
||||
JsonObject: TypeAlias = dict[str, JsonValue]
|
||||
|
||||
DEVELOPMENT_VERSION_LABEL = "development build"
|
||||
SUMMARY_FILE = "generation-summary.json"
|
||||
|
||||
|
||||
def version_filename_fragment(version: str) -> str:
|
||||
"""Return a safe version fragment for generated filenames."""
|
||||
|
||||
fragment = re.sub(r"[^a-z0-9.-]+", "-", version.strip().lower()).strip("-")
|
||||
if fragment.startswith("v"):
|
||||
fragment = fragment[1:]
|
||||
return fragment or "unversioned"
|
||||
|
||||
|
||||
def expected_catalog_file_name(release_version: str | None = None) -> str:
|
||||
"""Return the expected generated OSCAL catalog filename."""
|
||||
|
||||
if release_version:
|
||||
source_version = version_filename_fragment(release_version)
|
||||
return (
|
||||
f"scubagoggles-v{source_version}-oscal-catalog-"
|
||||
f"oscal-{DEFAULT_OSCAL_VERSION}.json"
|
||||
)
|
||||
return f"scubagoggles-oscal-catalog-oscal-{DEFAULT_OSCAL_VERSION}.json"
|
||||
|
||||
|
||||
def expected_metadata_version(release_version: str | None = None) -> str:
|
||||
"""Return the expected OSCAL metadata version."""
|
||||
|
||||
if release_version:
|
||||
return release_version
|
||||
return f"{SCUBAGOGGLES_VERSION} ({DEVELOPMENT_VERSION_LABEL})"
|
||||
|
||||
|
||||
def read_json_object(path: Path) -> JsonObject:
|
||||
"""Read a JSON file and require its top-level value to be an object."""
|
||||
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError(f"{path} must contain a JSON object")
|
||||
return data
|
||||
|
||||
|
||||
def require_json_object(value: JsonValue | None, label: str) -> JsonObject:
|
||||
"""Require a JSON value to be an object."""
|
||||
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError(f"{label} must be a JSON object")
|
||||
return value
|
||||
|
||||
|
||||
def require_string(value: JsonValue | None, label: str) -> str:
|
||||
"""Require a JSON value to be a string."""
|
||||
|
||||
if not isinstance(value, str):
|
||||
raise ValueError(f"{label} must be a string")
|
||||
return value
|
||||
|
||||
|
||||
def validate_catalog_output(
|
||||
output_dir: Path,
|
||||
release_version: str | None = None,
|
||||
) -> Path:
|
||||
"""Validate generated catalog summary and metadata, returning catalog path."""
|
||||
|
||||
summary_path = output_dir / SUMMARY_FILE
|
||||
if not summary_path.exists():
|
||||
raise FileNotFoundError(f"Missing OSCAL generation summary: {summary_path}")
|
||||
|
||||
summary = read_json_object(summary_path)
|
||||
output_name = require_string(summary.get("output"), "summary.output")
|
||||
expected_output_name = expected_catalog_file_name(release_version)
|
||||
if output_name != expected_output_name:
|
||||
raise ValueError(
|
||||
f"Expected generated catalog {expected_output_name}, got {output_name}"
|
||||
)
|
||||
|
||||
catalog_path = output_dir / output_name
|
||||
if not catalog_path.exists():
|
||||
raise FileNotFoundError(f"Missing generated OSCAL catalog: {catalog_path}")
|
||||
|
||||
catalog_json = read_json_object(catalog_path)
|
||||
catalog = require_json_object(catalog_json.get("catalog"), "catalog")
|
||||
metadata = require_json_object(catalog.get("metadata"), "catalog.metadata")
|
||||
|
||||
oscal_version = require_string(
|
||||
metadata.get("oscal-version"),
|
||||
"catalog.metadata.oscal-version",
|
||||
)
|
||||
if oscal_version != DEFAULT_OSCAL_VERSION:
|
||||
raise ValueError(
|
||||
f"Generated OSCAL catalog uses OSCAL {oscal_version}; "
|
||||
f"expected {DEFAULT_OSCAL_VERSION}"
|
||||
)
|
||||
|
||||
metadata_version = require_string(
|
||||
metadata.get("version"),
|
||||
"catalog.metadata.version",
|
||||
)
|
||||
expected_version = expected_metadata_version(release_version)
|
||||
if metadata_version != expected_version:
|
||||
raise ValueError(
|
||||
f"Generated OSCAL catalog metadata.version is {metadata_version}; "
|
||||
f"expected {expected_version}"
|
||||
)
|
||||
|
||||
return catalog_path
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
"""Parse command-line arguments."""
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Validate a generated SCuBA OSCAL baseline catalog."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
type=Path,
|
||||
default=REPO_ROOT / "dist" / "oscal-baseline-catalog",
|
||||
help="Directory containing generated OSCAL catalog files.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--release-version",
|
||||
default=None,
|
||||
help="Release version expected in the generated OSCAL catalog.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--print-catalog-path",
|
||||
action="store_true",
|
||||
help="Print only the validated catalog path.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Run the validator."""
|
||||
|
||||
args = parse_args()
|
||||
catalog_path = validate_catalog_output(args.output_dir, args.release_version)
|
||||
if args.print_catalog_path:
|
||||
print(catalog_path)
|
||||
else:
|
||||
print(f"Validated OSCAL catalog JSON: {catalog_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in new issue
Block a user