mirror of
https://github.com/mukul975/Anthropic-Cybersecurity-Skills.git
synced 2026-08-01 00:37:42 +03:00
Initial commit - 611 cybersecurity skills across all subdomains
This commit is contained in:
@@ -0,0 +1,318 @@
|
||||
---
|
||||
name: integrating-sast-into-github-actions-pipeline
|
||||
description: >
|
||||
This skill covers integrating Static Application Security Testing (SAST) tools—CodeQL
|
||||
and Semgrep—into GitHub Actions CI/CD pipelines. It addresses configuring automated code
|
||||
scanning on pull requests and pushes, tuning rules to reduce false positives, uploading
|
||||
SARIF results to GitHub Advanced Security, and establishing quality gates that block merges
|
||||
when high-severity vulnerabilities are detected.
|
||||
domain: cybersecurity
|
||||
subdomain: devsecops
|
||||
tags: [devsecops, cicd, sast, codeql, semgrep, secure-sdlc]
|
||||
version: 1.0.0
|
||||
author: mahipal
|
||||
license: MIT
|
||||
---
|
||||
|
||||
# Integrating SAST into GitHub Actions Pipeline
|
||||
|
||||
## When to Use
|
||||
|
||||
- When development teams need automated code-level vulnerability detection on every pull request
|
||||
- When security teams require consistent SAST enforcement across all repositories in an organization
|
||||
- When migrating from manual or periodic security reviews to continuous security testing
|
||||
- When compliance frameworks (SOC 2, PCI DSS, NIST SSDF) require evidence of automated code analysis
|
||||
- When multiple languages coexist in a monorepo and need unified scanning under one workflow
|
||||
|
||||
**Do not use** for runtime vulnerability detection (use DAST instead), for scanning third-party dependencies (use SCA tools like Snyk), or for infrastructure-as-code scanning (use Checkov or tfsec).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- GitHub repository with GitHub Actions enabled
|
||||
- GitHub Advanced Security license (required for CodeQL on private repos; free for public repos)
|
||||
- Semgrep account for managed rules and Semgrep App dashboard (free tier available)
|
||||
- Repository code in a supported language: Python, JavaScript/TypeScript, Java, C/C++, C#, Go, Ruby, Swift, Kotlin
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Configure CodeQL Analysis Workflow
|
||||
|
||||
Create a CodeQL workflow that runs on pull requests and on a weekly schedule to catch vulnerabilities in existing code.
|
||||
|
||||
```yaml
|
||||
# .github/workflows/codeql-analysis.yml
|
||||
name: "CodeQL Analysis"
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main, develop]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
schedule:
|
||||
- cron: '30 2 * * 1' # Weekly Monday 2:30 AM
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
name: Analyze (${{ matrix.language }})
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
actions: read
|
||||
contents: read
|
||||
security-events: write
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
language: ['javascript', 'python']
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v3
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
queries: security-extended,security-and-quality
|
||||
|
||||
- name: Autobuild
|
||||
uses: github/codeql-action/autobuild@v3
|
||||
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@v3
|
||||
with:
|
||||
category: "/language:${{ matrix.language }}"
|
||||
```
|
||||
|
||||
### Step 2: Add Semgrep Scanning for Custom Rules
|
||||
|
||||
Semgrep complements CodeQL with faster scans and support for custom pattern-based rules. Configure it to upload SARIF results to the same GitHub Security tab.
|
||||
|
||||
```yaml
|
||||
# .github/workflows/semgrep.yml
|
||||
name: "Semgrep SAST Scan"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main, develop]
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
semgrep:
|
||||
name: Semgrep Scan
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
security-events: write
|
||||
contents: read
|
||||
|
||||
container:
|
||||
image: semgrep/semgrep:latest
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Run Semgrep
|
||||
run: |
|
||||
semgrep ci \
|
||||
--config auto \
|
||||
--config p/owasp-top-ten \
|
||||
--config p/cwe-top-25 \
|
||||
--sarif --output semgrep-results.sarif \
|
||||
--severity ERROR \
|
||||
--error
|
||||
env:
|
||||
SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}
|
||||
|
||||
- name: Upload SARIF
|
||||
if: always()
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
with:
|
||||
sarif_file: semgrep-results.sarif
|
||||
category: semgrep
|
||||
```
|
||||
|
||||
### Step 3: Create Custom Semgrep Rules for Organization Patterns
|
||||
|
||||
Write organization-specific rules to catch patterns unique to your codebase, such as deprecated internal APIs or insecure configuration patterns.
|
||||
|
||||
```yaml
|
||||
# .semgrep/custom-rules.yml
|
||||
rules:
|
||||
- id: hardcoded-database-url
|
||||
patterns:
|
||||
- pattern: |
|
||||
$DB_URL = "...$PROTO://...:...$PASS@..."
|
||||
message: |
|
||||
Hardcoded database connection string with credentials detected.
|
||||
Use environment variables or a secrets manager instead.
|
||||
languages: [python, javascript, typescript]
|
||||
severity: ERROR
|
||||
metadata:
|
||||
cwe: "CWE-798: Use of Hard-coded Credentials"
|
||||
owasp: "A07:2021 - Identification and Authentication Failures"
|
||||
|
||||
- id: unsafe-deserialization
|
||||
patterns:
|
||||
- pattern-either:
|
||||
- pattern: pickle.loads(...)
|
||||
- pattern: yaml.load(..., Loader=yaml.Loader)
|
||||
- pattern: yaml.load(..., Loader=yaml.FullLoader)
|
||||
message: |
|
||||
Unsafe deserialization detected. Use safe alternatives to prevent
|
||||
remote code execution vulnerabilities.
|
||||
languages: [python]
|
||||
severity: ERROR
|
||||
metadata:
|
||||
cwe: "CWE-502: Deserialization of Untrusted Data"
|
||||
|
||||
- id: missing-csrf-protection
|
||||
patterns:
|
||||
- pattern: |
|
||||
@app.route("...", methods=["POST"])
|
||||
def $FUNC(...):
|
||||
...
|
||||
- pattern-not-inside: |
|
||||
@csrf.exempt
|
||||
...
|
||||
message: "POST endpoint may lack CSRF protection."
|
||||
languages: [python]
|
||||
severity: WARNING
|
||||
```
|
||||
|
||||
### Step 4: Establish Quality Gates with Branch Protection
|
||||
|
||||
Configure branch protection rules that require SAST checks to pass before merging, preventing vulnerable code from reaching production branches.
|
||||
|
||||
```bash
|
||||
# Use GitHub CLI to set branch protection requiring SAST checks
|
||||
gh api repos/{owner}/{repo}/branches/main/protection \
|
||||
--method PUT \
|
||||
--field required_status_checks='{"strict":true,"contexts":["Analyze (javascript)","Analyze (python)","Semgrep Scan"]}' \
|
||||
--field enforce_admins=true \
|
||||
--field required_pull_request_reviews='{"required_approving_review_count":1}'
|
||||
```
|
||||
|
||||
### Step 5: Tune and Suppress False Positives
|
||||
|
||||
Manage false positives through CodeQL query filters and Semgrep nosemgrep annotations to maintain developer trust in scan results.
|
||||
|
||||
```yaml
|
||||
# codeql-config.yml - Custom CodeQL configuration
|
||||
name: "Custom CodeQL Config"
|
||||
queries:
|
||||
- uses: security-extended
|
||||
- uses: security-and-quality
|
||||
- excludes:
|
||||
id: js/unused-local-variable
|
||||
paths-ignore:
|
||||
- '**/test/**'
|
||||
- '**/tests/**'
|
||||
- '**/vendor/**'
|
||||
- '**/node_modules/**'
|
||||
- '**/*.test.js'
|
||||
- '**/*.spec.py'
|
||||
```
|
||||
|
||||
```python
|
||||
# Example: Suppressing a known false positive in Semgrep
|
||||
import subprocess
|
||||
|
||||
def run_safe_command(cmd_list):
|
||||
# nosemgrep: python.lang.security.audit.dangerous-subprocess-use
|
||||
result = subprocess.run(cmd_list, capture_output=True, text=True, shell=False)
|
||||
return result.stdout
|
||||
```
|
||||
|
||||
### Step 6: Aggregate and Report Findings
|
||||
|
||||
Use the GitHub Security Overview dashboard and configure notifications for security alerts across repositories.
|
||||
|
||||
```bash
|
||||
# Query SARIF results via GitHub API for reporting
|
||||
gh api repos/{owner}/{repo}/code-scanning/alerts \
|
||||
--jq '.[] | select(.state=="open") | {rule: .rule.id, severity: .rule.security_severity_level, file: .most_recent_instance.location.path, line: .most_recent_instance.location.start_line}'
|
||||
|
||||
# Count open alerts by severity
|
||||
gh api repos/{owner}/{repo}/code-scanning/alerts \
|
||||
--jq '[.[] | select(.state=="open")] | group_by(.rule.security_severity_level) | map({severity: .[0].rule.security_severity_level, count: length})'
|
||||
```
|
||||
|
||||
## Key Concepts
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| SAST | Static Application Security Testing — analyzes source code without executing it to find security vulnerabilities |
|
||||
| SARIF | Static Analysis Results Interchange Format — standardized JSON format for expressing results from static analysis tools |
|
||||
| CodeQL | GitHub's semantic code analysis engine that treats code as data and queries it for vulnerability patterns |
|
||||
| Semgrep | Lightweight static analysis tool using pattern matching to find bugs and security issues across many languages |
|
||||
| Security Extended | CodeQL query suite that includes additional security queries beyond the default set for deeper analysis |
|
||||
| Quality Gate | Automated checkpoint that blocks code from progressing through the pipeline unless security criteria are met |
|
||||
| False Positive | A scan finding that incorrectly identifies secure code as vulnerable, requiring suppression or tuning |
|
||||
|
||||
## Tools & Systems
|
||||
|
||||
- **CodeQL**: GitHub's semantic code analysis engine with deep dataflow and taint tracking analysis
|
||||
- **Semgrep**: Fast, lightweight pattern-matching SAST tool with 3000+ community rules and custom rule support
|
||||
- **GitHub Advanced Security**: Platform providing code scanning, secret scanning, and dependency review in GitHub
|
||||
- **SARIF Viewer**: VS Code extension for reviewing SARIF results locally during development
|
||||
- **GitHub Security Overview**: Organization-level dashboard aggregating security alerts across all repositories
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
### Scenario: Monorepo with Multiple Languages Needs Unified SAST
|
||||
|
||||
**Context**: A platform team manages a monorepo containing Python microservices, TypeScript frontends, and Go infrastructure tools. Security reviews happen manually every quarter, missing vulnerabilities between reviews.
|
||||
|
||||
**Approach**:
|
||||
1. Configure CodeQL with a matrix strategy covering Python, JavaScript, and Go languages
|
||||
2. Add Semgrep with `--config auto` to detect language automatically and apply relevant rulesets
|
||||
3. Create path-based triggers so only changed language directories trigger their respective scans
|
||||
4. Upload all SARIF results to GitHub Security tab with unique categories per tool and language
|
||||
5. Set branch protection requiring all SAST jobs to pass before merge
|
||||
6. Schedule weekly full-repository scans to catch issues in unchanged code from newly published CVE patterns
|
||||
|
||||
**Pitfalls**: Setting CodeQL to analyze all languages on every PR increases CI time significantly. Use path filters to trigger only relevant language scans. Semgrep's `--config auto` may enable rules that conflict with CodeQL findings, creating duplicate alerts.
|
||||
|
||||
### Scenario: Reducing Alert Fatigue from High False Positive Rate
|
||||
|
||||
**Context**: After enabling SAST, developers ignore findings because 40% are false positives, undermining the security program.
|
||||
|
||||
**Approach**:
|
||||
1. Export all current alerts and categorize them as true positive, false positive, or informational
|
||||
2. Create a custom CodeQL config excluding noisy query IDs that produce the most false positives
|
||||
3. Write `.semgrepignore` patterns for test files, generated code, and vendored dependencies
|
||||
4. Establish a weekly triage meeting where security and development leads review new rule additions
|
||||
5. Track false positive rate as a metric and target below 15% for developer trust
|
||||
|
||||
**Pitfalls**: Over-suppressing rules to reduce noise can create blind spots. Always validate suppressions against the OWASP Top 10 and CWE Top 25 to ensure critical vulnerability classes remain covered.
|
||||
|
||||
## Output Format
|
||||
|
||||
```
|
||||
SAST Pipeline Scan Report
|
||||
==========================
|
||||
Repository: org/web-application
|
||||
Branch: feature/user-auth-refactor
|
||||
Scan Date: 2026-02-23
|
||||
Commit: a1b2c3d4
|
||||
|
||||
CodeQL Results:
|
||||
Language Queries Run Findings Critical High Medium
|
||||
javascript 312 4 1 2 1
|
||||
python 287 2 0 1 1
|
||||
|
||||
Semgrep Results:
|
||||
Ruleset Rules Matched Findings Errors Warnings
|
||||
auto 1,847 3 1 2
|
||||
owasp-top-ten 186 2 1 1
|
||||
custom-rules 12 1 0 1
|
||||
|
||||
QUALITY GATE: FAILED
|
||||
Blocking findings: 2 Critical/High severity issues
|
||||
- [CRITICAL] CWE-89: SQL Injection in src/api/users.py:47
|
||||
- [HIGH] CWE-79: Cross-site Scripting in src/components/Search.tsx:123
|
||||
|
||||
Action Required: Fix blocking findings before merge is permitted.
|
||||
```
|
||||
@@ -0,0 +1,216 @@
|
||||
# SAST Pipeline Configuration Templates
|
||||
|
||||
## GitHub Actions: Combined CodeQL + Semgrep Workflow
|
||||
|
||||
```yaml
|
||||
# .github/workflows/sast-pipeline.yml
|
||||
name: "SAST Security Pipeline"
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main, develop]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
schedule:
|
||||
- cron: '0 3 * * 1'
|
||||
|
||||
concurrency:
|
||||
group: sast-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
# ─────────────── CodeQL Analysis ───────────────
|
||||
codeql:
|
||||
name: CodeQL (${{ matrix.language }})
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
actions: read
|
||||
contents: read
|
||||
security-events: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
language: ['javascript', 'python']
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v3
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
queries: security-extended
|
||||
config-file: .github/codeql/codeql-config.yml
|
||||
|
||||
- name: Autobuild
|
||||
uses: github/codeql-action/autobuild@v3
|
||||
|
||||
- name: Perform Analysis
|
||||
uses: github/codeql-action/analyze@v3
|
||||
with:
|
||||
category: "/language:${{ matrix.language }}"
|
||||
|
||||
# ─────────────── Semgrep Scan ───────────────
|
||||
semgrep:
|
||||
name: Semgrep Scan
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
security-events: write
|
||||
contents: read
|
||||
container:
|
||||
image: semgrep/semgrep:latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Run Semgrep
|
||||
run: |
|
||||
semgrep ci \
|
||||
--config auto \
|
||||
--config p/owasp-top-ten \
|
||||
--config p/cwe-top-25 \
|
||||
--config .semgrep/ \
|
||||
--sarif --output semgrep.sarif \
|
||||
--severity ERROR \
|
||||
--error
|
||||
env:
|
||||
SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}
|
||||
|
||||
- name: Upload SARIF
|
||||
if: always()
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
with:
|
||||
sarif_file: semgrep.sarif
|
||||
category: semgrep
|
||||
|
||||
# ─────────────── Quality Gate ───────────────
|
||||
security-gate:
|
||||
name: Security Quality Gate
|
||||
needs: [codeql, semgrep]
|
||||
runs-on: ubuntu-latest
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check SAST Results
|
||||
run: |
|
||||
if [ "${{ needs.codeql.result }}" == "failure" ] || [ "${{ needs.semgrep.result }}" == "failure" ]; then
|
||||
echo "::error::SAST security gate failed. Review findings in the Security tab."
|
||||
exit 1
|
||||
fi
|
||||
echo "Security gate passed."
|
||||
```
|
||||
|
||||
## CodeQL Custom Configuration
|
||||
|
||||
```yaml
|
||||
# .github/codeql/codeql-config.yml
|
||||
name: "Organization CodeQL Config"
|
||||
|
||||
queries:
|
||||
- uses: security-extended
|
||||
- uses: security-and-quality
|
||||
|
||||
paths-ignore:
|
||||
- '**/test/**'
|
||||
- '**/tests/**'
|
||||
- '**/spec/**'
|
||||
- '**/vendor/**'
|
||||
- '**/node_modules/**'
|
||||
- '**/__mocks__/**'
|
||||
- '**/*.test.js'
|
||||
- '**/*.test.ts'
|
||||
- '**/*.spec.py'
|
||||
- '**/migrations/**'
|
||||
|
||||
query-filters:
|
||||
- exclude:
|
||||
id: js/unused-local-variable
|
||||
- exclude:
|
||||
id: py/unused-import
|
||||
```
|
||||
|
||||
## Semgrep Ignore File
|
||||
|
||||
```
|
||||
# .semgrepignore
|
||||
# Test files
|
||||
*_test.go
|
||||
*_test.py
|
||||
*.test.js
|
||||
*.test.ts
|
||||
*.spec.js
|
||||
*.spec.ts
|
||||
test/
|
||||
tests/
|
||||
__tests__/
|
||||
spec/
|
||||
|
||||
# Generated code
|
||||
*_generated.go
|
||||
*.pb.go
|
||||
**/generated/**
|
||||
|
||||
# Vendored dependencies
|
||||
vendor/
|
||||
node_modules/
|
||||
third_party/
|
||||
|
||||
# Build artifacts
|
||||
dist/
|
||||
build/
|
||||
out/
|
||||
```
|
||||
|
||||
## Branch Protection Configuration (Terraform)
|
||||
|
||||
```hcl
|
||||
# branch-protection.tf
|
||||
resource "github_branch_protection" "main" {
|
||||
repository_id = github_repository.app.node_id
|
||||
pattern = "main"
|
||||
|
||||
required_status_checks {
|
||||
strict = true
|
||||
contexts = [
|
||||
"CodeQL (javascript)",
|
||||
"CodeQL (python)",
|
||||
"Semgrep Scan",
|
||||
"Security Quality Gate"
|
||||
]
|
||||
}
|
||||
|
||||
required_pull_request_reviews {
|
||||
required_approving_review_count = 1
|
||||
dismiss_stale_reviews = true
|
||||
}
|
||||
|
||||
enforce_admins = true
|
||||
|
||||
allows_deletions = false
|
||||
allows_force_pushes = false
|
||||
}
|
||||
```
|
||||
|
||||
## SARIF Report Aggregation Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# aggregate-sarif.sh - Merge multiple SARIF files for unified upload
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT="merged-results.sarif"
|
||||
SARIF_FILES=($(find . -name "*.sarif" -type f))
|
||||
|
||||
if [ ${#SARIF_FILES[@]} -eq 0 ]; then
|
||||
echo "No SARIF files found"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Use jq to merge SARIF runs
|
||||
jq -s '{
|
||||
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/main/sarif-2.1/schema/sarif-schema-2.1.0.json",
|
||||
"version": "2.1.0",
|
||||
"runs": [.[].runs[]]
|
||||
}' "${SARIF_FILES[@]}" > "$OUTPUT"
|
||||
|
||||
echo "Merged ${#SARIF_FILES[@]} SARIF files into $OUTPUT"
|
||||
TOTAL=$(jq '[.runs[].results | length] | add' "$OUTPUT")
|
||||
echo "Total findings: $TOTAL"
|
||||
```
|
||||
@@ -0,0 +1,65 @@
|
||||
# Standards Reference: SAST in GitHub Actions
|
||||
|
||||
## OWASP SAMM - Verification: Security Testing
|
||||
|
||||
### Maturity Level 1
|
||||
- Perform automated SAST scanning with default rulesets on all application code
|
||||
- Results are visible to development teams through IDE or CI/CD integration
|
||||
|
||||
### Maturity Level 2
|
||||
- Customize SAST rules to reduce false positives below 20%
|
||||
- Track and triage all findings with defined SLAs per severity
|
||||
- Integrate SAST results into a centralized vulnerability management system
|
||||
|
||||
### Maturity Level 3
|
||||
- Correlate SAST findings with DAST and SCA results for comprehensive coverage
|
||||
- Measure and improve detection accuracy through benchmarking against known vulnerabilities
|
||||
- Custom rules cover organization-specific vulnerability patterns and deprecated APIs
|
||||
|
||||
## NIST SSDF (SP 800-218) - Produce Well-Secured Software
|
||||
|
||||
### PW.7: Review and Analyze Code
|
||||
- PW.7.1: Determine whether SAST tools should be used and select appropriate tools
|
||||
- PW.7.2: Use SAST tools to analyze source code and identify vulnerabilities
|
||||
- Configure tools to analyze code for compliance with secure coding standards
|
||||
|
||||
### PW.8: Test Executable Code
|
||||
- Integration of SAST into CI/CD ensures code is tested before deployment
|
||||
- Findings are tracked and remediated according to organizational policy
|
||||
|
||||
## CIS Software Supply Chain Security Guide
|
||||
|
||||
### Source Code (SC) Controls
|
||||
- SC-2: Enforce branch protection requiring SAST checks to pass
|
||||
- SC-3: Require code review in addition to automated scanning
|
||||
- SC-4: Automate security testing in the build pipeline
|
||||
|
||||
### Build (BD) Controls
|
||||
- BD-1: Define and enforce security requirements for build processes
|
||||
- BD-2: Integrate multiple security testing tools for defense in depth
|
||||
|
||||
## OWASP Top 10 Coverage Matrix
|
||||
|
||||
| OWASP Category | CodeQL | Semgrep | Combined |
|
||||
|----------------|--------|---------|----------|
|
||||
| A01: Broken Access Control | Partial | Yes | Yes |
|
||||
| A02: Cryptographic Failures | Yes | Yes | Yes |
|
||||
| A03: Injection | Yes | Yes | Yes |
|
||||
| A04: Insecure Design | No | Partial | Partial |
|
||||
| A05: Security Misconfiguration | Partial | Yes | Yes |
|
||||
| A06: Vulnerable Components | No | No | No (Use SCA) |
|
||||
| A07: Auth Failures | Yes | Yes | Yes |
|
||||
| A08: Software Integrity | No | Partial | Partial |
|
||||
| A09: Logging Failures | Partial | Yes | Yes |
|
||||
| A10: SSRF | Yes | Yes | Yes |
|
||||
|
||||
## PCI DSS v4.0 Mapping
|
||||
|
||||
- Requirement 6.2.4: Software engineering techniques or automated methods prevent or mitigate common software attacks
|
||||
- Requirement 6.3.2: An inventory of bespoke and custom software and third-party software components is maintained
|
||||
- Requirement 6.5.4: SAST tools are run as part of the software development lifecycle
|
||||
|
||||
## SOC 2 Trust Service Criteria
|
||||
|
||||
- CC7.1: Deploy detection and monitoring mechanisms for anomalies indicative of actual or attempted attacks
|
||||
- CC8.1: Authorize, design, develop or acquire, configure, document, test, approve, and implement changes to infrastructure, data, software, and procedures
|
||||
@@ -0,0 +1,137 @@
|
||||
# Workflow Reference: SAST in GitHub Actions Pipeline
|
||||
|
||||
## End-to-End SAST Integration Workflow
|
||||
|
||||
```
|
||||
Developer Push/PR
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ GitHub Actions │
|
||||
│ Trigger │
|
||||
└──────┬───────────┘
|
||||
│
|
||||
├──────────────────────┐
|
||||
▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ CodeQL Init │ │ Semgrep CI │
|
||||
│ + Autobuild │ │ + Custom │
|
||||
│ + Analyze │ │ Rules │
|
||||
└──────┬───────┘ └──────┬───────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ SARIF Upload │ │ SARIF Upload │
|
||||
│ (CodeQL) │ │ (Semgrep) │
|
||||
└──────┬───────┘ └──────┬───────┘
|
||||
│ │
|
||||
└──────────┬─────────┘
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ GitHub Security │
|
||||
│ Tab / Dashboard │
|
||||
└──────┬───────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ Branch Protection│
|
||||
│ Quality Gate │
|
||||
└──────┬───────────┘
|
||||
│
|
||||
┌─────────┴──────────┐
|
||||
▼ ▼
|
||||
PASS: Merge FAIL: Block
|
||||
Permitted + Notify Dev
|
||||
```
|
||||
|
||||
## CodeQL Analysis Deep Dive
|
||||
|
||||
### Database Creation Phase
|
||||
1. CodeQL extracts source code into a relational database
|
||||
2. For compiled languages (Java, C++, C#, Go), the build process is intercepted
|
||||
3. For interpreted languages (Python, JavaScript, Ruby), source files are parsed directly
|
||||
4. The database captures the full AST, data flow, and control flow of the program
|
||||
|
||||
### Query Execution Phase
|
||||
1. Security queries analyze the database for known vulnerability patterns
|
||||
2. Taint tracking follows data from untrusted sources to dangerous sinks
|
||||
3. Dataflow analysis tracks variable assignments across method boundaries
|
||||
4. Results are deduplicated and ranked by confidence and severity
|
||||
|
||||
### Query Suites
|
||||
- **default**: Core security queries with high precision and low false positive rate
|
||||
- **security-extended**: Additional queries covering more vulnerability types
|
||||
- **security-and-quality**: All security queries plus code quality checks
|
||||
|
||||
## Semgrep Rule Authoring Process
|
||||
|
||||
### Rule Development Lifecycle
|
||||
1. Identify a vulnerability pattern from a recent security incident or code review
|
||||
2. Write the pattern using Semgrep syntax with `pattern`, `pattern-either`, `pattern-not`
|
||||
3. Test the rule against known vulnerable and safe code samples
|
||||
4. Add metadata: CWE ID, OWASP category, severity, remediation guidance
|
||||
5. Deploy via `.semgrep/` directory or Semgrep App registry
|
||||
6. Monitor false positive rate and refine patterns
|
||||
|
||||
### Pattern Operators Reference
|
||||
| Operator | Purpose |
|
||||
|----------|---------|
|
||||
| `pattern` | Match a single code pattern |
|
||||
| `pattern-either` | Match any of multiple patterns (OR) |
|
||||
| `pattern-not` | Exclude specific patterns from matches |
|
||||
| `pattern-inside` | Match only within a containing pattern |
|
||||
| `pattern-not-inside` | Exclude matches within a containing pattern |
|
||||
| `metavariable-regex` | Constrain metavariable values with regex |
|
||||
| `metavariable-comparison` | Compare metavariable values numerically |
|
||||
|
||||
## SARIF Processing Pipeline
|
||||
|
||||
### SARIF Structure
|
||||
```json
|
||||
{
|
||||
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/main/sarif-2.1/schema/sarif-schema-2.1.0.json",
|
||||
"version": "2.1.0",
|
||||
"runs": [
|
||||
{
|
||||
"tool": {
|
||||
"driver": {
|
||||
"name": "Semgrep",
|
||||
"rules": []
|
||||
}
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"ruleId": "hardcoded-database-url",
|
||||
"level": "error",
|
||||
"message": { "text": "..." },
|
||||
"locations": [
|
||||
{
|
||||
"physicalLocation": {
|
||||
"artifactLocation": { "uri": "src/config.py" },
|
||||
"region": { "startLine": 42 }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Triage and Remediation Workflow
|
||||
|
||||
### Severity-Based SLA
|
||||
| Severity | Triage SLA | Remediation SLA | Escalation |
|
||||
|----------|-----------|-----------------|------------|
|
||||
| Critical | 1 business day | 3 business days | Security Lead + VP Eng |
|
||||
| High | 3 business days | 10 business days | Security Lead |
|
||||
| Medium | 5 business days | 30 business days | Team Lead |
|
||||
| Low | 10 business days | 90 business days | Backlog |
|
||||
|
||||
### Finding States
|
||||
1. **Open**: New finding not yet reviewed
|
||||
2. **Confirmed**: Finding validated as true positive
|
||||
3. **False Positive**: Finding dismissed with justification
|
||||
4. **Fixed**: Remediation committed and verified by rescan
|
||||
5. **Won't Fix**: Accepted risk with documented justification and risk owner
|
||||
@@ -0,0 +1,380 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
SAST Pipeline Orchestration Script
|
||||
|
||||
Runs CodeQL and Semgrep scans, aggregates SARIF results, evaluates quality gates,
|
||||
and produces a consolidated report. Designed to be invoked from GitHub Actions
|
||||
or any CI/CD platform.
|
||||
|
||||
Usage:
|
||||
python process.py --repo-path /path/to/repo --output report.json
|
||||
python process.py --repo-path . --severity-threshold high --fail-on-findings
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
class ScanFinding:
|
||||
rule_id: str
|
||||
severity: str
|
||||
message: str
|
||||
file_path: str
|
||||
start_line: int
|
||||
end_line: int
|
||||
tool: str
|
||||
cwe: str = ""
|
||||
owasp: str = ""
|
||||
fingerprint: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class ScanResult:
|
||||
tool: str
|
||||
findings: list = field(default_factory=list)
|
||||
rules_evaluated: int = 0
|
||||
scan_duration_seconds: float = 0.0
|
||||
exit_code: int = 0
|
||||
error_message: str = ""
|
||||
|
||||
|
||||
SEVERITY_ORDER = {"critical": 0, "high": 1, "medium": 2, "low": 3, "note": 4, "none": 5}
|
||||
|
||||
|
||||
def run_semgrep(repo_path: str, config: str = "auto", extra_configs: Optional[list] = None) -> ScanResult:
|
||||
"""Run Semgrep scan and return structured results."""
|
||||
result = ScanResult(tool="semgrep")
|
||||
sarif_output = os.path.join(repo_path, "semgrep-results.sarif")
|
||||
|
||||
cmd = [
|
||||
"semgrep", "ci",
|
||||
"--config", config,
|
||||
"--sarif",
|
||||
"--output", sarif_output,
|
||||
"--json",
|
||||
"--quiet"
|
||||
]
|
||||
|
||||
if extra_configs:
|
||||
for cfg in extra_configs:
|
||||
cmd.extend(["--config", cfg])
|
||||
|
||||
start_time = datetime.now(timezone.utc)
|
||||
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
cmd,
|
||||
cwd=repo_path,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=600
|
||||
)
|
||||
result.exit_code = proc.returncode
|
||||
|
||||
if proc.returncode not in (0, 1):
|
||||
result.error_message = proc.stderr[:500]
|
||||
return result
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
result.error_message = "Semgrep scan timed out after 600 seconds"
|
||||
result.exit_code = -1
|
||||
return result
|
||||
except FileNotFoundError:
|
||||
result.error_message = "semgrep binary not found. Install with: pip install semgrep"
|
||||
result.exit_code = -1
|
||||
return result
|
||||
|
||||
result.scan_duration_seconds = (datetime.now(timezone.utc) - start_time).total_seconds()
|
||||
|
||||
if os.path.exists(sarif_output):
|
||||
result.findings = parse_sarif(sarif_output, "semgrep")
|
||||
with open(sarif_output, "r") as f:
|
||||
sarif_data = json.load(f)
|
||||
for run in sarif_data.get("runs", []):
|
||||
result.rules_evaluated = len(run.get("tool", {}).get("driver", {}).get("rules", []))
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def run_codeql_query(repo_path: str, language: str, database_path: str) -> ScanResult:
|
||||
"""Run CodeQL analysis on a pre-created database and return structured results."""
|
||||
result = ScanResult(tool=f"codeql-{language}")
|
||||
sarif_output = os.path.join(repo_path, f"codeql-{language}-results.sarif")
|
||||
|
||||
cmd = [
|
||||
"codeql", "database", "analyze",
|
||||
database_path,
|
||||
f"codeql/{language}-queries:codeql-suites/{language}-security-extended.qls",
|
||||
"--format=sarifv2.1.0",
|
||||
f"--output={sarif_output}",
|
||||
"--threads=0"
|
||||
]
|
||||
|
||||
start_time = datetime.now(timezone.utc)
|
||||
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
cmd,
|
||||
cwd=repo_path,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=1200
|
||||
)
|
||||
result.exit_code = proc.returncode
|
||||
|
||||
if proc.returncode != 0:
|
||||
result.error_message = proc.stderr[:500]
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
result.error_message = "CodeQL analysis timed out after 1200 seconds"
|
||||
result.exit_code = -1
|
||||
return result
|
||||
except FileNotFoundError:
|
||||
result.error_message = "codeql binary not found. Install from https://github.com/github/codeql-cli-binaries"
|
||||
result.exit_code = -1
|
||||
return result
|
||||
|
||||
result.scan_duration_seconds = (datetime.now(timezone.utc) - start_time).total_seconds()
|
||||
|
||||
if os.path.exists(sarif_output):
|
||||
result.findings = parse_sarif(sarif_output, f"codeql-{language}")
|
||||
with open(sarif_output, "r") as f:
|
||||
sarif_data = json.load(f)
|
||||
for run in sarif_data.get("runs", []):
|
||||
result.rules_evaluated = len(run.get("tool", {}).get("driver", {}).get("rules", []))
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def parse_sarif(sarif_path: str, tool_name: str) -> list:
|
||||
"""Parse a SARIF file and extract findings as ScanFinding objects."""
|
||||
findings = []
|
||||
|
||||
with open(sarif_path, "r") as f:
|
||||
sarif_data = json.load(f)
|
||||
|
||||
for run in sarif_data.get("runs", []):
|
||||
rules_map = {}
|
||||
for rule in run.get("tool", {}).get("driver", {}).get("rules", []):
|
||||
rule_id = rule.get("id", "")
|
||||
properties = rule.get("properties", {})
|
||||
cwe_tags = [t for t in properties.get("tags", []) if t.startswith("CWE")]
|
||||
owasp_tags = [t for t in properties.get("tags", []) if "owasp" in t.lower()]
|
||||
rules_map[rule_id] = {
|
||||
"cwe": cwe_tags[0] if cwe_tags else "",
|
||||
"owasp": owasp_tags[0] if owasp_tags else "",
|
||||
"severity": rule.get("defaultConfiguration", {}).get("level", "warning")
|
||||
}
|
||||
|
||||
for result in run.get("results", []):
|
||||
rule_id = result.get("ruleId", "unknown")
|
||||
rule_info = rules_map.get(rule_id, {})
|
||||
|
||||
level = result.get("level", rule_info.get("severity", "warning"))
|
||||
severity_map = {"error": "high", "warning": "medium", "note": "low", "none": "none"}
|
||||
severity = severity_map.get(level, "medium")
|
||||
|
||||
security_severity = None
|
||||
for rule in run.get("tool", {}).get("driver", {}).get("rules", []):
|
||||
if rule.get("id") == rule_id:
|
||||
security_severity = rule.get("properties", {}).get("security-severity")
|
||||
break
|
||||
|
||||
if security_severity:
|
||||
score = float(security_severity)
|
||||
if score >= 9.0:
|
||||
severity = "critical"
|
||||
elif score >= 7.0:
|
||||
severity = "high"
|
||||
elif score >= 4.0:
|
||||
severity = "medium"
|
||||
else:
|
||||
severity = "low"
|
||||
|
||||
locations = result.get("locations", [{}])
|
||||
physical = locations[0].get("physicalLocation", {}) if locations else {}
|
||||
artifact = physical.get("artifactLocation", {})
|
||||
region = physical.get("region", {})
|
||||
|
||||
findings.append(ScanFinding(
|
||||
rule_id=rule_id,
|
||||
severity=severity,
|
||||
message=result.get("message", {}).get("text", ""),
|
||||
file_path=artifact.get("uri", "unknown"),
|
||||
start_line=region.get("startLine", 0),
|
||||
end_line=region.get("endLine", region.get("startLine", 0)),
|
||||
tool=tool_name,
|
||||
cwe=rule_info.get("cwe", ""),
|
||||
owasp=rule_info.get("owasp", ""),
|
||||
fingerprint=str(result.get("fingerprints", {}).get("primaryLocationLineHash", ""))
|
||||
))
|
||||
|
||||
return findings
|
||||
|
||||
|
||||
def evaluate_quality_gate(findings: list, severity_threshold: str) -> dict:
|
||||
"""Evaluate quality gate based on finding severities."""
|
||||
threshold_level = SEVERITY_ORDER.get(severity_threshold.lower(), 1)
|
||||
|
||||
blocking_findings = [
|
||||
f for f in findings
|
||||
if SEVERITY_ORDER.get(f.severity.lower(), 5) <= threshold_level
|
||||
]
|
||||
|
||||
severity_counts = {}
|
||||
for f in findings:
|
||||
sev = f.severity.lower()
|
||||
severity_counts[sev] = severity_counts.get(sev, 0) + 1
|
||||
|
||||
return {
|
||||
"passed": len(blocking_findings) == 0,
|
||||
"threshold": severity_threshold,
|
||||
"total_findings": len(findings),
|
||||
"blocking_findings": len(blocking_findings),
|
||||
"severity_counts": severity_counts,
|
||||
"blocking_details": [
|
||||
{
|
||||
"rule_id": f.rule_id,
|
||||
"severity": f.severity,
|
||||
"file": f.file_path,
|
||||
"line": f.start_line,
|
||||
"tool": f.tool,
|
||||
"message": f.message[:200]
|
||||
}
|
||||
for f in blocking_findings
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
def generate_report(scan_results: list, quality_gate: dict, repo_path: str) -> dict:
|
||||
"""Generate a consolidated SAST report."""
|
||||
all_findings = []
|
||||
for sr in scan_results:
|
||||
all_findings.extend(sr.findings)
|
||||
|
||||
cwe_counts = {}
|
||||
for f in all_findings:
|
||||
if f.cwe:
|
||||
cwe_counts[f.cwe] = cwe_counts.get(f.cwe, 0) + 1
|
||||
|
||||
report = {
|
||||
"report_metadata": {
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"repository": repo_path,
|
||||
"report_version": "1.0.0"
|
||||
},
|
||||
"scan_summary": [
|
||||
{
|
||||
"tool": sr.tool,
|
||||
"findings_count": len(sr.findings),
|
||||
"rules_evaluated": sr.rules_evaluated,
|
||||
"duration_seconds": sr.scan_duration_seconds,
|
||||
"status": "success" if sr.exit_code in (0, 1) else "error",
|
||||
"error": sr.error_message
|
||||
}
|
||||
for sr in scan_results
|
||||
],
|
||||
"quality_gate": quality_gate,
|
||||
"top_cwes": sorted(cwe_counts.items(), key=lambda x: x[1], reverse=True)[:10],
|
||||
"findings": [
|
||||
{
|
||||
"rule_id": f.rule_id,
|
||||
"severity": f.severity,
|
||||
"tool": f.tool,
|
||||
"file": f.file_path,
|
||||
"line": f.start_line,
|
||||
"cwe": f.cwe,
|
||||
"owasp": f.owasp,
|
||||
"message": f.message[:300]
|
||||
}
|
||||
for f in sorted(all_findings, key=lambda x: SEVERITY_ORDER.get(x.severity.lower(), 5))
|
||||
]
|
||||
}
|
||||
|
||||
return report
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="SAST Pipeline Orchestration")
|
||||
parser.add_argument("--repo-path", required=True, help="Path to the repository to scan")
|
||||
parser.add_argument("--output", default="sast-report.json", help="Output report file path")
|
||||
parser.add_argument("--severity-threshold", default="high",
|
||||
choices=["critical", "high", "medium", "low"],
|
||||
help="Minimum severity to block pipeline")
|
||||
parser.add_argument("--fail-on-findings", action="store_true",
|
||||
help="Exit with non-zero code if quality gate fails")
|
||||
parser.add_argument("--semgrep-config", default="auto",
|
||||
help="Semgrep configuration (default: auto)")
|
||||
parser.add_argument("--semgrep-extra-configs", nargs="*",
|
||||
help="Additional Semgrep config paths")
|
||||
parser.add_argument("--skip-semgrep", action="store_true", help="Skip Semgrep scan")
|
||||
parser.add_argument("--skip-codeql", action="store_true", help="Skip CodeQL scan")
|
||||
parser.add_argument("--codeql-language", default=None, help="Language for CodeQL analysis")
|
||||
parser.add_argument("--codeql-db-path", default=None, help="Path to CodeQL database")
|
||||
parser.add_argument("--sarif-only", nargs="*",
|
||||
help="Only parse existing SARIF files instead of running scans")
|
||||
args = parser.parse_args()
|
||||
|
||||
repo_path = os.path.abspath(args.repo_path)
|
||||
scan_results = []
|
||||
|
||||
if args.sarif_only:
|
||||
for sarif_file in args.sarif_only:
|
||||
tool_name = Path(sarif_file).stem
|
||||
findings = parse_sarif(sarif_file, tool_name)
|
||||
sr = ScanResult(tool=tool_name, findings=findings)
|
||||
scan_results.append(sr)
|
||||
else:
|
||||
if not args.skip_semgrep:
|
||||
print("[*] Running Semgrep scan...")
|
||||
semgrep_result = run_semgrep(
|
||||
repo_path,
|
||||
config=args.semgrep_config,
|
||||
extra_configs=args.semgrep_extra_configs
|
||||
)
|
||||
scan_results.append(semgrep_result)
|
||||
print(f" Found {len(semgrep_result.findings)} findings in {semgrep_result.scan_duration_seconds:.1f}s")
|
||||
|
||||
if semgrep_result.error_message:
|
||||
print(f" Warning: {semgrep_result.error_message}")
|
||||
|
||||
if not args.skip_codeql and args.codeql_language and args.codeql_db_path:
|
||||
print(f"[*] Running CodeQL analysis for {args.codeql_language}...")
|
||||
codeql_result = run_codeql_query(repo_path, args.codeql_language, args.codeql_db_path)
|
||||
scan_results.append(codeql_result)
|
||||
print(f" Found {len(codeql_result.findings)} findings in {codeql_result.scan_duration_seconds:.1f}s")
|
||||
|
||||
all_findings = []
|
||||
for sr in scan_results:
|
||||
all_findings.extend(sr.findings)
|
||||
|
||||
quality_gate = evaluate_quality_gate(all_findings, args.severity_threshold)
|
||||
|
||||
report = generate_report(scan_results, quality_gate, repo_path)
|
||||
|
||||
output_path = os.path.abspath(args.output)
|
||||
with open(output_path, "w") as f:
|
||||
json.dump(report, f, indent=2)
|
||||
print(f"\n[*] Report written to {output_path}")
|
||||
|
||||
if quality_gate["passed"]:
|
||||
print(f"[PASS] Quality gate passed. {quality_gate['total_findings']} findings, none blocking.")
|
||||
else:
|
||||
print(f"[FAIL] Quality gate failed. {quality_gate['blocking_findings']} blocking findings:")
|
||||
for detail in quality_gate["blocking_details"]:
|
||||
print(f" - [{detail['severity'].upper()}] {detail['rule_id']} in {detail['file']}:{detail['line']}")
|
||||
|
||||
if args.fail_on_findings and not quality_gate["passed"]:
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user