mirror of
https://github.com/mukul975/Anthropic-Cybersecurity-Skills.git
synced 2026-06-11 13:44:56 +03:00
Add 10 new cybersecurity skills with full folder anatomy
Skills added: - implementing-privileged-access-workstation (IAM, PAW hardening) - detecting-suspicious-oauth-application-consent (cloud security, Graph API) - performing-hardware-security-module-integration (cryptography, PKCS#11) - analyzing-android-malware-with-apktool (malware analysis, androguard) - hunting-for-unusual-service-installations (threat hunting, T1543.003) - detecting-shadow-it-cloud-usage (cloud security, proxy/DNS log analysis) - performing-active-directory-forest-trust-attack (red team, impacket) - implementing-deception-based-detection-with-canarytoken (deception, Canary API) - analyzing-office365-audit-logs-for-compromise (cloud security, BEC detection) - hunting-for-startup-folder-persistence (threat hunting, T1547.001) Each skill includes SKILL.md, LICENSE, scripts/agent.py, references/api-reference.md
This commit is contained in:
mukul975
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: analyzing-android-malware-with-apktool
|
||||
description: Perform static analysis of Android APK malware samples using apktool for decompilation, jadx for Java source recovery, and androguard for permission analysis, manifest inspection, and suspicious API call detection.
|
||||
domain: cybersecurity
|
||||
subdomain: malware-analysis
|
||||
tags: [Android, APK, apktool, jadx, androguard, mobile-malware, static-analysis, reverse-engineering]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Analyzing Android Malware with Apktool
|
||||
|
||||
## Overview
|
||||
|
||||
Android malware distributed as APK files can be statically analyzed to extract permissions, activities, services, broadcast receivers, and suspicious API calls without executing the sample. This skill uses androguard for programmatic APK analysis, identifying dangerous permission combinations, obfuscated code patterns, dynamic code loading, reflection-based API calls, and network communication indicators.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+ with `androguard`
|
||||
- apktool (for resource decompilation)
|
||||
- jadx (for Java source recovery, optional)
|
||||
- Isolated analysis environment (VM or sandbox)
|
||||
- Sample APK files for analysis
|
||||
|
||||
## Steps
|
||||
|
||||
1. Parse APK with androguard to extract manifest metadata
|
||||
2. Enumerate requested permissions and flag dangerous combinations
|
||||
3. List activities, services, receivers, and providers from manifest
|
||||
4. Scan for suspicious API calls (reflection, crypto, SMS, telephony)
|
||||
5. Detect dynamic code loading patterns (DexClassLoader, Runtime.exec)
|
||||
6. Extract hardcoded URLs, IPs, and C2 indicators from strings
|
||||
7. Generate risk assessment report with MITRE ATT&CK mobile mappings
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report with permission analysis, component listing, suspicious API calls, network indicators, and risk score
|
||||
- Extracted strings and potential IOCs from the APK
|
||||
@@ -0,0 +1,69 @@
|
||||
# API Reference — Analyzing Android Malware with Apktool
|
||||
|
||||
## Libraries Used
|
||||
- **androguard**: Python APK/DEX analysis — `AnalyzeAPK()`, permission enumeration, API call scanning
|
||||
- **re**: Regex extraction of URLs, IPs, base64 patterns from DEX strings
|
||||
- **json**: JSON serialization for analysis reports
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py sample.apk permissions
|
||||
python agent.py sample.apk manifest
|
||||
python agent.py sample.apk apis
|
||||
python agent.py sample.apk strings
|
||||
python agent.py sample.apk full
|
||||
python agent.py sample.apk # defaults to full analysis
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `analyze_permissions(apk)` — Permission risk assessment
|
||||
Calls `apk.get_permissions()`. Flags 20 dangerous permissions including
|
||||
SEND_SMS, READ_CONTACTS, BIND_DEVICE_ADMIN, BIND_ACCESSIBILITY_SERVICE.
|
||||
Risk: CRITICAL >= 8 dangerous, HIGH >= 5, MEDIUM >= 2, LOW < 2.
|
||||
|
||||
### `analyze_manifest(apk)` — Manifest component extraction
|
||||
Calls `apk.get_activities()`, `get_services()`, `get_receivers()`, `get_providers()`.
|
||||
Returns package name, version, SDK levels, and all component lists.
|
||||
|
||||
### `scan_suspicious_apis(dx)` — Suspicious API call detection
|
||||
Searches DEX analysis for 14 patterns including:
|
||||
- `Runtime.exec`, `ProcessBuilder.start` — command execution
|
||||
- `DexClassLoader.loadClass` — dynamic code loading
|
||||
- `Method.invoke`, `Class.forName` — reflection
|
||||
- `Cipher.getInstance` — cryptographic operations
|
||||
- `SmsManager.sendTextMessage` — SMS abuse
|
||||
|
||||
### `extract_strings(dx, apk)` — IOC extraction from DEX strings
|
||||
Regex extraction of HTTP/HTTPS URLs, external IP addresses, and base64 strings.
|
||||
Filters out private IP ranges (10.x, 192.168.x, 172.16.x, 127.x).
|
||||
|
||||
### `detect_obfuscation(apk, dx)` — Obfuscation indicator detection
|
||||
Checks for single-letter class names (ProGuard), multi-DEX, native libraries.
|
||||
|
||||
### `full_analysis(apk_path)` — Comprehensive malware assessment
|
||||
|
||||
## Androguard API
|
||||
| Method | Returns |
|
||||
|--------|---------|
|
||||
| `AnalyzeAPK(path)` | `(APK, list[DEX], Analysis)` tuple |
|
||||
| `apk.get_permissions()` | List of Android permissions |
|
||||
| `apk.get_activities()` | Activity component names |
|
||||
| `apk.get_services()` | Service component names |
|
||||
| `apk.get_receivers()` | BroadcastReceiver names |
|
||||
| `apk.get_package()` | Package name string |
|
||||
| `dx.find_methods(classname, methodname)` | Matching method analysis objects |
|
||||
| `dx.get_strings()` | All strings from DEX files |
|
||||
| `dx.get_classes()` | All class analysis objects |
|
||||
|
||||
## Risk Scoring
|
||||
| Factor | Max Points |
|
||||
|--------|-----------|
|
||||
| Dangerous permissions (8 pts each) | 40 |
|
||||
| Suspicious API calls (10 pts each) | 30 |
|
||||
| External IPs (5 pts each) | 15 |
|
||||
| Obfuscation detected | 15 |
|
||||
|
||||
## Dependencies
|
||||
- `androguard` >= 3.4.0
|
||||
- Isolated analysis environment recommended
|
||||
@@ -0,0 +1,228 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for static analysis of Android APK malware using androguard."""
|
||||
|
||||
import json
|
||||
import re
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
|
||||
try:
|
||||
from androguard.core.apk import APK
|
||||
from androguard.core.dex import DEX
|
||||
from androguard.misc import AnalyzeAPK
|
||||
except ImportError:
|
||||
APK = None
|
||||
AnalyzeAPK = None
|
||||
|
||||
DANGEROUS_PERMISSIONS = [
|
||||
"android.permission.SEND_SMS", "android.permission.READ_SMS",
|
||||
"android.permission.RECEIVE_SMS", "android.permission.READ_CONTACTS",
|
||||
"android.permission.READ_CALL_LOG", "android.permission.RECORD_AUDIO",
|
||||
"android.permission.CAMERA", "android.permission.ACCESS_FINE_LOCATION",
|
||||
"android.permission.READ_PHONE_STATE", "android.permission.CALL_PHONE",
|
||||
"android.permission.WRITE_EXTERNAL_STORAGE", "android.permission.READ_EXTERNAL_STORAGE",
|
||||
"android.permission.INSTALL_PACKAGES", "android.permission.REQUEST_INSTALL_PACKAGES",
|
||||
"android.permission.SYSTEM_ALERT_WINDOW", "android.permission.BIND_ACCESSIBILITY_SERVICE",
|
||||
"android.permission.BIND_DEVICE_ADMIN", "android.permission.RECEIVE_BOOT_COMPLETED",
|
||||
"android.permission.WRITE_SETTINGS", "android.permission.CHANGE_WIFI_STATE",
|
||||
]
|
||||
|
||||
SUSPICIOUS_API_PATTERNS = [
|
||||
r"Ljava/lang/Runtime;->exec",
|
||||
r"Ljava/lang/ProcessBuilder;->start",
|
||||
r"Ldalvik/system/DexClassLoader;->loadClass",
|
||||
r"Ljava/lang/reflect/Method;->invoke",
|
||||
r"Ljava/lang/Class;->forName",
|
||||
r"Ljavax/crypto/Cipher;->getInstance",
|
||||
r"Landroid/telephony/SmsManager;->sendTextMessage",
|
||||
r"Landroid/app/admin/DevicePolicyManager;->lockNow",
|
||||
r"Landroid/content/pm/PackageManager;->setComponentEnabledSetting",
|
||||
r"Ljava/net/HttpURLConnection;->connect",
|
||||
r"Lokhttp3/OkHttpClient;->newCall",
|
||||
r"Landroid/webkit/WebView;->loadUrl",
|
||||
r"Landroid/os/Build;->SERIAL",
|
||||
r"Landroid/provider/Settings\$Secure;->getString",
|
||||
]
|
||||
|
||||
|
||||
def analyze_permissions(apk):
|
||||
"""Analyze requested permissions and flag dangerous ones."""
|
||||
permissions = apk.get_permissions()
|
||||
dangerous = [p for p in permissions if p in DANGEROUS_PERMISSIONS]
|
||||
return {
|
||||
"total_permissions": len(permissions),
|
||||
"permissions": permissions,
|
||||
"dangerous_permissions": dangerous,
|
||||
"dangerous_count": len(dangerous),
|
||||
"permission_risk": "CRITICAL" if len(dangerous) >= 8 else "HIGH" if len(dangerous) >= 5 else "MEDIUM" if len(dangerous) >= 2 else "LOW",
|
||||
}
|
||||
|
||||
|
||||
def analyze_manifest(apk):
|
||||
"""Extract manifest components: activities, services, receivers, providers."""
|
||||
activities = apk.get_activities()
|
||||
services = apk.get_services()
|
||||
receivers = apk.get_receivers()
|
||||
providers = apk.get_providers()
|
||||
return {
|
||||
"package_name": apk.get_package(),
|
||||
"app_name": apk.get_app_name(),
|
||||
"version_name": apk.get_androidversion_name(),
|
||||
"version_code": apk.get_androidversion_code(),
|
||||
"min_sdk": apk.get_min_sdk_version(),
|
||||
"target_sdk": apk.get_target_sdk_version(),
|
||||
"activities": list(activities),
|
||||
"services": list(services),
|
||||
"receivers": list(receivers),
|
||||
"providers": list(providers),
|
||||
"activity_count": len(activities),
|
||||
"service_count": len(services),
|
||||
"receiver_count": len(receivers),
|
||||
"provider_count": len(providers),
|
||||
}
|
||||
|
||||
|
||||
def scan_suspicious_apis(dx):
|
||||
"""Scan DEX analysis for suspicious API calls."""
|
||||
findings = []
|
||||
if not dx:
|
||||
return findings
|
||||
for pattern in SUSPICIOUS_API_PATTERNS:
|
||||
class_name = pattern.split(";->")[0] + ";"
|
||||
method_name = pattern.split(";->")[1] if ";->" in pattern else None
|
||||
for method in dx.find_methods(classname=class_name, methodname=method_name):
|
||||
xrefs = list(method.get_xref_from())
|
||||
if xrefs:
|
||||
findings.append({
|
||||
"api": pattern,
|
||||
"callers": len(xrefs),
|
||||
"first_caller_class": str(xrefs[0][0].name) if xrefs else None,
|
||||
})
|
||||
return findings
|
||||
|
||||
|
||||
def extract_strings(dx, apk):
|
||||
"""Extract suspicious strings: URLs, IPs, base64 patterns."""
|
||||
url_pattern = re.compile(r'https?://[\w\-._~:/?#\[\]@!$&\'()*+,;=]+', re.IGNORECASE)
|
||||
ip_pattern = re.compile(r'\b(?:\d{1,3}\.){3}\d{1,3}\b')
|
||||
b64_pattern = re.compile(r'[A-Za-z0-9+/]{30,}={0,2}')
|
||||
|
||||
urls = set()
|
||||
ips = set()
|
||||
b64_strings = []
|
||||
|
||||
if dx:
|
||||
for s in dx.get_strings():
|
||||
val = str(s)
|
||||
urls.update(url_pattern.findall(val))
|
||||
ips.update(ip_pattern.findall(val))
|
||||
b64_matches = b64_pattern.findall(val)
|
||||
b64_strings.extend(b64_matches[:5])
|
||||
|
||||
private_ips = {"10.", "192.168.", "172.16.", "127.0."}
|
||||
external_ips = [ip for ip in ips if not any(ip.startswith(p) for p in private_ips)]
|
||||
|
||||
return {
|
||||
"urls": sorted(urls)[:30],
|
||||
"external_ips": sorted(external_ips)[:20],
|
||||
"suspicious_base64": b64_strings[:10],
|
||||
"url_count": len(urls),
|
||||
"external_ip_count": len(external_ips),
|
||||
}
|
||||
|
||||
|
||||
def detect_obfuscation(apk, dx):
|
||||
"""Detect code obfuscation indicators."""
|
||||
indicators = []
|
||||
if dx:
|
||||
short_class_names = 0
|
||||
for cls in dx.get_classes():
|
||||
name = str(cls.name)
|
||||
parts = name.replace("/", ".").split(".")
|
||||
if any(len(p) == 1 and p.isalpha() for p in parts):
|
||||
short_class_names += 1
|
||||
if short_class_names > 10:
|
||||
indicators.append({"type": "single_letter_classes", "count": short_class_names})
|
||||
|
||||
dex_files = [f for f in apk.get_files() if f.endswith(".dex")]
|
||||
if len(dex_files) > 1:
|
||||
indicators.append({"type": "multi_dex", "dex_count": len(dex_files)})
|
||||
|
||||
native_libs = [f for f in apk.get_files() if f.endswith(".so")]
|
||||
if native_libs:
|
||||
indicators.append({"type": "native_libraries", "libs": native_libs[:10]})
|
||||
|
||||
return {
|
||||
"obfuscation_indicators": indicators,
|
||||
"likely_obfuscated": len(indicators) > 0,
|
||||
}
|
||||
|
||||
|
||||
def full_analysis(apk_path):
|
||||
"""Run comprehensive APK malware analysis."""
|
||||
if not APK or not AnalyzeAPK:
|
||||
return {"error": "androguard not installed: pip install androguard"}
|
||||
|
||||
a, d, dx = AnalyzeAPK(apk_path)
|
||||
|
||||
perm_analysis = analyze_permissions(a)
|
||||
manifest = analyze_manifest(a)
|
||||
suspicious_apis = scan_suspicious_apis(dx)
|
||||
strings = extract_strings(dx, a)
|
||||
obfuscation = detect_obfuscation(a, dx)
|
||||
|
||||
risk_score = 0
|
||||
risk_score += min(perm_analysis["dangerous_count"] * 8, 40)
|
||||
risk_score += min(len(suspicious_apis) * 10, 30)
|
||||
risk_score += min(strings["external_ip_count"] * 5, 15)
|
||||
risk_score += 15 if obfuscation["likely_obfuscated"] else 0
|
||||
risk_score = min(risk_score, 100)
|
||||
|
||||
return {
|
||||
"analysis_type": "Android APK Static Analysis",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"file": apk_path,
|
||||
"manifest": manifest,
|
||||
"permissions": perm_analysis,
|
||||
"suspicious_apis": suspicious_apis[:20],
|
||||
"strings": strings,
|
||||
"obfuscation": obfuscation,
|
||||
"risk_score": risk_score,
|
||||
"risk_level": "CRITICAL" if risk_score >= 70 else "HIGH" if risk_score >= 50 else "MEDIUM" if risk_score >= 25 else "LOW",
|
||||
"mitre_techniques": [
|
||||
{"id": "T1418", "name": "Software Discovery"} if manifest["service_count"] > 5 else None,
|
||||
{"id": "T1417", "name": "Input Capture"} if "android.permission.BIND_ACCESSIBILITY_SERVICE" in perm_analysis["permissions"] else None,
|
||||
{"id": "T1582", "name": "SMS Control"} if "android.permission.SEND_SMS" in perm_analysis["permissions"] else None,
|
||||
{"id": "T1404", "name": "Exploitation for Privilege Escalation"} if any("DevicePolicyManager" in a.get("api", "") for a in suspicious_apis) else None,
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Android APK Malware Analysis Agent")
|
||||
parser.add_argument("apk", help="Path to APK file")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("permissions", help="Analyze permissions")
|
||||
sub.add_parser("manifest", help="Extract manifest components")
|
||||
sub.add_parser("apis", help="Scan for suspicious API calls")
|
||||
sub.add_parser("strings", help="Extract URLs, IPs, and encoded strings")
|
||||
sub.add_parser("full", help="Full malware analysis")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "full" or args.command is None:
|
||||
result = full_analysis(args.apk)
|
||||
else:
|
||||
a, d, dx = AnalyzeAPK(args.apk)
|
||||
if args.command == "permissions":
|
||||
result = analyze_permissions(a)
|
||||
elif args.command == "manifest":
|
||||
result = analyze_manifest(a)
|
||||
elif args.command == "apis":
|
||||
result = scan_suspicious_apis(dx)
|
||||
elif args.command == "strings":
|
||||
result = extract_strings(dx, a)
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: analyzing-office365-audit-logs-for-compromise
|
||||
description: Parse Office 365 Unified Audit Logs via Microsoft Graph API to detect email forwarding rule creation, inbox delegation, suspicious OAuth app grants, and other indicators of account compromise.
|
||||
domain: cybersecurity
|
||||
subdomain: cloud-security
|
||||
tags: [Office365, Microsoft-Graph, audit-logs, email-compromise, inbox-rules, OAuth, BEC]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Analyzing Office 365 Audit Logs for Compromise
|
||||
|
||||
## Overview
|
||||
|
||||
Business Email Compromise (BEC) attacks often leave traces in Office 365 audit logs: suspicious inbox rule creation, email forwarding to external addresses, mailbox delegation changes, and unauthorized OAuth application consent grants. This skill uses the Microsoft Graph API to query the Unified Audit Log, enumerate inbox rules across mailboxes, detect forwarding configurations, and identify compromised account indicators.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Azure AD app registration with `AuditLog.Read.All`, `MailboxSettings.Read`, `Mail.Read` (application permissions)
|
||||
- Python 3.9+ with `msal`, `requests`
|
||||
- Client secret or certificate for authentication
|
||||
- Global Reader or Security Reader role
|
||||
|
||||
## Steps
|
||||
|
||||
1. Authenticate to Microsoft Graph using MSAL client credentials flow
|
||||
2. Query Unified Audit Log for suspicious operations (Set-Mailbox, New-InboxRule)
|
||||
3. Enumerate inbox rules across mailboxes and flag forwarding rules
|
||||
4. Detect mailbox delegation changes (Add-MailboxPermission)
|
||||
5. Identify OAuth consent grants to suspicious applications
|
||||
6. Check for suspicious sign-in patterns from audit logs
|
||||
7. Generate compromise indicator report with timeline
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing forwarding rules, delegation changes, OAuth grants, and suspicious audit events with risk scores
|
||||
- Timeline of compromise indicators with affected mailboxes
|
||||
@@ -0,0 +1,66 @@
|
||||
# API Reference — Analyzing Office 365 Audit Logs for Compromise
|
||||
|
||||
## Libraries Used
|
||||
- **msal**: Microsoft Authentication Library — client credentials flow for Graph API
|
||||
- **requests**: HTTP client for Graph API calls with pagination
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py --tenant-id T --client-id C --client-secret S audit-logs --days 7
|
||||
python agent.py --tenant-id T --client-id C --client-secret S inbox-rules --user user@domain.com
|
||||
python agent.py --tenant-id T --client-id C --client-secret S forwarding --user user@domain.com
|
||||
python agent.py --tenant-id T --client-id C --client-secret S oauth
|
||||
python agent.py --tenant-id T --client-id C --client-secret S full --users user1@d.com user2@d.com --days 7
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `get_access_token(tenant_id, client_id, client_secret)` — MSAL auth
|
||||
Uses `ConfidentialClientApplication.acquire_token_for_client()` with
|
||||
`https://graph.microsoft.com/.default` scope.
|
||||
|
||||
### `query_audit_logs(token, days)` — Suspicious operation detection
|
||||
Queries `/auditLogs/directoryAudits` for 11 suspicious operations:
|
||||
New-InboxRule, Set-InboxRule, Set-Mailbox, Add-MailboxPermission,
|
||||
UpdateInboxRules, New-TransportRule, etc.
|
||||
|
||||
### `check_inbox_rules(token, user_id)` — Forwarding rule detection
|
||||
GET `/users/{id}/mailFolders/inbox/messageRules`. Checks `actions` for
|
||||
`forwardTo`, `forwardAsAttachmentTo`, `redirectTo`. Flags external forwards
|
||||
and delete-after-forward patterns.
|
||||
|
||||
### `check_mailbox_forwarding(token, user_id)` — SMTP forwarding check
|
||||
GET `/users/{id}/mailboxSettings`. Checks auto-reply status and external audience.
|
||||
|
||||
### `check_oauth_grants(token)` — OAuth consent audit
|
||||
GET `/oauth2PermissionGrants`. Flags grants with Mail.Read, Mail.ReadWrite,
|
||||
Mail.Send, Files.ReadWrite.All, MailboxSettings.ReadWrite.
|
||||
|
||||
### `full_audit(token, users, days)` — Comprehensive compromise analysis
|
||||
|
||||
## Microsoft Graph Endpoints
|
||||
| Endpoint | Method | Permission |
|
||||
|----------|--------|-----------|
|
||||
| `/auditLogs/directoryAudits` | GET | `AuditLog.Read.All` |
|
||||
| `/users/{id}/mailFolders/inbox/messageRules` | GET | `MailboxSettings.Read` |
|
||||
| `/users/{id}/mailboxSettings` | GET | `MailboxSettings.Read` |
|
||||
| `/oauth2PermissionGrants` | GET | `Directory.Read.All` |
|
||||
|
||||
## Inbox Rule Risk Scoring
|
||||
| Factor | Points |
|
||||
|--------|--------|
|
||||
| Forwarding rule present | +40 |
|
||||
| Delete after forward | +25 |
|
||||
| External forward address | +20 |
|
||||
| Mark as read + forward | +15 |
|
||||
|
||||
## Suspicious Operations Monitored
|
||||
New-InboxRule, Set-InboxRule, Set-Mailbox, Add-MailboxPermission,
|
||||
Set-MailboxJunkEmailConfiguration, Set-OwaMailboxPolicy, New-TransportRule,
|
||||
Add-RecipientPermission, Set-TransportRule, UpdateInboxRules,
|
||||
Set-MailboxAutoReplyConfiguration
|
||||
|
||||
## Dependencies
|
||||
- `msal` >= 1.24.0
|
||||
- `requests` >= 2.28.0
|
||||
- Azure AD app with `AuditLog.Read.All`, `MailboxSettings.Read`, `Mail.Read`
|
||||
@@ -0,0 +1,253 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for analyzing Office 365 audit logs for compromise indicators via Microsoft Graph."""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
try:
|
||||
import msal
|
||||
except ImportError:
|
||||
msal = None
|
||||
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
requests = None
|
||||
|
||||
GRAPH_BASE = "https://graph.microsoft.com/v1.0"
|
||||
|
||||
SUSPICIOUS_OPERATIONS = [
|
||||
"New-InboxRule", "Set-InboxRule", "Set-Mailbox",
|
||||
"Add-MailboxPermission", "Set-MailboxJunkEmailConfiguration",
|
||||
"Set-OwaMailboxPolicy", "New-TransportRule",
|
||||
"Add-RecipientPermission", "Set-TransportRule",
|
||||
"UpdateInboxRules", "Set-MailboxAutoReplyConfiguration",
|
||||
]
|
||||
|
||||
|
||||
def get_access_token(tenant_id, client_id, client_secret):
|
||||
"""Authenticate via MSAL client credentials flow."""
|
||||
if not msal:
|
||||
return None
|
||||
authority = f"https://login.microsoftonline.com/{tenant_id}"
|
||||
app = msal.ConfidentialClientApplication(
|
||||
client_id, authority=authority, client_credential=client_secret
|
||||
)
|
||||
result = app.acquire_token_for_client(scopes=["https://graph.microsoft.com/.default"])
|
||||
if "access_token" in result:
|
||||
return result["access_token"]
|
||||
raise RuntimeError(f"Auth failed: {result.get('error_description', result.get('error'))}")
|
||||
|
||||
|
||||
def graph_get(token, endpoint, params=None):
|
||||
"""Paginated GET against Microsoft Graph API."""
|
||||
headers = {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
|
||||
url = f"{GRAPH_BASE}{endpoint}"
|
||||
all_items = []
|
||||
while url:
|
||||
resp = requests.get(url, headers=headers, params=params, timeout=30)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
all_items.extend(data.get("value", []))
|
||||
url = data.get("@odata.nextLink")
|
||||
params = None
|
||||
return all_items
|
||||
|
||||
|
||||
def query_audit_logs(token, days=7):
|
||||
"""Query Unified Audit Log for suspicious mail operations."""
|
||||
since = (datetime.utcnow() - timedelta(days=days)).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
events = []
|
||||
for operation in SUSPICIOUS_OPERATIONS:
|
||||
filter_str = (
|
||||
f"activityDisplayName eq '{operation}' "
|
||||
f"and activityDateTime ge {since}"
|
||||
)
|
||||
try:
|
||||
logs = graph_get(token, "/auditLogs/directoryAudits",
|
||||
params={"$filter": filter_str, "$top": "100"})
|
||||
for log in logs:
|
||||
initiated = log.get("initiatedBy", {}).get("user", {})
|
||||
events.append({
|
||||
"operation": operation,
|
||||
"timestamp": log.get("activityDateTime"),
|
||||
"result": log.get("result"),
|
||||
"user": initiated.get("userPrincipalName"),
|
||||
"ip_address": initiated.get("ipAddress"),
|
||||
"target": [t.get("displayName") for t in log.get("targetResources", [])],
|
||||
"details": log.get("additionalDetails"),
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
events.sort(key=lambda x: x.get("timestamp", ""), reverse=True)
|
||||
return events
|
||||
|
||||
|
||||
def check_inbox_rules(token, user_id):
|
||||
"""List inbox rules for a specific mailbox and flag forwarding rules."""
|
||||
rules = graph_get(token, f"/users/{user_id}/mailFolders/inbox/messageRules")
|
||||
findings = []
|
||||
for rule in rules:
|
||||
is_forwarding = False
|
||||
forward_to = []
|
||||
actions = rule.get("actions", {})
|
||||
if actions.get("forwardTo"):
|
||||
is_forwarding = True
|
||||
forward_to = [r.get("emailAddress", {}).get("address", "")
|
||||
for r in actions["forwardTo"]]
|
||||
if actions.get("forwardAsAttachmentTo"):
|
||||
is_forwarding = True
|
||||
forward_to += [r.get("emailAddress", {}).get("address", "")
|
||||
for r in actions["forwardAsAttachmentTo"]]
|
||||
if actions.get("redirectTo"):
|
||||
is_forwarding = True
|
||||
forward_to += [r.get("emailAddress", {}).get("address", "")
|
||||
for r in actions["redirectTo"]]
|
||||
delete_after = actions.get("delete", False)
|
||||
mark_read = actions.get("markAsRead", False)
|
||||
|
||||
risk = 0
|
||||
if is_forwarding:
|
||||
risk += 40
|
||||
if delete_after:
|
||||
risk += 25
|
||||
if mark_read and is_forwarding:
|
||||
risk += 15
|
||||
external = [f for f in forward_to if f and not f.endswith(user_id.split("@")[-1])]
|
||||
if external:
|
||||
risk += 20
|
||||
|
||||
findings.append({
|
||||
"rule_id": rule.get("id"),
|
||||
"display_name": rule.get("displayName"),
|
||||
"enabled": rule.get("isEnabled"),
|
||||
"is_forwarding": is_forwarding,
|
||||
"forward_to": forward_to,
|
||||
"external_forwards": external,
|
||||
"delete_after_forward": delete_after,
|
||||
"mark_as_read": mark_read,
|
||||
"conditions": rule.get("conditions"),
|
||||
"risk_score": min(risk, 100),
|
||||
"risk_level": "CRITICAL" if risk >= 70 else "HIGH" if risk >= 50 else "MEDIUM" if risk >= 25 else "LOW",
|
||||
})
|
||||
return findings
|
||||
|
||||
|
||||
def check_mailbox_forwarding(token, user_id):
|
||||
"""Check mailbox-level SMTP forwarding configuration."""
|
||||
try:
|
||||
headers = {"Authorization": f"Bearer {token}"}
|
||||
resp = requests.get(
|
||||
f"{GRAPH_BASE}/users/{user_id}/mailboxSettings",
|
||||
headers=headers, timeout=30
|
||||
)
|
||||
resp.raise_for_status()
|
||||
settings = resp.json()
|
||||
auto_reply = settings.get("automaticRepliesSetting", {})
|
||||
return {
|
||||
"user": user_id,
|
||||
"auto_reply_enabled": auto_reply.get("status") != "disabled",
|
||||
"auto_reply_external_audience": auto_reply.get("externalAudience"),
|
||||
"language": settings.get("language", {}).get("locale"),
|
||||
"timezone": settings.get("timeZone"),
|
||||
}
|
||||
except Exception as e:
|
||||
return {"user": user_id, "error": str(e)}
|
||||
|
||||
|
||||
def check_oauth_grants(token):
|
||||
"""Check for suspicious OAuth application consent grants."""
|
||||
grants = graph_get(token, "/oauth2PermissionGrants")
|
||||
high_risk_scopes = {"Mail.Read", "Mail.ReadWrite", "Mail.Send",
|
||||
"Files.ReadWrite.All", "MailboxSettings.ReadWrite"}
|
||||
suspicious = []
|
||||
for g in grants:
|
||||
scopes = g.get("scope", "").split()
|
||||
risky = [s for s in scopes if s in high_risk_scopes]
|
||||
if risky:
|
||||
suspicious.append({
|
||||
"client_id": g.get("clientId"),
|
||||
"consent_type": g.get("consentType"),
|
||||
"principal_id": g.get("principalId"),
|
||||
"high_risk_scopes": risky,
|
||||
"all_scopes": scopes,
|
||||
"risk_score": min(len(risky) * 20, 100),
|
||||
})
|
||||
suspicious.sort(key=lambda x: x["risk_score"], reverse=True)
|
||||
return suspicious
|
||||
|
||||
|
||||
def full_audit(token, users=None, days=7):
|
||||
"""Run comprehensive O365 compromise audit."""
|
||||
audit_events = query_audit_logs(token, days)
|
||||
oauth_findings = check_oauth_grants(token)
|
||||
|
||||
inbox_findings = []
|
||||
forwarding_findings = []
|
||||
if users:
|
||||
for user in users:
|
||||
rules = check_inbox_rules(token, user)
|
||||
inbox_findings.extend([{**r, "mailbox": user} for r in rules])
|
||||
fwd = check_mailbox_forwarding(token, user)
|
||||
forwarding_findings.append(fwd)
|
||||
|
||||
suspicious_rules = [r for r in inbox_findings if r.get("is_forwarding")]
|
||||
external_forwards = [r for r in inbox_findings if r.get("external_forwards")]
|
||||
|
||||
return {
|
||||
"audit_type": "Office 365 Compromise Analysis",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"lookback_days": days,
|
||||
"summary": {
|
||||
"suspicious_audit_events": len(audit_events),
|
||||
"oauth_high_risk_grants": len(oauth_findings),
|
||||
"forwarding_rules_found": len(suspicious_rules),
|
||||
"external_forwarding_rules": len(external_forwards),
|
||||
"mailboxes_scanned": len(users) if users else 0,
|
||||
},
|
||||
"audit_events": audit_events[:25],
|
||||
"oauth_findings": oauth_findings[:15],
|
||||
"forwarding_rules": suspicious_rules[:20],
|
||||
"mailbox_settings": forwarding_findings[:20],
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Office 365 Audit Log Compromise Analysis Agent")
|
||||
parser.add_argument("--tenant-id", required=True, help="Azure AD tenant ID")
|
||||
parser.add_argument("--client-id", required=True, help="App registration client ID")
|
||||
parser.add_argument("--client-secret", required=True, help="App client secret")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
p_audit = sub.add_parser("audit-logs", help="Query suspicious audit events")
|
||||
p_audit.add_argument("--days", type=int, default=7)
|
||||
p_rules = sub.add_parser("inbox-rules", help="Check inbox rules for forwarding")
|
||||
p_rules.add_argument("--user", required=True, help="User principal name")
|
||||
p_fwd = sub.add_parser("forwarding", help="Check mailbox forwarding settings")
|
||||
p_fwd.add_argument("--user", required=True)
|
||||
sub.add_parser("oauth", help="Check OAuth consent grants")
|
||||
p_full = sub.add_parser("full", help="Full compromise audit")
|
||||
p_full.add_argument("--users", nargs="+", help="User principal names to scan")
|
||||
p_full.add_argument("--days", type=int, default=7)
|
||||
args = parser.parse_args()
|
||||
|
||||
token = get_access_token(args.tenant_id, args.client_id, args.client_secret)
|
||||
|
||||
if args.command == "audit-logs":
|
||||
result = query_audit_logs(token, args.days)
|
||||
elif args.command == "inbox-rules":
|
||||
result = check_inbox_rules(token, args.user)
|
||||
elif args.command == "forwarding":
|
||||
result = check_mailbox_forwarding(token, args.user)
|
||||
elif args.command == "oauth":
|
||||
result = check_oauth_grants(token)
|
||||
elif args.command == "full" or args.command is None:
|
||||
result = full_audit(token, getattr(args, "users", None), getattr(args, "days", 7))
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: detecting-shadow-it-cloud-usage
|
||||
description: Detect unauthorized SaaS and cloud service usage (shadow IT) by analyzing proxy logs, DNS query logs, and netflow data using Python pandas for traffic pattern analysis and domain classification.
|
||||
domain: cybersecurity
|
||||
subdomain: cloud-security
|
||||
tags: [shadow-IT, SaaS-discovery, proxy-logs, DNS-analysis, netflow, cloud-security, pandas]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Detecting Shadow IT Cloud Usage
|
||||
|
||||
## Overview
|
||||
|
||||
Shadow IT refers to unauthorized SaaS applications and cloud services used without IT approval. This skill analyzes proxy logs, DNS query logs, and firewall/netflow data to identify unauthorized cloud service usage, classify discovered domains against known SaaS categories, measure data transfer volumes, and flag high-risk services based on security posture and compliance requirements.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+ with `pandas`, `tldextract`
|
||||
- Proxy logs (Squid, Zscaler, or Palo Alto format) or DNS query logs
|
||||
- SaaS application catalog/blocklist for classification
|
||||
- Network firewall logs with FQDN resolution (optional)
|
||||
|
||||
## Steps
|
||||
|
||||
1. Parse proxy access logs and extract destination domains with traffic volumes
|
||||
2. Parse DNS query logs to identify resolved cloud service domains
|
||||
3. Aggregate traffic by domain using pandas — total bytes, request counts, unique users
|
||||
4. Classify domains against known SaaS categories (storage, email, dev tools, AI)
|
||||
5. Flag unauthorized services not on the approved application list
|
||||
6. Calculate risk scores based on data volume, user count, and service category
|
||||
7. Generate shadow IT discovery report with remediation recommendations
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing discovered cloud services with traffic volumes, user counts, risk scores, and approval status
|
||||
- Top unauthorized services ranked by data exfiltration risk
|
||||
@@ -0,0 +1,61 @@
|
||||
# API Reference — Detecting Shadow IT Cloud Usage
|
||||
|
||||
## Libraries Used
|
||||
- **pandas**: DataFrame aggregation for traffic analysis — groupby, agg, nunique
|
||||
- **tldextract**: Accurate registered domain extraction from URLs/hostnames
|
||||
- **csv**: CSV log parsing with DictReader
|
||||
- **re**: Regex parsing for Squid proxy and BIND DNS query log formats
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py access.log --type proxy parse
|
||||
python agent.py access.log --type proxy analyze
|
||||
python agent.py dns-queries.log --type dns full
|
||||
python agent.py traffic.csv --type csv --approved approved.txt full
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `parse_proxy_log(filepath)` — Parse Squid/common proxy access logs
|
||||
Regex pattern matches Squid format: `timestamp duration client_ip status bytes method url`.
|
||||
Falls back to Apache Common Log Format parsing.
|
||||
|
||||
### `parse_dns_log(filepath)` — Parse BIND/named DNS query logs
|
||||
Extracts query name and type from `query: DOMAIN IN TYPE` patterns.
|
||||
Strips trailing dots from FQDNs.
|
||||
|
||||
### `parse_csv_log(filepath)` — Parse generic CSV traffic logs
|
||||
Expects columns: timestamp, src_ip, dst_domain, bytes_out, bytes_in.
|
||||
|
||||
### `analyze_traffic(records)` — Aggregate and classify traffic
|
||||
Uses pandas groupby on domain: total_bytes (sum), request_count (count),
|
||||
unique_users (nunique). Falls back to collections.defaultdict if pandas unavailable.
|
||||
|
||||
### `classify_domain(domain)` — Categorize against SaaS database
|
||||
Categories: storage, email, dev_tools, ai_ml, messaging, file_sharing, vpn_proxy.
|
||||
|
||||
### `full_audit(log_path, log_type, approved_list)` — Complete shadow IT audit
|
||||
|
||||
## Risk Scoring
|
||||
| Factor | Points |
|
||||
|--------|--------|
|
||||
| Unapproved domain | +30 |
|
||||
| Storage/file-sharing/VPN category | +25 |
|
||||
| Email category | +15 |
|
||||
| Data volume (per 10 MB) | +1 (max 20) |
|
||||
| Unique users (per user) | +3 (max 15) |
|
||||
|
||||
## SaaS Category Database
|
||||
| Category | Example Domains |
|
||||
|----------|----------------|
|
||||
| storage | dropbox.com, box.com, mega.nz, wetransfer.com |
|
||||
| email | protonmail.com, tutanota.com, guerrillamail.com |
|
||||
| dev_tools | github.com, gitlab.com, replit.com |
|
||||
| ai_ml | chat.openai.com, claude.ai, huggingface.co |
|
||||
| messaging | telegram.org, discord.com, signal.org |
|
||||
| file_sharing | pastebin.com, file.io, gofile.io |
|
||||
| vpn_proxy | nordvpn.com, expressvpn.com, protonvpn.com |
|
||||
|
||||
## Dependencies
|
||||
- `pandas` >= 1.5.0
|
||||
- `tldextract` >= 3.4.0 (optional, improves domain extraction accuracy)
|
||||
@@ -0,0 +1,285 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for detecting shadow IT cloud usage via proxy logs, DNS queries, and netflow."""
|
||||
|
||||
import json
|
||||
import csv
|
||||
import re
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
from collections import defaultdict
|
||||
|
||||
try:
|
||||
import pandas as pd
|
||||
except ImportError:
|
||||
pd = None
|
||||
|
||||
try:
|
||||
import tldextract
|
||||
except ImportError:
|
||||
tldextract = None
|
||||
|
||||
KNOWN_SAAS_DOMAINS = {
|
||||
"storage": ["dropbox.com", "box.com", "mega.nz", "wetransfer.com", "mediafire.com",
|
||||
"pcloud.com", "sync.com", "icloud.com"],
|
||||
"email": ["protonmail.com", "tutanota.com", "guerrillamail.com", "yandex.com",
|
||||
"mail.ru", "zoho.com"],
|
||||
"dev_tools": ["github.com", "gitlab.com", "bitbucket.org", "replit.com",
|
||||
"codepen.io", "stackblitz.com", "vercel.app", "netlify.app"],
|
||||
"ai_ml": ["chat.openai.com", "claude.ai", "bard.google.com", "huggingface.co",
|
||||
"midjourney.com", "perplexity.ai"],
|
||||
"messaging": ["telegram.org", "web.telegram.org", "signal.org", "discord.com",
|
||||
"slack.com", "whatsapp.com"],
|
||||
"file_sharing": ["pastebin.com", "hastebin.com", "justpaste.it", "file.io",
|
||||
"anonfiles.com", "gofile.io"],
|
||||
"vpn_proxy": ["nordvpn.com", "expressvpn.com", "surfshark.com", "hide.me",
|
||||
"windscribe.com", "protonvpn.com"],
|
||||
}
|
||||
|
||||
APPROVED_DOMAINS = set()
|
||||
|
||||
|
||||
def load_approved_list(filepath):
|
||||
"""Load approved SaaS domain list from a text file."""
|
||||
global APPROVED_DOMAINS
|
||||
try:
|
||||
with open(filepath, "r") as f:
|
||||
APPROVED_DOMAINS = {line.strip().lower() for line in f if line.strip()}
|
||||
except FileNotFoundError:
|
||||
APPROVED_DOMAINS = set()
|
||||
|
||||
|
||||
def extract_domain(url_or_host):
|
||||
"""Extract registered domain from URL or hostname."""
|
||||
if tldextract:
|
||||
ext = tldextract.extract(url_or_host)
|
||||
return f"{ext.domain}.{ext.suffix}".lower() if ext.suffix else url_or_host.lower()
|
||||
host = re.sub(r'^https?://', '', url_or_host).split('/')[0].split(':')[0]
|
||||
parts = host.lower().split('.')
|
||||
return '.'.join(parts[-2:]) if len(parts) >= 2 else host
|
||||
|
||||
|
||||
def parse_proxy_log(filepath):
|
||||
"""Parse proxy access log (Squid/common format) into structured records."""
|
||||
records = []
|
||||
squid_pattern = re.compile(
|
||||
r'^(\S+)\s+(\d+)\s+(\S+)\s+\w+/(\d+)\s+(\d+)\s+(\w+)\s+(\S+)\s+'
|
||||
)
|
||||
with open(filepath, "r") as f:
|
||||
for line in f:
|
||||
m = squid_pattern.match(line)
|
||||
if m:
|
||||
records.append({
|
||||
"timestamp": m.group(1),
|
||||
"duration_ms": int(m.group(2)),
|
||||
"client_ip": m.group(3),
|
||||
"status_code": int(m.group(4)),
|
||||
"bytes": int(m.group(5)),
|
||||
"method": m.group(6),
|
||||
"url": m.group(7),
|
||||
"domain": extract_domain(m.group(7)),
|
||||
})
|
||||
else:
|
||||
parts = line.strip().split()
|
||||
if len(parts) >= 7:
|
||||
url = parts[6] if parts[6].startswith("http") else parts[5]
|
||||
records.append({
|
||||
"client_ip": parts[0],
|
||||
"timestamp": parts[3].lstrip("["),
|
||||
"method": parts[5].lstrip('"'),
|
||||
"url": url,
|
||||
"domain": extract_domain(url),
|
||||
"status_code": int(parts[8]) if len(parts) > 8 and parts[8].isdigit() else 0,
|
||||
"bytes": int(parts[9]) if len(parts) > 9 and parts[9].isdigit() else 0,
|
||||
})
|
||||
return records
|
||||
|
||||
|
||||
def parse_dns_log(filepath):
|
||||
"""Parse DNS query log (named/bind query log format)."""
|
||||
records = []
|
||||
dns_pattern = re.compile(r'query:\s+(\S+)\s+IN\s+(\w+)')
|
||||
with open(filepath, "r") as f:
|
||||
for line in f:
|
||||
m = dns_pattern.search(line)
|
||||
if m:
|
||||
queried = m.group(1).rstrip(".")
|
||||
records.append({
|
||||
"query_name": queried,
|
||||
"query_type": m.group(2),
|
||||
"domain": extract_domain(queried),
|
||||
"raw_line": line.strip()[:200],
|
||||
})
|
||||
return records
|
||||
|
||||
|
||||
def parse_csv_log(filepath):
|
||||
"""Parse generic CSV log with columns: timestamp, src_ip, dst_domain, bytes_out, bytes_in."""
|
||||
records = []
|
||||
with open(filepath, "r") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
domain = extract_domain(row.get("dst_domain", row.get("domain", row.get("url", ""))))
|
||||
records.append({
|
||||
"timestamp": row.get("timestamp", ""),
|
||||
"client_ip": row.get("src_ip", row.get("client_ip", "")),
|
||||
"domain": domain,
|
||||
"bytes_out": int(row.get("bytes_out", row.get("bytes", 0)) or 0),
|
||||
"bytes_in": int(row.get("bytes_in", 0) or 0),
|
||||
})
|
||||
return records
|
||||
|
||||
|
||||
def classify_domain(domain):
|
||||
"""Classify a domain against known SaaS categories."""
|
||||
for category, domains in KNOWN_SAAS_DOMAINS.items():
|
||||
if domain in domains:
|
||||
return category
|
||||
return "unknown"
|
||||
|
||||
|
||||
def analyze_traffic(records):
|
||||
"""Aggregate traffic by domain using pandas and classify."""
|
||||
if not pd:
|
||||
agg = defaultdict(lambda: {"bytes": 0, "requests": 0, "users": set()})
|
||||
for r in records:
|
||||
d = r.get("domain", "")
|
||||
if not d:
|
||||
continue
|
||||
agg[d]["bytes"] += r.get("bytes", 0) + r.get("bytes_out", 0)
|
||||
agg[d]["requests"] += 1
|
||||
agg[d]["users"].add(r.get("client_ip", "unknown"))
|
||||
results = []
|
||||
for domain, stats in agg.items():
|
||||
cat = classify_domain(domain)
|
||||
approved = domain in APPROVED_DOMAINS
|
||||
risk = 0
|
||||
if not approved:
|
||||
risk += 30
|
||||
if cat in ("storage", "file_sharing", "vpn_proxy"):
|
||||
risk += 25
|
||||
if cat == "email":
|
||||
risk += 15
|
||||
risk += min(stats["bytes"] // (10 * 1024 * 1024), 20)
|
||||
risk += min(len(stats["users"]) * 3, 15)
|
||||
risk = min(risk, 100)
|
||||
results.append({
|
||||
"domain": domain,
|
||||
"category": cat,
|
||||
"approved": approved,
|
||||
"total_bytes": stats["bytes"],
|
||||
"total_bytes_mb": round(stats["bytes"] / (1024 * 1024), 2),
|
||||
"request_count": stats["requests"],
|
||||
"unique_users": len(stats["users"]),
|
||||
"risk_score": risk,
|
||||
"risk_level": "CRITICAL" if risk >= 70 else "HIGH" if risk >= 50 else "MEDIUM" if risk >= 25 else "LOW",
|
||||
})
|
||||
results.sort(key=lambda x: x["risk_score"], reverse=True)
|
||||
return results
|
||||
|
||||
df = pd.DataFrame(records)
|
||||
if "bytes" not in df.columns:
|
||||
df["bytes"] = df.get("bytes_out", 0)
|
||||
df["bytes"] = pd.to_numeric(df["bytes"], errors="coerce").fillna(0)
|
||||
grouped = df.groupby("domain").agg(
|
||||
total_bytes=("bytes", "sum"),
|
||||
request_count=("domain", "count"),
|
||||
unique_users=("client_ip", "nunique") if "client_ip" in df.columns else ("domain", "count"),
|
||||
).reset_index()
|
||||
results = []
|
||||
for _, row in grouped.iterrows():
|
||||
domain = row["domain"]
|
||||
cat = classify_domain(domain)
|
||||
approved = domain in APPROVED_DOMAINS
|
||||
risk = 0
|
||||
if not approved:
|
||||
risk += 30
|
||||
if cat in ("storage", "file_sharing", "vpn_proxy"):
|
||||
risk += 25
|
||||
if cat == "email":
|
||||
risk += 15
|
||||
risk += min(int(row["total_bytes"]) // (10 * 1024 * 1024), 20)
|
||||
risk += min(int(row["unique_users"]) * 3, 15)
|
||||
risk = min(risk, 100)
|
||||
results.append({
|
||||
"domain": domain,
|
||||
"category": cat,
|
||||
"approved": approved,
|
||||
"total_bytes": int(row["total_bytes"]),
|
||||
"total_bytes_mb": round(row["total_bytes"] / (1024 * 1024), 2),
|
||||
"request_count": int(row["request_count"]),
|
||||
"unique_users": int(row["unique_users"]),
|
||||
"risk_score": risk,
|
||||
"risk_level": "CRITICAL" if risk >= 70 else "HIGH" if risk >= 50 else "MEDIUM" if risk >= 25 else "LOW",
|
||||
})
|
||||
results.sort(key=lambda x: x["risk_score"], reverse=True)
|
||||
return results
|
||||
|
||||
|
||||
def full_audit(log_path, log_type="proxy", approved_list=None):
|
||||
"""Run full shadow IT discovery audit."""
|
||||
if approved_list:
|
||||
load_approved_list(approved_list)
|
||||
if log_type == "proxy":
|
||||
records = parse_proxy_log(log_path)
|
||||
elif log_type == "dns":
|
||||
records = parse_dns_log(log_path)
|
||||
elif log_type == "csv":
|
||||
records = parse_csv_log(log_path)
|
||||
else:
|
||||
return {"error": f"Unknown log type: {log_type}"}
|
||||
analysis = analyze_traffic(records)
|
||||
unauthorized = [a for a in analysis if not a["approved"] and a["category"] != "unknown"]
|
||||
return {
|
||||
"audit_type": "Shadow IT Cloud Usage Discovery",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"log_file": log_path,
|
||||
"log_type": log_type,
|
||||
"total_records_parsed": len(records),
|
||||
"unique_domains": len(analysis),
|
||||
"unauthorized_saas_services": len(unauthorized),
|
||||
"critical_findings": sum(1 for a in analysis if a["risk_level"] == "CRITICAL"),
|
||||
"high_findings": sum(1 for a in analysis if a["risk_level"] == "HIGH"),
|
||||
"top_shadow_it_services": unauthorized[:20],
|
||||
"all_services": analysis[:50],
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Shadow IT Cloud Usage Detection Agent")
|
||||
parser.add_argument("log_file", help="Path to log file")
|
||||
parser.add_argument("--type", choices=["proxy", "dns", "csv"], default="proxy", help="Log file format")
|
||||
parser.add_argument("--approved", help="Path to approved domains list (one per line)")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("parse", help="Parse log file and show raw records")
|
||||
sub.add_parser("analyze", help="Analyze traffic patterns")
|
||||
sub.add_parser("full", help="Full shadow IT audit")
|
||||
args = parser.parse_args()
|
||||
|
||||
if approved := args.approved:
|
||||
load_approved_list(approved)
|
||||
|
||||
if args.command == "parse":
|
||||
if args.type == "proxy":
|
||||
result = parse_proxy_log(args.log_file)
|
||||
elif args.type == "dns":
|
||||
result = parse_dns_log(args.log_file)
|
||||
else:
|
||||
result = parse_csv_log(args.log_file)
|
||||
elif args.command == "analyze":
|
||||
if args.type == "proxy":
|
||||
records = parse_proxy_log(args.log_file)
|
||||
elif args.type == "dns":
|
||||
records = parse_dns_log(args.log_file)
|
||||
else:
|
||||
records = parse_csv_log(args.log_file)
|
||||
result = analyze_traffic(records)
|
||||
elif args.command == "full" or args.command is None:
|
||||
result = full_audit(args.log_file, args.type, args.approved)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: detecting-suspicious-oauth-application-consent
|
||||
description: Detect risky OAuth application consent grants in Azure AD / Microsoft Entra ID using Microsoft Graph API, audit logs, and permission analysis to identify illicit consent grant attacks.
|
||||
domain: cybersecurity
|
||||
subdomain: cloud-security
|
||||
tags: [OAuth, Azure-AD, Entra-ID, Microsoft-Graph, illicit-consent, cloud-security, application-permissions]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Detecting Suspicious OAuth Application Consent
|
||||
|
||||
## Overview
|
||||
|
||||
Illicit consent grant attacks trick users into granting excessive permissions to malicious OAuth applications in Azure AD / Microsoft Entra ID. This skill uses the Microsoft Graph API to enumerate OAuth2 permission grants, analyze application permissions for overly broad scopes, review directory audit logs for consent events, and flag high-risk applications based on publisher verification status and permission scope.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Azure AD / Entra ID tenant with Global Reader or Security Reader role
|
||||
- Microsoft Graph API access with `Application.Read.All`, `AuditLog.Read.All`, `Directory.Read.All`
|
||||
- Python 3.9+ with `msal`, `requests`
|
||||
- App registration with client secret or certificate for authentication
|
||||
|
||||
## Steps
|
||||
|
||||
1. Authenticate to Microsoft Graph using MSAL client credentials flow
|
||||
2. Enumerate all OAuth2 permission grants via `/oauth2PermissionGrants`
|
||||
3. List service principals and their assigned application permissions
|
||||
4. Query directory audit logs for `Consent to application` events
|
||||
5. Flag applications with high-risk scopes (Mail.Read, Files.ReadWrite.All, etc.)
|
||||
6. Check publisher verification status for each application
|
||||
7. Generate risk report with remediation recommendations
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing all OAuth apps with granted permissions, risk scores, unverified publishers, and suspicious consent patterns
|
||||
- Audit trail of consent grant events with user and IP details
|
||||
@@ -0,0 +1,63 @@
|
||||
# API Reference — Detecting Suspicious OAuth Application Consent
|
||||
|
||||
## Libraries Used
|
||||
- **msal**: Microsoft Authentication Library — client credentials flow for Microsoft Graph
|
||||
- **requests**: HTTP client for Graph API calls with pagination
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py --tenant-id TENANT --client-id CLIENT --client-secret SECRET grants
|
||||
python agent.py --tenant-id TENANT --client-id CLIENT --client-secret SECRET apps
|
||||
python agent.py --tenant-id TENANT --client-id CLIENT --client-secret SECRET audit-logs
|
||||
python agent.py --tenant-id TENANT --client-id CLIENT --client-secret SECRET full --days 30
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `get_access_token(tenant_id, client_id, client_secret)` — MSAL client credentials auth
|
||||
Uses `ConfidentialClientApplication.acquire_token_for_client()` with scope
|
||||
`https://graph.microsoft.com/.default` to obtain a bearer token.
|
||||
|
||||
### `graph_get(token, endpoint, params)` — Paginated Graph API GET
|
||||
Handles `@odata.nextLink` pagination. Returns aggregated `value` array.
|
||||
|
||||
### `enumerate_oauth_grants(token)` — List delegated permission grants
|
||||
Calls `GET /oauth2PermissionGrants`. Parses scope strings, flags high-risk scopes.
|
||||
|
||||
### `list_service_principals(token)` — Enumerate apps with verification status
|
||||
Calls `GET /servicePrincipals?$top=999`. Checks `verifiedPublisher` field.
|
||||
|
||||
### `query_consent_audit_logs(token, days)` — Consent event audit trail
|
||||
Calls `GET /auditLogs/directoryAudits` filtered by `activityDisplayName eq 'Consent to application'`.
|
||||
Returns user principal, IP, target app, and result.
|
||||
|
||||
### `analyze_risk(grants, service_principals)` — Risk scoring engine
|
||||
Scoring: +15 per high-risk scope, +25 for unverified publisher, +20 for AllPrincipals consent.
|
||||
Levels: CRITICAL >= 70, HIGH >= 50, MEDIUM >= 25, LOW < 25.
|
||||
|
||||
### `full_audit(token, days)` — Comprehensive consent audit
|
||||
|
||||
## Microsoft Graph Endpoints
|
||||
| Endpoint | Method | Required Permission |
|
||||
|----------|--------|-------------------|
|
||||
| `/oauth2PermissionGrants` | GET | `Directory.Read.All` |
|
||||
| `/servicePrincipals` | GET | `Application.Read.All` |
|
||||
| `/auditLogs/directoryAudits` | GET | `AuditLog.Read.All` |
|
||||
|
||||
## High-Risk OAuth Scopes
|
||||
Mail.Read, Mail.ReadWrite, Mail.Send, Files.ReadWrite.All, Files.Read.All,
|
||||
User.ReadWrite.All, Directory.ReadWrite.All, Sites.ReadWrite.All,
|
||||
Contacts.ReadWrite, MailboxSettings.ReadWrite, People.Read.All,
|
||||
Calendars.ReadWrite, Notes.ReadWrite.All
|
||||
|
||||
## Risk Scoring
|
||||
| Factor | Points |
|
||||
|--------|--------|
|
||||
| Each high-risk scope | +15 |
|
||||
| Unverified publisher | +25 |
|
||||
| Admin consent (AllPrincipals) | +20 |
|
||||
|
||||
## Dependencies
|
||||
- `msal` >= 1.24.0
|
||||
- `requests` >= 2.28.0
|
||||
- Azure AD app registration with `Application.Read.All`, `AuditLog.Read.All`, `Directory.Read.All`
|
||||
@@ -0,0 +1,208 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for detecting suspicious OAuth application consent grants in Azure AD / Entra ID."""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
try:
|
||||
import msal
|
||||
except ImportError:
|
||||
msal = None
|
||||
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
requests = None
|
||||
|
||||
GRAPH_BASE = "https://graph.microsoft.com/v1.0"
|
||||
HIGH_RISK_SCOPES = [
|
||||
"Mail.Read", "Mail.ReadWrite", "Mail.Send",
|
||||
"Files.ReadWrite.All", "Files.Read.All",
|
||||
"User.ReadWrite.All", "Directory.ReadWrite.All",
|
||||
"Sites.ReadWrite.All", "Contacts.ReadWrite",
|
||||
"MailboxSettings.ReadWrite", "People.Read.All",
|
||||
"Calendars.ReadWrite", "Notes.ReadWrite.All",
|
||||
]
|
||||
|
||||
|
||||
def get_access_token(tenant_id, client_id, client_secret):
|
||||
"""Authenticate via MSAL client credentials flow and return access token."""
|
||||
if not msal:
|
||||
return None
|
||||
authority = f"https://login.microsoftonline.com/{tenant_id}"
|
||||
app = msal.ConfidentialClientApplication(
|
||||
client_id, authority=authority, client_credential=client_secret
|
||||
)
|
||||
result = app.acquire_token_for_client(scopes=["https://graph.microsoft.com/.default"])
|
||||
if "access_token" in result:
|
||||
return result["access_token"]
|
||||
raise RuntimeError(f"Auth failed: {result.get('error_description', result.get('error'))}")
|
||||
|
||||
|
||||
def graph_get(token, endpoint, params=None):
|
||||
"""Make authenticated GET request to Microsoft Graph API."""
|
||||
headers = {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
|
||||
url = f"{GRAPH_BASE}{endpoint}"
|
||||
all_items = []
|
||||
while url:
|
||||
resp = requests.get(url, headers=headers, params=params, timeout=30)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
all_items.extend(data.get("value", []))
|
||||
url = data.get("@odata.nextLink")
|
||||
params = None
|
||||
return all_items
|
||||
|
||||
|
||||
def enumerate_oauth_grants(token):
|
||||
"""List all delegated OAuth2 permission grants in the tenant."""
|
||||
grants = graph_get(token, "/oauth2PermissionGrants")
|
||||
results = []
|
||||
for g in grants:
|
||||
scope_list = g.get("scope", "").split()
|
||||
risky = [s for s in scope_list if s in HIGH_RISK_SCOPES]
|
||||
results.append({
|
||||
"id": g.get("id"),
|
||||
"clientId": g.get("clientId"),
|
||||
"consentType": g.get("consentType"),
|
||||
"principalId": g.get("principalId"),
|
||||
"resourceId": g.get("resourceId"),
|
||||
"scopes": scope_list,
|
||||
"high_risk_scopes": risky,
|
||||
"risk_score": len(risky) * 15,
|
||||
})
|
||||
return results
|
||||
|
||||
|
||||
def list_service_principals(token):
|
||||
"""List service principals with their app roles and permissions."""
|
||||
sps = graph_get(token, "/servicePrincipals", params={"$top": "999"})
|
||||
results = []
|
||||
for sp in sps:
|
||||
app_roles = sp.get("appRoles", [])
|
||||
verified = sp.get("verifiedPublisher", {})
|
||||
results.append({
|
||||
"id": sp.get("id"),
|
||||
"appId": sp.get("appId"),
|
||||
"displayName": sp.get("displayName"),
|
||||
"publisherName": sp.get("publisherName"),
|
||||
"verifiedPublisher": verified.get("displayName") if verified else None,
|
||||
"isVerified": bool(verified.get("verifiedPublisherId")),
|
||||
"appRoleCount": len(app_roles),
|
||||
"accountEnabled": sp.get("accountEnabled"),
|
||||
"signInAudience": sp.get("signInAudience"),
|
||||
})
|
||||
return results
|
||||
|
||||
|
||||
def query_consent_audit_logs(token, days=30):
|
||||
"""Query directory audit logs for consent grant events."""
|
||||
since = (datetime.utcnow() - timedelta(days=days)).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
filter_str = (
|
||||
f"activityDisplayName eq 'Consent to application' "
|
||||
f"and activityDateTime ge {since}"
|
||||
)
|
||||
logs = graph_get(token, "/auditLogs/directoryAudits", params={"$filter": filter_str})
|
||||
events = []
|
||||
for log in logs:
|
||||
initiated = log.get("initiatedBy", {}).get("user", {})
|
||||
targets = log.get("targetResources", [])
|
||||
events.append({
|
||||
"activityDateTime": log.get("activityDateTime"),
|
||||
"activityDisplayName": log.get("activityDisplayName"),
|
||||
"result": log.get("result"),
|
||||
"initiatedByUser": initiated.get("userPrincipalName"),
|
||||
"initiatedByIp": initiated.get("ipAddress"),
|
||||
"targetApp": targets[0].get("displayName") if targets else None,
|
||||
"targetAppId": targets[0].get("id") if targets else None,
|
||||
"additionalDetails": log.get("additionalDetails"),
|
||||
})
|
||||
return events
|
||||
|
||||
|
||||
def analyze_risk(grants, service_principals):
|
||||
"""Correlate grants with service principals to produce risk assessment."""
|
||||
sp_map = {sp["id"]: sp for sp in service_principals}
|
||||
findings = []
|
||||
for grant in grants:
|
||||
sp = sp_map.get(grant["clientId"], {})
|
||||
risk = grant["risk_score"]
|
||||
if not sp.get("isVerified"):
|
||||
risk += 25
|
||||
if grant.get("consentType") == "AllPrincipals":
|
||||
risk += 20
|
||||
risk = min(risk, 100)
|
||||
level = "CRITICAL" if risk >= 70 else "HIGH" if risk >= 50 else "MEDIUM" if risk >= 25 else "LOW"
|
||||
findings.append({
|
||||
"appDisplayName": sp.get("displayName", "Unknown"),
|
||||
"appId": sp.get("appId"),
|
||||
"publisherVerified": sp.get("isVerified", False),
|
||||
"consentType": grant.get("consentType"),
|
||||
"highRiskScopes": grant.get("high_risk_scopes"),
|
||||
"riskScore": risk,
|
||||
"riskLevel": level,
|
||||
"recommendation": "Revoke consent and investigate" if risk >= 50
|
||||
else "Review scopes and publisher" if risk >= 25
|
||||
else "Monitor",
|
||||
})
|
||||
findings.sort(key=lambda x: x["riskScore"], reverse=True)
|
||||
return findings
|
||||
|
||||
|
||||
def full_audit(token, days=30):
|
||||
"""Run comprehensive OAuth consent audit."""
|
||||
grants = enumerate_oauth_grants(token)
|
||||
sps = list_service_principals(token)
|
||||
audit_events = query_consent_audit_logs(token, days)
|
||||
risk_findings = analyze_risk(grants, sps)
|
||||
critical = sum(1 for f in risk_findings if f["riskLevel"] == "CRITICAL")
|
||||
high = sum(1 for f in risk_findings if f["riskLevel"] == "HIGH")
|
||||
unverified = sum(1 for sp in sps if not sp.get("isVerified"))
|
||||
return {
|
||||
"audit_type": "OAuth Application Consent Audit",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"summary": {
|
||||
"total_grants": len(grants),
|
||||
"total_service_principals": len(sps),
|
||||
"consent_events_last_n_days": len(audit_events),
|
||||
"critical_findings": critical,
|
||||
"high_findings": high,
|
||||
"unverified_publishers": unverified,
|
||||
},
|
||||
"risk_findings": risk_findings[:25],
|
||||
"recent_consent_events": audit_events[:20],
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="OAuth Application Consent Audit Agent")
|
||||
parser.add_argument("--tenant-id", required=True, help="Azure AD tenant ID")
|
||||
parser.add_argument("--client-id", required=True, help="App registration client ID")
|
||||
parser.add_argument("--client-secret", required=True, help="App client secret")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("grants", help="Enumerate OAuth2 permission grants")
|
||||
sub.add_parser("apps", help="List service principals")
|
||||
sub.add_parser("audit-logs", help="Query consent audit logs")
|
||||
p_full = sub.add_parser("full", help="Full OAuth consent audit")
|
||||
p_full.add_argument("--days", type=int, default=30, help="Audit log lookback days")
|
||||
args = parser.parse_args()
|
||||
|
||||
token = get_access_token(args.tenant_id, args.client_id, args.client_secret)
|
||||
|
||||
if args.command == "grants":
|
||||
result = enumerate_oauth_grants(token)
|
||||
elif args.command == "apps":
|
||||
result = list_service_principals(token)
|
||||
elif args.command == "audit-logs":
|
||||
result = query_consent_audit_logs(token)
|
||||
elif args.command == "full":
|
||||
result = full_audit(token, args.days)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: hunting-for-startup-folder-persistence
|
||||
description: Detect T1547.001 startup folder persistence by monitoring Windows startup directories for suspicious file creation, analyzing autoruns entries, and using Python watchdog for real-time filesystem monitoring.
|
||||
domain: cybersecurity
|
||||
subdomain: threat-hunting
|
||||
tags: [threat-hunting, T1547.001, startup-folder, persistence, autoruns, watchdog, filesystem-monitoring]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Hunting for Startup Folder Persistence
|
||||
|
||||
## Overview
|
||||
|
||||
Attackers use Windows startup folders for persistence (MITRE ATT&CK T1547.001 — Boot or Logon Autostart Execution: Registry Run Keys / Startup Folder). Files placed in `%APPDATA%\Microsoft\Windows\Start Menu\Programs\Startup` or `C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup` execute automatically at user logon. This skill scans startup directories for suspicious files, monitors for real-time changes using Python watchdog, and analyzes file metadata to detect persistence implants.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+ with `watchdog`, `pefile` (optional for PE analysis)
|
||||
- Access to Windows startup folders (user and all-users)
|
||||
- Windows Event Logs for Event ID 4663 correlation (optional)
|
||||
|
||||
## Steps
|
||||
|
||||
1. Enumerate all files in user and system startup directories
|
||||
2. Analyze file types, creation timestamps, and digital signatures
|
||||
3. Flag suspicious file extensions (.bat, .vbs, .ps1, .lnk, .exe)
|
||||
4. Check for recently created files (< 7 days) as potential implants
|
||||
5. Monitor startup folders in real-time using watchdog FileSystemEventHandler
|
||||
6. Correlate with known legitimate startup entries
|
||||
7. Generate threat hunting report with T1547.001 MITRE mapping
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing all startup folder contents with risk scores, file metadata, and suspicious indicators
|
||||
- Real-time monitoring alerts for new file creation in startup directories
|
||||
@@ -0,0 +1,84 @@
|
||||
# API Reference — Hunting for Startup Folder Persistence
|
||||
|
||||
## Libraries Used
|
||||
- **watchdog**: Real-time filesystem monitoring — `Observer`, `FileSystemEventHandler`
|
||||
- **hashlib**: SHA-256 file hashing
|
||||
- **subprocess**: Registry Run key queries via `reg query`
|
||||
- **pathlib**: Cross-platform path handling and file metadata
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py scan
|
||||
python agent.py registry
|
||||
python agent.py monitor --duration 120
|
||||
python agent.py full
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `get_startup_paths()` — Enumerate startup directories
|
||||
Returns user startup (`%APPDATA%\...\Startup`) and all-users startup
|
||||
(`%PROGRAMDATA%\...\Startup`) paths.
|
||||
|
||||
### `analyze_file(filepath, scope)` — Single file risk analysis
|
||||
Computes SHA-256 hash, checks extension against risk table, evaluates
|
||||
file age, size, baseline membership. Risk scoring by extension, recency,
|
||||
and scope.
|
||||
|
||||
### `scan_startup_folders()` — Full startup directory scan
|
||||
Iterates all files in both startup paths. Returns sorted by risk score.
|
||||
|
||||
### `check_registry_run_keys()` — Registry autostart audit
|
||||
Queries 4 Registry Run keys via `reg query`:
|
||||
- `HKCU\...\Run`, `HKCU\...\RunOnce`
|
||||
- `HKLM\...\Run`, `HKLM\...\RunOnce`
|
||||
Flags entries containing powershell, cmd.exe, temp paths, encoded commands.
|
||||
|
||||
### `StartupMonitorHandler` — Watchdog event handler
|
||||
Subclasses `FileSystemEventHandler`. Handles `on_created`, `on_modified`,
|
||||
`on_deleted`. Runs `analyze_file()` on new files and prints JSON alerts.
|
||||
|
||||
### `monitor_startup(duration_seconds)` — Real-time monitoring
|
||||
Creates `Observer`, schedules handler on all startup paths. Monitors for
|
||||
specified duration. Returns detected events.
|
||||
|
||||
### `full_hunt()` — Comprehensive persistence hunt
|
||||
|
||||
## Startup Folder Paths
|
||||
| Scope | Path |
|
||||
|-------|------|
|
||||
| Current User | `%APPDATA%\Microsoft\Windows\Start Menu\Programs\Startup` |
|
||||
| All Users | `%PROGRAMDATA%\Microsoft\Windows\Start Menu\Programs\Startup` |
|
||||
|
||||
## File Extension Risk Scores
|
||||
| Extension | Base Score | Notes |
|
||||
|-----------|-----------|-------|
|
||||
| .ps1 | 45 | PowerShell script |
|
||||
| .hta | 45 | HTML Application |
|
||||
| .pif | 45 | Program Information File |
|
||||
| .vbs, .vbe | 40 | VBScript |
|
||||
| .js, .jse | 40 | JScript |
|
||||
| .wsf, .wsh | 35-40 | Windows Script |
|
||||
| .bat, .cmd | 35 | Batch file |
|
||||
| .exe | 30 | Executable |
|
||||
| .scr | 40 | Screen saver (executable) |
|
||||
| .url | 20 | Internet shortcut |
|
||||
| .lnk | 15 | Shortcut (often legitimate) |
|
||||
|
||||
## Additional Risk Factors
|
||||
| Factor | Points |
|
||||
|--------|--------|
|
||||
| Created within 7 days | +25 |
|
||||
| Created within 24 hours | +15 |
|
||||
| Zero-byte file | +10 |
|
||||
| File > 10 MB | +10 |
|
||||
| Not in baseline | +10 |
|
||||
| All-users scope | +10 |
|
||||
|
||||
## MITRE ATT&CK Mapping
|
||||
- **T1547.001** — Boot or Logon Autostart Execution: Registry Run Keys / Startup Folder
|
||||
- Tactics: Persistence, Privilege Escalation
|
||||
|
||||
## Dependencies
|
||||
- `watchdog` >= 3.0.0
|
||||
- Windows OS (startup folder paths are Windows-specific)
|
||||
@@ -0,0 +1,290 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for hunting T1547.001 startup folder persistence on Windows."""
|
||||
|
||||
import json
|
||||
import os
|
||||
import hashlib
|
||||
import argparse
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from watchdog.observers import Observer
|
||||
from watchdog.events import FileSystemEventHandler
|
||||
except ImportError:
|
||||
Observer = None
|
||||
FileSystemEventHandler = object
|
||||
|
||||
SUSPICIOUS_EXTENSIONS = {
|
||||
".exe": 30, ".bat": 35, ".cmd": 35, ".vbs": 40, ".vbe": 40,
|
||||
".js": 40, ".jse": 40, ".wsf": 40, ".wsh": 35, ".ps1": 45,
|
||||
".pif": 45, ".scr": 40, ".hta": 45, ".lnk": 15, ".url": 20,
|
||||
}
|
||||
|
||||
LEGITIMATE_ENTRIES = [
|
||||
"desktop.ini", "Send to OneNote.lnk", "OneNote 2016.lnk",
|
||||
"Microsoft Teams.lnk", "Outlook.lnk", "OneDrive.lnk",
|
||||
"Cisco AnyConnect Secure Mobility Client.lnk",
|
||||
"Skype for Business.lnk", "Zoom.lnk",
|
||||
]
|
||||
|
||||
|
||||
def get_startup_paths():
|
||||
"""Return user-specific and all-users startup folder paths."""
|
||||
paths = []
|
||||
user_startup = os.path.join(
|
||||
os.environ.get("APPDATA", ""),
|
||||
"Microsoft", "Windows", "Start Menu", "Programs", "Startup"
|
||||
)
|
||||
if os.path.isdir(user_startup):
|
||||
paths.append({"path": user_startup, "scope": "current_user"})
|
||||
all_users_startup = os.path.join(
|
||||
os.environ.get("PROGRAMDATA", r"C:\ProgramData"),
|
||||
"Microsoft", "Windows", "Start Menu", "Programs", "Startup"
|
||||
)
|
||||
if os.path.isdir(all_users_startup):
|
||||
paths.append({"path": all_users_startup, "scope": "all_users"})
|
||||
return paths
|
||||
|
||||
|
||||
def compute_file_hash(filepath):
|
||||
"""Compute SHA-256 hash of a file."""
|
||||
sha256 = hashlib.sha256()
|
||||
try:
|
||||
with open(filepath, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(8192), b""):
|
||||
sha256.update(chunk)
|
||||
return sha256.hexdigest()
|
||||
except (PermissionError, OSError):
|
||||
return "access_denied"
|
||||
|
||||
|
||||
def analyze_file(filepath, scope="unknown"):
|
||||
"""Analyze a single file in a startup directory."""
|
||||
path = Path(filepath)
|
||||
name = path.name
|
||||
ext = path.suffix.lower()
|
||||
try:
|
||||
stat = path.stat()
|
||||
created = datetime.fromtimestamp(stat.st_ctime)
|
||||
modified = datetime.fromtimestamp(stat.st_mtime)
|
||||
size = stat.st_size
|
||||
except (PermissionError, OSError):
|
||||
return {"file": str(filepath), "error": "access_denied"}
|
||||
|
||||
is_legitimate = name in LEGITIMATE_ENTRIES
|
||||
age_days = (datetime.now() - created).days
|
||||
|
||||
risk = 0
|
||||
indicators = []
|
||||
risk += SUSPICIOUS_EXTENSIONS.get(ext, 0)
|
||||
if ext in SUSPICIOUS_EXTENSIONS and ext != ".lnk":
|
||||
indicators.append(f"suspicious_extension_{ext}")
|
||||
if age_days < 7:
|
||||
risk += 25
|
||||
indicators.append("recently_created")
|
||||
if age_days < 1:
|
||||
risk += 15
|
||||
indicators.append("created_within_24h")
|
||||
if size == 0:
|
||||
risk += 10
|
||||
indicators.append("zero_byte_file")
|
||||
if size > 10 * 1024 * 1024:
|
||||
risk += 10
|
||||
indicators.append("large_file_over_10mb")
|
||||
if not is_legitimate:
|
||||
risk += 10
|
||||
indicators.append("not_in_baseline")
|
||||
if scope == "all_users" and ext in SUSPICIOUS_EXTENSIONS:
|
||||
risk += 10
|
||||
indicators.append("all_users_startup")
|
||||
|
||||
risk = min(risk, 100)
|
||||
|
||||
return {
|
||||
"file": str(filepath),
|
||||
"filename": name,
|
||||
"extension": ext,
|
||||
"scope": scope,
|
||||
"size_bytes": size,
|
||||
"created": created.isoformat(),
|
||||
"modified": modified.isoformat(),
|
||||
"age_days": age_days,
|
||||
"sha256": compute_file_hash(filepath),
|
||||
"is_legitimate_baseline": is_legitimate,
|
||||
"suspicious_indicators": indicators,
|
||||
"risk_score": risk,
|
||||
"risk_level": "CRITICAL" if risk >= 70 else "HIGH" if risk >= 50 else "MEDIUM" if risk >= 25 else "LOW",
|
||||
}
|
||||
|
||||
|
||||
def scan_startup_folders():
|
||||
"""Scan all startup directories and analyze contents."""
|
||||
startup_paths = get_startup_paths()
|
||||
results = []
|
||||
for sp in startup_paths:
|
||||
folder = sp["path"]
|
||||
scope = sp["scope"]
|
||||
try:
|
||||
for entry in os.listdir(folder):
|
||||
full_path = os.path.join(folder, entry)
|
||||
if os.path.isfile(full_path):
|
||||
analysis = analyze_file(full_path, scope)
|
||||
results.append(analysis)
|
||||
except PermissionError:
|
||||
results.append({"path": folder, "error": "access_denied"})
|
||||
results.sort(key=lambda x: x.get("risk_score", 0), reverse=True)
|
||||
return results
|
||||
|
||||
|
||||
def check_registry_run_keys():
|
||||
"""Check Registry Run keys for autostart entries."""
|
||||
import subprocess
|
||||
run_keys = [
|
||||
r"HKCU\Software\Microsoft\Windows\CurrentVersion\Run",
|
||||
r"HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnce",
|
||||
r"HKLM\Software\Microsoft\Windows\CurrentVersion\Run",
|
||||
r"HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce",
|
||||
]
|
||||
entries = []
|
||||
for key in run_keys:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["reg", "query", key],
|
||||
capture_output=True, text=True, timeout=10
|
||||
)
|
||||
if result.returncode == 0:
|
||||
for line in result.stdout.strip().split("\n"):
|
||||
line = line.strip()
|
||||
if line and not line.startswith("HK") and "REG_" in line:
|
||||
parts = line.split("REG_SZ", 1) if "REG_SZ" in line else line.split("REG_EXPAND_SZ", 1)
|
||||
name = parts[0].strip() if parts else line
|
||||
value = parts[1].strip() if len(parts) > 1 else ""
|
||||
entries.append({
|
||||
"registry_key": key,
|
||||
"name": name,
|
||||
"value": value,
|
||||
"suspicious": any(p in value.lower() for p in
|
||||
["powershell", "cmd.exe", "\\temp\\", "\\appdata\\",
|
||||
"mshta", "-enc", "downloadstring"]),
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
return entries
|
||||
|
||||
|
||||
class StartupMonitorHandler(FileSystemEventHandler):
|
||||
"""Watchdog handler for monitoring startup folder changes."""
|
||||
|
||||
def __init__(self):
|
||||
self.events = []
|
||||
|
||||
def on_created(self, event):
|
||||
if not event.is_directory:
|
||||
analysis = analyze_file(event.src_path)
|
||||
alert = {
|
||||
"event": "FILE_CREATED",
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"file": event.src_path,
|
||||
"risk_score": analysis.get("risk_score", 0),
|
||||
"risk_level": analysis.get("risk_level", "UNKNOWN"),
|
||||
"indicators": analysis.get("suspicious_indicators", []),
|
||||
}
|
||||
self.events.append(alert)
|
||||
print(json.dumps(alert, indent=2))
|
||||
|
||||
def on_modified(self, event):
|
||||
if not event.is_directory:
|
||||
alert = {
|
||||
"event": "FILE_MODIFIED",
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"file": event.src_path,
|
||||
}
|
||||
self.events.append(alert)
|
||||
print(json.dumps(alert, indent=2))
|
||||
|
||||
def on_deleted(self, event):
|
||||
if not event.is_directory:
|
||||
alert = {
|
||||
"event": "FILE_DELETED",
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"file": event.src_path,
|
||||
}
|
||||
self.events.append(alert)
|
||||
print(json.dumps(alert, indent=2))
|
||||
|
||||
|
||||
def monitor_startup(duration_seconds=60):
|
||||
"""Monitor startup folders in real-time using watchdog."""
|
||||
if not Observer:
|
||||
return {"error": "watchdog not installed: pip install watchdog"}
|
||||
handler = StartupMonitorHandler()
|
||||
observer = Observer()
|
||||
startup_paths = get_startup_paths()
|
||||
for sp in startup_paths:
|
||||
observer.schedule(handler, sp["path"], recursive=False)
|
||||
observer.start()
|
||||
try:
|
||||
time.sleep(duration_seconds)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
observer.stop()
|
||||
observer.join()
|
||||
return {"monitored_seconds": duration_seconds, "events_detected": handler.events}
|
||||
|
||||
|
||||
def full_hunt():
|
||||
"""Run comprehensive startup persistence threat hunt."""
|
||||
scan_results = scan_startup_folders()
|
||||
registry_entries = check_registry_run_keys()
|
||||
suspicious_files = [r for r in scan_results if r.get("risk_score", 0) >= 25]
|
||||
suspicious_reg = [e for e in registry_entries if e.get("suspicious")]
|
||||
return {
|
||||
"hunt_type": "Startup Folder Persistence (T1547.001)",
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"startup_paths": get_startup_paths(),
|
||||
"statistics": {
|
||||
"total_startup_files": len(scan_results),
|
||||
"suspicious_files": len(suspicious_files),
|
||||
"registry_run_entries": len(registry_entries),
|
||||
"suspicious_registry_entries": len(suspicious_reg),
|
||||
},
|
||||
"file_analysis": scan_results[:30],
|
||||
"registry_analysis": registry_entries[:20],
|
||||
"mitre_technique": {
|
||||
"id": "T1547.001",
|
||||
"name": "Boot or Logon Autostart Execution: Registry Run Keys / Startup Folder",
|
||||
"tactic": "Persistence, Privilege Escalation",
|
||||
},
|
||||
"recommendation": "Investigate CRITICAL and HIGH files. Verify hashes against known-good baselines."
|
||||
if suspicious_files else "No suspicious startup entries detected.",
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Startup Folder Persistence Hunting Agent (T1547.001)")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("scan", help="Scan startup folders for suspicious files")
|
||||
sub.add_parser("registry", help="Check Registry Run keys")
|
||||
p_mon = sub.add_parser("monitor", help="Monitor startup folders in real-time")
|
||||
p_mon.add_argument("--duration", type=int, default=60, help="Monitor duration in seconds")
|
||||
sub.add_parser("full", help="Full persistence threat hunt")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "scan":
|
||||
result = scan_startup_folders()
|
||||
elif args.command == "registry":
|
||||
result = check_registry_run_keys()
|
||||
elif args.command == "monitor":
|
||||
result = monitor_startup(args.duration)
|
||||
elif args.command == "full" or args.command is None:
|
||||
result = full_hunt()
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: hunting-for-unusual-service-installations
|
||||
description: Detect suspicious Windows service installations (MITRE ATT&CK T1543.003) by parsing System event logs for Event ID 7045, analyzing service binary paths, and identifying indicators of persistence mechanisms.
|
||||
domain: cybersecurity
|
||||
subdomain: threat-hunting
|
||||
tags: [threat-hunting, T1543.003, service-installation, persistence, Event-7045, Sysmon, Windows-services]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Hunting for Unusual Service Installations
|
||||
|
||||
## Overview
|
||||
|
||||
Attackers frequently install malicious Windows services for persistence and privilege escalation (MITRE ATT&CK T1543.003 — Create or Modify System Process: Windows Service). Event ID 7045 in the System event log records every new service installation. This skill parses .evtx log files to extract service installation events, flags suspicious binary paths (temp directories, PowerShell, cmd.exe, encoded commands), and correlates with known attack patterns.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+ with `python-evtx`, `lxml`
|
||||
- Windows System event log (.evtx) files
|
||||
- Access to live System event log (optional, for real-time monitoring)
|
||||
- Sysmon logs for enhanced process tracking (optional)
|
||||
|
||||
## Steps
|
||||
|
||||
1. Parse System.evtx for Event ID 7045 (new service installed)
|
||||
2. Extract service name, binary path, service type, and account
|
||||
3. Flag services with suspicious binary paths (temp dirs, encoded commands)
|
||||
4. Detect PowerShell-based service creation patterns
|
||||
5. Identify services running as LocalSystem with unusual paths
|
||||
6. Cross-reference with known legitimate service baselines
|
||||
7. Generate threat hunting report with MITRE ATT&CK T1543.003 mapping
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing all new service installations with risk scores, suspicious indicators, and remediation recommendations
|
||||
- Timeline of service installation events with binary path analysis
|
||||
@@ -0,0 +1,73 @@
|
||||
# API Reference — Hunting for Unusual Service Installations
|
||||
|
||||
## Libraries Used
|
||||
- **python-evtx**: Parse Windows .evtx event log files
|
||||
- **lxml**: XML parsing of EVTX record XML
|
||||
- **re**: Regex matching for suspicious binary path patterns
|
||||
- **collections.Counter**: Statistics aggregation
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py System.evtx parse
|
||||
python agent.py System.evtx hunt
|
||||
python agent.py System.evtx stats
|
||||
python agent.py System.evtx full
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `parse_evtx_events(evtx_path)` — Extract Event ID 7045 records
|
||||
Opens .evtx file with `evtx.Evtx()`, iterates records, parses XML with lxml.
|
||||
Extracts: ServiceName, ImagePath, ServiceType, StartType, AccountName, timestamp.
|
||||
Namespace: `http://schemas.microsoft.com/win/2004/08/events/event`
|
||||
|
||||
### `analyze_service_path(image_path)` — Binary path risk analysis
|
||||
Matches against 17 suspicious patterns (temp dirs, PowerShell, encoded commands,
|
||||
LOLBins, download patterns). Checks against 5 legitimate path prefixes.
|
||||
Scoring: +20 for non-standard path, +15 per suspicious indicator. Max 100.
|
||||
|
||||
### `hunt_suspicious_services(evtx_path)` — Main hunting engine
|
||||
Combines parsing and analysis. Extra +20 risk for LocalSystem account with
|
||||
non-standard binary path. Results sorted by risk score descending.
|
||||
|
||||
### `generate_statistics(results)` — Summary statistics
|
||||
Counts risk distribution, top indicators, service account usage.
|
||||
|
||||
### `full_hunt(evtx_path)` — Comprehensive threat hunt report
|
||||
|
||||
## Suspicious Path Patterns
|
||||
| Pattern | Indicator |
|
||||
|---------|-----------|
|
||||
| `\temp\`, `\tmp\` | temp_directory |
|
||||
| `\appdata\` | appdata_directory |
|
||||
| `\users\public\` | public_user_directory |
|
||||
| `powershell.exe` | powershell_execution |
|
||||
| `cmd.exe /c` | cmd_execution |
|
||||
| `-enc`, `-encodedcommand` | encoded_command |
|
||||
| `downloadstring`, `webclient` | download_pattern |
|
||||
| `invoke-expression`, `iex` | invoke_expression |
|
||||
| `mshta`, `regsvr32`, `rundll32` | lolbin_execution |
|
||||
|
||||
## Legitimate Service Path Prefixes
|
||||
- `C:\Windows\System32\`
|
||||
- `C:\Windows\SysWOW64\`
|
||||
- `C:\Program Files\`
|
||||
- `C:\Program Files (x86)\`
|
||||
- `C:\Windows\Microsoft.NET\`
|
||||
|
||||
## Event ID 7045 Fields
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| ServiceName | Display name of installed service |
|
||||
| ImagePath | Binary path and arguments |
|
||||
| ServiceType | user mode service, kernel driver, etc. |
|
||||
| StartType | auto start, demand start, boot start |
|
||||
| AccountName | Service account (LocalSystem, etc.) |
|
||||
|
||||
## MITRE ATT&CK Mapping
|
||||
- **T1543.003** — Create or Modify System Process: Windows Service
|
||||
- Tactics: Persistence, Privilege Escalation
|
||||
|
||||
## Dependencies
|
||||
- `python-evtx` >= 0.7.4
|
||||
- `lxml` >= 4.9.0
|
||||
@@ -0,0 +1,194 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for hunting suspicious Windows service installations (T1543.003) via Event ID 7045."""
|
||||
|
||||
import json
|
||||
import re
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
from collections import Counter
|
||||
|
||||
try:
|
||||
from lxml import etree
|
||||
except ImportError:
|
||||
etree = None
|
||||
|
||||
try:
|
||||
import Evtx.Evtx as evtx
|
||||
import Evtx.Views as evtx_views
|
||||
except ImportError:
|
||||
evtx = None
|
||||
|
||||
SUSPICIOUS_PATH_PATTERNS = [
|
||||
(r"\\temp\\", "temp_directory"),
|
||||
(r"\\tmp\\", "temp_directory"),
|
||||
(r"\\appdata\\", "appdata_directory"),
|
||||
(r"\\users\\public\\", "public_user_directory"),
|
||||
(r"\\programdata\\[^\\]+\.[^\\]+$", "loose_file_in_programdata"),
|
||||
(r"powershell\.exe", "powershell_execution"),
|
||||
(r"cmd\.exe\s*/c", "cmd_execution"),
|
||||
(r"-enc\s+", "encoded_command"),
|
||||
(r"-encodedcommand", "encoded_command"),
|
||||
(r"frombase64string", "base64_decode"),
|
||||
(r"downloadstring|downloadfile|webclient", "download_pattern"),
|
||||
(r"invoke-expression|iex\s", "invoke_expression"),
|
||||
(r"\\windows\\temp\\", "windows_temp"),
|
||||
(r"mshta\.exe|regsvr32\.exe|rundll32\.exe|msiexec\.exe", "lolbin_execution"),
|
||||
(r"sc\.exe\s+create", "sc_create_chain"),
|
||||
(r"\\\$recycle\.bin\\", "recycle_bin"),
|
||||
(r"\\perflog", "perflog_abuse"),
|
||||
]
|
||||
|
||||
LEGITIMATE_SERVICE_PATHS = [
|
||||
r"^\"?C:\\Windows\\System32\\",
|
||||
r"^\"?C:\\Windows\\SysWOW64\\",
|
||||
r"^\"?C:\\Program Files\\",
|
||||
r"^\"?C:\\Program Files \(x86\)\\",
|
||||
r"^\"?C:\\Windows\\Microsoft\.NET\\",
|
||||
]
|
||||
|
||||
|
||||
def parse_evtx_events(evtx_path):
|
||||
"""Parse System.evtx and extract Event ID 7045 records."""
|
||||
if not evtx:
|
||||
raise RuntimeError("python-evtx not installed: pip install python-evtx")
|
||||
events = []
|
||||
ns = {"e": "http://schemas.microsoft.com/win/2004/08/events/event"}
|
||||
with evtx.Evtx(evtx_path) as log:
|
||||
for record in log.records():
|
||||
try:
|
||||
xml = record.xml()
|
||||
root = etree.fromstring(xml.encode("utf-8") if isinstance(xml, str) else xml)
|
||||
event_id_el = root.find(".//e:System/e:EventID", ns)
|
||||
if event_id_el is None or event_id_el.text != "7045":
|
||||
continue
|
||||
system = root.find(".//e:System", ns)
|
||||
event_data = root.find(".//e:EventData", ns)
|
||||
data_fields = {}
|
||||
if event_data is not None:
|
||||
for data in event_data.findall("e:Data", ns):
|
||||
name = data.get("Name", "")
|
||||
data_fields[name] = data.text or ""
|
||||
time_el = system.find("e:TimeCreated", ns)
|
||||
timestamp = time_el.get("SystemTime", "") if time_el is not None else ""
|
||||
computer_el = system.find("e:Computer", ns)
|
||||
computer = computer_el.text if computer_el is not None else ""
|
||||
events.append({
|
||||
"timestamp": timestamp,
|
||||
"event_id": 7045,
|
||||
"computer": computer,
|
||||
"service_name": data_fields.get("ServiceName", ""),
|
||||
"image_path": data_fields.get("ImagePath", ""),
|
||||
"service_type": data_fields.get("ServiceType", ""),
|
||||
"start_type": data_fields.get("StartType", ""),
|
||||
"account_name": data_fields.get("AccountName", ""),
|
||||
})
|
||||
except Exception:
|
||||
continue
|
||||
return events
|
||||
|
||||
|
||||
def analyze_service_path(image_path):
|
||||
"""Analyze a service binary path for suspicious indicators."""
|
||||
findings = []
|
||||
path_lower = image_path.lower()
|
||||
for pattern, indicator in SUSPICIOUS_PATH_PATTERNS:
|
||||
if re.search(pattern, path_lower, re.IGNORECASE):
|
||||
findings.append(indicator)
|
||||
is_legitimate = any(re.match(p, image_path, re.IGNORECASE) for p in LEGITIMATE_SERVICE_PATHS)
|
||||
risk_score = 0
|
||||
if not is_legitimate:
|
||||
risk_score += 20
|
||||
risk_score += len(findings) * 15
|
||||
risk_score = min(risk_score, 100)
|
||||
return {
|
||||
"suspicious_indicators": findings,
|
||||
"legitimate_path": is_legitimate,
|
||||
"risk_score": risk_score,
|
||||
"risk_level": "CRITICAL" if risk_score >= 70 else "HIGH" if risk_score >= 50 else "MEDIUM" if risk_score >= 20 else "LOW",
|
||||
}
|
||||
|
||||
|
||||
def hunt_suspicious_services(evtx_path):
|
||||
"""Main hunting function: parse events and analyze each service."""
|
||||
events = parse_evtx_events(evtx_path)
|
||||
results = []
|
||||
for event in events:
|
||||
analysis = analyze_service_path(event.get("image_path", ""))
|
||||
entry = {**event, **analysis}
|
||||
if entry.get("account_name", "").lower() == "localsystem" and not analysis["legitimate_path"]:
|
||||
entry["risk_score"] = min(entry["risk_score"] + 20, 100)
|
||||
entry["risk_level"] = "CRITICAL" if entry["risk_score"] >= 70 else entry["risk_level"]
|
||||
entry["suspicious_indicators"].append("localsystem_nonstandard_path")
|
||||
results.append(entry)
|
||||
results.sort(key=lambda x: x.get("risk_score", 0), reverse=True)
|
||||
return results
|
||||
|
||||
|
||||
def generate_statistics(results):
|
||||
"""Generate summary statistics from hunting results."""
|
||||
total = len(results)
|
||||
risk_counts = Counter(r.get("risk_level", "LOW") for r in results)
|
||||
indicator_counts = Counter()
|
||||
account_counts = Counter()
|
||||
for r in results:
|
||||
for ind in r.get("suspicious_indicators", []):
|
||||
indicator_counts[ind] += 1
|
||||
account_counts[r.get("account_name", "Unknown")] += 1
|
||||
return {
|
||||
"total_service_installations": total,
|
||||
"risk_distribution": dict(risk_counts),
|
||||
"top_indicators": dict(indicator_counts.most_common(10)),
|
||||
"service_accounts": dict(account_counts.most_common(10)),
|
||||
"critical_count": risk_counts.get("CRITICAL", 0),
|
||||
"high_count": risk_counts.get("HIGH", 0),
|
||||
}
|
||||
|
||||
|
||||
def full_hunt(evtx_path):
|
||||
"""Run comprehensive service installation threat hunt."""
|
||||
results = hunt_suspicious_services(evtx_path)
|
||||
stats = generate_statistics(results)
|
||||
suspicious = [r for r in results if r.get("risk_score", 0) >= 20]
|
||||
return {
|
||||
"hunt_type": "Unusual Service Installation (T1543.003)",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"evtx_file": evtx_path,
|
||||
"statistics": stats,
|
||||
"suspicious_services": suspicious[:30],
|
||||
"mitre_technique": {
|
||||
"id": "T1543.003",
|
||||
"name": "Create or Modify System Process: Windows Service",
|
||||
"tactic": "Persistence, Privilege Escalation",
|
||||
},
|
||||
"recommendation": "Investigate CRITICAL and HIGH services. Verify binary hashes against known-good baselines." if stats["critical_count"] + stats["high_count"] > 0
|
||||
else "No high-risk service installations detected.",
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Service Installation Threat Hunting Agent (T1543.003)")
|
||||
parser.add_argument("evtx", help="Path to System.evtx file")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("parse", help="Parse and list all Event ID 7045 records")
|
||||
sub.add_parser("hunt", help="Hunt for suspicious service installations")
|
||||
sub.add_parser("stats", help="Generate hunting statistics")
|
||||
sub.add_parser("full", help="Full threat hunt report")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "parse":
|
||||
result = parse_evtx_events(args.evtx)
|
||||
elif args.command == "hunt":
|
||||
result = hunt_suspicious_services(args.evtx)
|
||||
elif args.command == "stats":
|
||||
results = hunt_suspicious_services(args.evtx)
|
||||
result = generate_statistics(results)
|
||||
elif args.command == "full" or args.command is None:
|
||||
result = full_hunt(args.evtx)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: implementing-deception-based-detection-with-canarytoken
|
||||
description: Deploy and monitor Canary Tokens via the Thinkst Canary API for deception-based breach detection using web bug tokens, DNS tokens, document tokens, and AWS key tokens.
|
||||
domain: cybersecurity
|
||||
subdomain: deception-technology
|
||||
tags: [canarytoken, deception, honeytokens, breach-detection, Thinkst-Canary, tripwire, early-warning]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Implementing Deception-Based Detection with Canarytoken
|
||||
|
||||
## Overview
|
||||
|
||||
Canary Tokens are lightweight tripwire mechanisms that alert when an attacker accesses a resource. This skill uses the Thinkst Canary REST API to programmatically create tokens (web bugs, DNS tokens, MS Word documents, AWS API keys), deploy them to strategic locations, monitor for triggered alerts, and generate deception coverage reports.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Thinkst Canary Console or canarytokens.org account
|
||||
- API auth token from Canary Console
|
||||
- Python 3.9+ with `requests`
|
||||
- File system access for deploying document and file tokens
|
||||
|
||||
## Steps
|
||||
|
||||
1. Authenticate to the Canary Console API using auth_token
|
||||
2. Create web bug (HTTP) tokens for embedding in documents and web pages
|
||||
3. Create DNS tokens for monitoring DNS resolution attempts
|
||||
4. Create MS Word document tokens for file share deployment
|
||||
5. List all active tokens and their trigger history
|
||||
6. Query recent alerts for triggered token events
|
||||
7. Generate deception coverage report with deployment recommendations
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing all deployed Canary Tokens, trigger history, alert details, and coverage analysis
|
||||
- Deployment map showing token types across network segments
|
||||
+73
@@ -0,0 +1,73 @@
|
||||
# API Reference — Implementing Deception-Based Detection with Canarytoken
|
||||
|
||||
## Libraries Used
|
||||
- **requests**: HTTP client for Thinkst Canary Console REST API
|
||||
- **json**: JSON serialization for audit reports
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py --console abc123 --auth-token TOKEN ping
|
||||
python agent.py --console abc123 --auth-token TOKEN list
|
||||
python agent.py --console abc123 --auth-token TOKEN alerts
|
||||
python agent.py --console abc123 --auth-token TOKEN create --kind http --memo "Web server token"
|
||||
python agent.py --console abc123 --auth-token TOKEN create --kind dns --memo "DNS honeypot"
|
||||
python agent.py --console abc123 --auth-token TOKEN coverage
|
||||
python agent.py --console abc123 --auth-token TOKEN full
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `CanaryClient(console_domain, auth_token)` — API client
|
||||
Base URL: `https://{console_domain}.canary.tools/api/v1`
|
||||
Auth: `auth_token` parameter on every request.
|
||||
|
||||
### `create_token(kind, memo, **kwargs)` — Create Canarytoken
|
||||
POST `/canarytoken/create` with `kind`, `memo`, `auth_token`.
|
||||
For doc-msword: uploads file via multipart form with MIME type
|
||||
`application/vnd.openxmlformats-officedocument.wordprocessingml.document`.
|
||||
|
||||
### `list_tokens()` — List all deployed tokens
|
||||
GET `/canarytokens/fetch`. Returns array of token objects with kind, memo, url, enabled.
|
||||
|
||||
### `get_alerts(newer_than)` — Fetch triggered token alerts
|
||||
GET `/incidents/all`. Optional `newer_than` timestamp filter.
|
||||
Returns src_host (source IP), description, timestamp, acknowledged status.
|
||||
|
||||
### `ack_alert(incident_id)` — Acknowledge an alert
|
||||
POST `/incident/acknowledge` with incident ID.
|
||||
|
||||
### `audit_token_coverage(client)` — Coverage analysis
|
||||
Calculates: tokens by kind, triggered vs untriggered, missing token types,
|
||||
coverage score as percentage of TOKEN_KINDS deployed.
|
||||
|
||||
### `full_audit(client)` — Comprehensive deception audit
|
||||
|
||||
## Canary Console API Endpoints
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/ping` | GET | Test API connectivity |
|
||||
| `/canarytoken/create` | POST | Create new token |
|
||||
| `/canarytokens/fetch` | GET | List all tokens |
|
||||
| `/canarytoken/fetch` | GET | Get specific token |
|
||||
| `/canarytoken/delete` | POST | Delete a token |
|
||||
| `/incidents/all` | GET | Fetch all alerts |
|
||||
| `/canarytoken/incidents` | GET | Alerts for specific token |
|
||||
| `/incident/acknowledge` | POST | Acknowledge alert |
|
||||
|
||||
## Supported Token Types
|
||||
| Kind | Description |
|
||||
|------|-------------|
|
||||
| http | Web bug — triggers on HTTP request |
|
||||
| dns | DNS token — triggers on DNS resolution |
|
||||
| doc-msword | MS Word document with embedded beacon |
|
||||
| pdf-acrobat-reader | PDF with embedded beacon |
|
||||
| aws-id | Fake AWS API key pair |
|
||||
| web-image | Image with tracking pixel |
|
||||
| cloned-web | Cloned website detection |
|
||||
| qr-code | QR code with tracking URL |
|
||||
| sensitive-cmd | Triggers on command execution |
|
||||
| windows-dir | Windows folder open detection |
|
||||
|
||||
## Dependencies
|
||||
- `requests` >= 2.28.0
|
||||
- Thinkst Canary Console account with API auth token
|
||||
@@ -0,0 +1,232 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for deploying and monitoring Canary Tokens via the Thinkst Canary API."""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
requests = None
|
||||
|
||||
TOKEN_KINDS = {
|
||||
"http": "http",
|
||||
"dns": "dns",
|
||||
"doc-msword": "doc-msword",
|
||||
"pdf-acrobat-reader": "pdf-acrobat-reader",
|
||||
"web-image": "web-image",
|
||||
"cloned-web": "cloned-web",
|
||||
"aws-id": "aws-id",
|
||||
"qr-code": "qr-code",
|
||||
"sql": "sql",
|
||||
"svn": "svn",
|
||||
"smtp": "smtp",
|
||||
"windows-dir": "windows-dir",
|
||||
"sensitive-cmd": "sensitive-cmd",
|
||||
}
|
||||
|
||||
|
||||
class CanaryClient:
|
||||
"""Client for the Thinkst Canary Console REST API."""
|
||||
|
||||
def __init__(self, console_domain, auth_token):
|
||||
self.base_url = f"https://{console_domain}.canary.tools/api/v1"
|
||||
self.auth_token = auth_token
|
||||
|
||||
def _get(self, endpoint, params=None):
|
||||
params = params or {}
|
||||
params["auth_token"] = self.auth_token
|
||||
resp = requests.get(f"{self.base_url}{endpoint}", params=params, timeout=30)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
def _post(self, endpoint, data=None, files=None):
|
||||
data = data or {}
|
||||
data["auth_token"] = self.auth_token
|
||||
resp = requests.post(f"{self.base_url}{endpoint}", data=data, files=files, timeout=30)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
def ping(self):
|
||||
"""Test API connectivity."""
|
||||
return self._get("/ping")
|
||||
|
||||
def create_token(self, kind, memo, **kwargs):
|
||||
"""Create a new Canarytoken.
|
||||
|
||||
Args:
|
||||
kind: Token type (http, dns, doc-msword, aws-id, etc.)
|
||||
memo: Description/reminder for the token
|
||||
**kwargs: Additional parameters (e.g., cloned_web for cloned-web type)
|
||||
"""
|
||||
data = {"kind": kind, "memo": memo}
|
||||
data.update(kwargs)
|
||||
files = None
|
||||
if kind == "doc-msword" and "doc" in kwargs:
|
||||
doc_path = kwargs.pop("doc")
|
||||
data.pop("doc", None)
|
||||
files = {"doc": (doc_path, open(doc_path, "rb"),
|
||||
"application/vnd.openxmlformats-officedocument.wordprocessingml.document")}
|
||||
return self._post("/canarytoken/create", data=data, files=files)
|
||||
|
||||
def list_tokens(self):
|
||||
"""List all Canarytokens on the console."""
|
||||
return self._get("/canarytokens/fetch")
|
||||
|
||||
def get_token(self, canarytoken):
|
||||
"""Get details for a specific token."""
|
||||
return self._get("/canarytoken/fetch", params={"canarytoken": canarytoken})
|
||||
|
||||
def delete_token(self, canarytoken):
|
||||
"""Delete a Canarytoken."""
|
||||
return self._post("/canarytoken/delete", data={"canarytoken": canarytoken})
|
||||
|
||||
def get_alerts(self, newer_than=None):
|
||||
"""Fetch recent alerts from triggered tokens."""
|
||||
params = {}
|
||||
if newer_than:
|
||||
params["newer_than"] = newer_than
|
||||
return self._get("/incidents/all", params=params)
|
||||
|
||||
def get_token_alerts(self, canarytoken):
|
||||
"""Fetch alerts for a specific token."""
|
||||
return self._get("/canarytoken/incidents", params={"canarytoken": canarytoken})
|
||||
|
||||
def ack_alert(self, incident_id):
|
||||
"""Acknowledge an alert."""
|
||||
return self._post("/incident/acknowledge", data={"incident": incident_id})
|
||||
|
||||
|
||||
def create_deployment(client, deployment_plan):
|
||||
"""Create multiple tokens based on a deployment plan."""
|
||||
results = []
|
||||
for token_spec in deployment_plan:
|
||||
kind = token_spec.get("kind", "http")
|
||||
memo = token_spec.get("memo", f"Canarytoken - {kind}")
|
||||
extra = {k: v for k, v in token_spec.items() if k not in ("kind", "memo")}
|
||||
try:
|
||||
resp = client.create_token(kind, memo, **extra)
|
||||
results.append({
|
||||
"kind": kind,
|
||||
"memo": memo,
|
||||
"status": "CREATED",
|
||||
"canarytoken": resp.get("canarytoken", {}).get("canarytoken", ""),
|
||||
"url": resp.get("canarytoken", {}).get("url", ""),
|
||||
})
|
||||
except Exception as e:
|
||||
results.append({"kind": kind, "memo": memo, "status": "FAILED", "error": str(e)})
|
||||
return results
|
||||
|
||||
|
||||
def audit_token_coverage(client):
|
||||
"""Audit deployed token coverage and generate report."""
|
||||
tokens_resp = client.list_tokens()
|
||||
tokens = tokens_resp.get("tokens", [])
|
||||
alerts_resp = client.get_alerts()
|
||||
alerts = alerts_resp.get("incidents", [])
|
||||
|
||||
kind_counts = {}
|
||||
triggered_tokens = set()
|
||||
for token in tokens:
|
||||
kind = token.get("kind", "unknown")
|
||||
kind_counts[kind] = kind_counts.get(kind, 0) + 1
|
||||
|
||||
for alert in alerts:
|
||||
triggered_tokens.add(alert.get("canarytoken", ""))
|
||||
|
||||
untriggered = [t for t in tokens if t.get("canarytoken", "") not in triggered_tokens]
|
||||
recommended_types = []
|
||||
for kind_name in TOKEN_KINDS:
|
||||
if kind_name not in kind_counts:
|
||||
recommended_types.append(kind_name)
|
||||
|
||||
return {
|
||||
"total_tokens": len(tokens),
|
||||
"total_alerts": len(alerts),
|
||||
"tokens_by_kind": kind_counts,
|
||||
"triggered_token_count": len(triggered_tokens),
|
||||
"untriggered_tokens": len(untriggered),
|
||||
"missing_token_types": recommended_types,
|
||||
"coverage_score": round(len(kind_counts) / len(TOKEN_KINDS) * 100, 1),
|
||||
}
|
||||
|
||||
|
||||
def full_audit(client):
|
||||
"""Run comprehensive Canarytoken deployment audit."""
|
||||
coverage = audit_token_coverage(client)
|
||||
tokens_resp = client.list_tokens()
|
||||
tokens = tokens_resp.get("tokens", [])
|
||||
alerts_resp = client.get_alerts()
|
||||
alerts = alerts_resp.get("incidents", [])
|
||||
|
||||
token_details = []
|
||||
for t in tokens[:30]:
|
||||
token_details.append({
|
||||
"canarytoken": t.get("canarytoken"),
|
||||
"kind": t.get("kind"),
|
||||
"memo": t.get("memo"),
|
||||
"created": t.get("created_printable"),
|
||||
"enabled": t.get("enabled"),
|
||||
"url": t.get("url", ""),
|
||||
})
|
||||
|
||||
alert_details = []
|
||||
for a in alerts[:20]:
|
||||
alert_details.append({
|
||||
"incident_id": a.get("id"),
|
||||
"description": a.get("description"),
|
||||
"source_ip": a.get("src_host"),
|
||||
"timestamp": a.get("created_printable"),
|
||||
"canarytoken": a.get("canarytoken"),
|
||||
"acknowledged": a.get("acknowledged"),
|
||||
})
|
||||
|
||||
return {
|
||||
"audit_type": "Canarytoken Deception Coverage Audit",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"coverage": coverage,
|
||||
"deployed_tokens": token_details,
|
||||
"recent_alerts": alert_details,
|
||||
"recommendation": "Deploy missing token types to improve coverage"
|
||||
if coverage["coverage_score"] < 50 else "Good coverage — review untriggered tokens",
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Canarytoken Deception Detection Agent")
|
||||
parser.add_argument("--console", required=True, help="Canary Console domain (e.g., abc123)")
|
||||
parser.add_argument("--auth-token", required=True, help="API auth token")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("ping", help="Test API connectivity")
|
||||
sub.add_parser("list", help="List all deployed tokens")
|
||||
sub.add_parser("alerts", help="Fetch recent alerts")
|
||||
p_create = sub.add_parser("create", help="Create a new token")
|
||||
p_create.add_argument("--kind", required=True, choices=list(TOKEN_KINDS.keys()))
|
||||
p_create.add_argument("--memo", required=True)
|
||||
sub.add_parser("coverage", help="Audit token coverage")
|
||||
sub.add_parser("full", help="Full deception audit")
|
||||
args = parser.parse_args()
|
||||
|
||||
client = CanaryClient(args.console, args.auth_token)
|
||||
|
||||
if args.command == "ping":
|
||||
result = client.ping()
|
||||
elif args.command == "list":
|
||||
result = client.list_tokens()
|
||||
elif args.command == "alerts":
|
||||
result = client.get_alerts()
|
||||
elif args.command == "create":
|
||||
result = client.create_token(args.kind, args.memo)
|
||||
elif args.command == "coverage":
|
||||
result = audit_token_coverage(client)
|
||||
elif args.command == "full" or args.command is None:
|
||||
result = full_audit(client)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: implementing-privileged-access-workstation
|
||||
description: Design and implement Privileged Access Workstations (PAWs) with device hardening, just-in-time access, and integration with CyberArk or BeyondTrust for secure administrative operations.
|
||||
domain: cybersecurity
|
||||
subdomain: identity-and-access-management
|
||||
tags: [privileged-access, PAW, zero-trust, device-hardening, CyberArk, BeyondTrust, just-in-time-access]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Implementing Privileged Access Workstation
|
||||
|
||||
## Overview
|
||||
|
||||
A Privileged Access Workstation (PAW) is a hardened device dedicated to performing sensitive administrative tasks. This skill covers PAW design using the tiered administration model, device compliance enforcement via Microsoft Intune or Group Policy, just-in-time (JIT) access provisioning, and integration with privileged access management (PAM) platforms like CyberArk and BeyondTrust.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Windows 10/11 Enterprise with Virtualization Based Security (VBS)
|
||||
- Microsoft Intune or Active Directory Group Policy
|
||||
- CyberArk Privileged Access Security or BeyondTrust Password Safe (optional)
|
||||
- Python 3.9+ with `requests`, `subprocess`, `json`
|
||||
- Administrative access to target endpoints
|
||||
|
||||
## Steps
|
||||
|
||||
1. Audit current privileged access patterns and identify Tier 0/1/2 assets
|
||||
2. Configure device hardening baselines (AppLocker, Credential Guard, Device Guard)
|
||||
3. Enforce compliance policies via Intune or GPO
|
||||
4. Implement just-in-time access with time-limited admin group membership
|
||||
5. Integrate with CyberArk/BeyondTrust for credential vaulting
|
||||
6. Validate PAW configuration against CIS and Microsoft PAW guidance
|
||||
7. Monitor privileged sessions and generate compliance reports
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing device compliance status, hardening checks, JIT access windows, and PAM integration verification
|
||||
- Risk scoring per workstation with remediation recommendations
|
||||
@@ -0,0 +1,52 @@
|
||||
# API Reference — Implementing Privileged Access Workstation
|
||||
|
||||
## Libraries Used
|
||||
- **subprocess**: Execute PowerShell cmdlets for device hardening, group membership, software inventory
|
||||
- **json**: Parse PowerShell ConvertTo-Json output
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py harden
|
||||
python agent.py admins
|
||||
python agent.py software
|
||||
python agent.py network
|
||||
python agent.py full
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `check_device_hardening()` — Audit 7 PAW hardening controls
|
||||
Checks: Credential Guard, VBS status, Secure Boot, BitLocker, AppLocker,
|
||||
Windows Firewall profiles, UAC level via registry.
|
||||
|
||||
### `check_local_admin_group()` — JIT access audit
|
||||
Enumerates local Administrators group via `Get-LocalGroupMember`.
|
||||
Flags unexpected members not matching known admin accounts.
|
||||
|
||||
### `check_installed_software()` — Software allowlist enforcement
|
||||
Queries installed software from registry. Checks against blocked list:
|
||||
browsers (Chrome, Firefox), personal apps (Spotify, Steam, Slack, Zoom, Dropbox).
|
||||
|
||||
### `check_network_restrictions()` — Network isolation verification
|
||||
Counts outbound firewall block rules. Tests general internet connectivity.
|
||||
PAW Tier 0 should block internet — only management endpoints allowed.
|
||||
|
||||
### `full_paw_audit()` — Comprehensive compliance report
|
||||
|
||||
## PAW Hardening Checks
|
||||
| Check | PowerShell Source | Pass Criteria |
|
||||
|-------|------------------|---------------|
|
||||
| Credential Guard | Win32_DeviceGuard | SecurityServicesRunning > 0 |
|
||||
| VBS | Win32_DeviceGuard | VirtualizationBasedSecurityStatus = 2 |
|
||||
| Secure Boot | Confirm-SecureBootUEFI | Returns True |
|
||||
| BitLocker | Get-BitLockerVolume | ProtectionStatus = On |
|
||||
| AppLocker | Get-AppLockerPolicy | RuleCollection count > 0 |
|
||||
| Firewall | Get-NetFirewallProfile | All profiles enabled |
|
||||
| UAC | Registry query | ConsentPromptBehaviorAdmin >= 2 |
|
||||
|
||||
## Blocked Software Patterns
|
||||
chrome, firefox, spotify, steam, vlc, zoom, slack, dropbox, itunes, whatsapp, telegram
|
||||
|
||||
## Dependencies
|
||||
No external packages — Python standard library only.
|
||||
Requires: Windows 10/11 Enterprise with PowerShell 5.1+
|
||||
@@ -0,0 +1,170 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for implementing and auditing Privileged Access Workstation (PAW) configurations."""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
import subprocess
|
||||
import re
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def check_device_hardening():
|
||||
"""Audit local device hardening controls for PAW compliance."""
|
||||
checks = {}
|
||||
hardening_cmds = {
|
||||
"credential_guard": ["powershell", "-Command",
|
||||
"(Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\\Microsoft\\Windows\\DeviceGuard).SecurityServicesRunning"],
|
||||
"vbs_status": ["powershell", "-Command",
|
||||
"(Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\\Microsoft\\Windows\\DeviceGuard).VirtualizationBasedSecurityStatus"],
|
||||
"secure_boot": ["powershell", "-Command", "Confirm-SecureBootUEFI"],
|
||||
"bitlocker": ["powershell", "-Command",
|
||||
"(Get-BitLockerVolume -MountPoint C:).ProtectionStatus"],
|
||||
"applocker_status": ["powershell", "-Command",
|
||||
"(Get-AppLockerPolicy -Effective -Xml | Select-String 'RuleCollection').Count"],
|
||||
"firewall_enabled": ["powershell", "-Command",
|
||||
"(Get-NetFirewallProfile | Where-Object {$_.Enabled -eq $true}).Name -join ','"],
|
||||
"uac_level": ["reg", "query",
|
||||
"HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Policies\\System",
|
||||
"/v", "ConsentPromptBehaviorAdmin"],
|
||||
}
|
||||
for name, cmd in hardening_cmds.items():
|
||||
try:
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=15)
|
||||
output = result.stdout.strip()
|
||||
checks[name] = {"value": output, "status": "PASS" if output and output not in ("0", "False", "") else "FAIL"}
|
||||
except Exception as e:
|
||||
checks[name] = {"value": str(e), "status": "ERROR"}
|
||||
passed = sum(1 for c in checks.values() if c["status"] == "PASS")
|
||||
return {
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"checks": checks,
|
||||
"passed": passed,
|
||||
"total": len(checks),
|
||||
"compliance_pct": round(passed / len(checks) * 100, 1),
|
||||
"risk": "LOW" if passed >= 6 else "MEDIUM" if passed >= 4 else "HIGH",
|
||||
}
|
||||
|
||||
|
||||
def check_local_admin_group():
|
||||
"""Enumerate local Administrators group membership for JIT audit."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["powershell", "-Command",
|
||||
"Get-LocalGroupMember -Group Administrators | Select-Object Name,ObjectClass,PrincipalSource | ConvertTo-Json"],
|
||||
capture_output=True, text=True, timeout=15)
|
||||
members = json.loads(result.stdout) if result.stdout.strip() else []
|
||||
if isinstance(members, dict):
|
||||
members = [members]
|
||||
expected_admins = ["Administrator", "Domain Admins"]
|
||||
unexpected = [m for m in members if not any(e in m.get("Name", "") for e in expected_admins)]
|
||||
return {
|
||||
"total_members": len(members),
|
||||
"members": members,
|
||||
"unexpected_admins": unexpected,
|
||||
"jit_finding": "UNEXPECTED_ADMINS" if unexpected else "CLEAN",
|
||||
"severity": "HIGH" if unexpected else "INFO",
|
||||
}
|
||||
except Exception as e:
|
||||
return {"error": str(e)}
|
||||
|
||||
|
||||
def check_installed_software():
|
||||
"""Audit installed software against PAW allowlist."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["powershell", "-Command",
|
||||
"Get-ItemProperty HKLM:\\Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\* | "
|
||||
"Select-Object DisplayName,Publisher,InstallDate | Where-Object {$_.DisplayName -ne $null} | "
|
||||
"ConvertTo-Json -Depth 2"],
|
||||
capture_output=True, text=True, timeout=30)
|
||||
software = json.loads(result.stdout) if result.stdout.strip() else []
|
||||
if isinstance(software, dict):
|
||||
software = [software]
|
||||
blocked_patterns = [
|
||||
"chrome", "firefox", "spotify", "steam", "vlc", "zoom", "slack",
|
||||
"dropbox", "onedrive personal", "itunes", "whatsapp", "telegram",
|
||||
]
|
||||
violations = []
|
||||
for sw in software:
|
||||
name = (sw.get("DisplayName") or "").lower()
|
||||
if any(p in name for p in blocked_patterns):
|
||||
violations.append({"name": sw.get("DisplayName"), "publisher": sw.get("Publisher")})
|
||||
return {
|
||||
"total_installed": len(software),
|
||||
"paw_violations": len(violations),
|
||||
"violations": violations[:20],
|
||||
"finding": "BLOCKED_SOFTWARE_FOUND" if violations else "COMPLIANT",
|
||||
}
|
||||
except Exception as e:
|
||||
return {"error": str(e)}
|
||||
|
||||
|
||||
def check_network_restrictions():
|
||||
"""Verify PAW network isolation and restrictions."""
|
||||
checks = {}
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["powershell", "-Command",
|
||||
"Get-NetFirewallRule | Where-Object {$_.Enabled -eq 'True' -and $_.Direction -eq 'Outbound' -and $_.Action -eq 'Block'} | "
|
||||
"Measure-Object | Select-Object -ExpandProperty Count"],
|
||||
capture_output=True, text=True, timeout=15)
|
||||
checks["outbound_block_rules"] = int(result.stdout.strip() or 0)
|
||||
except Exception:
|
||||
checks["outbound_block_rules"] = 0
|
||||
try:
|
||||
result = subprocess.run(["powershell", "-Command",
|
||||
"Test-NetConnection -ComputerName google.com -Port 80 -WarningAction SilentlyContinue | Select-Object TcpTestSucceeded | ConvertTo-Json"],
|
||||
capture_output=True, text=True, timeout=15)
|
||||
data = json.loads(result.stdout) if result.stdout.strip() else {}
|
||||
checks["internet_access"] = data.get("TcpTestSucceeded", True)
|
||||
except Exception:
|
||||
checks["internet_access"] = "unknown"
|
||||
paw_should_block_internet = checks.get("internet_access") is False
|
||||
return {
|
||||
"network_checks": checks,
|
||||
"internet_blocked": paw_should_block_internet,
|
||||
"finding": "INTERNET_BLOCKED" if paw_should_block_internet else "INTERNET_ALLOWED",
|
||||
"severity": "INFO" if paw_should_block_internet else "MEDIUM",
|
||||
"recommendation": "PAW Tier 0 should block general internet — restrict to management endpoints only",
|
||||
}
|
||||
|
||||
|
||||
def full_paw_audit():
|
||||
"""Run comprehensive PAW compliance audit."""
|
||||
return {
|
||||
"audit_type": "Privileged Access Workstation",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"device_hardening": check_device_hardening(),
|
||||
"admin_group": check_local_admin_group(),
|
||||
"software_compliance": check_installed_software(),
|
||||
"network_isolation": check_network_restrictions(),
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Privileged Access Workstation Audit Agent")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("harden", help="Check device hardening controls")
|
||||
sub.add_parser("admins", help="Audit local admin group membership")
|
||||
sub.add_parser("software", help="Check installed software against allowlist")
|
||||
sub.add_parser("network", help="Verify network isolation")
|
||||
sub.add_parser("full", help="Full PAW compliance audit")
|
||||
args = parser.parse_args()
|
||||
if args.command == "harden":
|
||||
result = check_device_hardening()
|
||||
elif args.command == "admins":
|
||||
result = check_local_admin_group()
|
||||
elif args.command == "software":
|
||||
result = check_installed_software()
|
||||
elif args.command == "network":
|
||||
result = check_network_restrictions()
|
||||
elif args.command == "full":
|
||||
result = full_paw_audit()
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: performing-active-directory-forest-trust-attack
|
||||
description: Enumerate and audit Active Directory forest trust relationships using impacket for SID filtering analysis, trust key extraction, cross-forest SID history abuse detection, and inter-realm Kerberos ticket assessment.
|
||||
domain: cybersecurity
|
||||
subdomain: red-team
|
||||
tags: [active-directory, forest-trust, impacket, SID-filtering, kerberos, red-team, trust-enumeration]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Performing Active Directory Forest Trust Attack
|
||||
|
||||
## Overview
|
||||
|
||||
Active Directory forest trusts enable authentication across organizational boundaries but introduce attack surface if misconfigured. This skill uses impacket to enumerate trust relationships, analyze SID filtering configuration, detect SID history abuse vectors, perform cross-forest SID lookups via LSA/LSAT RPC calls, and assess inter-realm Kerberos ticket configurations for trust ticket forgery risks.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+ with `impacket`, `ldap3`
|
||||
- Domain credentials with read access to AD trust objects
|
||||
- Network access to Domain Controllers (ports 389, 445, 88)
|
||||
- Authorized penetration testing engagement or lab environment
|
||||
|
||||
## Steps
|
||||
|
||||
1. Enumerate forest trust relationships via LDAP trusted domain objects
|
||||
2. Query trust attributes and SID filtering status for each trust
|
||||
3. Perform SID lookups across trust boundaries using LsarLookupNames3
|
||||
4. Enumerate foreign security principals in trusted domains
|
||||
5. Check for SID history on cross-forest accounts
|
||||
6. Assess trust direction and transitivity for lateral movement paths
|
||||
7. Generate trust security audit report with risk findings
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing all trust relationships, SID filtering status, foreign principals, trust direction/transitivity, and risk assessment
|
||||
- Cross-forest attack path analysis with remediation recommendations
|
||||
@@ -0,0 +1,63 @@
|
||||
# API Reference — Performing Active Directory Forest Trust Attack
|
||||
|
||||
## Libraries Used
|
||||
- **impacket**: SMB/RPC transport for LSA SID lookups via `lsat.hLsarLookupSids2()`
|
||||
- **ldap3**: LDAP queries against `trustedDomain` objects and `foreignSecurityPrincipal` containers
|
||||
- **json**: JSON serialization for audit reports
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py --dc 10.0.0.1 --domain corp.local --username admin --password Pass123 trusts
|
||||
python agent.py --dc 10.0.0.1 --domain corp.local --username admin --password Pass123 foreign
|
||||
python agent.py --dc 10.0.0.1 --domain corp.local --username admin --password Pass123 lookup-sid --sid S-1-5-21-...
|
||||
python agent.py --dc 10.0.0.1 --domain corp.local --username admin --password Pass123 full
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `enumerate_trusts_ldap(dc_host, domain, username, password)` — Trust enumeration
|
||||
LDAP search: `(objectClass=trustedDomain)` under `CN=System,DC=...`.
|
||||
Attributes: trustPartner, trustDirection, trustType, trustAttributes, flatName.
|
||||
Decodes trust attribute bitmask for SID filtering, forest transitivity, RC4 encryption.
|
||||
|
||||
### `enumerate_foreign_principals(dc_host, domain, username, password)` — Cross-forest members
|
||||
LDAP search: `(objectClass=foreignSecurityPrincipal)` under `CN=ForeignSecurityPrincipals`.
|
||||
Filters well-known SIDs (S-1-5-x with 3 dashes). Returns group memberships.
|
||||
|
||||
### `lookup_sid_cross_forest(dc_host, domain, username, password, target_sid)` — LSA SID resolution
|
||||
Opens SMB transport to `\lsarpc`, binds MSRPC_UUID_LSAT, calls `hLsarLookupSids2()`.
|
||||
Resolves SIDs across trust boundaries.
|
||||
|
||||
### `assess_trust_risk(trusts, foreign_principals)` — Risk scoring
|
||||
Scoring: +40 SID filtering disabled, +20 RC4 encryption, +15 bidirectional trust,
|
||||
+10 forest transitive.
|
||||
|
||||
### `full_audit(dc_host, domain, username, password)` — Comprehensive audit
|
||||
|
||||
## Trust Direction Values
|
||||
| Value | Direction |
|
||||
|-------|-----------|
|
||||
| 0 | Disabled |
|
||||
| 1 | Inbound |
|
||||
| 2 | Outbound |
|
||||
| 3 | Bidirectional |
|
||||
|
||||
## Trust Attribute Flags
|
||||
| Flag | Hex | Description |
|
||||
|------|-----|-------------|
|
||||
| NON_TRANSITIVE | 0x01 | Trust does not extend transitively |
|
||||
| QUARANTINED_DOMAIN | 0x04 | SID filtering enabled |
|
||||
| FOREST_TRANSITIVE | 0x08 | Forest-wide transitive trust |
|
||||
| USES_RC4_ENCRYPTION | 0x80 | RC4 trust key (weaker than AES) |
|
||||
|
||||
## Impacket RPC Calls
|
||||
| Call | Module | Purpose |
|
||||
|------|--------|---------|
|
||||
| `hLsarOpenPolicy2` | lsad | Open LSA policy handle |
|
||||
| `hLsarLookupSids2` | lsat | Resolve SIDs to names across trust |
|
||||
| SMBTransport(`\lsarpc`) | transport | RPC transport over SMB |
|
||||
|
||||
## Dependencies
|
||||
- `impacket` >= 0.11.0
|
||||
- `ldap3` >= 2.9.0
|
||||
- Network access to DC ports 389 (LDAP), 445 (SMB), 88 (Kerberos)
|
||||
@@ -0,0 +1,229 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for AD forest trust enumeration and security assessment using impacket."""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
|
||||
try:
|
||||
from impacket.dcerpc.v5 import transport, lsat, lsad
|
||||
from impacket.dcerpc.v5.dtypes import MAXIMUM_ALLOWED
|
||||
from impacket.dcerpc.v5.samr import SID_NAME_USE
|
||||
from impacket.smbconnection import SMBConnection
|
||||
except ImportError:
|
||||
transport = None
|
||||
|
||||
try:
|
||||
import ldap3
|
||||
from ldap3 import Server, Connection, ALL, SUBTREE
|
||||
except ImportError:
|
||||
ldap3 = None
|
||||
|
||||
TRUST_DIRECTION = {0: "Disabled", 1: "Inbound", 2: "Outbound", 3: "Bidirectional"}
|
||||
TRUST_TYPE = {1: "Downlevel (Windows NT)", 2: "Uplevel (Windows 2000+)", 3: "MIT Kerberos", 4: "DCE"}
|
||||
TRUST_ATTRIBUTES = {
|
||||
0x00000001: "NON_TRANSITIVE",
|
||||
0x00000002: "UPLEVEL_ONLY",
|
||||
0x00000004: "QUARANTINED_DOMAIN (SID Filtering Enabled)",
|
||||
0x00000008: "FOREST_TRANSITIVE",
|
||||
0x00000010: "CROSS_ORGANIZATION",
|
||||
0x00000020: "WITHIN_FOREST",
|
||||
0x00000040: "TREAT_AS_EXTERNAL",
|
||||
0x00000080: "USES_RC4_ENCRYPTION",
|
||||
0x00000200: "CROSS_ORGANIZATION_NO_TGT_DELEGATION",
|
||||
0x00000400: "PIM_TRUST",
|
||||
}
|
||||
|
||||
|
||||
def enumerate_trusts_ldap(dc_host, domain, username, password):
|
||||
"""Enumerate AD trust relationships via LDAP trustedDomain objects."""
|
||||
if not ldap3:
|
||||
return {"error": "ldap3 not installed: pip install ldap3"}
|
||||
server = Server(dc_host, get_info=ALL, use_ssl=False)
|
||||
base_dn = ",".join(f"DC={p}" for p in domain.split("."))
|
||||
conn = Connection(server, user=f"{domain}\\{username}", password=password, auto_bind=True)
|
||||
conn.search(
|
||||
search_base=f"CN=System,{base_dn}",
|
||||
search_filter="(objectClass=trustedDomain)",
|
||||
search_scope=SUBTREE,
|
||||
attributes=[
|
||||
"cn", "trustPartner", "trustDirection", "trustType",
|
||||
"trustAttributes", "securityIdentifier", "whenCreated",
|
||||
"flatName", "trustPosixOffset",
|
||||
],
|
||||
)
|
||||
trusts = []
|
||||
for entry in conn.entries:
|
||||
attrs = entry.entry_attributes_as_dict
|
||||
direction_val = int(attrs.get("trustDirection", [0])[0])
|
||||
type_val = int(attrs.get("trustType", [0])[0])
|
||||
attr_val = int(attrs.get("trustAttributes", [0])[0])
|
||||
decoded_attrs = []
|
||||
for bit, name in TRUST_ATTRIBUTES.items():
|
||||
if attr_val & bit:
|
||||
decoded_attrs.append(name)
|
||||
sid_filtering = bool(attr_val & 0x00000004)
|
||||
forest_trust = bool(attr_val & 0x00000008)
|
||||
trusts.append({
|
||||
"trust_partner": str(attrs.get("trustPartner", [""])[0]),
|
||||
"flat_name": str(attrs.get("flatName", [""])[0]),
|
||||
"trust_direction": TRUST_DIRECTION.get(direction_val, str(direction_val)),
|
||||
"trust_type": TRUST_TYPE.get(type_val, str(type_val)),
|
||||
"trust_attributes_raw": attr_val,
|
||||
"trust_attributes": decoded_attrs,
|
||||
"sid_filtering_enabled": sid_filtering,
|
||||
"forest_transitive": forest_trust,
|
||||
"when_created": str(attrs.get("whenCreated", [""])[0]),
|
||||
})
|
||||
conn.unbind()
|
||||
return trusts
|
||||
|
||||
|
||||
def enumerate_foreign_principals(dc_host, domain, username, password):
|
||||
"""Find foreign security principals (cross-forest members) in the domain."""
|
||||
if not ldap3:
|
||||
return {"error": "ldap3 not installed"}
|
||||
server = Server(dc_host, get_info=ALL, use_ssl=False)
|
||||
base_dn = ",".join(f"DC={p}" for p in domain.split("."))
|
||||
conn = Connection(server, user=f"{domain}\\{username}", password=password, auto_bind=True)
|
||||
conn.search(
|
||||
search_base=f"CN=ForeignSecurityPrincipals,{base_dn}",
|
||||
search_filter="(objectClass=foreignSecurityPrincipal)",
|
||||
search_scope=SUBTREE,
|
||||
attributes=["cn", "objectSid", "whenCreated", "memberOf"],
|
||||
)
|
||||
principals = []
|
||||
for entry in conn.entries:
|
||||
attrs = entry.entry_attributes_as_dict
|
||||
sid = str(attrs.get("cn", [""])[0])
|
||||
member_of = [str(g) for g in attrs.get("memberOf", [])]
|
||||
principals.append({
|
||||
"sid": sid,
|
||||
"member_of_groups": member_of,
|
||||
"when_created": str(attrs.get("whenCreated", [""])[0]),
|
||||
"is_well_known": sid.startswith("S-1-5-") and sid.count("-") == 3,
|
||||
})
|
||||
conn.unbind()
|
||||
custom_principals = [p for p in principals if not p["is_well_known"]]
|
||||
return {
|
||||
"total_foreign_principals": len(principals),
|
||||
"custom_foreign_principals": len(custom_principals),
|
||||
"principals": custom_principals[:30],
|
||||
}
|
||||
|
||||
|
||||
def lookup_sid_cross_forest(dc_host, domain, username, password, target_sid):
|
||||
"""Resolve a SID across forest trust using LSA LookupSids RPC call."""
|
||||
if not transport:
|
||||
return {"error": "impacket not installed: pip install impacket"}
|
||||
rpctransport = transport.SMBTransport(dc_host, filename=r"\lsarpc")
|
||||
rpctransport.set_credentials(username, password, domain)
|
||||
dce = rpctransport.get_dce_rpc()
|
||||
dce.connect()
|
||||
dce.bind(lsat.MSRPC_UUID_LSAT)
|
||||
resp = lsad.hLsarOpenPolicy2(dce, MAXIMUM_ALLOWED)
|
||||
policy_handle = resp["PolicyHandle"]
|
||||
try:
|
||||
from impacket.dcerpc.v5.dtypes import RPC_SID
|
||||
sid = RPC_SID()
|
||||
sid.fromCanonical(target_sid)
|
||||
resp = lsat.hLsarLookupSids2(dce, policy_handle, [sid])
|
||||
names = []
|
||||
for item in resp["TranslatedNames"]["Names"]:
|
||||
names.append({
|
||||
"name": item["Name"],
|
||||
"sid_type": SID_NAME_USE.enumItems(item["Use"]).name if hasattr(SID_NAME_USE, 'enumItems') else str(item["Use"]),
|
||||
"domain_index": item["DomainIndex"],
|
||||
})
|
||||
return {"target_sid": target_sid, "resolved_names": names}
|
||||
except Exception as e:
|
||||
return {"target_sid": target_sid, "error": str(e)}
|
||||
finally:
|
||||
dce.disconnect()
|
||||
|
||||
|
||||
def assess_trust_risk(trusts, foreign_principals):
|
||||
"""Assess security risk of trust relationships."""
|
||||
findings = []
|
||||
for trust in trusts:
|
||||
risk = 0
|
||||
issues = []
|
||||
if not trust.get("sid_filtering_enabled"):
|
||||
risk += 40
|
||||
issues.append("SID filtering DISABLED — SID history attacks possible")
|
||||
if trust.get("trust_direction") == "Bidirectional":
|
||||
risk += 15
|
||||
issues.append("Bidirectional trust increases attack surface")
|
||||
if trust.get("forest_transitive"):
|
||||
risk += 10
|
||||
issues.append("Forest transitive trust — all domains reachable")
|
||||
if "USES_RC4_ENCRYPTION" in trust.get("trust_attributes", []):
|
||||
risk += 20
|
||||
issues.append("RC4 encryption — vulnerable to trust key cracking")
|
||||
risk = min(risk, 100)
|
||||
findings.append({
|
||||
"trust_partner": trust.get("trust_partner"),
|
||||
"risk_score": risk,
|
||||
"risk_level": "CRITICAL" if risk >= 70 else "HIGH" if risk >= 50 else "MEDIUM" if risk >= 25 else "LOW",
|
||||
"issues": issues,
|
||||
"recommendation": "Enable SID filtering and migrate to AES encryption"
|
||||
if risk >= 50 else "Review trust configuration",
|
||||
})
|
||||
return findings
|
||||
|
||||
|
||||
def full_audit(dc_host, domain, username, password):
|
||||
"""Run comprehensive forest trust security audit."""
|
||||
trusts = enumerate_trusts_ldap(dc_host, domain, username, password)
|
||||
foreign = enumerate_foreign_principals(dc_host, domain, username, password)
|
||||
risk = assess_trust_risk(trusts if isinstance(trusts, list) else [], foreign)
|
||||
critical = sum(1 for r in risk if r["risk_level"] == "CRITICAL")
|
||||
high = sum(1 for r in risk if r["risk_level"] == "HIGH")
|
||||
no_sid_filter = sum(1 for t in (trusts if isinstance(trusts, list) else []) if not t.get("sid_filtering_enabled"))
|
||||
return {
|
||||
"audit_type": "AD Forest Trust Security Assessment",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"domain": domain,
|
||||
"summary": {
|
||||
"total_trusts": len(trusts) if isinstance(trusts, list) else 0,
|
||||
"trusts_without_sid_filtering": no_sid_filter,
|
||||
"foreign_principals": foreign.get("custom_foreign_principals", 0),
|
||||
"critical_findings": critical,
|
||||
"high_findings": high,
|
||||
},
|
||||
"trusts": trusts,
|
||||
"foreign_principals": foreign,
|
||||
"risk_assessment": risk,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="AD Forest Trust Security Audit Agent")
|
||||
parser.add_argument("--dc", required=True, help="Domain Controller hostname or IP")
|
||||
parser.add_argument("--domain", required=True, help="Domain name (e.g., corp.local)")
|
||||
parser.add_argument("--username", required=True, help="Domain username")
|
||||
parser.add_argument("--password", required=True, help="Domain password")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("trusts", help="Enumerate trust relationships")
|
||||
sub.add_parser("foreign", help="List foreign security principals")
|
||||
p_sid = sub.add_parser("lookup-sid", help="Cross-forest SID lookup")
|
||||
p_sid.add_argument("--sid", required=True, help="Target SID to resolve")
|
||||
sub.add_parser("full", help="Full trust security audit")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "trusts":
|
||||
result = enumerate_trusts_ldap(args.dc, args.domain, args.username, args.password)
|
||||
elif args.command == "foreign":
|
||||
result = enumerate_foreign_principals(args.dc, args.domain, args.username, args.password)
|
||||
elif args.command == "lookup-sid":
|
||||
result = lookup_sid_cross_forest(args.dc, args.domain, args.username, args.password, args.sid)
|
||||
elif args.command == "full" or args.command is None:
|
||||
result = full_audit(args.dc, args.domain, args.username, args.password)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mahipal
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: performing-hardware-security-module-integration
|
||||
description: Integrate Hardware Security Modules (HSMs) using PKCS#11 interface for cryptographic key management, signing operations, and secure key storage with python-pkcs11, AWS CloudHSM, and YubiHSM2.
|
||||
domain: cybersecurity
|
||||
subdomain: cryptography
|
||||
tags: [HSM, PKCS11, CloudHSM, YubiHSM2, key-management, cryptographic-operations, hardware-security]
|
||||
version: "1.0"
|
||||
author: mahipal
|
||||
license: Apache-2.0
|
||||
---
|
||||
|
||||
# Performing Hardware Security Module Integration
|
||||
|
||||
## Overview
|
||||
|
||||
Hardware Security Modules (HSMs) provide tamper-resistant cryptographic key storage and operations. This skill covers integrating with HSMs via the PKCS#11 standard interface using python-pkcs11, performing key generation, signing, encryption, and verification operations, querying token and slot information, and validating HSM configuration for compliance with FIPS 140-2/3 requirements.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- HSM device or software HSM (SoftHSM2 for testing)
|
||||
- PKCS#11 shared library (.so/.dll) for the HSM vendor
|
||||
- Python 3.9+ with `python-pkcs11`
|
||||
- Token initialized with SO PIN and user PIN
|
||||
- For AWS CloudHSM: `cloudhsm-pkcs11` provider configured
|
||||
|
||||
## Steps
|
||||
|
||||
1. Load PKCS#11 library and enumerate available slots and tokens
|
||||
2. Open session and authenticate with user PIN
|
||||
3. Generate RSA 2048-bit or EC P-256 key pairs on the HSM
|
||||
4. Perform signing and verification using on-device keys
|
||||
5. List all objects (keys, certificates) stored on the token
|
||||
6. Query mechanism list to verify supported algorithms
|
||||
7. Generate compliance report with key inventory and algorithm audit
|
||||
|
||||
## Expected Output
|
||||
|
||||
- JSON report listing HSM slots, tokens, stored keys, supported mechanisms, and compliance status
|
||||
- Signing test results with key metadata and algorithm details
|
||||
@@ -0,0 +1,70 @@
|
||||
# API Reference — Performing Hardware Security Module Integration
|
||||
|
||||
## Libraries Used
|
||||
- **python-pkcs11**: Python PKCS#11 wrapper for HSM cryptographic operations
|
||||
- **json**: JSON serialization for audit reports
|
||||
|
||||
## CLI Interface
|
||||
```
|
||||
python agent.py --lib /usr/lib/softhsm/libsofthsm2.so --token MyToken --pin 1234 slots
|
||||
python agent.py --lib /usr/lib/softhsm/libsofthsm2.so --token MyToken --pin 1234 objects
|
||||
python agent.py --lib /usr/lib/softhsm/libsofthsm2.so --token MyToken --pin 1234 gen-rsa --label mykey --bits 2048
|
||||
python agent.py --lib /usr/lib/softhsm/libsofthsm2.so --token MyToken --pin 1234 gen-ec --label myec
|
||||
python agent.py --lib /usr/lib/softhsm/libsofthsm2.so --token MyToken --pin 1234 sign-verify --key-label mykey
|
||||
python agent.py --lib /usr/lib/softhsm/libsofthsm2.so --token MyToken --pin 1234 mechanisms
|
||||
python agent.py --lib /usr/lib/softhsm/libsofthsm2.so --token MyToken --pin 1234 full
|
||||
```
|
||||
|
||||
## Core Functions
|
||||
|
||||
### `load_library(lib_path)` — Load PKCS#11 shared library
|
||||
Calls `pkcs11.lib(lib_path)` to initialize the PKCS#11 provider.
|
||||
|
||||
### `enumerate_slots(lib)` — List slots and token info
|
||||
Iterates `lib.get_slots(token_present=True)`. Returns token label, manufacturer,
|
||||
model, serial, initialization status, and supported mechanism list.
|
||||
|
||||
### `list_objects(lib, token_label, pin)` — Inventory stored keys
|
||||
Opens authenticated session, calls `session.get_objects()`. Returns object class,
|
||||
label, key type, key length, and object ID.
|
||||
|
||||
### `generate_rsa_keypair(lib, token_label, pin, key_label, bits)` — RSA key generation
|
||||
Calls `session.generate_keypair(KeyType.RSA, bits, store=True, label=key_label)`.
|
||||
|
||||
### `generate_ec_keypair(lib, token_label, pin, key_label)` — EC P-256 key generation
|
||||
Creates domain parameters for secp256r1 via `encode_named_curve_parameters`,
|
||||
then calls `ecparams.generate_keypair()`.
|
||||
|
||||
### `sign_and_verify(lib, token_label, pin, key_label)` — Signing test
|
||||
Signs with `priv.sign(data, mechanism=Mechanism.SHA256_RSA_PKCS)`.
|
||||
Verifies with `pub.verify(data, signature, mechanism=Mechanism.SHA256_RSA_PKCS)`.
|
||||
|
||||
### `query_mechanisms(lib, token_label)` — Algorithm support audit
|
||||
Enumerates all mechanisms with min/max key sizes from the slot.
|
||||
|
||||
### `full_audit(lib, token_label, pin)` — Comprehensive compliance report
|
||||
|
||||
## PKCS#11 Object Classes
|
||||
| Class | Description |
|
||||
|-------|-------------|
|
||||
| PUBLIC_KEY | RSA/EC public keys |
|
||||
| PRIVATE_KEY | RSA/EC private keys (non-extractable) |
|
||||
| SECRET_KEY | Symmetric keys (AES, DES3) |
|
||||
| CERTIFICATE | X.509 certificates |
|
||||
|
||||
## FIPS 140-2 Required Mechanisms
|
||||
RSA_PKCS, SHA256_RSA_PKCS, SHA384_RSA_PKCS, SHA512_RSA_PKCS,
|
||||
ECDSA, ECDSA_SHA256, AES_CBC, AES_GCM, SHA256, SHA384, SHA512
|
||||
|
||||
## Common PKCS#11 Libraries
|
||||
| HSM | Library Path |
|
||||
|-----|-------------|
|
||||
| SoftHSM2 | `/usr/lib/softhsm/libsofthsm2.so` |
|
||||
| AWS CloudHSM | `/opt/cloudhsm/lib/libcloudhsm_pkcs11.so` |
|
||||
| YubiHSM2 | `/usr/lib/x86_64-linux-gnu/pkcs11/yubihsm_pkcs11.so` |
|
||||
| Thales Luna | `/usr/safenet/lunaclient/lib/libCryptoki2_64.so` |
|
||||
|
||||
## Dependencies
|
||||
- `python-pkcs11` >= 0.7.0
|
||||
- PKCS#11 shared library for target HSM
|
||||
- Initialized token with user PIN
|
||||
@@ -0,0 +1,223 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Agent for Hardware Security Module integration via PKCS#11 interface."""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
|
||||
try:
|
||||
import pkcs11
|
||||
from pkcs11 import KeyType, ObjectClass, Mechanism
|
||||
from pkcs11.util.rsa import encode_rsa_public_key
|
||||
except ImportError:
|
||||
pkcs11 = None
|
||||
|
||||
|
||||
def load_library(lib_path):
|
||||
"""Load PKCS#11 shared library."""
|
||||
if not pkcs11:
|
||||
raise RuntimeError("python-pkcs11 not installed: pip install python-pkcs11")
|
||||
return pkcs11.lib(lib_path)
|
||||
|
||||
|
||||
def enumerate_slots(lib):
|
||||
"""List all available PKCS#11 slots and token info."""
|
||||
slots = []
|
||||
for slot in lib.get_slots(token_present=True):
|
||||
token = slot.get_token()
|
||||
mechs = slot.get_mechanisms()
|
||||
slots.append({
|
||||
"slot_id": slot.slot_id,
|
||||
"slot_description": slot.slot_description.strip() if hasattr(slot, 'slot_description') else str(slot),
|
||||
"token_label": token.label.strip(),
|
||||
"token_manufacturer": token.manufacturer_id.strip(),
|
||||
"token_model": token.model.strip(),
|
||||
"token_serial": token.serial.strip(),
|
||||
"token_initialized": token.flags & pkcs11.TokenFlag.TOKEN_INITIALIZED != 0,
|
||||
"mechanism_count": len(list(mechs)),
|
||||
"supported_mechanisms": sorted([m.name for m in mechs])[:30],
|
||||
})
|
||||
return slots
|
||||
|
||||
|
||||
def list_objects(lib, token_label, pin):
|
||||
"""List all cryptographic objects stored on the HSM token."""
|
||||
token = lib.get_token(token_label=token_label)
|
||||
with token.open(user_pin=pin) as session:
|
||||
objects = []
|
||||
for obj in session.get_objects():
|
||||
obj_info = {
|
||||
"object_class": obj.object_class.name if hasattr(obj.object_class, 'name') else str(obj.object_class),
|
||||
"label": getattr(obj, 'label', 'N/A'),
|
||||
}
|
||||
if hasattr(obj, 'key_type'):
|
||||
obj_info["key_type"] = obj.key_type.name if hasattr(obj.key_type, 'name') else str(obj.key_type)
|
||||
if hasattr(obj, 'key_length'):
|
||||
obj_info["key_length"] = obj.key_length
|
||||
if hasattr(obj, 'id'):
|
||||
obj_info["id"] = obj.id.hex() if isinstance(obj.id, bytes) else str(obj.id)
|
||||
objects.append(obj_info)
|
||||
return objects
|
||||
|
||||
|
||||
def generate_rsa_keypair(lib, token_label, pin, key_label="agent-rsa-2048", bits=2048):
|
||||
"""Generate an RSA key pair on the HSM."""
|
||||
token = lib.get_token(token_label=token_label)
|
||||
with token.open(rw=True, user_pin=pin) as session:
|
||||
pub, priv = session.generate_keypair(
|
||||
KeyType.RSA, bits,
|
||||
store=True,
|
||||
label=key_label,
|
||||
)
|
||||
return {
|
||||
"action": "generate_rsa_keypair",
|
||||
"key_label": key_label,
|
||||
"key_size": bits,
|
||||
"public_key_class": pub.object_class.name,
|
||||
"private_key_class": priv.object_class.name,
|
||||
"status": "SUCCESS",
|
||||
}
|
||||
|
||||
|
||||
def generate_ec_keypair(lib, token_label, pin, key_label="agent-ec-p256"):
|
||||
"""Generate an EC P-256 key pair on the HSM."""
|
||||
token = lib.get_token(token_label=token_label)
|
||||
with token.open(rw=True, user_pin=pin) as session:
|
||||
ecparams = session.create_domain_parameters(
|
||||
KeyType.EC,
|
||||
{pkcs11.Attribute.EC_PARAMS: pkcs11.util.ec.encode_named_curve_parameters("secp256r1")},
|
||||
local=True,
|
||||
)
|
||||
pub, priv = ecparams.generate_keypair(store=True, label=key_label)
|
||||
return {
|
||||
"action": "generate_ec_keypair",
|
||||
"key_label": key_label,
|
||||
"curve": "secp256r1 (P-256)",
|
||||
"public_key_class": pub.object_class.name,
|
||||
"private_key_class": priv.object_class.name,
|
||||
"status": "SUCCESS",
|
||||
}
|
||||
|
||||
|
||||
def sign_and_verify(lib, token_label, pin, key_label, data=b"HSM test message"):
|
||||
"""Sign data with an RSA private key and verify with the public key."""
|
||||
token = lib.get_token(token_label=token_label)
|
||||
with token.open(user_pin=pin) as session:
|
||||
priv_keys = list(session.get_objects({
|
||||
pkcs11.Attribute.CLASS: ObjectClass.PRIVATE_KEY,
|
||||
pkcs11.Attribute.LABEL: key_label,
|
||||
}))
|
||||
if not priv_keys:
|
||||
return {"error": f"Private key '{key_label}' not found"}
|
||||
priv = priv_keys[0]
|
||||
signature = priv.sign(data, mechanism=Mechanism.SHA256_RSA_PKCS)
|
||||
|
||||
pub_keys = list(session.get_objects({
|
||||
pkcs11.Attribute.CLASS: ObjectClass.PUBLIC_KEY,
|
||||
pkcs11.Attribute.LABEL: key_label,
|
||||
}))
|
||||
if not pub_keys:
|
||||
return {"error": f"Public key '{key_label}' not found"}
|
||||
pub = pub_keys[0]
|
||||
try:
|
||||
pub.verify(data, signature, mechanism=Mechanism.SHA256_RSA_PKCS)
|
||||
verified = True
|
||||
except Exception:
|
||||
verified = False
|
||||
|
||||
return {
|
||||
"action": "sign_and_verify",
|
||||
"key_label": key_label,
|
||||
"data_length": len(data),
|
||||
"signature_length": len(signature),
|
||||
"signature_hex": signature[:32].hex() + "...",
|
||||
"mechanism": "SHA256_RSA_PKCS",
|
||||
"verified": verified,
|
||||
}
|
||||
|
||||
|
||||
def query_mechanisms(lib, token_label):
|
||||
"""List all supported mechanisms for the token's slot."""
|
||||
token = lib.get_token(token_label=token_label)
|
||||
slot = token.slot
|
||||
mechs = []
|
||||
for m in slot.get_mechanisms():
|
||||
info = slot.get_mechanism_info(m)
|
||||
mechs.append({
|
||||
"mechanism": m.name,
|
||||
"min_key_size": info.min_key_size if hasattr(info, 'min_key_size') else None,
|
||||
"max_key_size": info.max_key_size if hasattr(info, 'max_key_size') else None,
|
||||
})
|
||||
return mechs
|
||||
|
||||
|
||||
def full_audit(lib, token_label, pin):
|
||||
"""Run comprehensive HSM compliance audit."""
|
||||
slots = enumerate_slots(lib)
|
||||
objects = list_objects(lib, token_label, pin)
|
||||
mechanisms = query_mechanisms(lib, token_label)
|
||||
rsa_keys = [o for o in objects if o.get("key_type") == "RSA"]
|
||||
ec_keys = [o for o in objects if o.get("key_type") == "EC"]
|
||||
weak_keys = [o for o in objects if o.get("key_type") == "RSA" and (o.get("key_length") or 2048) < 2048]
|
||||
fips_mechs = {"RSA_PKCS", "SHA256_RSA_PKCS", "SHA384_RSA_PKCS", "SHA512_RSA_PKCS",
|
||||
"ECDSA", "ECDSA_SHA256", "AES_CBC", "AES_GCM", "SHA256", "SHA384", "SHA512"}
|
||||
supported_names = {m["mechanism"] for m in mechanisms}
|
||||
fips_coverage = len(fips_mechs & supported_names)
|
||||
return {
|
||||
"audit_type": "HSM PKCS#11 Compliance Audit",
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"slots": slots,
|
||||
"stored_objects": len(objects),
|
||||
"objects": objects[:30],
|
||||
"rsa_key_count": len(rsa_keys),
|
||||
"ec_key_count": len(ec_keys),
|
||||
"weak_rsa_keys": len(weak_keys),
|
||||
"total_mechanisms": len(mechanisms),
|
||||
"fips_mechanism_coverage": f"{fips_coverage}/{len(fips_mechs)}",
|
||||
"compliance": "PASS" if not weak_keys and fips_coverage >= 6 else "REVIEW",
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="HSM PKCS#11 Integration Agent")
|
||||
parser.add_argument("--lib", required=True, help="Path to PKCS#11 shared library")
|
||||
parser.add_argument("--token", required=True, help="Token label")
|
||||
parser.add_argument("--pin", required=True, help="User PIN")
|
||||
sub = parser.add_subparsers(dest="command")
|
||||
sub.add_parser("slots", help="Enumerate PKCS#11 slots and tokens")
|
||||
sub.add_parser("objects", help="List stored cryptographic objects")
|
||||
p_rsa = sub.add_parser("gen-rsa", help="Generate RSA key pair")
|
||||
p_rsa.add_argument("--label", default="agent-rsa-2048")
|
||||
p_rsa.add_argument("--bits", type=int, default=2048)
|
||||
p_ec = sub.add_parser("gen-ec", help="Generate EC P-256 key pair")
|
||||
p_ec.add_argument("--label", default="agent-ec-p256")
|
||||
p_sign = sub.add_parser("sign-verify", help="Sign and verify test data")
|
||||
p_sign.add_argument("--key-label", required=True)
|
||||
sub.add_parser("mechanisms", help="List supported mechanisms")
|
||||
sub.add_parser("full", help="Full HSM compliance audit")
|
||||
args = parser.parse_args()
|
||||
|
||||
lib = load_library(args.lib)
|
||||
|
||||
if args.command == "slots":
|
||||
result = enumerate_slots(lib)
|
||||
elif args.command == "objects":
|
||||
result = list_objects(lib, args.token, args.pin)
|
||||
elif args.command == "gen-rsa":
|
||||
result = generate_rsa_keypair(lib, args.token, args.pin, args.label, args.bits)
|
||||
elif args.command == "gen-ec":
|
||||
result = generate_ec_keypair(lib, args.token, args.pin, args.label)
|
||||
elif args.command == "sign-verify":
|
||||
result = sign_and_verify(lib, args.token, args.pin, args.key_label)
|
||||
elif args.command == "mechanisms":
|
||||
result = query_mechanisms(lib, args.token)
|
||||
elif args.command == "full":
|
||||
result = full_audit(lib, args.token, args.pin)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user