Initial commit - 611 cybersecurity skills across all subdomains

This commit is contained in:
mukul975
2026-02-25 10:47:44 +01:00
commit 22a7ab1462
1765 changed files with 280648 additions and 0 deletions
@@ -0,0 +1,104 @@
---
name: implementing-proofpoint-email-security-gateway
description: Deploy and configure Proofpoint Email Protection as a secure email gateway to detect and block phishing, malware, BEC, and spam before messages reach user inboxes.
domain: cybersecurity
subdomain: phishing-defense
tags: [email-security, proofpoint, secure-email-gateway, phishing, anti-spam, anti-malware, bec, email-filtering]
version: "1.0"
author: mahipal
license: MIT
---
# Implementing Proofpoint Email Security Gateway
## Overview
Proofpoint Email Protection is a cloud-native secure email gateway (SEG) that acts as a security checkpoint where all inbound and outbound mail traffic routes through the gateway before reaching user inboxes. It combines signature-based detection for known malware, machine learning algorithms for emerging threats, real-time threat intelligence feeds, URL rewriting with time-of-click sandboxing, and behavioral analysis for BEC detection. Proofpoint processes over 2.8 billion emails daily and blocks over 1 million extortion attempts per day.
## Prerequisites
- Proofpoint Email Protection license (PPS on-premises or Proofpoint on Demand cloud)
- Administrative access to DNS management for MX record changes
- Microsoft 365 or Google Workspace email environment
- Understanding of mail flow architecture and SPF/DKIM/DMARC
- Network firewall rules permitting Proofpoint IP ranges
## Key Concepts
### Deployment Models
1. **MX-Based Gateway (Traditional SEG)**: All mail routes through Proofpoint via MX record changes; intercepts threats before delivery
2. **API-Based Integration**: Connects directly to Microsoft 365 or Google Workspace via API; no MX changes required; can be operational within 48 hours
3. **Hybrid Deployment**: Combines gateway and API for layered protection
### Core Detection Technologies
- **Impostor Classifier**: ML model detecting BEC/impersonation with no malicious URLs or attachments
- **URL Defense**: Rewrites URLs and performs real-time sandboxing at time of click
- **Attachment Defense**: Sandboxes suspicious attachments in virtual environments
- **Nexus Threat Graph**: Cross-customer threat intelligence correlation engine
- **Supplier Threat Detection**: Identifies compromised vendor email accounts
### Protection Layers
| Layer | Technology | Threat Type |
|---|---|---|
| Connection | IP reputation, rate limiting | Spam botnets |
| Authentication | SPF, DKIM, DMARC enforcement | Spoofing |
| Content | ML classifiers, NLP analysis | BEC, phishing |
| URL | Rewriting + time-of-click sandbox | Credential theft |
| Attachment | Static + dynamic sandboxing | Malware, ransomware |
| Post-delivery | TRAP (auto-retraction) | Weaponized after delivery |
## Implementation Steps
### Step 1: Plan Mail Flow Architecture
- Document current MX records and mail flow path
- Identify all legitimate sending sources (marketing platforms, CRM, ticketing systems)
- Map inbound connectors and transport rules in Microsoft 365 or Google Workspace
- Plan IP allowlisting for Proofpoint egress IPs on receiving infrastructure
- Configure SPF record to include Proofpoint: `v=spf1 include:spf.protection.outlook.com include:spf-a.proofpoint.com -all`
### Step 2: Configure Proofpoint Policies
- Create organizational units matching business structure
- Define inbound mail policies: anti-spam, anti-virus, impostor detection
- Configure Smart Search quarantine with end-user digest notifications
- Set up Proofpoint Encryption for sensitive outbound messages
- Enable Targeted Attack Protection (TAP) for URL and attachment sandboxing
### Step 3: Deploy Email Authentication
- Configure DKIM signing through Proofpoint for outbound messages
- Set DMARC policy to monitor mode initially: `v=DMARC1; p=none; rua=mailto:dmarc@company.com`
- Enable inbound DMARC enforcement to reject spoofed messages
- Configure anti-spoofing rules for executive impersonation protection
### Step 4: Enable Advanced Threat Protection
- Activate URL Defense with rewriting enabled for all inbound messages
- Configure Attachment Defense sandbox policies (safe attachment mode)
- Enable Threat Response Auto-Pull (TRAP) for post-delivery remediation
- Set up TAP Dashboard alerts for targeted attack campaigns
- Configure Supplier Risk monitoring for vendor email compromise
### Step 5: Migrate MX Records
- Lower MX record TTL to 300 seconds 48 hours before cutover
- Update MX records to point to Proofpoint: `company-com.mail.protection.proofpoint.com`
- Configure connector restrictions in Microsoft 365 to accept mail only from Proofpoint IPs
- Monitor mail flow through Proofpoint Message Trace for 48-72 hours
- Verify no legitimate mail is being blocked or delayed
### Step 6: Tune and Optimize
- Review quarantine and false positive/negative rates weekly for first month
- Adjust spam thresholds based on organizational tolerance
- Add approved senders and safe lists for legitimate bulk mail
- Configure data loss prevention (DLP) rules for outbound sensitive content
- Enable email warning banners for external sender identification
## Tools & Resources
- **Proofpoint TAP Dashboard**: Real-time threat visibility and campaign tracking
- **Proofpoint TRAP**: Automated post-delivery email retraction
- **Proofpoint SER (Spam/End-user Release)**: Self-service quarantine management
- **Proofpoint Closed-Loop Email Analysis (CLEAR)**: Phishing report button integration
- **MX Toolbox**: DNS record verification and mail flow testing
## Validation
- All inbound email routes through Proofpoint (verify MX records and message headers)
- TAP Dashboard shows threat detections and blocked campaigns
- URL Defense rewrites links in test messages and sandboxes at click time
- Attachment Defense detonates test malware samples in sandbox
- TRAP successfully retracts test phishing message from inboxes post-delivery
- False positive rate below 0.1% after initial tuning period
- DMARC/SPF/DKIM authentication passes for all legitimate outbound mail
@@ -0,0 +1,45 @@
# Proofpoint Email Security Gateway Deployment Template
## Pre-Deployment Checklist
- [ ] Proofpoint license type confirmed (PPS / PoD)
- [ ] Current MX records documented
- [ ] All legitimate sending sources inventoried
- [ ] SPF record updated with Proofpoint include
- [ ] DKIM keys generated and DNS records published
- [ ] DMARC record configured in monitoring mode
- [ ] Firewall rules updated for Proofpoint IP ranges
- [ ] Microsoft 365 / Google Workspace connector configured
## Policy Configuration
| Policy | Scope | Action | Status |
|---|---|---|---|
| Anti-spam (inbound) | All users | Quarantine high confidence | |
| Anti-virus | All users | Block + notify admin | |
| Impostor detection | VIP list | Quarantine + SOC alert | |
| URL Defense | All users | Rewrite + sandbox at click | |
| Attachment Defense | All users | Sandbox suspicious types | |
| TRAP auto-pull | All users | Retract post-delivery threats | |
| DLP (outbound) | All users | Block + manager notify | |
## VIP Protection List
| Name | Title | Email | Protected |
|---|---|---|---|
| | CEO | | Yes |
| | CFO | | Yes |
| | CTO | | Yes |
| | VP Finance | | Yes |
## MX Record Migration
| Record Type | Priority | Old Value | New Value |
|---|---|---|---|
| MX | 10 | | {org}.mail.protection.proofpoint.com |
## Post-Deployment Validation
- [ ] Test inbound mail delivery through Proofpoint
- [ ] Verify message headers show Proofpoint processing
- [ ] Test URL Defense rewriting on inbound links
- [ ] Test Attachment Defense with EICAR test file
- [ ] Verify TRAP can retract delivered message
- [ ] Confirm quarantine digest notifications working
- [ ] Validate SPF/DKIM/DMARC pass for outbound mail
- [ ] Review false positive rate after 48 hours
@@ -0,0 +1,37 @@
# Standards & References: Implementing Proofpoint Email Security Gateway
## Industry Standards
- **NIST SP 800-177 Rev.1**: Trustworthy Email - guidelines for email security deployment
- **RFC 7208**: Sender Policy Framework (SPF) for authorizing use of domains in email
- **RFC 6376**: DomainKeys Identified Mail (DKIM) Signatures
- **RFC 7489**: Domain-based Message Authentication, Reporting & Conformance (DMARC)
- **CIS Controls v8 Control 9**: Email and Web Browser Protections
## MITRE ATT&CK References
- **T1566.001**: Phishing: Spearphishing Attachment
- **T1566.002**: Phishing: Spearphishing Link
- **T1566.003**: Phishing: Spearphishing via Service
- **T1534**: Internal Spearphishing
- **T1598**: Phishing for Information
- **T1114.003**: Email Collection: Email Forwarding Rule
## Proofpoint-Specific References
- **Proofpoint Email Protection (PPS)**: On-premises protection platform
- **Proofpoint on Demand (PoD)**: Cloud-hosted email security service
- **Proofpoint TAP (Targeted Attack Protection)**: Advanced threat detection
- **Proofpoint TRAP (Threat Response Auto-Pull)**: Post-delivery remediation
- **Proofpoint Nexus Threat Graph**: Cross-customer threat intelligence
## Compliance Alignment
| Framework | Control | Description |
|---|---|---|
| SOC 2 | CC6.1 | Logical and physical access controls |
| HIPAA | 164.312(a)(1) | Access control for ePHI |
| PCI DSS 4.0 | 5.2 | Anti-malware solutions |
| NIST CSF 2.0 | PR.DS-1 | Data-at-rest and data-in-transit protection |
| ISO 27001 | A.8.23 | Web filtering |
## Email Security Gateway Market
- Proofpoint processes 2.8+ billion messages daily (2024)
- Gartner Magic Quadrant Leader for Email Security (2019-2024)
- Over 50% of Fortune 100 companies use Proofpoint
@@ -0,0 +1,107 @@
# Workflows: Implementing Proofpoint Email Security Gateway
## Workflow 1: Inbound Mail Processing Pipeline
```
External sender sends email
|
v
[DNS MX lookup resolves to Proofpoint]
|
v
[Connection-level filtering]
+-- IP reputation check (Proofpoint Nexus)
+-- Rate limiting and connection throttling
+-- REJECT if known-bad IP
|
v
[Authentication checks]
+-- SPF validation
+-- DKIM signature verification
+-- DMARC policy evaluation
+-- FAIL actions: quarantine or reject per policy
|
v
[Content analysis]
+-- Anti-spam scoring (ML classifier)
+-- Anti-virus scanning (multi-engine)
+-- Impostor classifier (BEC detection)
+-- NLP analysis for social engineering language
|
v
[URL Defense]
+-- Extract all URLs from body and attachments
+-- Rewrite URLs through Proofpoint proxy
+-- Pre-delivery URL reputation check
+-- BLOCK if known malicious
|
v
[Attachment Defense]
+-- Static analysis (signatures, heuristics)
+-- Dynamic sandbox detonation (if suspicious)
+-- Wait for sandbox verdict (up to 7 minutes)
+-- QUARANTINE if malicious
|
v
[Policy action]
+-- DELIVER: Clean email to mailbox
+-- TAG: Add warning banner for external/suspicious
+-- QUARANTINE: Hold for admin/user review
+-- REJECT: Block with NDR to sender
```
## Workflow 2: Post-Delivery Threat Response (TRAP)
```
Threat intelligence update received
|
v
[TRAP scans delivered messages retroactively]
+-- URL becomes malicious after delivery
+-- New malware signature matches delivered attachment
|
v
[Auto-Pull action triggered]
+-- Move message from user inbox to quarantine
+-- Log retraction in TRAP dashboard
+-- Notify SOC team of post-delivery threat
|
v
[SOC investigation]
+-- Review TRAP alert and threat details
+-- Check if user clicked URL before retraction
+-- If clicked: initiate incident response
+-- If not clicked: close as contained
|
v
[Update policies]
+-- Add sender/domain to block list if needed
+-- Create detection rule for similar campaigns
+-- Update TAP Dashboard threat tracking
```
## Workflow 3: Phishing Report and CLEAR Integration
```
User receives suspicious email
|
v
[User clicks "Report Phishing" button (Proofpoint CLEAR)]
|
v
[Email forwarded to Proofpoint analysis pipeline]
+-- Automated classification (phishing/spam/clean)
+-- URL and attachment analysis
|
v
[CLEAR verdict]
+-- MALICIOUS: Auto-retract from all inboxes that received it
+-- SPAM: Move to junk for all recipients
+-- CLEAN: Return to inbox, thank reporter
|
v
[Metrics and feedback]
+-- Track reporter accuracy rate
+-- Update user risk score
+-- Feed into security awareness metrics
```
@@ -0,0 +1,335 @@
#!/usr/bin/env python3
"""
Proofpoint Email Security Gateway Configuration Validator
Validates MX records, SPF/DKIM/DMARC alignment, and Proofpoint
mail flow configuration for a given domain. Uses DNS lookups to
verify that email is routing through Proofpoint correctly.
Usage:
python process.py check-mx --domain example.com
python process.py check-auth --domain example.com
python process.py validate-headers --eml-file message.eml
"""
import argparse
import json
import re
import sys
from dataclasses import dataclass, field, asdict
from typing import Optional
try:
import dns.resolver
HAS_DNS = True
except ImportError:
HAS_DNS = False
@dataclass
class MXCheckResult:
"""MX record validation result."""
domain: str = ""
mx_records: list = field(default_factory=list)
routes_through_proofpoint: bool = False
proofpoint_mx: str = ""
issues: list = field(default_factory=list)
@dataclass
class AuthCheckResult:
"""Email authentication check result."""
domain: str = ""
spf_record: str = ""
spf_includes_proofpoint: bool = False
dmarc_record: str = ""
dmarc_policy: str = ""
dkim_selector: str = ""
issues: list = field(default_factory=list)
@dataclass
class HeaderAnalysis:
"""Email header forensic analysis."""
from_address: str = ""
return_path: str = ""
received_chain: list = field(default_factory=list)
spf_result: str = ""
dkim_result: str = ""
dmarc_result: str = ""
proofpoint_headers: list = field(default_factory=list)
x_proofpoint_spam_details: str = ""
x_proofpoint_virus_version: str = ""
passed_through_proofpoint: bool = False
issues: list = field(default_factory=list)
PROOFPOINT_MX_PATTERNS = [
r'\.pphosted\.com$',
r'\.proofpoint\.com$',
r'mail\.protection\.proofpoint\.com$',
]
PROOFPOINT_SPF_INCLUDES = [
'spf-a.proofpoint.com',
'spf-b.proofpoint.com',
'spf.proofpoint.com',
'pphosted.com',
]
PROOFPOINT_HEADER_MARKERS = [
'X-Proofpoint-Spam-Details',
'X-Proofpoint-Virus-Version',
'X-Proofpoint-GUID',
'X-Proofpoint-ORIG-GUID',
]
def check_mx_records(domain: str) -> MXCheckResult:
"""Check if domain MX records route through Proofpoint."""
result = MXCheckResult(domain=domain)
if not HAS_DNS:
result.issues.append("dnspython not installed. Install with: pip install dnspython")
return result
try:
answers = dns.resolver.resolve(domain, 'MX')
for rdata in sorted(answers, key=lambda r: r.preference):
mx_host = str(rdata.exchange).rstrip('.')
result.mx_records.append({
"priority": rdata.preference,
"host": mx_host
})
for pattern in PROOFPOINT_MX_PATTERNS:
if re.search(pattern, mx_host, re.IGNORECASE):
result.routes_through_proofpoint = True
result.proofpoint_mx = mx_host
break
except dns.resolver.NXDOMAIN:
result.issues.append(f"Domain {domain} does not exist")
except dns.resolver.NoAnswer:
result.issues.append(f"No MX records found for {domain}")
except Exception as e:
result.issues.append(f"DNS query failed: {str(e)}")
if not result.routes_through_proofpoint and result.mx_records:
result.issues.append(
"MX records do not point to Proofpoint. "
"Expected pattern: *.pphosted.com or *.proofpoint.com"
)
return result
def check_authentication(domain: str) -> AuthCheckResult:
"""Check SPF, DKIM, and DMARC records for Proofpoint alignment."""
result = AuthCheckResult(domain=domain)
if not HAS_DNS:
result.issues.append("dnspython not installed. Install with: pip install dnspython")
return result
# Check SPF
try:
answers = dns.resolver.resolve(domain, 'TXT')
for rdata in answers:
txt = str(rdata).strip('"')
if txt.startswith('v=spf1'):
result.spf_record = txt
for include in PROOFPOINT_SPF_INCLUDES:
if include in txt:
result.spf_includes_proofpoint = True
break
break
except Exception:
result.issues.append("Could not retrieve SPF record")
if result.spf_record and not result.spf_includes_proofpoint:
result.issues.append(
"SPF record does not include Proofpoint. "
"Add: include:spf-a.proofpoint.com"
)
# Check DMARC
try:
dmarc_domain = f"_dmarc.{domain}"
answers = dns.resolver.resolve(dmarc_domain, 'TXT')
for rdata in answers:
txt = str(rdata).strip('"')
if txt.startswith('v=DMARC1'):
result.dmarc_record = txt
policy_match = re.search(r'p=(\w+)', txt)
if policy_match:
result.dmarc_policy = policy_match.group(1)
break
except Exception:
result.issues.append("No DMARC record found")
if result.dmarc_policy == "none":
result.issues.append(
"DMARC policy is set to 'none' (monitoring only). "
"Plan rollout to 'quarantine' then 'reject'"
)
# Check Proofpoint DKIM selector
try:
dkim_domain = f"proofpoint._domainkey.{domain}"
answers = dns.resolver.resolve(dkim_domain, 'TXT')
for rdata in answers:
result.dkim_selector = "proofpoint"
break
except Exception:
pass
return result
def analyze_headers(eml_content: str) -> HeaderAnalysis:
"""Analyze email headers for Proofpoint routing and authentication."""
analysis = HeaderAnalysis()
# Extract From header
from_match = re.search(r'^From:\s*(.+)$', eml_content, re.MULTILINE | re.IGNORECASE)
if from_match:
analysis.from_address = from_match.group(1).strip()
# Extract Return-Path
rp_match = re.search(r'^Return-Path:\s*(.+)$', eml_content, re.MULTILINE | re.IGNORECASE)
if rp_match:
analysis.return_path = rp_match.group(1).strip()
# Extract Received chain
received_headers = re.findall(
r'^Received:\s*(.*?)(?=\n\S|\nReceived:|\n\n)',
eml_content, re.MULTILINE | re.DOTALL | re.IGNORECASE
)
for hdr in received_headers:
clean = ' '.join(hdr.split())
analysis.received_chain.append(clean)
if any(p.replace(r'$', '').replace(r'\.', '.') in clean.lower()
for p in ['pphosted.com', 'proofpoint.com']):
analysis.passed_through_proofpoint = True
# Extract Authentication-Results
auth_match = re.search(
r'^Authentication-Results:\s*(.*?)(?=\n\S)',
eml_content, re.MULTILINE | re.DOTALL | re.IGNORECASE
)
if auth_match:
auth_text = auth_match.group(1)
spf_match = re.search(r'spf=(\w+)', auth_text)
if spf_match:
analysis.spf_result = spf_match.group(1)
dkim_match = re.search(r'dkim=(\w+)', auth_text)
if dkim_match:
analysis.dkim_result = dkim_match.group(1)
dmarc_match = re.search(r'dmarc=(\w+)', auth_text)
if dmarc_match:
analysis.dmarc_result = dmarc_match.group(1)
# Check for Proofpoint-specific headers
for marker in PROOFPOINT_HEADER_MARKERS:
marker_match = re.search(
rf'^{re.escape(marker)}:\s*(.+)$',
eml_content, re.MULTILINE | re.IGNORECASE
)
if marker_match:
analysis.proofpoint_headers.append({
"header": marker,
"value": marker_match.group(1).strip()
})
if marker == 'X-Proofpoint-Spam-Details':
analysis.x_proofpoint_spam_details = marker_match.group(1).strip()
elif marker == 'X-Proofpoint-Virus-Version':
analysis.x_proofpoint_virus_version = marker_match.group(1).strip()
if not analysis.passed_through_proofpoint and not analysis.proofpoint_headers:
analysis.issues.append("Email does not appear to have routed through Proofpoint")
if analysis.spf_result and analysis.spf_result != 'pass':
analysis.issues.append(f"SPF check failed: {analysis.spf_result}")
if analysis.dkim_result and analysis.dkim_result != 'pass':
analysis.issues.append(f"DKIM check failed: {analysis.dkim_result}")
if analysis.dmarc_result and analysis.dmarc_result != 'pass':
analysis.issues.append(f"DMARC check failed: {analysis.dmarc_result}")
return analysis
def format_report(title: str, data: dict) -> str:
"""Format check results as a readable report."""
lines = []
lines.append("=" * 60)
lines.append(f" {title}")
lines.append("=" * 60)
for key, value in data.items():
if key == 'issues':
if value:
lines.append(f"\n [ISSUES]")
for i, issue in enumerate(value, 1):
lines.append(f" {i}. {issue}")
elif isinstance(value, list):
lines.append(f"\n {key}:")
for item in value:
if isinstance(item, dict):
lines.append(f" - {json.dumps(item)}")
else:
lines.append(f" - {item}")
elif isinstance(value, bool):
status = "YES" if value else "NO"
lines.append(f" {key}: {status}")
else:
lines.append(f" {key}: {value}")
lines.append("=" * 60)
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(description="Proofpoint Email Gateway Validator")
subparsers = parser.add_subparsers(dest="command")
mx_parser = subparsers.add_parser("check-mx", help="Check MX records for Proofpoint routing")
mx_parser.add_argument("--domain", required=True, help="Domain to check")
auth_parser = subparsers.add_parser("check-auth", help="Check email authentication records")
auth_parser.add_argument("--domain", required=True, help="Domain to check")
hdr_parser = subparsers.add_parser("validate-headers", help="Analyze email headers")
hdr_parser.add_argument("--eml-file", required=True, help="Path to .eml file")
parser.add_argument("--json", action="store_true", help="Output as JSON")
args = parser.parse_args()
if args.command == "check-mx":
result = check_mx_records(args.domain)
if args.json:
print(json.dumps(asdict(result), indent=2))
else:
print(format_report("PROOFPOINT MX RECORD CHECK", asdict(result)))
elif args.command == "check-auth":
result = check_authentication(args.domain)
if args.json:
print(json.dumps(asdict(result), indent=2))
else:
print(format_report("EMAIL AUTHENTICATION CHECK", asdict(result)))
elif args.command == "validate-headers":
with open(args.eml_file, 'r', errors='replace') as f:
content = f.read()
result = analyze_headers(content)
if args.json:
print(json.dumps(asdict(result), indent=2))
else:
print(format_report("EMAIL HEADER ANALYSIS", asdict(result)))
else:
parser.print_help()
if __name__ == "__main__":
main()