Initial commit - 611 cybersecurity skills across all subdomains

2026-07-14 03:15:16 +03:00 · 2026-02-25 10:47:44 +01:00
commit 22a7ab1462
1765 changed files with 280648 additions and 0 deletions
@@ -0,0 +1,105 @@
+---
+name: performing-open-source-intelligence-gathering
+description: Open Source Intelligence (OSINT) gathering is the first active phase of a red team engagement, where operators collect publicly available information about the target organization to identify attack s
+domain: cybersecurity
+subdomain: red-teaming
+tags: [red-team, adversary-simulation, mitre-attack, exploitation, post-exploitation, osint, reconnaissance]
+version: "1.0"
+author: mahipal
+license: MIT
+---
+# Performing Open Source Intelligence Gathering
+
+## Overview
+
+Open Source Intelligence (OSINT) gathering is the first active phase of a red team engagement, where operators collect publicly available information about the target organization to identify attack surfaces, potential targets for social engineering, technology stacks, and credential exposures. Effective OSINT directly shapes initial access strategies and reduces operational risk.
+
+## Objectives
+
+- Enumerate the target organization's external attack surface (domains, IPs, cloud assets)
+- Identify employees and their roles for social engineering targeting
+- Discover leaked credentials, API keys, and sensitive documents
+- Map the organization's technology stack and vendors
+- Identify physical locations, office layouts, and access control details
+- Build target profiles for spearphishing campaign development
+
+## Core Concepts
+
+### OSINT Categories
+
+| Category | Sources | Value |
+|----------|---------|-------|
+| Domain Intelligence | DNS records, WHOIS, CT logs, subdomain enumeration | Network attack surface |
+| Personnel Intelligence | LinkedIn, social media, conference talks, publications | Social engineering targets |
+| Credential Intelligence | Breach databases, paste sites, GitHub leaks | Valid credential discovery |
+| Technology Intelligence | Job postings, Wappalyzer, Shodan, Censys | Vulnerability identification |
+| Physical Intelligence | Google Maps, social media photos, Glassdoor | Physical access planning |
+| Document Intelligence | SEC filings, public documents, metadata extraction | Organizational structure |
+
+### MITRE ATT&CK Mapping
+
+- **T1595.001** - Active Scanning: Scanning IP Blocks
+- **T1595.002** - Active Scanning: Vulnerability Scanning
+- **T1592** - Gather Victim Host Information
+- **T1589** - Gather Victim Identity Information
+- **T1590** - Gather Victim Network Information
+- **T1591** - Gather Victim Org Information
+- **T1593** - Search Open Websites/Domains
+- **T1594** - Search Victim-Owned Websites
+- **T1596** - Search Open Technical Databases
+
+## Implementation Steps
+
+### Phase 1: Domain and Network Reconnaissance
+1. Perform WHOIS lookups for target domains
+2. Enumerate subdomains using Certificate Transparency logs, DNS brute-force, and web scraping
+3. Identify IP ranges and ASN ownership
+4. Scan for exposed services using Shodan/Censys
+5. Check for cloud storage buckets (S3, Azure Blob, GCS)
+6. Map CDN and hosting providers
+
+### Phase 2: Personnel and Social Intelligence
+1. Enumerate employees via LinkedIn, company website, and conference speaker lists
+2. Identify email naming conventions
+3. Discover personal social media accounts of key targets
+4. Map organizational hierarchy and reporting structure
+5. Identify recently hired IT/security personnel
+6. Check for conference presentations and technical publications
+
+### Phase 3: Credential and Data Leak Discovery
+1. Search breach databases (Have I Been Pwned, DeHashed)
+2. Check paste sites (Pastebin, GitHub Gists)
+3. Search GitHub/GitLab for leaked secrets and API keys
+4. Look for exposed configuration files and backups
+5. Check for leaked internal documents via Google dorking
+
+### Phase 4: Technology Stack Identification
+1. Analyze job postings for technology mentions
+2. Use Wappalyzer/BuiltWith for web technology fingerprinting
+3. Check for exposed admin panels and development environments
+4. Identify VPN and remote access technologies
+5. Map cloud services and SaaS applications
+
+## Tools and Resources
+
+| Tool | Purpose | Type |
+|------|---------|------|
+| Amass | Subdomain enumeration and network mapping | Open Source |
+| Subfinder | Passive subdomain discovery | Open Source |
+| theHarvester | Email, subdomain, and name harvesting | Open Source |
+| Maltego | Visual link analysis and data correlation | Commercial |
+| SpiderFoot | Automated OSINT collection | Open Source |
+| Shodan | Internet-connected device search | Commercial |
+| Censys | Internet asset discovery | Commercial |
+| Recon-ng | Web reconnaissance framework | Open Source |
+| GitDorker | GitHub secret scanning | Open Source |
+| Photon | Web crawler for OSINT | Open Source |
+
+## Validation Criteria
+
+- [ ] Complete list of target domains and subdomains
+- [ ] Employee list with roles and email addresses
+- [ ] Technology stack identified
+- [ ] Credential leak assessment completed
+- [ ] Attack surface map documented
+- [ ] OSINT report compiled for engagement team
@@ -0,0 +1,189 @@
+# OSINT Collection Report Template
+
+## Document Control
+
+| Field | Value |
+|-------|-------|
+| Target Organization | [NAME] |
+| Target Domain(s) | [DOMAINS] |
+| Engagement ID | [ID] |
+| Collector | [NAME] |
+| Date | [DATE] |
+| Classification | CONFIDENTIAL |
+
+---
+
+## 1. Executive Summary
+
+[Brief overview of OSINT findings and their impact on attack planning]
+
+**Key Findings:**
+- [Finding 1]
+- [Finding 2]
+- [Finding 3]
+
+**Recommended Attack Vectors:**
+1. [Vector 1 based on OSINT]
+2. [Vector 2 based on OSINT]
+
+---
+
+## 2. External Attack Surface
+
+### 2.1 Domain Inventory
+
+| Domain | Registrar | Nameservers | Expiry Date |
+|--------|-----------|-------------|-------------|
+| | | | |
+
+### 2.2 Subdomain Enumeration
+
+| Subdomain | IP Address | Service | Status |
+|-----------|-----------|---------|--------|
+| | | | Active/Inactive |
+
+### 2.3 IP Range and ASN
+
+| ASN | Organization | IP Range | Country |
+|-----|-------------|----------|---------|
+| | | | |
+
+### 2.4 Cloud Assets
+
+| Provider | Asset Type | Identifier | Public Access |
+|----------|-----------|------------|---------------|
+| AWS | S3 Bucket | | Yes/No |
+| Azure | Blob Storage | | Yes/No |
+| GCP | Storage | | Yes/No |
+
+---
+
+## 3. Personnel Intelligence
+
+### 3.1 Key Personnel
+
+| Name | Title | Department | LinkedIn | Email |
+|------|-------|------------|----------|-------|
+| | | | | |
+
+### 3.2 Email Format
+- **Confirmed Format:** [first.last@domain.com]
+- **Validation Method:** [How confirmed]
+
+### 3.3 Social Engineering Targets
+
+| Target | Role | Justification | Risk Level |
+|--------|------|---------------|------------|
+| | | | High/Medium/Low |
+
+---
+
+## 4. Technology Stack
+
+### 4.1 Web Technologies
+
+| Component | Technology | Version | Notes |
+|-----------|-----------|---------|-------|
+| Web Server | | | |
+| Framework | | | |
+| CMS | | | |
+| CDN | | | |
+| WAF | | | |
+
+### 4.2 Security Posture
+
+| Security Header | Status | Rating |
+|----------------|--------|--------|
+| HSTS | Present/Missing | |
+| CSP | Present/Missing | |
+| X-Frame-Options | Present/Missing | |
+| X-Content-Type-Options | Present/Missing | |
+
+### 4.3 Infrastructure
+
+| Service | Product | Version | Port |
+|---------|---------|---------|------|
+| VPN | | | |
+| Email | | | |
+| Remote Access | | | |
+
+---
+
+## 5. Credential Exposure
+
+### 5.1 Breach Database Results
+
+| Source | Date | Records | Type |
+|--------|------|---------|------|
+| | | | Passwords/Hashes/Emails |
+
+### 5.2 Code Repository Leaks
+
+| Repository | File | Type of Secret | Status |
+|-----------|------|----------------|--------|
+| | | API Key/Password/Token | Active/Rotated |
+
+### 5.3 Paste Site Findings
+
+| Site | Date | Content Type | Relevance |
+|------|------|-------------|-----------|
+| | | | High/Medium/Low |
+
+---
+
+## 6. Physical Intelligence
+
+### 6.1 Office Locations
+
+| Address | Type | Access Controls | Notes |
+|---------|------|-----------------|-------|
+| | HQ/Branch/DC | | |
+
+### 6.2 Physical Security Observations
+
+| Observation | Location | Photo Reference |
+|-------------|----------|-----------------|
+| | | |
+
+---
+
+## 7. Attack Surface Summary
+
+### 7.1 Priority Targets
+
+| # | Target | Type | Rationale | Risk |
+|---|--------|------|-----------|------|
+| 1 | | Technical/Human/Physical | | Critical/High/Medium |
+
+### 7.2 Recommended Initial Access Methods
+
+| Method | MITRE ATT&CK | Likelihood | Required Resources |
+|--------|-------------|------------|-------------------|
+| Spearphishing | T1566.001 | | |
+| Credential Stuffing | T1078 | | |
+| External Exploit | T1190 | | |
+
+---
+
+## Appendix A: Raw Data Files
+
+| File | Description | Location |
+|------|-------------|----------|
+| subdomains.txt | Full subdomain list | |
+| emails.txt | Discovered email addresses | |
+| dorks.txt | Google dorking queries | |
+| tech_fingerprint.json | Technology details | |
+
+## Appendix B: Tools Used
+
+| Tool | Version | Purpose |
+|------|---------|---------|
+| | | |
+
+## Appendix C: MITRE ATT&CK Techniques Used
+
+| Technique ID | Name | Result |
+|-------------|------|--------|
+| T1593.002 | Search Engines | |
+| T1596.005 | Scan Databases | |
+| T1589.002 | Email Addresses | |
@@ -0,0 +1,67 @@
+# Standards and Framework References
+
+## MITRE ATT&CK - Reconnaissance (TA0043)
+
+| Technique ID | Name | Description |
+|-------------|------|-------------|
+| T1595.001 | Active Scanning: Scanning IP Blocks | Scanning target IP ranges for active hosts |
+| T1595.002 | Active Scanning: Vulnerability Scanning | Scanning for vulnerabilities on discovered hosts |
+| T1592.001 | Gather Victim Host Information: Hardware | Identifying target hardware configurations |
+| T1592.002 | Gather Victim Host Information: Software | Identifying installed software and versions |
+| T1592.004 | Gather Victim Host Information: Client Configurations | Discovering client-side configurations |
+| T1589.001 | Gather Victim Identity Information: Credentials | Searching for exposed credentials |
+| T1589.002 | Gather Victim Identity Information: Email Addresses | Harvesting email addresses |
+| T1589.003 | Gather Victim Identity Information: Employee Names | Collecting employee information |
+| T1590.001 | Gather Victim Network Information: Domain Properties | DNS and domain enumeration |
+| T1590.002 | Gather Victim Network Information: DNS | DNS record collection |
+| T1590.004 | Gather Victim Network Information: Network Topology | Mapping network architecture |
+| T1590.005 | Gather Victim Network Information: IP Addresses | Identifying target IP addresses |
+| T1591.001 | Gather Victim Org Information: Determine Physical Locations | Physical location mapping |
+| T1591.002 | Gather Victim Org Information: Business Relationships | Identifying vendors and partners |
+| T1591.004 | Gather Victim Org Information: Identify Roles | Mapping organizational roles |
+| T1593.001 | Search Open Websites/Domains: Social Media | Social media intelligence |
+| T1593.002 | Search Open Websites/Domains: Search Engines | Google dorking and search engine recon |
+| T1594 | Search Victim-Owned Websites | Analyzing target websites |
+| T1596.001 | Search Open Technical Databases: DNS/Passive DNS | Passive DNS intelligence |
+| T1596.005 | Search Open Technical Databases: Scan Databases | Shodan, Censys, ZoomEye queries |
+| T1597.001 | Search Closed Sources: Threat Intel Vendors | Threat intelligence platform queries |
+
+## PTES - Intelligence Gathering
+
+### Level 1: Passive Information Gathering
+- WHOIS lookups
+- DNS enumeration
+- Search engine queries
+- Social media analysis
+- Public records review
+
+### Level 2: Semi-Passive Information Gathering
+- Website analysis and spidering
+- Metadata extraction from documents
+- Job posting analysis
+- Technology stack identification
+
+### Level 3: Active Information Gathering
+- Port scanning
+- Service enumeration
+- Web application fingerprinting
+- Active subdomain brute-forcing
+
+## OSSTMM - Information Security Testing
+
+### Section 5: Human Security Testing
+- Social engineering reconnaissance
+- Personnel profiling
+- Communication channel mapping
+
+### Section 6: Physical Security Testing
+- Location reconnaissance
+- Access control assessment
+- Surveillance analysis
+
+## NIST SP 800-115 Section 3: Review Techniques
+- Documentation review
+- Log review
+- Ruleset review
+- System configuration review
+- Network sniffing (passive)
@@ -0,0 +1,253 @@
+# OSINT Gathering Workflows
+
+## Workflow 1: Domain and Infrastructure Reconnaissance
+
+### Step 1: Passive DNS and WHOIS
+```bash
+# WHOIS lookup
+whois targetdomain.com
+
+# DNS record enumeration
+dig targetdomain.com ANY
+dig targetdomain.com MX
+dig targetdomain.com TXT
+dig targetdomain.com NS
+
+# Reverse DNS
+dig -x <IP_ADDRESS>
+
+# Zone transfer attempt
+dig axfr @ns1.targetdomain.com targetdomain.com
+```
+
+### Step 2: Subdomain Enumeration
+```bash
+# Using Subfinder for passive enumeration
+subfinder -d targetdomain.com -o subdomains.txt
+
+# Using Amass for comprehensive enumeration
+amass enum -passive -d targetdomain.com -o amass_results.txt
+
+# Certificate Transparency log search
+curl -s "https://crt.sh/?q=%.targetdomain.com&output=json" | jq -r '.[].name_value' | sort -u
+
+# Using httpx to probe discovered subdomains
+cat subdomains.txt | httpx -status-code -title -tech-detect -o live_subdomains.txt
+```
+
+### Step 3: IP Range and ASN Discovery
+```bash
+# ASN lookup
+whois -h whois.radb.net -- '-i origin AS12345'
+
+# BGP prefix lookup via Hurricane Electric
+curl -s "https://bgp.he.net/AS12345#_prefixes"
+
+# Shodan search for organization
+shodan search "org:Target Corporation" --fields ip_str,port,product
+```
+
+### Step 4: Cloud Asset Discovery
+```bash
+# AWS S3 bucket enumeration
+python3 cloud_enum.py -k targetcorp -l cloud_results.txt
+
+# Azure blob storage check
+for name in targetcorp targetcorp-dev targetcorp-backup; do
+  curl -s -o /dev/null -w "%{http_code}" "https://${name}.blob.core.windows.net/"
+done
+
+# GCP bucket check
+gsutil ls gs://targetcorp-*
+```
+
+## Workflow 2: Personnel Intelligence
+
+### Step 1: Employee Enumeration
+```bash
+# theHarvester for email and name harvesting
+theHarvester -d targetdomain.com -b all -l 500 -f harvest_results
+
+# LinkedIn enumeration (manual + tools)
+# Use LinkedIn search operators:
+# site:linkedin.com/in "targetcorp" "security engineer"
+# site:linkedin.com/in "targetcorp" "system administrator"
+
+# CrossLinked for LinkedIn name harvesting
+python3 crosslinked.py -f '{first}.{last}@targetdomain.com' "Target Corporation"
+```
+
+### Step 2: Email Validation
+```bash
+# Verify email format using Hunter.io API
+curl "https://api.hunter.io/v2/domain-search?domain=targetdomain.com&api_key=YOUR_KEY"
+
+# SMTP verification (careful - can be logged)
+# Use tools like EmailHippo or NeverBounce for passive verification
+```
+
+### Step 3: Social Media Profiling
+```bash
+# Sherlock for username enumeration across platforms
+python3 sherlock username --timeout 5 --output sherlock_results.txt
+
+# Social media searching
+# Twitter advanced search: from:username targetcorp
+# Instagram: #targetcorp
+# GitHub: org:targetcorp
+```
+
+## Workflow 3: Credential and Data Leak Discovery
+
+### Step 1: Breach Database Search
+```bash
+# Have I Been Pwned API check
+curl "https://haveibeenpwned.com/api/v3/breachedaccount/user@targetdomain.com" \
+  -H "hibp-api-key: YOUR_KEY"
+
+# DeHashed search (requires subscription)
+curl "https://api.dehashed.com/search?query=domain:targetdomain.com" \
+  -u email:api_key
+```
+
+### Step 2: GitHub Secret Scanning
+```bash
+# GitDorker for GitHub dorking
+python3 GitDorker.py -tf tokens.txt -d dorks/alldorksv3 -q targetdomain.com
+
+# truffleHog for repository scanning
+trufflehog github --org=targetcorp --only-verified
+
+# Manual GitHub dorking
+# Search: "targetdomain.com" password
+# Search: "targetdomain.com" api_key
+# Search: "targetcorp" filename:.env
+# Search: "targetcorp" filename:wp-config.php
+```
+
+### Step 3: Google Dorking
+```
+# Sensitive files
+site:targetdomain.com filetype:pdf
+site:targetdomain.com filetype:xlsx
+site:targetdomain.com filetype:docx confidential
+
+# Configuration files
+site:targetdomain.com filetype:xml
+site:targetdomain.com filetype:conf
+site:targetdomain.com filetype:env
+
+# Login pages and admin panels
+site:targetdomain.com inurl:admin
+site:targetdomain.com inurl:login
+site:targetdomain.com intitle:"index of"
+
+# Error messages with sensitive info
+site:targetdomain.com "error" "sql" "syntax"
+site:targetdomain.com "php error" "on line"
+```
+
+## Workflow 4: Technology Stack Identification
+
+### Step 1: Web Technology Fingerprinting
+```bash
+# Wappalyzer CLI
+wappalyzer https://targetdomain.com
+
+# WhatWeb for technology identification
+whatweb targetdomain.com -v
+
+# Nuclei for technology detection
+nuclei -u https://targetdomain.com -t technologies/
+```
+
+### Step 2: Service and Version Detection
+```bash
+# Nmap service detection (active - requires authorization)
+nmap -sV -sC -p- targetdomain.com -oA nmap_results
+
+# Shodan host lookup
+shodan host <IP_ADDRESS>
+
+# Censys host search
+censys search "services.tls.certificates.leaf_data.subject.organization:Target Corp"
+```
+
+### Step 3: Job Posting Analysis
+```
+# Search job boards for technology mentions:
+# LinkedIn Jobs: "Target Corporation" AND ("AWS" OR "Azure" OR "GCP")
+# Indeed: "Target Corporation" "security" tools
+# Glassdoor: Target Corporation technology stack
+
+# Look for mentions of:
+# - Cloud platforms (AWS, Azure, GCP)
+# - Security tools (CrowdStrike, Carbon Black, Splunk)
+# - Development languages and frameworks
+# - Network equipment vendors (Cisco, Palo Alto, Fortinet)
+# - Identity providers (Okta, Azure AD, Ping Identity)
+```
+
+## Workflow 5: Physical Intelligence
+
+### Step 1: Location Mapping
+```
+# Google Maps reconnaissance:
+# - Office locations and building layouts
+# - Parking areas and entry points
+# - Nearby businesses for staging
+# - Delivery entrance locations
+
+# Google Street View:
+# - Access control systems (card readers, turnstiles)
+# - Security camera locations
+# - Badge/lanyard colors and designs
+# - Building signage
+```
+
+### Step 2: Document Metadata Extraction
+```bash
+# ExifTool for document metadata
+exiftool -r -ext pdf -ext docx -ext xlsx ./downloaded_documents/
+
+# FOCA for metadata analysis (Windows)
+# Import documents and analyze:
+# - Author names and usernames
+# - Software versions
+# - Internal file paths
+# - Printer names and network paths
+```
+
+## Workflow 6: OSINT Report Compilation
+
+### Report Structure
+```
+1. Executive Summary
+   - Key findings overview
+   - Risk assessment
+
+2. Attack Surface Map
+   - External infrastructure diagram
+   - Domain and subdomain inventory
+   - Exposed services and applications
+
+3. Personnel Intelligence
+   - Key personnel profiles
+   - Email address list
+   - Organizational chart
+
+4. Credential Exposure
+   - Breach database findings
+   - Leaked secrets and API keys
+   - Password pattern analysis
+
+5. Technology Stack
+   - Identified technologies and versions
+   - Known vulnerabilities for detected versions
+   - Security tool coverage gaps
+
+6. Recommended Attack Vectors
+   - Prioritized initial access options
+   - Social engineering target list
+   - Technical vulnerability targets
+```
@@ -0,0 +1,604 @@
+#!/usr/bin/env python3
+"""
+OSINT Gathering Automation Tool
+
+Performs automated open source intelligence collection including:
+- Subdomain enumeration via Certificate Transparency logs
+- DNS record collection
+- WHOIS information gathering
+- Technology fingerprinting
+- Google dorking query generation
+- Email pattern discovery
+- Shodan/Censys integration
+
+Usage:
+    python process.py --domain targetdomain.com --output ./osint_report
+    python process.py --domain targetdomain.com --modules all
+    python process.py --domain targetdomain.com --modules dns,subdomains,emails
+
+Requirements:
+    pip install requests dnspython whois beautifulsoup4 rich
+"""
+
+import argparse
+import json
+import re
+import socket
+import sys
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+from urllib.parse import urlparse
+
+try:
+    import dns.resolver
+    import requests
+    from bs4 import BeautifulSoup
+    from rich.console import Console
+    from rich.table import Table
+    from rich.panel import Panel
+    from rich.progress import Progress, SpinnerColumn, TextColumn
+except ImportError:
+    print("[!] Missing dependencies. Install with:")
+    print("    pip install requests dnspython beautifulsoup4 rich")
+    sys.exit(1)
+
+console = Console()
+SESSION = requests.Session()
+SESSION.headers.update(
+    {
+        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
+    }
+)
+
+
+def resolve_dns_records(domain: str) -> dict:
+    """Collect DNS records for a domain."""
+    records = {}
+    record_types = ["A", "AAAA", "MX", "NS", "TXT", "SOA", "CNAME", "SRV"]
+
+    for rtype in record_types:
+        try:
+            answers = dns.resolver.resolve(domain, rtype)
+            records[rtype] = [str(rdata) for rdata in answers]
+        except (dns.resolver.NoAnswer, dns.resolver.NXDOMAIN, dns.resolver.NoNameservers):
+            pass
+        except Exception:
+            pass
+
+    return records
+
+
+def enumerate_subdomains_ct(domain: str) -> list[str]:
+    """Enumerate subdomains using Certificate Transparency logs via crt.sh."""
+    subdomains = set()
+
+    try:
+        url = f"https://crt.sh/?q=%.{domain}&output=json"
+        response = SESSION.get(url, timeout=30)
+        if response.status_code == 200:
+            data = response.json()
+            for entry in data:
+                name_value = entry.get("name_value", "")
+                for name in name_value.split("\n"):
+                    name = name.strip().lower()
+                    if name.endswith(f".{domain}") or name == domain:
+                        # Skip wildcard entries
+                        if not name.startswith("*"):
+                            subdomains.add(name)
+    except Exception as e:
+        console.print(f"[yellow][!] crt.sh query failed: {e}[/yellow]")
+
+    # Also try common subdomain prefixes
+    common_prefixes = [
+        "www", "mail", "ftp", "smtp", "pop", "imap", "webmail",
+        "vpn", "remote", "portal", "admin", "dev", "staging",
+        "test", "api", "app", "blog", "shop", "store", "cdn",
+        "ns1", "ns2", "dns", "mx", "exchange", "owa", "autodiscover",
+        "sso", "login", "auth", "git", "gitlab", "jenkins",
+        "jira", "confluence", "wiki", "docs", "support", "help",
+    ]
+
+    for prefix in common_prefixes:
+        subdomain = f"{prefix}.{domain}"
+        try:
+            dns.resolver.resolve(subdomain, "A")
+            subdomains.add(subdomain)
+        except Exception:
+            pass
+
+    return sorted(subdomains)
+
+
+def perform_whois_lookup(domain: str) -> dict:
+    """Perform WHOIS lookup for a domain."""
+    try:
+        import whois as python_whois
+        w = python_whois.whois(domain)
+        result = {
+            "domain_name": str(w.domain_name) if w.domain_name else "N/A",
+            "registrar": str(w.registrar) if w.registrar else "N/A",
+            "creation_date": str(w.creation_date) if w.creation_date else "N/A",
+            "expiration_date": str(w.expiration_date) if w.expiration_date else "N/A",
+            "name_servers": w.name_servers if w.name_servers else [],
+            "registrant_org": str(w.org) if w.org else "N/A",
+            "registrant_country": str(w.country) if w.country else "N/A",
+            "emails": w.emails if w.emails else [],
+        }
+        return result
+    except Exception as e:
+        console.print(f"[yellow][!] WHOIS lookup failed: {e}[/yellow]")
+        return {}
+
+
+def discover_email_format(domain: str) -> dict:
+    """Attempt to discover email format patterns using Hunter.io free tier."""
+    result = {
+        "domain": domain,
+        "format_guess": [],
+        "discovered_emails": [],
+    }
+
+    # Common email format patterns
+    common_formats = [
+        "{first}.{last}@" + domain,
+        "{first}{last}@" + domain,
+        "{f}{last}@" + domain,
+        "{first}_{last}@" + domain,
+        "{first}@" + domain,
+        "{last}@" + domain,
+    ]
+    result["format_guess"] = common_formats
+
+    # Try to discover emails from web pages
+    try:
+        response = SESSION.get(f"https://{domain}", timeout=10)
+        if response.status_code == 200:
+            # Extract emails from page content
+            email_pattern = re.compile(
+                rf"[a-zA-Z0-9._%+-]+@{re.escape(domain)}",
+                re.IGNORECASE
+            )
+            emails = email_pattern.findall(response.text)
+            result["discovered_emails"] = list(set(emails))
+    except Exception:
+        pass
+
+    # Search for emails on common pages
+    common_pages = ["/contact", "/about", "/team", "/about-us", "/contact-us"]
+    for page in common_pages:
+        try:
+            response = SESSION.get(f"https://{domain}{page}", timeout=10)
+            if response.status_code == 200:
+                email_pattern = re.compile(
+                    rf"[a-zA-Z0-9._%+-]+@{re.escape(domain)}",
+                    re.IGNORECASE
+                )
+                emails = email_pattern.findall(response.text)
+                result["discovered_emails"].extend(emails)
+        except Exception:
+            pass
+
+    result["discovered_emails"] = list(set(result["discovered_emails"]))
+    return result
+
+
+def fingerprint_web_technologies(domain: str) -> dict:
+    """Identify web technologies using HTTP headers and response analysis."""
+    tech = {
+        "server": None,
+        "powered_by": None,
+        "frameworks": [],
+        "security_headers": {},
+        "cookies": [],
+        "cdn": None,
+        "cms": None,
+    }
+
+    try:
+        response = SESSION.get(f"https://{domain}", timeout=10, allow_redirects=True)
+        headers = response.headers
+
+        # Server identification
+        tech["server"] = headers.get("Server", "Not disclosed")
+        tech["powered_by"] = headers.get("X-Powered-By", "Not disclosed")
+
+        # Security headers
+        security_headers = [
+            "Strict-Transport-Security",
+            "Content-Security-Policy",
+            "X-Frame-Options",
+            "X-Content-Type-Options",
+            "X-XSS-Protection",
+            "Referrer-Policy",
+            "Permissions-Policy",
+            "Cross-Origin-Opener-Policy",
+            "Cross-Origin-Resource-Policy",
+        ]
+
+        for header in security_headers:
+            value = headers.get(header)
+            tech["security_headers"][header] = value if value else "MISSING"
+
+        # Cookie analysis
+        for cookie in response.cookies:
+            tech["cookies"].append(
+                {
+                    "name": cookie.name,
+                    "secure": cookie.secure,
+                    "httponly": "httponly" in cookie._rest,
+                    "domain": cookie.domain,
+                }
+            )
+
+        # CDN detection
+        cdn_indicators = {
+            "cloudflare": ["cf-ray", "cf-cache-status"],
+            "akamai": ["x-akamai-transformed"],
+            "cloudfront": ["x-amz-cf-id", "x-amz-cf-pop"],
+            "fastly": ["x-served-by", "x-fastly-request-id"],
+            "incapsula": ["x-iinfo"],
+        }
+
+        for cdn, indicators in cdn_indicators.items():
+            for indicator in indicators:
+                if indicator in [h.lower() for h in headers]:
+                    tech["cdn"] = cdn
+                    break
+
+        # CMS detection from HTML
+        html = response.text.lower()
+        if "wp-content" in html or "wordpress" in html:
+            tech["cms"] = "WordPress"
+        elif "drupal" in html:
+            tech["cms"] = "Drupal"
+        elif "joomla" in html:
+            tech["cms"] = "Joomla"
+        elif "shopify" in html:
+            tech["cms"] = "Shopify"
+
+        # Framework detection
+        if "react" in html or "reactdom" in html:
+            tech["frameworks"].append("React")
+        if "angular" in html or "ng-" in html:
+            tech["frameworks"].append("Angular")
+        if "vue" in html or "vuejs" in html:
+            tech["frameworks"].append("Vue.js")
+        if "jquery" in html:
+            tech["frameworks"].append("jQuery")
+        if "bootstrap" in html:
+            tech["frameworks"].append("Bootstrap")
+
+    except Exception as e:
+        console.print(f"[yellow][!] Web fingerprinting failed: {e}[/yellow]")
+
+    return tech
+
+
+def generate_google_dorks(domain: str) -> list[str]:
+    """Generate Google dorking queries for the target domain."""
+    dorks = [
+        # Sensitive files
+        f'site:{domain} filetype:pdf',
+        f'site:{domain} filetype:xlsx',
+        f'site:{domain} filetype:docx',
+        f'site:{domain} filetype:csv',
+        f'site:{domain} filetype:sql',
+        f'site:{domain} filetype:log',
+        f'site:{domain} filetype:bak',
+        f'site:{domain} filetype:conf',
+        f'site:{domain} filetype:env',
+        f'site:{domain} filetype:xml',
+        # Configuration and credentials
+        f'site:{domain} inurl:admin',
+        f'site:{domain} inurl:login',
+        f'site:{domain} inurl:wp-admin',
+        f'site:{domain} inurl:wp-login',
+        f'site:{domain} intitle:"index of"',
+        f'site:{domain} intitle:"dashboard"',
+        f'site:{domain} inurl:config',
+        f'site:{domain} inurl:setup',
+        # Error messages
+        f'site:{domain} "error" "sql syntax"',
+        f'site:{domain} "php error" "on line"',
+        f'site:{domain} "ORA-" "error"',
+        f'site:{domain} "mysql" "error"',
+        f'site:{domain} "stack trace" "at"',
+        # Sensitive information
+        f'site:{domain} "confidential"',
+        f'site:{domain} "internal use only"',
+        f'site:{domain} "not for distribution"',
+        f'site:{domain} "password" filetype:txt',
+        f'site:{domain} "api_key" OR "apikey" OR "api-key"',
+        # Infrastructure
+        f'site:{domain} inurl:vpn',
+        f'site:{domain} inurl:remote',
+        f'site:{domain} inurl:portal',
+        f'site:{domain} inurl:citrix',
+        f'site:{domain} inurl:owa',
+        # GitHub leaks
+        f'"{domain}" password site:github.com',
+        f'"{domain}" api_key site:github.com',
+        f'"{domain}" secret site:github.com',
+        f'"{domain}" token site:github.com',
+        f'"{domain}" site:pastebin.com',
+        # Cloud storage
+        f'site:s3.amazonaws.com "{domain.split(".")[0]}"',
+        f'site:blob.core.windows.net "{domain.split(".")[0]}"',
+        f'site:storage.googleapis.com "{domain.split(".")[0]}"',
+    ]
+    return dorks
+
+
+def check_security_txt(domain: str) -> dict | None:
+    """Check for security.txt file per RFC 9116."""
+    urls = [
+        f"https://{domain}/.well-known/security.txt",
+        f"https://{domain}/security.txt",
+    ]
+
+    for url in urls:
+        try:
+            response = SESSION.get(url, timeout=10)
+            if response.status_code == 200 and "contact" in response.text.lower():
+                return {
+                    "url": url,
+                    "content": response.text[:2000],
+                }
+        except Exception:
+            pass
+
+    return None
+
+
+def check_robots_txt(domain: str) -> dict | None:
+    """Check robots.txt for interesting paths."""
+    try:
+        response = SESSION.get(f"https://{domain}/robots.txt", timeout=10)
+        if response.status_code == 200:
+            disallowed = []
+            for line in response.text.split("\n"):
+                if line.strip().lower().startswith("disallow:"):
+                    path = line.split(":", 1)[1].strip()
+                    if path and path != "/":
+                        disallowed.append(path)
+            return {
+                "content": response.text[:2000],
+                "disallowed_paths": disallowed,
+            }
+    except Exception:
+        pass
+    return None
+
+
+def generate_report(domain: str, results: dict, output_dir: Path):
+    """Generate comprehensive OSINT report."""
+    report = f"""# OSINT Report: {domain}
+## Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
+
+---
+
+## 1. Domain Information
+
+### WHOIS Data
+"""
+
+    whois_data = results.get("whois", {})
+    if whois_data:
+        for key, value in whois_data.items():
+            report += f"- **{key}**: {value}\n"
+    else:
+        report += "WHOIS data not available.\n"
+
+    report += "\n### DNS Records\n\n"
+    dns_data = results.get("dns", {})
+    if dns_data:
+        for rtype, records in dns_data.items():
+            report += f"#### {rtype} Records\n"
+            for record in records:
+                report += f"- `{record}`\n"
+            report += "\n"
+
+    report += "## 2. Subdomain Enumeration\n\n"
+    subdomains = results.get("subdomains", [])
+    report += f"**Total subdomains discovered:** {len(subdomains)}\n\n"
+    for sub in subdomains:
+        try:
+            ip = socket.gethostbyname(sub)
+            report += f"- `{sub}` -> `{ip}`\n"
+        except Exception:
+            report += f"- `{sub}` -> [unresolvable]\n"
+
+    report += "\n## 3. Web Technology Fingerprint\n\n"
+    tech = results.get("technology", {})
+    if tech:
+        report += f"- **Server:** {tech.get('server', 'N/A')}\n"
+        report += f"- **Powered By:** {tech.get('powered_by', 'N/A')}\n"
+        report += f"- **CMS:** {tech.get('cms', 'None detected')}\n"
+        report += f"- **CDN:** {tech.get('cdn', 'None detected')}\n"
+        report += f"- **Frameworks:** {', '.join(tech.get('frameworks', [])) or 'None detected'}\n\n"
+
+        report += "### Security Headers\n\n"
+        report += "| Header | Status |\n|--------|--------|\n"
+        for header, value in tech.get("security_headers", {}).items():
+            status = "MISSING" if value == "MISSING" else "Present"
+            report += f"| {header} | {status} |\n"
+
+    report += "\n## 4. Email Intelligence\n\n"
+    email_data = results.get("emails", {})
+    if email_data:
+        report += "### Discovered Emails\n"
+        for email in email_data.get("discovered_emails", []):
+            report += f"- `{email}`\n"
+        report += "\n### Likely Email Formats\n"
+        for fmt in email_data.get("format_guess", []):
+            report += f"- `{fmt}`\n"
+
+    report += "\n## 5. Google Dorking Queries\n\n"
+    dorks = results.get("dorks", [])
+    for dork in dorks[:20]:
+        report += f"- `{dork}`\n"
+
+    report += "\n## 6. Additional Findings\n\n"
+    security_txt = results.get("security_txt")
+    if security_txt:
+        report += f"### security.txt Found\n- URL: {security_txt['url']}\n\n"
+
+    robots_txt = results.get("robots_txt")
+    if robots_txt:
+        report += "### Interesting robots.txt Paths\n"
+        for path in robots_txt.get("disallowed_paths", []):
+            report += f"- `{path}`\n"
+
+    report += f"""
+---
+
+## 7. Recommendations for Attack Planning
+
+### Priority Initial Access Vectors
+1. Review discovered subdomains for vulnerable web applications
+2. Validate credential leaks against target systems
+3. Use discovered email format for spearphishing campaign
+4. Investigate missing security headers for potential exploitation
+5. Check disallowed paths from robots.txt for sensitive content
+
+### Social Engineering Targets
+- Use discovered personnel for targeted phishing
+- Leverage technology stack knowledge for pretexting
+- Utilize physical location data for physical access testing
+
+---
+
+*Report generated by OSINT Automation Tool*
+*Classification: CONFIDENTIAL - Red Team Use Only*
+"""
+
+    report_path = output_dir / f"osint_report_{domain}.md"
+    with open(report_path, "w") as f:
+        f.write(report)
+
+    console.print(f"[green][+] Report saved to: {report_path}[/green]")
+
+    # Save raw data as JSON
+    json_path = output_dir / f"osint_data_{domain}.json"
+    serializable_results = {}
+    for key, value in results.items():
+        try:
+            json.dumps(value)
+            serializable_results[key] = value
+        except (TypeError, ValueError):
+            serializable_results[key] = str(value)
+
+    with open(json_path, "w") as f:
+        json.dump(serializable_results, f, indent=2, default=str)
+
+    console.print(f"[green][+] Raw data saved to: {json_path}[/green]")
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="OSINT Gathering Automation Tool"
+    )
+    parser.add_argument("--domain", required=True, help="Target domain")
+    parser.add_argument(
+        "--output", default="./osint_output", help="Output directory"
+    )
+    parser.add_argument(
+        "--modules",
+        default="all",
+        help="Comma-separated modules: dns,subdomains,whois,emails,tech,dorks,all",
+    )
+
+    args = parser.parse_args()
+    output_dir = Path(args.output)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    modules = args.modules.split(",") if args.modules != "all" else [
+        "dns", "subdomains", "whois", "emails", "tech", "dorks", "security_txt", "robots_txt"
+    ]
+
+    console.print(
+        Panel(
+            f"[bold red]OSINT Gathering Tool[/bold red]\n"
+            f"Target: {args.domain}\n"
+            f"Modules: {', '.join(modules)}\n"
+            f"Output: {args.output}",
+            title="Configuration",
+        )
+    )
+
+    results = {}
+
+    with Progress(
+        SpinnerColumn(),
+        TextColumn("[progress.description]{task.description}"),
+        console=console,
+    ) as progress:
+
+        if "dns" in modules:
+            task = progress.add_task("[cyan]Collecting DNS records...", total=None)
+            results["dns"] = resolve_dns_records(args.domain)
+            progress.update(task, completed=True, description="[green]DNS records collected")
+
+        if "subdomains" in modules:
+            task = progress.add_task("[cyan]Enumerating subdomains...", total=None)
+            results["subdomains"] = enumerate_subdomains_ct(args.domain)
+            progress.update(
+                task,
+                completed=True,
+                description=f"[green]Found {len(results['subdomains'])} subdomains",
+            )
+
+        if "whois" in modules:
+            task = progress.add_task("[cyan]Performing WHOIS lookup...", total=None)
+            results["whois"] = perform_whois_lookup(args.domain)
+            progress.update(task, completed=True, description="[green]WHOIS data collected")
+
+        if "emails" in modules:
+            task = progress.add_task("[cyan]Discovering email formats...", total=None)
+            results["emails"] = discover_email_format(args.domain)
+            progress.update(task, completed=True, description="[green]Email discovery complete")
+
+        if "tech" in modules:
+            task = progress.add_task("[cyan]Fingerprinting technologies...", total=None)
+            results["technology"] = fingerprint_web_technologies(args.domain)
+            progress.update(task, completed=True, description="[green]Technology fingerprint complete")
+
+        if "dorks" in modules:
+            task = progress.add_task("[cyan]Generating Google dorks...", total=None)
+            results["dorks"] = generate_google_dorks(args.domain)
+            progress.update(
+                task,
+                completed=True,
+                description=f"[green]Generated {len(results['dorks'])} dork queries",
+            )
+
+        if "security_txt" in modules:
+            task = progress.add_task("[cyan]Checking security.txt...", total=None)
+            results["security_txt"] = check_security_txt(args.domain)
+            progress.update(task, completed=True, description="[green]security.txt check complete")
+
+        if "robots_txt" in modules:
+            task = progress.add_task("[cyan]Checking robots.txt...", total=None)
+            results["robots_txt"] = check_robots_txt(args.domain)
+            progress.update(task, completed=True, description="[green]robots.txt check complete")
+
+    generate_report(args.domain, results, output_dir)
+
+    # Display summary table
+    table = Table(title=f"OSINT Summary: {args.domain}")
+    table.add_column("Category", style="cyan")
+    table.add_column("Count/Status", style="green")
+
+    table.add_row("DNS Record Types", str(len(results.get("dns", {}))))
+    table.add_row("Subdomains Found", str(len(results.get("subdomains", []))))
+    table.add_row("Emails Discovered", str(len(results.get("emails", {}).get("discovered_emails", []))))
+    table.add_row("Google Dorks Generated", str(len(results.get("dorks", []))))
+    table.add_row("security.txt", "Found" if results.get("security_txt") else "Not Found")
+    table.add_row("robots.txt", "Found" if results.get("robots_txt") else "Not Found")
+
+    console.print(table)
+
+
+if __name__ == "__main__":
+    main()