Initial commit - 611 cybersecurity skills across all subdomains

2026-07-15 04:05:17 +03:00 · 2026-02-25 10:47:44 +01:00
commit 22a7ab1462
1765 changed files with 280648 additions and 0 deletions
@@ -0,0 +1,57 @@
+---
+name: implementing-end-to-end-encryption-for-messaging
+description: End-to-end encryption (E2EE) ensures that only the communicating parties can read messages, with no intermediary (including the server) able to decrypt them. This skill implements a simplified version
+domain: cybersecurity
+subdomain: cryptography
+tags: [cryptography, encryption, e2e, messaging, signal-protocol]
+version: "1.0"
+author: mahipal
+license: MIT
+---
+# Implementing End-to-End Encryption for Messaging
+
+## Overview
+
+End-to-end encryption (E2EE) ensures that only the communicating parties can read messages, with no intermediary (including the server) able to decrypt them. This skill implements a simplified version of the Signal Protocol's Double Ratchet algorithm, using X25519 for key exchange, HKDF for key derivation, and AES-256-GCM for message encryption.
+
+## Objectives
+
+- Implement X25519 Diffie-Hellman key exchange for session establishment
+- Build the Double Ratchet key management algorithm
+- Encrypt and decrypt messages with per-message keys
+- Implement forward secrecy (compromise of current key does not reveal past messages)
+- Handle out-of-order message delivery
+- Implement key agreement using X3DH (Extended Triple Diffie-Hellman)
+
+## Key Concepts
+
+### Signal Protocol Components
+
+| Component | Purpose | Algorithm |
+|-----------|---------|-----------|
+| X3DH | Initial key agreement | X25519 |
+| Double Ratchet | Ongoing key management | X25519 + HKDF + AES-GCM |
+| Sending Chain | Per-message encryption keys | HMAC-SHA256 chain |
+| Receiving Chain | Per-message decryption keys | HMAC-SHA256 chain |
+| Root Chain | Derives new chain keys on DH ratchet | HKDF |
+
+### Forward Secrecy
+
+Each message uses a unique encryption key derived from a ratcheting chain. After a key is used, it is deleted, ensuring that compromise of the current state does not reveal previously sent/received messages.
+
+## Security Considerations
+
+- Delete message keys immediately after decryption
+- Implement message ordering and replay protection
+- Use authenticated encryption (AES-GCM) for all messages
+- Protect identity keys with device-level security
+- Verify identity keys out-of-band (safety numbers)
+
+## Validation Criteria
+
+- [ ] X25519 key exchange produces shared secret
+- [ ] Messages encrypt and decrypt correctly between two parties
+- [ ] Different messages produce different ciphertexts
+- [ ] Forward secrecy: old keys cannot decrypt new messages
+- [ ] Out-of-order messages can be decrypted
+- [ ] Tampered messages are rejected by authentication
@@ -0,0 +1,41 @@
+# E2E Encryption for Messaging Template
+
+## Protocol Summary
+
+| Phase | Algorithm | Purpose |
+|-------|-----------|---------|
+| Key Exchange | X3DH (X25519) | Initial shared secret |
+| Key Ratchet | Double Ratchet | Per-message key derivation |
+| Encryption | AES-256-GCM | Message confidentiality + integrity |
+| Key Derivation | HKDF-SHA256 | Derive keys from DH outputs |
+| Chain KDF | HMAC-SHA256 | Advance symmetric ratchet |
+
+## Security Properties Checklist
+
+- [ ] Forward secrecy: Past messages safe if current keys compromised
+- [ ] Post-compromise security: Recovery after temporary key compromise
+- [ ] Deniability: No cryptographic proof of message authorship
+- [ ] Authenticated encryption: Tampered messages detected and rejected
+- [ ] Replay protection: Message counters prevent replay attacks
+- [ ] Out-of-order delivery: Skipped keys stored for late messages
+
+## Message Format
+
+```
+[Header (40 bytes)]
+  - DH Public Key: 32 bytes
+  - Previous Chain Length: 4 bytes (big-endian)
+  - Message Number: 4 bytes (big-endian)
+
+[Encrypted Payload]
+  - Nonce: 12 bytes
+  - Ciphertext + Tag: variable
+```
+
+## Integration Notes
+
+- Identity keys should be stored in secure device storage (Keychain, TEE)
+- Implement safety number verification for identity key comparison
+- Handle device changes by re-running X3DH with new identity keys
+- Store skipped message keys with a maximum limit (e.g., 1000)
+- Delete message keys immediately after successful decryption
@@ -0,0 +1,40 @@
+# Standards and References - End-to-End Encryption for Messaging
+
+## Signal Protocol Specifications
+
+### The Double Ratchet Algorithm
+- **URL**: https://signal.org/docs/specifications/doubleratchet/
+- **Description**: Core key management algorithm for E2EE messaging
+
+### The X3DH Key Agreement Protocol
+- **URL**: https://signal.org/docs/specifications/x3dh/
+- **Description**: Initial key exchange using Extended Triple Diffie-Hellman
+
+### The Sesame Algorithm
+- **URL**: https://signal.org/docs/specifications/sesame/
+- **Description**: Multi-device session management
+
+## Cryptographic Standards
+
+### RFC 7748 - Elliptic Curves for Security (X25519)
+- **URL**: https://www.rfc-editor.org/rfc/rfc7748
+- **Description**: X25519 Diffie-Hellman key exchange
+
+### RFC 5869 - HKDF (HMAC-based Key Derivation Function)
+- **URL**: https://www.rfc-editor.org/rfc/rfc5869
+- **Description**: Key derivation for chain key updates
+
+### RFC 8032 - Edwards-Curve Digital Signature Algorithm (Ed25519)
+- **URL**: https://www.rfc-editor.org/rfc/rfc8032
+- **Description**: Identity key signatures
+
+### NIST SP 800-38D - AES-GCM
+- **URL**: https://csrc.nist.gov/publications/detail/sp/800-38d/final
+- **Description**: Authenticated encryption for messages
+
+## Python Libraries
+
+### cryptography
+- **X25519**: `cryptography.hazmat.primitives.asymmetric.x25519`
+- **HKDF**: `cryptography.hazmat.primitives.kdf.hkdf`
+- **AES-GCM**: `cryptography.hazmat.primitives.ciphers.aead.AESGCM`
@@ -0,0 +1,87 @@
+# Workflows - End-to-End Encryption for Messaging
+
+## Workflow 1: X3DH Key Agreement
+
+```
+Alice (initiator)                 Server                  Bob (responder)
+  |                                 |                         |
+  |                                 |<-- Register:            |
+  |                                 |    Identity Key (IK_B)  |
+  |                                 |    Signed PreKey (SPK_B)|
+  |                                 |    One-Time PreKeys     |
+  |                                 |                         |
+  |-- Fetch Bob's Keys ----------->|                         |
+  |<-- IK_B, SPK_B, OPK_B --------|                         |
+  |                                 |                         |
+  [Compute shared secret]:                                    |
+  DH1 = DH(IK_A, SPK_B)                                     |
+  DH2 = DH(EK_A, IK_B)                                      |
+  DH3 = DH(EK_A, SPK_B)                                     |
+  DH4 = DH(EK_A, OPK_B)                                     |
+  SK = HKDF(DH1 || DH2 || DH3 || DH4)                      |
+  |                                 |                         |
+  |-- Send Initial Message ------->|-- Forward to Bob ------>|
+  |   (IK_A, EK_A, OPK_id, msg)   |                         |
+  |                                 |   [Bob computes same SK]|
+```
+
+## Workflow 2: Double Ratchet (Sending)
+
+```
+[Message to Send]
+      |
+[Check: Do we have recipient's new DH public key?]
+  YES --> [DH Ratchet Step]
+          - Generate new DH key pair
+          - Compute DH shared secret
+          - Derive new root key + sending chain key via HKDF
+  NO  --> [Continue with current sending chain]
+      |
+[Symmetric Ratchet: Derive message key from sending chain]
+(chain_key, message_key) = HMAC(chain_key, constants)
+      |
+[Encrypt message with AES-256-GCM using message_key]
+      |
+[Include header: DH public key, previous chain length, message number]
+      |
+[Delete message_key from memory]
+```
+
+## Workflow 3: Double Ratchet (Receiving)
+
+```
+[Received Encrypted Message + Header]
+      |
+[Check DH public key in header]
+  [New key?]
+    YES --> [DH Ratchet Step]
+            - Compute DH shared secret
+            - Derive new root key + receiving chain key
+    NO  --> [Use current receiving chain]
+      |
+[Symmetric Ratchet: Derive message key]
+      |
+[Decrypt message with AES-256-GCM]
+      |
+[Verify authentication tag]
+  FAIL --> Reject message
+  PASS --> Return plaintext
+      |
+[Delete message_key from memory]
+```
+
+## Workflow 4: Session Lifecycle
+
+```
+[Initial Contact] --> [X3DH Key Exchange]
+                            |
+                      [Initialize Double Ratchet]
+                            |
+                      [Exchange Messages]
+                      (DH ratchet + symmetric ratchet)
+                            |
+                      [Periodic DH Ratchet]
+                      (every N messages or on reply)
+                            |
+                      [Session End / Archive]
+```
@@ -0,0 +1,358 @@
+#!/usr/bin/env python3
+"""
+End-to-End Encryption for Messaging (Simplified Double Ratchet)
+
+Implements a simplified version of the Signal Protocol's Double Ratchet
+algorithm using X25519 key exchange, HKDF key derivation, and AES-256-GCM.
+
+Requirements:
+    pip install cryptography
+
+Usage:
+    python process.py demo
+    python process.py benchmark --messages 1000
+"""
+
+import os
+import sys
+import json
+import time
+import struct
+import hashlib
+import argparse
+import logging
+from dataclasses import dataclass, field
+from typing import Dict, Optional, Tuple, List
+
+from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey, X25519PublicKey
+from cryptography.hazmat.primitives.kdf.hkdf import HKDF
+from cryptography.hazmat.primitives.ciphers.aead import AESGCM
+from cryptography.hazmat.primitives import hashes, hmac, serialization
+from cryptography.hazmat.backends import default_backend
+
+logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
+logger = logging.getLogger(__name__)
+
+INFO_ROOT_KEY = b"DoubleRatchetRootKey"
+INFO_CHAIN_KEY = b"DoubleRatchetChainKey"
+CHAIN_KEY_CONSTANT = b"\x01"
+MESSAGE_KEY_CONSTANT = b"\x02"
+
+
+def generate_x25519_keypair() -> Tuple[X25519PrivateKey, bytes]:
+    """Generate an X25519 key pair, returning (private_key, public_key_bytes)."""
+    private_key = X25519PrivateKey.generate()
+    public_bytes = private_key.public_key().public_bytes(
+        serialization.Encoding.Raw, serialization.PublicFormat.Raw
+    )
+    return private_key, public_bytes
+
+
+def dh(private_key: X25519PrivateKey, public_key_bytes: bytes) -> bytes:
+    """Perform X25519 Diffie-Hellman key exchange."""
+    public_key = X25519PublicKey.from_public_bytes(public_key_bytes)
+    return private_key.exchange(public_key)
+
+
+def hkdf_derive(input_key: bytes, info: bytes, length: int = 64) -> bytes:
+    """Derive key material using HKDF-SHA256."""
+    derived = HKDF(
+        algorithm=hashes.SHA256(),
+        length=length,
+        salt=b"\x00" * 32,
+        info=info,
+        backend=default_backend(),
+    ).derive(input_key)
+    return derived
+
+
+def hmac_derive(key: bytes, constant: bytes) -> bytes:
+    """Derive a key using HMAC-SHA256."""
+    h = hmac.HMAC(key, hashes.SHA256(), backend=default_backend())
+    h.update(constant)
+    return h.finalize()
+
+
+def kdf_rk(root_key: bytes, dh_output: bytes) -> Tuple[bytes, bytes]:
+    """Root key KDF: derive new root key and chain key from DH output."""
+    derived = hkdf_derive(dh_output + root_key, INFO_ROOT_KEY, 64)
+    new_root_key = derived[:32]
+    new_chain_key = derived[32:]
+    return new_root_key, new_chain_key
+
+
+def kdf_ck(chain_key: bytes) -> Tuple[bytes, bytes]:
+    """Chain key KDF: derive next chain key and message key."""
+    new_chain_key = hmac_derive(chain_key, CHAIN_KEY_CONSTANT)
+    message_key = hmac_derive(chain_key, MESSAGE_KEY_CONSTANT)
+    return new_chain_key, message_key
+
+
+def encrypt_message(message_key: bytes, plaintext: bytes, associated_data: bytes = b"") -> bytes:
+    """Encrypt a message using AES-256-GCM."""
+    nonce = os.urandom(12)
+    aesgcm = AESGCM(message_key)
+    ciphertext = aesgcm.encrypt(nonce, plaintext, associated_data)
+    return nonce + ciphertext
+
+
+def decrypt_message(message_key: bytes, data: bytes, associated_data: bytes = b"") -> bytes:
+    """Decrypt a message using AES-256-GCM."""
+    nonce = data[:12]
+    ciphertext = data[12:]
+    aesgcm = AESGCM(message_key)
+    return aesgcm.decrypt(nonce, ciphertext, associated_data)
+
+
+@dataclass
+class MessageHeader:
+    """Header included with each encrypted message."""
+    dh_public_key: bytes
+    previous_chain_length: int
+    message_number: int
+
+    def serialize(self) -> bytes:
+        return (
+            self.dh_public_key
+            + struct.pack(">II", self.previous_chain_length, self.message_number)
+        )
+
+    @classmethod
+    def deserialize(cls, data: bytes) -> "MessageHeader":
+        dh_public_key = data[:32]
+        prev_chain_len, msg_num = struct.unpack(">II", data[32:40])
+        return cls(dh_public_key, prev_chain_len, msg_num)
+
+
+@dataclass
+class DoubleRatchetState:
+    """State for one side of the Double Ratchet."""
+    dh_self_private: Optional[X25519PrivateKey] = None
+    dh_self_public: bytes = b""
+    dh_remote_public: bytes = b""
+    root_key: bytes = b""
+    sending_chain_key: Optional[bytes] = None
+    receiving_chain_key: Optional[bytes] = None
+    send_count: int = 0
+    recv_count: int = 0
+    previous_send_count: int = 0
+    skipped_keys: Dict[Tuple[bytes, int], bytes] = field(default_factory=dict)
+    max_skip: int = 100
+
+
+def initialize_alice(shared_secret: bytes, bob_dh_public: bytes) -> DoubleRatchetState:
+    """Initialize the ratchet for Alice (initiator)."""
+    state = DoubleRatchetState()
+    state.dh_remote_public = bob_dh_public
+
+    state.dh_self_private, state.dh_self_public = generate_x25519_keypair()
+
+    dh_output = dh(state.dh_self_private, bob_dh_public)
+    state.root_key, state.sending_chain_key = kdf_rk(shared_secret, dh_output)
+    state.receiving_chain_key = None
+    state.send_count = 0
+    state.recv_count = 0
+    state.previous_send_count = 0
+
+    return state
+
+
+def initialize_bob(shared_secret: bytes, bob_dh_keypair: Tuple[X25519PrivateKey, bytes]) -> DoubleRatchetState:
+    """Initialize the ratchet for Bob (responder)."""
+    state = DoubleRatchetState()
+    state.dh_self_private = bob_dh_keypair[0]
+    state.dh_self_public = bob_dh_keypair[1]
+    state.root_key = shared_secret
+    state.sending_chain_key = None
+    state.receiving_chain_key = None
+    state.send_count = 0
+    state.recv_count = 0
+    state.previous_send_count = 0
+
+    return state
+
+
+def ratchet_encrypt(state: DoubleRatchetState, plaintext: bytes) -> Tuple[MessageHeader, bytes]:
+    """Encrypt a message using the Double Ratchet."""
+    state.sending_chain_key, message_key = kdf_ck(state.sending_chain_key)
+
+    header = MessageHeader(
+        dh_public_key=state.dh_self_public,
+        previous_chain_length=state.previous_send_count,
+        message_number=state.send_count,
+    )
+
+    ciphertext = encrypt_message(message_key, plaintext, header.serialize())
+    state.send_count += 1
+
+    return header, ciphertext
+
+
+def dh_ratchet_step(state: DoubleRatchetState, header: MessageHeader):
+    """Perform a DH ratchet step when receiving a new public key."""
+    state.previous_send_count = state.send_count
+    state.send_count = 0
+    state.recv_count = 0
+    state.dh_remote_public = header.dh_public_key
+
+    dh_recv = dh(state.dh_self_private, state.dh_remote_public)
+    state.root_key, state.receiving_chain_key = kdf_rk(state.root_key, dh_recv)
+
+    state.dh_self_private, state.dh_self_public = generate_x25519_keypair()
+
+    dh_send = dh(state.dh_self_private, state.dh_remote_public)
+    state.root_key, state.sending_chain_key = kdf_rk(state.root_key, dh_send)
+
+
+def skip_message_keys(state: DoubleRatchetState, until: int):
+    """Skip and store message keys for out-of-order messages."""
+    if state.receiving_chain_key is None:
+        return
+    while state.recv_count < until:
+        state.receiving_chain_key, mk = kdf_ck(state.receiving_chain_key)
+        state.skipped_keys[(state.dh_remote_public, state.recv_count)] = mk
+        state.recv_count += 1
+        if len(state.skipped_keys) > state.max_skip:
+            oldest = next(iter(state.skipped_keys))
+            del state.skipped_keys[oldest]
+
+
+def ratchet_decrypt(state: DoubleRatchetState, header: MessageHeader, ciphertext: bytes) -> bytes:
+    """Decrypt a message using the Double Ratchet."""
+    # Check skipped keys
+    skip_key = (header.dh_public_key, header.message_number)
+    if skip_key in state.skipped_keys:
+        mk = state.skipped_keys.pop(skip_key)
+        return decrypt_message(mk, ciphertext, header.serialize())
+
+    # DH ratchet step if new public key
+    if header.dh_public_key != state.dh_remote_public:
+        if state.receiving_chain_key is not None:
+            skip_message_keys(state, header.previous_chain_length)
+        dh_ratchet_step(state, header)
+
+    skip_message_keys(state, header.message_number)
+    state.receiving_chain_key, message_key = kdf_ck(state.receiving_chain_key)
+    state.recv_count += 1
+
+    return decrypt_message(message_key, ciphertext, header.serialize())
+
+
+def demo_conversation():
+    """Demonstrate a complete E2EE conversation."""
+    print("=== End-to-End Encryption Demo ===\n")
+
+    # Simulate X3DH: both parties derive the same shared secret
+    alice_ik_private, alice_ik_public = generate_x25519_keypair()
+    bob_ik_private, bob_ik_public = generate_x25519_keypair()
+    bob_spk_private, bob_spk_public = generate_x25519_keypair()
+    alice_ek_private, alice_ek_public = generate_x25519_keypair()
+
+    # Alice computes shared secret
+    dh1 = dh(alice_ik_private, bob_spk_public)
+    dh2 = dh(alice_ek_private, bob_ik_public)
+    dh3 = dh(alice_ek_private, bob_spk_public)
+    alice_shared = hkdf_derive(dh1 + dh2 + dh3, b"X3DH", 32)
+
+    # Bob computes same shared secret
+    bob_ik_pub_obj = X25519PublicKey.from_public_bytes(alice_ik_public)
+    bob_ek_pub_obj = X25519PublicKey.from_public_bytes(alice_ek_public)
+    dh1_b = bob_spk_private.exchange(bob_ik_pub_obj)
+    dh2_b = bob_ik_private.exchange(bob_ek_pub_obj)
+    dh3_b = bob_spk_private.exchange(bob_ek_pub_obj)
+    bob_shared = hkdf_derive(dh1_b + dh2_b + dh3_b, b"X3DH", 32)
+
+    assert alice_shared == bob_shared, "X3DH shared secret mismatch!"
+    print("[OK] X3DH key agreement: shared secrets match\n")
+
+    # Initialize Double Ratchet
+    bob_dh_private, bob_dh_public = generate_x25519_keypair()
+    alice_state = initialize_alice(alice_shared, bob_dh_public)
+    bob_state = initialize_bob(bob_shared, (bob_dh_private, bob_dh_public))
+
+    # Alice sends messages to Bob
+    messages = [
+        b"Hello Bob! This is an encrypted message.",
+        b"Can you read this?",
+        b"This uses the Double Ratchet algorithm.",
+    ]
+
+    print("--- Alice sends to Bob ---")
+    for msg in messages:
+        header, ct = ratchet_encrypt(alice_state, msg)
+        pt = ratchet_decrypt(bob_state, header, ct)
+        print(f"  Alice -> Bob: {pt.decode()}")
+        assert pt == msg
+
+    # Bob replies to Alice
+    replies = [
+        b"Hi Alice! Yes, I can read your messages.",
+        b"The DH ratchet just advanced!",
+    ]
+
+    print("\n--- Bob sends to Alice ---")
+    for msg in replies:
+        header, ct = ratchet_encrypt(bob_state, msg)
+        pt = ratchet_decrypt(alice_state, header, ct)
+        print(f"  Bob -> Alice: {pt.decode()}")
+        assert pt == msg
+
+    # Alice sends again (another DH ratchet)
+    print("\n--- Alice sends again (new DH ratchet) ---")
+    msg = b"This message uses a new DH ratchet step."
+    header, ct = ratchet_encrypt(alice_state, msg)
+    pt = ratchet_decrypt(bob_state, header, ct)
+    print(f"  Alice -> Bob: {pt.decode()}")
+    assert pt == msg
+
+    print("\n[OK] All messages encrypted and decrypted successfully")
+    print("[OK] Forward secrecy: each message uses a unique key")
+    print("[OK] DH ratchet advanced on direction change")
+
+
+def benchmark(num_messages: int = 1000):
+    """Benchmark encryption/decryption throughput."""
+    alice_ik_private, _ = generate_x25519_keypair()
+    bob_spk_private, bob_spk_public = generate_x25519_keypair()
+    shared = dh(alice_ik_private, bob_spk_public)
+    shared_key = hkdf_derive(shared, b"benchmark", 32)
+
+    bob_dh_private, bob_dh_public = generate_x25519_keypair()
+    alice_state = initialize_alice(shared_key, bob_dh_public)
+    bob_state = initialize_bob(shared_key, (bob_dh_private, bob_dh_public))
+
+    message = b"Benchmark message for throughput testing. " * 10
+
+    start = time.time()
+    for _ in range(num_messages):
+        header, ct = ratchet_encrypt(alice_state, message)
+        pt = ratchet_decrypt(bob_state, header, ct)
+    elapsed = time.time() - start
+
+    print(f"Messages: {num_messages}")
+    print(f"Time: {elapsed:.3f}s")
+    print(f"Throughput: {num_messages / elapsed:.0f} msg/s")
+    print(f"Latency: {elapsed / num_messages * 1000:.2f} ms/msg")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="E2EE Messaging Demo")
+    subparsers = parser.add_subparsers(dest="command")
+
+    subparsers.add_parser("demo", help="Run E2EE conversation demo")
+
+    bench = subparsers.add_parser("benchmark", help="Benchmark throughput")
+    bench.add_argument("--messages", type=int, default=1000, help="Number of messages")
+
+    args = parser.parse_args()
+
+    if args.command == "demo":
+        demo_conversation()
+    elif args.command == "benchmark":
+        benchmark(args.messages)
+    else:
+        parser.print_help()
+
+
+if __name__ == "__main__":
+    main()