# Project Architecture: AI-Enhanced CTF Toolchain

This document describes the architecture, directory layout, core modules, testing setup, and planned components of the CTF Toolchain project.

---

## 1. Directory Structure

The project follows a standard modern Python layout (utilizing `src/` directory layout) and is managed via the `uv` toolchain.

```text
├── config.toml           # Mock/Default configuration for local testing
├── pyproject.toml        # Hatchling build configuration & project dependencies
├── uv.lock               # uv lockfile for exact dependency resolution
├── src/
│   └── ctf/
│       ├── __init__.py   # Module initializer
│       ├── main.py       # CLI Entry Point
│       ├── commands.py   # CLI Commands and action functions
│       ├── utils.py      # Core utility functions (file parsing, config, paths)
│       └── forensics.py  # Forensics analysis tools
└── tests/
    ├── env/              # Sandboxed, persistent test environment directories
    └── test_utils.py     # Unit/Integration tests for utility functions
```

---

## 2. Core Modules & Configuration

### A. Configuration & State ([config.toml](file:///home/venus/code/ctf/config.toml))
The toolchain requires configuration of directories and active challenges:
*   `Competition`: Keeps track of the current `producer`, `competition`, `category`, and `challenge`.
*   `Environment`: Declares the base directory (`ctf_dir`) where all CTF files are stored.

### B. Utilities ([utils.py](file:///home/venus/code/ctf/src/ctf/utils.py))
Provides helper functions for filesystem management and configuration parsing:
*   `load_config(path)`: Loads and parses configuration data from a TOML file.
*   `write_config(data, path)`: Serializes/updates a dictionary to a TOML file.
*   `active_categories(path)`: Iterates over a competition path to return all active categories.
*   `active_competitions(dir)`: Scans the base directory for active competitions, skipping designated helper directories (like `tools`).
*   `suggest_patterns(flag, comp_name)`: Analyzes a provided sample flag string and generates a prioritized, deduplicated list of regular expression pattern candidates, optionally incorporating the competition name.

### C. Commands ([commands.py](file:///home/venus/code/ctf/src/ctf/commands.py))
Houses the logic for generic CLI context and active competition commands:
*   `Set_Challenge(comp, chal, setDirectory)`: Sets the current active challenge/competition context. *(Note: Currently has a `NameError` due to reference to an undefined `state` object.)*
*   `set-competition (name)`: Sets the name of the active competition in configuration.
*   `set-flag-format (pattern)`: Sets the flag regex format in configuration. Supports optional `PATTERN` positional argument or an interactive choice using `-o`/`--original` which takes a sample flag and prompts the user to select from suggested regex patterns, warning if the competition name is not found in the sample.


### D. Forensics ([forensics.py](file:///home/venus/code/ctf/src/ctf/forensics.py))
Implements specialized forensic inspection utilities registered as a nested subgroup under the CLI:
*   **Custom Extractor**: Implements our own pure-Python, zero-dependency parser for standard TIFF/EXIF tags, Adobe XMP XML blocks, Photoshop IPTC IIM records, and JPEG/PNG physical parameters, rather than importing large external libraries.
    > [!NOTE]
    > We designed and implemented our own custom EXIF scraper/parser from scratch in pure Python rather than importing an external library (like `exifread` or `Pillow`) to maintain zero dependencies and keep the toolchain highly lightweight and customizable.
*   `info`: Inspects target file sizes, reads magic bytes, and warns if extensions do not match detected signatures.
*   `flag-search`: Extracts printable string sequences (equivalent to GNU `strings`) and matches them against regular expression patterns to find potential flags.

---

## 3. CLI Entry Point ([main.py](file:///home/venus/code/ctf/src/ctf/main.py))

*   Serves as the central CLI entry point via the `main()` function.
*   Initializes the root Click `cli` group and registers nested sub-groups, such as `forensics_group`.

---

## 4. Test Infrastructure

*   **Framework**: `pytest` (run via `uv run pytest`).
*   **Sandbox**: [tests/env](file:///home/venus/code/ctf/tests/env) acts as a persistent mock directory tree containing temporary competition directories (like `comp1`, `comp2`) and mock files (e.g. valid PNGs, mismatching PDFs, text files with flag payloads) to safely test scanning, parsing, and CLI command execution without touching actual user data.

---

## 5. Planned Architectural Components

### A. Forensics Pipeline & Extended Metadata
*   **Pipeline Coordination**: A central runner that coordinates file inspections, signature verification, and string searches sequentially on target files.
*   **Carving & Extraction Toolkit**: Deep inspection and carving utilities (such as archive extracting, custom chunk/payload carvers, and EXIF/metadata header decoders).
*   **Chepy Machine Learning**: Plan to integrate the `chepy_ml` plugin later to perform automated heuristics, entropy analysis, and machine-learning-based format detection on unknown metadata payloads.
*   **Targeted Forensic Metadata Parsing**: Implement decoders/parsers for high-signal forensics structures:
    *   *PNG*: `tEXt`, `zTXt`, `iTXt` metadata chunks.
    *   *PDF*: `/Info` dictionaries and `/Metadata` XML streams.
    *   *GIF*: Comment extensions and application loop extensions.
    *   *ELF/PE*: Executable header offsets, architectures, and section headers.
    *   *MP3*: ID3v1 and ID3v2 tag blocks (Title, Artist, custom Comments).

### B. Core Solving Infrastructure & QOL
*   **Solving Assistants**: Standard templates and modular helper scripts for repeating CTF patterns (e.g., base conversions, cipher decoding, and request generation).
*   **Context & Progress Management**: Full CLI commands to organize active challenges, automate note tracking, and manage workspace folders dynamically.
*   **Per-Challenge Progress Logging**: A logging interface that creates separate progress logs for each challenge inside a configurable directory, tracking actions, milestones, attempts, and command history.

### C. Agentic Solving Orchestration
*   **Action Space & Sandboxing**: API boundaries and isolated docker/sandbox execution paths to run unknown commands and binaries safely.
*   **Autonomous Solving Agent**: Integration with LLM controllers that parse output from the forensics pipeline, suggest plans, iterate on solving steps, and verify flag captures.

---

## 6. CLI Data Flow & Presentation Guidelines

### A. Model-View Separation
All CLI command modules (such as [forensics.py](file:///home/venus/code/ctf/src/ctf/forensics.py)) must separate data extraction logic from command-line rendering.
*   **Data Models**: Standard Python `@dataclass` objects should be defined to house parsed metadata (e.g., file size, magic bytes, detected types, warnings, and format-specific attributes).
*   **Decoupled Parsers**: Extraction helper functions must return these dataclass instances instead of printing directly to standard output. This keeps the core parser functions purely functional and fully testable in unit tests.

### B. Console Rendering with `rich`
To provide a clean, modern, and easily readable console output without building a full terminal user interface (TUI):
*   **Tables**: Use `rich.table.Table` to align and structure multi-column metadata outputs.
*   **Formatting & Alerts**: Utilize `rich.console` or `rich.panel.Panel` to highlight warnings (such as signature/extension mismatches) with distinct styling and colors.
*   **JSON Serialization**: Dataclasses should be easily convertible to dictionaries to support raw JSON output options for scripting pipelines.

### C. Function Isolation & Pipeline Composition
To support fully automated or agentic solving, all processing functions must be isolated and highly composable:
*   **Pure Functions**: Core analysis and transformation functions (e.g., file carving, cipher decoders, and metadata extraction) must remain pure. They must accept standard Python types (e.g., `Path`, `bytes`, `str`) and return structured dataclasses without depending on global CLI state, click contexts, or interactive user prompts.
*   **Pipeline Chaining**: The output of one tool must serve as valid input for another. For example, a list of files or byte arrays carved from a file-carver must be pipeable directly into the string parser or decompression module.
*   **Programmatic API**: The entire core logic must be importable as standard Python APIs (`import ctf.utils`, `import ctf.forensics`) separate from CLI bindings. This allows external scripts, test cases, or autonomous agents to programmatically chain functions together to build complex, self-contained solving pipelines.
