Merge branch 'main' into fix_70

README.md

@@ -20,7 +20,6 @@ Support Aperi'Solve:
   <img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" height="40" />
 </a>
 
-
 ---
 
 Aperi'Solve is an open-source steganalysis web platform that performs automated analysis on images to detect and extract hidden data using common steganography tools and techniques.
@@ -41,15 +40,15 @@ Table of contents
 - Visualize each bit layer (LSB and other layers) per image channel (R/G/B/Alpha).
 - Browse and download each bit-layer image.
 - Integrates and displays outputs from:
-  - zsteg (LSB text/data extraction)
-  - steghide (extraction with password)
-  - outguess (extraction with password)
-  - openstego (extraction with password)
-  - exiftool (metadata and geolocation)
-  - binwalk (embedded archives)
-  - foremost (carved files)
-  - pngcheck
-  - strings
+  - [zsteg](https://github.com/zed-0xff/zsteg) (LSB text/data extraction)
+  - [steghide](https://steghide.sourceforge.net/) (extraction with password)
+  - [outguess](https://www.rbcafe.com/software/outguess/) (extraction with password)
+  - [openstego](https://www.openstego.com/) (extraction with password)
+  - [exiftool](https://exiftool.org/) (metadata and geolocation)
+  - [binwalk](https://github.com/ReFirmLabs/binwalk) (embedded archives)
+  - [foremost](https://foremost.sourceforge.net/) (carved files)
+  - [pngcheck](https://www.libpng.org/pub/png/apps/pngcheck.html)
+  - [strings](https://pubs.opengroup.org/onlinepubs/9799919799/utilities/strings.html)
 - Worker queue architecture for offloading heavy/slow analyzers (Redis + background workers).
 - Results stored for later browsing and download.
 
@@ -87,10 +86,12 @@ docker exec -it aperisolve-web bash
 docker exec -it postgres psql -U aperiuser -d aperisolve
 ```
 
-Note: If switching between dev and production compose files, remove the `results` directory or mounted volume to avoid conflicts:
-```bash
-rm -rf aperisolve/results
-```
+> [!WARNING]
+> If switching between dev and production compose files, remove the `results` directory or mounted volume to avoid conflicts:
+> ```bash
+> rm -rf aperisolve/results
+> ```
+
 ### Adding a new analyzer
 
 Adding a custom analyzer is straightforward:
@@ -129,9 +130,9 @@ Adding a custom analyzer is straightforward:
 
 4. Test locally: run the worker and submit jobs to ensure outputs are produced and displayed.
 
-Tips:
-- Keep analyzers idempotent and write outputs to the provided `results_dir`.
-- Return structured JSON so the frontend can render links/downloads automatically.
+> [!TIP]
+> - Keep analyzers idempotent and write outputs to the provided `results_dir`.
+> - Return structured JSON so the frontend can render links/downloads automatically.
 
 ## Configuration & environment variables
 
@@ -147,7 +148,8 @@ Main environment variables (examples):
 - SECRET_KEY=change_me
 - FLASK_ENV=production/development
 
-If using Docker Compose, defaults are set in the compose files. For production deployments, set secure secrets via your orchestrator or environment.
+> [!NOTE]
+> If using Docker Compose, defaults are set in the compose files. For production deployments, set secure secrets via your orchestrator or environment.
 
 ## Architecture
 
@@ -169,11 +171,16 @@ This separation keeps heavy tools (binwalk, foremost, zsteg, etc.) isolated and
 
 ## Roadmap
 
-- [ ] zsteg: full extraction (--all) and download of discovered files (mp3, etc.)
-- [ ] Mobile-friendly UI / test on mobile
-- [ ] i18n (internationalization)
-- [ ] Rootless / unprivileged analyzers
-- [ ] Improve analyzer sandboxing (e.g., per-analyzer containers)
+- [ ] **[Bug]** Error 500 when file already exist on disc
+- [ ] **[Bug]** Duplicate key value violates unique constraint "submission_pkey"
+- [ ] **[Bug]** Binwalk Extractor error
+- [ ] **[Feature]** Implement healthcheck and auto reboot
+- [ ] **[Feature]** Implement Sentry for better error handling
+- [ ] **[Feature]** Zsteg: full extraction (--all) and download of discovered files (mp3, etc.)
+- [ ] **[Feature]** Mobile-friendly UI / test on mobile
+- [ ] **[Feature]** i18n (internationalization)
+- [ ] **[Feature]** Rootless / unprivileged analyzers
+- [ ] **[Feature]** Improve analyzer sandboxing (e.g., per-analyzer containers)
 
 ## Contributing
 
@@ -182,20 +189,21 @@ Contributions are welcome! Please:
 2. Create a feature branch
 3. Open a pull request describing your change
 
-Please follow the code style and run linters before submitting.
-The project adheres to:
-- Black
-- Flake8 (ignoring E203, E501, W503)
-- Pylint (ignoring W0718, R0903, R0801)
-- Mypy (ignoring unused-awaitable)
+> [!IMPORTANT]
+> Please follow the code style and run linters before submitting. The project adheres to:
+> - Black
+> - Flake8 (ignoring E203, E501, W503)
+> - Pylint (ignoring W0718, R0903, R0801)
+> - Mypy (ignoring unused-awaitable)
 
 CI will run these checks on each PR.
 
 ## Security
 
-- If you discover a security issue, please report it privately to the repository owner instead of opening a public issue.
+> [!WARNING]
+> If you discover a security issue, please report it privately to the repository owner instead of opening a public issue.
 
 ## Credits
 
 Acknowledgements:
-- Thanks to contributors and the open-source community for the tools integrated (zsteg, steghide, openstego, binwalk, foremost, exiftool, ...).
+- Thanks to contributors and the open-source community for the tools integrated ([zsteg](https://github.com/zed-0xff/zsteg), [steghide](https://steghide.sourceforge.net/), [outguess](https://www.rbcafe.com/software/outguess/), [openstego](https://www.openstego.com/), [exiftool](https://exiftool.org/), [binwalk](https://github.com/ReFirmLabs/binwalk), [foremost](https://foremost.sourceforge.net/), [pngcheck](https://www.libpng.org/pub/png/apps/pngcheck.html), [strings](https://pubs.opengroup.org/onlinepubs/9799919799/utilities/strings.html), ...).

aperisolve/analyzers/decomposer.py

@@ -11,6 +11,12 @@ from .utils import update_data
 def analyze_decomposer(input_img: Path, output_dir: Path) -> None:
     """Analyze an image submission using bits decomposition."""
     img = Image.open(input_img)
+    converted = False
+
+    if img.mode == "P":
+        img = img.convert("RGB")
+        converted = True
+
     img_np = np.array(img)
 
     # Handle grayscale, RGB, RGBA, etc.
@@ -58,12 +64,15 @@ def analyze_decomposer(input_img: Path, output_dir: Path) -> None:
             bit_img.save(out_path)
         image_json[channel_label] = channel_json
 
-    update_data(
-        output_dir,
-        {
-            "decomposer": {
-                "status": "ok",
-                "images": image_json,
-            }
-        },
-    )
+    output = {
+        "decomposer": {
+            "status": "ok",
+            "images": image_json,
+        }
+    }
+    if converted:
+        output["decomposer"][
+            "note"
+        ] = "Image contains a color palette and was converted to RGB for processing."
+
+    update_data(output_dir, output)

aperisolve/analyzers/exiftool.py

@@ -18,21 +18,24 @@ def analyze_exiftool(input_img: Path, output_dir: Path) -> None:
             timeout=MAX_PENDING_TIME,
         )
 
-        if data.stderr:
-            update_data(
-                output_dir, {"exiftool": {"status": "error", "error": data.stderr}}
-            )
-            return
-
         metadata: dict[str, str] = {}
         for line in data.stdout.split("\n"):
             if ":" in line:
                 key, value = line.split(":", 1)
                 metadata[key.strip()] = value.strip()
 
-        # exiftool_dir = output_dir / "exiftool"
-        # exiftool_dir.mkdir(parents=True, exist_ok=True)
-        # update_data(exiftool_dir, metadata, "data.json")
+        if data.stderr:
+            update_data(
+                output_dir,
+                {
+                    "exiftool": {
+                        "status": "error",
+                        "output": metadata,
+                        "error": data.stderr,
+                    }
+                },
+            )
+            return
 
         update_data(
             output_dir,

aperisolve/analyzers/image_resize.py

@@ -146,6 +146,17 @@ def analyze_image_resize(input_img: Path, output_dir: Path) -> None:
 
         logs.append(f"Target CRC found: 0x{png.crc:08x}")
 
+        if png.crc == png.compute_crc():
+            note = (
+                f"PNG is already valid with dimensions {png.width}x{png.height} "
+                f"and crc 0x{png.crc:08x}."
+            )
+            output_no_compute: dict[str, Any] = {
+                "image_resize": {"status": "ok", "note": note}
+            }
+            update_data(output_dir, output_no_compute)
+            return
+
         saved_img_urls = []
 
         db_matches = lookup_crc(png)

aperisolve/analyzers/pngcheck.py

@@ -24,6 +24,19 @@ def analyze_pngcheck(input_img: Path, output_dir: Path) -> None:
         #     )
         #     return
 
+        if "this is neither a PNG or JNG image nor a MNG stream" in data.stdout:
+            update_data(
+                output_dir,
+                {
+                    "pngcheck": {
+                        "status": "error",
+                        "error": "ERRORS DETECTED: this is neither a PNG or JNG image nor a "
+                        "MNG stream.",
+                    }
+                },
+            )
+            return
+
         data_pngcheck = data.stdout.split("\n") if data else []
         data_pngcheck = [s for s in data_pngcheck if s]
 

aperisolve/analyzers/utils.py

@@ -180,17 +180,24 @@ class PNG:
         self.height = size[1]
         self.bit_depth = bit_depth
         self.color_type = color_type
-        self.crc = (
-            crc
-            or zlib.crc32(
+        self.interlace = interlace
+        self.crc = crc or self.compute_crc()
+
+    def compute_crc(self) -> int:
+        """Compute the CRC checksum for the PNG IHDR chunk.
+
+        Returns:
+            int: The computed CRC checksum.
+        """
+        return (
+            zlib.crc32(
                 b"IHDR"
                 + self.width.to_bytes(4, "big")
                 + self.height.to_bytes(4, "big")
-                + bytes((bit_depth, color_type, 0, 0, interlace))
+                + bytes((self.bit_depth, self.color_type, 0, 0, self.interlace))
             )
             & 0xFFFFFFFF
         )
-        self.interlace = interlace
 
     @classmethod
     def from_packed(cls, packed: int, crc: int | None = None) -> Self:

aperisolve/app.py

@@ -6,6 +6,7 @@ import os
 import time
 from datetime import datetime, timezone
 from pathlib import Path
+from shutil import rmtree
 from typing import Any, Optional
 
 from flask import Flask, Response, abort, jsonify, render_template, request, send_file
@@ -117,10 +118,22 @@ def upload_image() -> tuple[Response, int]:
         submission: Submission = Submission.query.filter_by(
             hash=submission_hash
         ).first()
-        return jsonify({"submission_hash": submission.hash}), 200
+
+        if submission is not None:  # File exist but not in DB (should not happen),
+            rmtree(submission_path)  # Removing file and continue processing as new
+            db.session.delete(submission)  # pylint: disable=no-member
+            db.session.commit()  # pylint: disable=no-member
+        else:
+            return jsonify({"submission_hash": submission.hash}), 200
 
     # If image is new, create a new Image entry
     new_img_path = RESULT_FOLDER / img_hash / image_name
+    sub_img = Image.query.filter_by(hash=img_hash).first()  # type: ignore
+
+    if new_img_path.parent.exists() and sub_img is None:
+        # Inconsistent state: image file exists but no DB entry
+        rmtree(new_img_path.parent)  # Remove the existing files and continue
+
     if not new_img_path.parent.exists():
         new_img_path.parent.mkdir(parents=True, exist_ok=True)
         with open(new_img_path, "wb") as f:  # Write file to disk

aperisolve/static/js/aperisolve.js

@@ -326,10 +326,13 @@ function parseResult(result) {
       }
     }
 
-    // Parse errors
-    if (result[tool]["status"] === "error") {
+    if (result[tool]["error"] && (/[^\s]/.test(result[tool]["error"]))) {
       showDanger(result[tool]["error"]);
     }
+
+    if (result[tool]["note"] && (/[^\s]/.test(result[tool]["note"]))) {
+      showInfo(result[tool]["note"]);
+    }
   }
 }