Independent Cybersecurity Lab · AI & Software Analysis

AI-Assisted Software Reverse Engineering & Behavioral Analysis

Python · Black-box testing · SHA-256 · JSON · AI-assisted hypothesis generation

Investigated an unfamiliar, benign file-manifest utility by treating it as a black box, designing controlled probes, recording observable behavior, and converting verified findings into a reproducible behavioral specification and regression tests.

Behavioral analysisEvidence collectionPython automationAI workflow

Project README Technical report (PDF) Probe evidence (JSON)

Executive Summary

This project demonstrates a practical workflow for understanding software behavior without assuming that its internal implementation is known. A small Python utility serves as the controlled target. The analysis examines valid input, error handling, nested files, exclusion rules, JSON output, and whether file-content changes affect reported digests.

AI is used as a hypothesis-generation and test-design aid. Its suggestions are treated as unverified until a controlled experiment supports them. The resulting findings are documented with evidence and converted into automated regression tests.

Scope note: This is a self-contained educational simulation using a benign script, not reverse engineering of a third-party commercial binary. The narrow scope makes the work reproducible and its claims easy to validate.

Objectives

Characterize behavior

Map accepted inputs, output structure, file-selection behavior, and failure modes.

Use AI responsibly

Generate competing explanations and discriminating tests without treating model output as evidence.

Capture reproducible evidence

Preserve return codes, stderr, observed file paths, and digest comparisons in machine-readable form.

Validate findings

Translate the behavioral specification into regression tests that can be rerun locally.

Methodology

Scope and baseline: use the included target and controlled sample fixtures only.
Probe design: test a valid directory, missing argument, nonexistent path, nested file, hidden file, temporary suffix, and changed file contents.
Evidence capture: record stdout/stderr, exit status, parsed output paths, and digest changes.
AI-assisted reasoning: ask an LLM for multiple hypotheses, confidence labels, and minimal tests that could disprove each hypothesis.
Independent verification: run probes and unit tests before promoting a hypothesis to a finding.
Documentation: produce a behavior specification, limitations, and a repeatable test suite.

Key Findings

Valid directory input produces a JSON manifest containing a root name, file count, and per-file records.
Records include relative file paths, byte sizes, and SHA-256 digests.
The tested nested file is included, while the supplied hidden-file and temporary-file fixtures are excluded.
Changing file bytes while retaining the filename changes the reported digest in the controlled probe.
Missing arguments produce a usage message and exit code 2; a nonexistent directory produces an error and exit code 1.

These findings apply to the tested target and fixtures. They are not claims of exhaustive behavior across all platforms, filesystems, or edge cases.

Technical Implementation

The project uses Python's standard library only. The probe harness invokes the target as a subprocess, captures output channels and return codes, parses JSON, and compares digests after changing test-file content.

python mystery_manifest.py sample_data
python probe_target.py
python -m unittest -v

Deliverables include a reusable probe script, sample fixtures, machine-readable evidence, a findings ledger, AI prompt templates, and five automated regression tests.

Security Relevance & Limitations

Behavioral analysis is useful when documentation is incomplete or when an analyst needs to verify what a utility actually does. Hash manifests can support integrity checks when compared against a trusted baseline, but hashes alone do not establish that a file is safe or authentic.

Educational target only; no exploitation, persistence, evasion, or network activity.
Permission edge cases, symlinks, special files, very large files, and cross-platform differences are not exhaustively tested.
AI-generated claims remain hypotheses until supported by independent tests.

Project Files

Summary

AI-Assisted Software Reverse Engineering & Behavioral Analysis

Developed a Python black-box testing workflow to characterize an unfamiliar utility through controlled input, output, and error-path probes.
Validated file filtering, recursive traversal, JSON output, exit-code behavior, and SHA-256 content sensitivity; translated observations into regression tests and an evidence-backed behavioral specification.
Documented an AI-assisted hypothesis workflow that separates model-generated ideas from experimentally verified conclusions.