PHANTOMTRACE：找出 AI 智能體執行追蹤中的異常

This is my project for the Hacktoberfest Open-Source AI Challenge — Week 4: Ghost in the Machine.

1. What I Built

I built PHANTOMTRACE, an AI Agent Trace Investigator and Debugging Dashboard.

AI agents perform multiple actions, such as searching the web, calling tools, and processing information. Sometimes they repeat the same action, encounter errors, or retry failed operations.

These problems can be difficult to understand by looking at raw execution logs.

PHANTOMTRACE helps developers investigate these execution traces and identify suspicious patterns.

Its goal is to make AI agent behavior easier to understand and debug.

2. Project Repository

GitHub Repository:

https://github.com/tk1981215/PHANTOMTRACE

The repository contains the Python application, JSONL sample logs, anomaly detection code, requirements and project documentation.

3. Demo

4. Technologies Used

I used the following technologies:

Python: For parsing and analyzing execution logs.
Streamlit: For building the interactive dashboard.
Pandas: For processing structured log data.
Plotly: For data visualization.
Ollama and Gemma 3 1B: Planned optional local open-weight AI integration. [Update this section after successfully testing the integration.]

5. How PHANTOMTRACE Works

PHANTOMTRACE accepts JSONL execution logs and analyzes them.

It identifies patterns such as:

Repeated tool calls.
Multiple failed operations.
Immediate retries after failures.

It uses deterministic Python rules to detect these patterns.

The optional local AI component is intended to explain detected patterns using evidence from the execution logs.

The aim is to distinguish actual observations from AI-generated explanations.

6. Why Does Open Innovation Matter?

Open innovation is important because it gives developers freedom to experiment, learn, modify and improve technology.

Open-weight AI models allow developers to explore AI without depending entirely on closed AI services.

With PHANTOMTRACE, my goal is to make AI agent debugging more transparent and understandable.

Developers should be able to inspect what happened instead of blindly trusting an AI-generated answer.

Local AI processing can also help developers keep their trace data on their own machines when configured and used locally.

7. Challenges I Faced

One of my biggest challenges was working with limited computer resources.

I focused on making the Python-based analysis lightweight and keeping the local AI integration optional.

I also learned how to work with JSONL files, identify suspicious patterns and build an interactive dashboard.

8. What I Learned

While developing PHANTOMTRACE, I learned:

How to parse JSONL files.
How to detect repeated tool calls.
How to identify failures and retries.
How to build a dashboard using Streamlit.
How deterministic rules can help debug AI agents.
How open-weight models can be integrated into local applications.

9. Future Improvements

In the future, I would like to add:

Interactive agent execution timelines.
More anomaly detection rules.
Better error analysis.
Improved local AI model support.
More visualizations.
Automated testing.

10. Final Thoughts

The idea behind PHANTOMTRACE is simple:

AI agents should not be mysterious black boxes.

Developers should be able to understand what happened, identify suspicious behavior and inspect the evidence.

I built this project as my interpretation of the theme Ghost in the Machine.

Thank you for checking out PHANTOMTRACE!