我想了解提示詞框以外的 AI：我的 DSPy 探索之旅

An exploration of open-source AI, the curiosity that started it, the errors that interrupted it, and the satisfaction of finally making something work.

There is something slightly inconvenient about being curious about too many things.

One day, I want to understand how language models work. Another day, I find myself exploring cybersecurity, quantitative finance, product management, or some entirely different corner of technology. Every field seems to have something interesting hiding underneath the surface, and I have this persistent feeling that I should at least try to understand it.

The problem is that trying everything and actually learning something are two very different things.

It is easy to watch a tutorial, understand the general idea, build something that works once, and move on to the next exciting thing. It is considerably harder to open someone else's codebase, navigate unfamiliar abstractions, figure out why something is failing, and stay long enough to understand what is happening beneath the interface.

For this exploration, I wanted to do the latter.

I decided to investigate DSPy, an open-source framework for programming language-model applications. I did not want to stop at understanding what the framework claimed to do. I wanted to see how it was structured, how its components interacted, and whether I could use what I learned to build a working demonstration of my own.

I started with questions. I ended up with source-code investigations, test runs, configuration errors, a locally running language model, and a much more concrete understanding of what it means to explore an open-source AI project.

This is that journey.

1. Why DSPy?

When we talk about building with language models, the conversation often revolves around prompts.

We write instructions, adjust the wording, test different variations, and hope the model produces the desired output. When the application becomes more complicated, managing those prompts and their interactions can become a problem of its own.

That made me interested in a different question:

What if I could think about a language-model application as a programmable system rather than a collection of prompts?

That question led me to DSPy.

DSPy provides abstractions for defining tasks, composing model-powered modules, and optimizing how those modules use language models. Instead of manually treating every prompt as an isolated piece of text, developers can describe the task they want to perform and build a structured program around it.

I found this interesting because it sits at the intersection of several things I wanted to understand: AI, software architecture, experimentation, and the process of turning an idea into a working system.

I could have chosen a framework simply because it was popular or built a small application using an API and stopped there. But I wanted to investigate the framework itself.

What happens when a prediction is called? What does a signature actually represent? How does a higher-level module build on a lower-level one? And what does optimization mean in a framework like this?

I did not have all the answers when I started.

That was precisely the point.

2. Before Building Anything, I Wanted to Understand the Architecture

My first step was to explore the repository and identify the main components I wanted to investigate.

Rather than trying to understand the entire codebase at once, I focused on a few concepts that appeared central to DSPy's programming model:

• Signatures
• The Predict module
• ChainOfThought
• Module composition
• BootstrapFewShot optimization

I treated these as connected pieces rather than independent features.

Signatures: Defining the task

A DSPy signature describes the relationship between the inputs a task receives and the outputs it is expected to produce.

For example, a simple classification task might look like this:

import dspy

class ClassifyIntent(dspy.Signature):
    """Classify a customer message into a support category."""

    query: str = dspy.InputField()
    intent: str = dspy.OutputField()

This defines a task that accepts a customer query and produces an intent.

The interesting part is what the code does not contain: I am not manually writing out the complete prompt that the model must follow. Instead, I am defining the task's interface and letting DSPy handle the prediction workflow.

That distinction became a useful starting point for understanding the rest of the framework.

Predict: Turning a specification into a prediction

The next component was dspy.Predict.

classifier = dspy.Predict(ClassifyIntent)

result = classifier(
    query="I was charged twice for my order."
)

print(result.intent)

The signature specifies the task. The predictor executes it using the configured language model.

I wanted to understand how that relationship was implemented internally, not just how to use it in a script. That meant looking beyond the example and into DSPy's source code.

This was where the investigation started becoming more interesting. A framework's public interface tells you how developers are expected to use it. Its implementation begins to explain how that interface actually works.

ChainOfThought: Building on existing abstractions

I also investigated dspy.ChainOfThought.

Conceptually, it builds on the prediction machinery by extending a supplied signature with a reasoning field. It then delegates prediction to a standard dspy.Predict instance.

What interested me here was the design pattern.

Rather than creating an entirely separate prediction system, DSPy composes existing functionality into a higher-level module. That makes the framework's abstractions easier to connect conceptually: more sophisticated behavior can be built on top of the underlying prediction mechanism.

Modules and composition

I explored DSPy's module structure to understand how these components fit into larger applications.

This mattered because real applications rarely consist of a single prediction. They combine multiple steps, pass outputs between components, and organize model calls into a workflow.

Thinking in terms of modules shifted my attention from individual prompts to the structure of the application around them.

BootstrapFewShot: Exploring optimization

The final major component I focused on was BootstrapFewShot.

DSPy's teleprompting and optimization machinery can use training examples, model-generated demonstrations, and an evaluation metric to construct a predictor with demonstrations selected through an optimization process.

This raised another question for me: how do you demonstrate optimization without confusing the existence of an optimizer with proof that it improves performance?

I would return to that question when I began testing the framework.

3. Moving From Reading Code to Running It

Understanding an architecture on paper is useful. Running the code gives you a different kind of understanding.

I created a small investigation demo covering the concepts I had been exploring. The goal was not to build an elaborate product. It was to make the framework's behavior observable through a few focused examples.

The demo covered: https://youtu.be/klimwICb1a4

1. Inspecting signatures.
2. Executing dspy.Predict.
3. Executing dspy.ChainOfThought.
4. Composing a customer-support pipeline.
5. Demonstrating the BootstrapFewShot workflow.

For the initial experiments, I used DSPy's DummyLM.

A dummy language model is useful when you want to test a program's structure without depending on a real model response. It allows you to investigate how components are called and how data moves through the workflow.

However, I needed to be precise about what that proved.

A successful mock-based demonstration is evidence that the workflow executes. It is not evidence that a real language model will produce accurate outputs.

I wanted to demonstrate both.

That meant moving beyond the initial tests and connecting DSPy to a real model.

4. The First Setback: My Code Would Not Even Import

Open-source exploration sounds particularly elegant when described after everything has worked.

The actual process is less elegant.

At one point, Python raised a SyntaxError inside:

dspy/predict/chain_of_thought.py

The error pointed to a line of explanatory prose that Python was interpreting as code.

I had managed to get ordinary article text into a Python source file while working between the investigation and its explanation. Naturally, Python was not impressed by my literary ambitions.

The immediate problem was not DSPy's architecture. The file itself was syntactically invalid.

I inspected the affected section, checked the repository's Git status, and verified the source file rather than immediately reinstalling packages or changing unrelated code.

Once the file was back to valid Python, I tested the import again.

dspy_investigation_demo/.venv/bin/python -c "import dspy; print('DSPy version:', dspy.__version__)"

The output was:

DSPy version: 3.4.1

That was a small but important lesson in debugging: identify the layer where the failure occurs before attempting to fix it.

A syntax error in a source file, a missing dependency, and a model connection failure are different problems. Treating them as interchangeable usually creates more problems than it solves.

I had not learned anything profound about language-model optimization from this incident. I had, however, practiced a more useful habit: inspect the evidence before changing the system.

5. Testing the Workflow

With DSPy importing successfully, I ran the investigation's test suite.

dspy_investigation_demo/.venv/bin/pytest dspy_investigation_demo/test_demo.py -v

All five tests passed:

• test_signature_declaration_and_fields
• test_basic_predict_execution
• test_chain_of_thought_execution
• test_composed_pipeline_execution
• test_optimization_with_bootstrap_fewshot

These tests covered the basic prediction workflow, signature declarations, module composition, and the optimization demonstration.

At this stage, I had evidence that the examples executed successfully under the test conditions.

But I still wanted to see the framework communicate with a real model.

That became the next stage of the investigation.

6. Connecting DSPy to Gemma Through Ollama

I already had Ollama installed and the gemma4:latest model available locally.

My next goal was straightforward: configure DSPy to use Gemma and create a small classification task.

I started with a customer-support example.

The input was:

I was charged twice for my order. Please refund the extra payment.

The expected category was billing.

The classification task itself was simple. The integration was where the debugging began.

The 404 error

My first DSPy execution failed with:

InvalidRequestError: 404 page not found (ollama, HTTP 404)

This was confusing because the model was installed and Ollama was available.

I needed to distinguish between two questions:

1. Was Ollama actually working?
2. Was DSPy communicating with it correctly?

I tested Ollama's API directly rather than assuming that the failure meant the model was broken.

Testing Ollama independently

I sent a direct request to the local chat endpoint, asking Gemma to reply with a fixed phrase.

My first attempt failed because the JSON contained an extra comma and a misplaced closing bracket.

The server responded with:

invalid character ']' looking for beginning of value

That was not a model failure or a DSPy failure. It was simply an invalid request body.

After correcting the JSON, the request succeeded:

{
  "model": "gemma4:latest",
  "message": {
    "role": "assistant",
    "content": "CONNECTION WORKS"
  },
  "done": true
}

The exact response structure returned by Ollama included additional metadata, but the important result was clear: Gemma responded successfully to a direct API call.

I now had evidence that Ollama was functioning independently of DSPy.

That narrowed the problem considerably.

Fixing the DSPy integration

The remaining issue was the way DSPy routed the model request.

I adjusted the model configuration to use DSPy's LiteLLM compatibility engine explicitly, keeping the Ollama endpoint and model name consistent with the working direct request.

The relevant configuration became:

lm = dspy.LM(
    "ollama_chat/gemma4:latest",
    api_base="http://localhost:11434",
    api_key="",
    model_type="chat",
    engine="litellm",
    cache=False,
)

dspy.configure(lm=lm)

With that configuration, the demo finally executed successfully.

The important part of this debugging process was not simply discovering which setting worked. It was separating the model server from the framework integration, testing them independently, and using the results to narrow down the failure.

I went from a generic HTTP 404 to a working local inference pipeline by breaking the problem into smaller, testable pieces.

7. The Moment the Demo Actually Worked

With the integration working, I ran the classification script again.

The output was:

--- DSPy + Gemma: Real Model Inference ---

Input: I was charged twice for my order. Please refund the extra payment.

Predicted intent: billing

Inference completed successfully.

It was a small result.

No complicated interface. No elaborate multi-agent system. No dramatic claim about changing the future of AI.

Just a customer message going into a program, a real local language model processing the request, and a category coming out.

But this result represented a different kind of progress from the earlier mock-based demonstration.

I had now verified that DSPy could execute the task using Gemma through Ollama on my machine.

I then extended the experiment into an interactive customer-support demo. Instead of supplying a single hard-coded input, I could enter different messages and have the program return a category and a suggested response.

For example, the inputs could describe:

• A duplicate charge.
• A delayed delivery.
• An application that crashes when uploading a file.

The program used the same signature and prediction module for each request.

That made the demonstration more useful because I could observe how the same task definition behaved with different inputs.

It also introduced an important limitation: the categories and responses were model-generated, not guaranteed answers. A working inference pipeline does not automatically establish classification accuracy, response quality, or reliability across a wider dataset.

Those questions would require a separate evaluation.

For this investigation, the goal was narrower: understand the framework's structure and demonstrate a functioning real-model integration.

I had achieved that.

8. What I Learned Beyond the Code

The technical results were useful, but the process taught me a few things that were not specific to DSPy.

First, understanding a framework is different from knowing how to call it

It is easy to learn the syntax for creating a predictor.

It takes more work to understand how signatures define task interfaces, how modules compose, how prediction execution is organized, and where optimization fits into the design.

Reading source code alongside running small experiments helped connect these ideas.

I did not need to understand every file in the repository. I needed to identify the components relevant to my questions and follow their relationships.

That is a much more manageable way to approach a large codebase.

Second, a successful execution is not the same as a successful evaluation

The five passing tests demonstrated that my mock-based workflows executed under the tested conditions.

The Gemma experiment demonstrated that real inference worked.

Neither result, on its own, proved that DSPy produced better outputs than a manually written prompt or that BootstrapFewShot improved task performance.

To establish that, I would need a defined dataset, an appropriate metric, a baseline, and a controlled comparison.

That distinction matters because AI projects can look impressive long before their actual performance has been measured.

I wanted my investigation to distinguish what I had demonstrated from what remained to be tested.

Third, errors can help narrow a problem down

The syntax error, malformed JSON, and HTTP 404 were different failures at different layers.

Fixing them required different approaches.

I learned to check whether the source file was valid, whether the local model server responded, whether the request itself was well formed, and whether DSPy's provider configuration was correct.

Debugging became less about randomly changing settings and more about asking which assumption had failed.

Finally, curiosity needs a direction

I started this exploration because I wanted to understand something unfamiliar.

But curiosity alone can easily become endless browsing: another repository, another framework, another concept that seems interesting enough to investigate.

The useful part was choosing a specific set of questions and following them far enough to produce evidence.

I examined the architecture, built a small demo, tested the workflow, connected a real model, and recorded the result.

I did not master DSPy in a day. That was never a realistic goal.

I developed a more concrete understanding of a framework by working through a small, defined investigation.

That is a better outcome than simply collecting another technology name to put on a profile.

9. What I Would Explore Next

This investigation gave me a working foundation, but it also raised questions I have not yet answered.

The most interesting next step would be to evaluate optimization rather than merely demonstrate that an optimizer runs.

I would want to:

1. Build a small, labeled customer-support dataset.
2. Define a metric for classification quality.
3. Measure a baseline predictor before optimization.
4. Run BootstrapFewShot with suitable training examples.
5. Evaluate the optimized predictor on held-out examples.
6. Compare the results under the same conditions.

That would help answer a more substantive question: under what conditions does DSPy's optimization process improve the task, and how much improvement can be measured?

I would also want to investigate failure cases, including ambiguous messages, overlapping categories, and inputs that do not fit the predefined labels.

Those experiments would move the project from an architectural investigation and working demonstration toward a more rigorous evaluation.

For now, I have deliberately kept the scope small. A clear experiment with honest limitations is more useful than a complicated demo whose results I cannot explain.

Conclusion: The Point Was Never to Know Everything Immediately

I started this investigation with a familiar kind of curiosity: I wanted to understand another part of AI, and I did not want to settle for a superficial explanation.

I chose DSPy because its approach to programming language-model applications gave me a concrete architectural question to explore. I investigated its signatures, prediction modules, composition, and optimization workflow. I created a demonstration, ran five tests using a dummy language model, and then connected DSPy to Gemma through Ollama for real inference.

Along the way, I encountered errors that forced me to distinguish between source-code problems, malformed requests, and integration issues. I fixed them, verified the working configuration, and extended the experiment into an interactive demo.

The final output was modest. The learning process was more valuable.

I am still interested in too many things to pretend that I have settled on one narrow path. I want to explore AI, cybersecurity, quantitative systems, software engineering, and the ways technology becomes useful to people.

But I am beginning to understand that exploring everything does not mean trying to master everything simultaneously.

Sometimes it means choosing one question, following it beyond the documentation, making something work, understanding why it works, and being honest about what you have not yet proved.

I don't need to know everything today.

I do want to become the kind of person who can approach something unfamiliar, investigate it seriously, and build a working understanding of it.

For this project, that meant going from What exactly is DSPy doing? to a working local inference pipeline.

For the next one, the question will probably be different.

The process, I hope, will be better.

Project Evidence

This investigation included:

• Source-code exploration of DSPy's signatures, prediction modules, and optimization machinery.
• A demonstration covering signatures, Predict, ChainOfThought, module composition, and BootstrapFewShot.
• Five passing tests using a dummy language model.
• A real-model inference script using DSPy 3.4.1, Ollama 0.40.10, and the locally installed gemma4:latest model.
• An interactive customer-support classification demonstration.

Scope note: The mock-based tests verified the example workflows. The Gemma integration verified that local inference worked for the demonstrated task. A controlled benchmark comparing prediction quality before and after optimization was outside the scope of this investigation.