讓 LLM 翻譯舊有程式碼前，先凍結其既有行為

Legacy modernization is mostly archaeology. A model can read a sprawling, undocumented codebase, explain modules, map dependencies, translate languages, and draft migrations, compressing days of careful reading into a reviewable draft. The catch is the business rules that live outside the syntax: say, an exception that exists because of a contract signed in 2011, the kind of rule that exists for a reason no doc explains. A model reads the code, not the intent, so it can fluently reproduce structure while getting a critical behavior wrong. A subtly wrong migration is worse than an obviously broken one, because it ships.

Rachelle Palmer of MongoDB made the case on Chain of Thought that modernization needs both speed and proof that the new code behaves like the old. You want the LLM to do the slow work (reading, explaining, drafting) and the engineer to own verification and the decisions that require understanding why the old system behaves the way it does. Before any translation starts, pin the behavior you must preserve.

What does a characterization test look like?

A characterization test captures the current output of a legacy function without judging whether that output is correct by spec. It is a snapshot: given this input, the system returns this result. Once frozen, any change the LLM produces that breaks the test is either a bug in the translation or a real design change that needs a human sign-off.

I would start with discrete legacy functions (a pricing calculator, a formatter, a validation rule) where inputs and outputs are easy to observe, and expect more work on UI-heavy flows or stateful systems without clean boundaries. Pick the functions that encode the business logic you are most afraid of losing.

How do I freeze behavior before translating?

Write a characterization suite first. Use the model to help draft tests by feeding it the legacy source and asking for representative input-output pairs, but review every one. Then freeze the suite in version control and do not change it during translation. Let the LLM translate, run the suite, and treat any failure as a design decision, not a nuisance to suppress.

Rachelle Palmer put it clearly:

In order to prove equivalence between a legacy application and a replacement application, you actually need to generate effectively equivalent tests between your legacy app and your new modernized app.

Rachelle Palmer, MongoDB, on Chain of Thought ep 14

That is the discipline: equivalent tests on both sides, signed off by a human.

Where does the 80/20 split actually land?

Palmer puts the model's share of modernization work at about 80 percent, with the remaining 20 percent purely manual. She does not itemize that 20 percent, and she says equivalent tests can themselves be generated with LLMs. In the workflow I am proposing, the human share includes deciding which behaviors matter, reviewing each failure after translation, and deciding whether a change is acceptable.

I do believe the eighty twenty rule applies here, which is that you can get LLMs to assist with 80, but you still have this remaining 20% that's like purely manual effort.

Rachelle Palmer, MongoDB, on Chain of Thought ep 14

This is the opposite of vibe coding, where you accept generated code by how it runs rather than by reading it. That loop is fine for a prototype. Code that might still carry an exception from, say, a 2011 contract deserves the frozen suite. The AI coding topic page and the best episodes on AI coding cover what changes when models write more of the code.

A short sketch for freezing before translating

This is an illustration of the order of operations, not production code.

# Sketch: freeze, translate, compare
legacy_tests = run_characterization(legacy_module)
new_code = llm_translate(legacy_module, spec=docs)
new_tests = run_characterization(new_code)
diffs = compare(legacy_tests, new_tests)
# Any diff is a design decision. Human signs off.

I would not run this unsupervised. The sketch shows the order, not the review step. Review every diff.

Checklist

• Pick the legacy functions that encode the business rules you are most afraid of losing.
• Write characterization tests that freeze current input-output pairs, not spec-correct behavior.
• Lock the suite in version control before any LLM translation starts.
• Translate with the model; compare against the frozen suite.
• Treat any changed test result as a design decision a human signs off on, not a suppression task.

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.