跳到正文
DEV Community· Md Shejan Mahamud·· 2 小時前AI 評分72

AI 會否令開發者思考能力變差?研究結果與 8 個有助保持技能的習慣

Is AI Making Developers Worse at Thinking? What the Research Says (and 8 Habits That Help)

AI 導讀

文章梳理研究後指出,尚無證據證明 AI 助手會降低開發者智力,但學習時把思考和任務交給 AI 可能妨礙技能形成。METR 隨機試驗中,16 名資深開源開發者使用 AI 工具完成工作所需時間增加 19%,但仍感覺更快;Anthropic 的 52 人試驗中,使用 AI 的工程師測驗平均約 50%,手寫組約 67%,速度差異未達統計顯著。

正文

 "AI is rotting developers' brains" makes a great headline. The evidence is more interesting than that. There is no proof that using an AI assistant lowers your intelligence. There is a small but growing set of randomized trials suggesting that how you use AI decides whether you learn or just ship.

Here is what the best studies found, where they fall short, and eight habits (plus a tiny script) you can use to keep your skills sharp.

TL;DR

  • In a randomized trial, experienced open-source developers were 19% slower with AI tools while believing they were faster.
  • In another randomized trial, developers learning a new library with AI scored 17 points lower on a quiz about code they had just written. The speed gain was not statistically significant.
  • The people who kept learning used AI to ask conceptual questions and get explanations, not to hand over the whole task.
  • Studies so far are small and use specific tools and tasks. Treat them as warning signs, not verdicts.

Study 1: METR and the perception gap

METR, an independent research nonprofit, ran a randomized controlled trial with 16 experienced open-source developers working on 246 real issues in repositories they knew well. Some issues allowed AI tools (mainly Cursor Pro with Claude models, in early 2025) and some did not.

The result: allowing AI increased completion time by 19%. Both the developers and outside experts had predicted a speedup of roughly 20 to 40 percent, and developers still felt faster after the study.

Why this matters for thinking: the gap between how fast it felt and how fast it was suggests we are poor judges of our own AI-assisted work. Fluent output feels like progress.

Caveats you should know:

  • It was only 16 developers, on mature codebases they knew deeply. That is the case where a human already has most of the context.
  • The tools were the early-2025 generation. METR has since run follow-up work with newer tools and described the data as unreliable, so don't treat 19% as a fixed number.
  • It measured speed, not code quality or learning.

Study 2: Anthropic and skill formation

Researchers Judy Hanwen Shen and Alex Tamkin ran a randomized trial with 52 mostly junior professional engineers who had to learn a new Python library (Trio). Half had an AI assistant, half coded by hand. Everyone then took a quiz on concepts they had used minutes earlier.

(Disclosure: Anthropic, which makes Claude, ran this study, and it also published the results, which found a downside to AI assistance for learning.)

The results:

AI group Hand-coding group
Average quiz score about 50% about 67%
Task speed slightly faster, not statistically significant baseline
Biggest gap debugging questions

That is a gap of roughly 17 percentage points, close to two letter grades. The hand-coders also hit more errors themselves, which seems to have built their debugging skills.

The interesting part: how people used AI

Not everyone in the AI group did badly. The researchers found distinct usage patterns:

Pattern What it looks like Learning outcome
Delegation "Write this whole thing for me" Low
Progressive reliance Starts independent, then hands over more Low
Debugging loops Pastes errors until it works Low
Conceptual questions "Why does this library work this way?" Higher
Code plus explanation Asks for code and an explanation Higher
Generate, then understand Gets code, then asks follow-ups until it makes sense Higher

The high-scoring patterns scored well above the delegating ones. The takeaway is simple: staying cognitively engaged is the variable that matters.

What other research adds

  • Knowledge workers. A Microsoft Research and Carnegie Mellon survey of 319 workers found that more confidence in AI went with less critical-thinking effort. Work shifted from producing things to verifying and integrating AI output.
  • Students. In a high school math experiment published in PNAS, a plain chatbot improved practice scores but hurt later unassisted test scores by about 17%. A tutor-style version that gave hints instead of answers avoided the harm.
  • Medicine. The 2026 International AI Safety Report notes a study where clinicians’ ability to detect tumours without AI dropped about 6% after three months of AI support.

The pattern is consistent: tools that hand you the answer help you now and can cost you later. Tools that make you work protect learning.

8 habits to stay sharp while using AI

1. Write the plan or the test first

Before you prompt, write the function signature, a rough plan, or a failing test. You are forcing yourself to understand the problem, and the AI then fills a gap you defined.

# I write this first, then ask for help with the implementation
def test_parse_duration():
    assert parse_duration("1h30m") == 5400
    assert parse_duration("45s") == 45
    assert parse_duration("") == 0

2. Ask for explanations, not just code

Use prompts that keep you in the loop.

Explain how X works and why this approach is better than Y.
Don't write the full solution. Give me the key idea and a small example.

3. Treat AI code like a junior's pull request

Read every line. If you can't explain a line to a teammate, either learn it or delete it. A good rule: no merge without an explanation in your own words.

4. Debug unaided first

Give yourself 15 minutes to form a hypothesis before asking AI. Write the hypothesis down. This is the skill the studies showed fading fastest.

Bug: checkout total is off by 1 cent
Hypothesis: float rounding in the discount step
Test: print Decimal values at each stage

5. Schedule AI-free reps

Athletes train without gear. Pick one regular session a week with no assistant: a kata, a small feature, or a code review. Think of it as maintaining a muscle.

6. Use AI as a reviewer, not an author

Write it yourself, then ask for critique.

Review this function for bugs, edge cases and readability.
List problems only. Do not rewrite the code.

7. Learn new tools from the docs first

When you meet a new library, read the official docs and examples first, then use AI to quiz you or fill gaps. In the Anthropic study, this is the situation where delegating cost people the most.

8. Measure yourself instead of trusting your feeling

The METR result is a warning that your sense of speed may be wrong. Run a small experiment.

Run the experiment on yourself (2 weeks)

Here is a tiny script that times tasks with and without AI so you can compare your own numbers.

#!/usr/bin/env python3
"""taskclock.py: time your tasks with and without AI, then compare."""
import csv
import statistics as st
import sys
import time
from pathlib import Path

LOG = Path.home() / ".taskclock.csv"


def log(name, ai, seconds):
    new = not LOG.exists()
    with LOG.open("a", newline="") as f:
        w = csv.writer(f)
        if new:
            w.writerow(["task", "ai", "seconds"])
        w.writerow([name, ai, round(seconds)])


def summary():
    if not LOG.exists():
        print("No data yet. Time a task first.")
        return
    with LOG.open() as f:
        rows = list(csv.DictReader(f))
    for flag in ("ai", "no-ai"):
        times = [int(r["seconds"]) for r in rows if r["ai"] == flag]
        if times:
            print(f"{flag:6} n={len(times):3}  median={st.median(times) / 60:.1f} min")


if __name__ == "__main__":
    if len(sys.argv) == 2 and sys.argv[1] == "summary":
        summary()
    elif len(sys.argv) == 3 and sys.argv[2] in ("ai", "no-ai"):
        start = time.time()
        input(f"Timing '{sys.argv[1]}' ({sys.argv[2]}). Press Enter when done...")
        log(sys.argv[1], sys.argv[2], time.time() - start)
    else:
        print("usage: taskclock.py 'task name' ai|no-ai   or   taskclock.py summary")

Usage:

python3 taskclock.py "add pagination" ai
python3 taskclock.py "fix flaky test" no-ai
python3 taskclock.py summary

To keep the test honest:

  1. Flip a coin to decide whether each task gets AI. Do not choose based on difficulty.
  2. Compare similar tasks. A bug fix and a new feature are not comparable.
  3. Add a retention check. The next day, explain the solution from memory, with no notes. If you can't, mark it.
  4. Remember it is a small sample. Ten tasks will not prove anything, but it will be your data instead of a vibe.

What these studies can't tell us

  • Tools change quickly. The METR trial used early-2025 tools, and agents are now far more capable.
  • Samples are small. 16 and 52 people are informative but not decisive.
  • Tasks are narrow. One library, one lab session, or open-source issues in familiar repos is not your whole career.
  • Learning is not always the goal. If a throwaway script saves you an hour, delegating can be a smart choice. The risk is delegating the skills you need to grow.

For team leads

  • Onboarding juniors: pair them with seniors and require explanations in PRs, not just working code.
  • Code review: ask "Why this approach?" instead of "Does it pass?" for AI-assisted changes.
  • Metrics: avoid measuring AI success by lines of code or tickets closed. Track defects, review time and onboarding speed.

Bottom line

AI is not making developers dumber across the board. The better reading of the research is that delegating thinking while you are still learning has a cost, and engaged use protects you. Write the plan first, ask for explanations, read the diffs, keep some AI-free reps and measure yourself.

I wrote a broader, non-developer version of this on my blog, covering thinking, memory and creativity, if you want the wider picture: Is AI Making Us Dumber?

Question for you: have you ever measured your own speed or understanding with and without an AI assistant? What did you find?

Sources: METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (2025); Shen and Tamkin, "How AI Impacts Skill Formation" (2026) and Anthropic's write-up "How AI assistance impacts the formation of coding skills"; Microsoft Research and Carnegie Mellon survey of knowledge workers (2025); the 2025 PNAS high school mathematics experiment; the International AI Safety Report 2026.

來源:DEV Community · dev.to