跳到正文
DEV Community· Amar·· 2 小時前AI 評分46

AI 與機器學習的真實侷限

The Honest Limits of AI and ML

AI 導讀

AI 與機器學習的侷限可分為數據、工程、信任、人類監管及前沿失誤五類,推測性邊界則另行標示。Amazon 2015 年履歷篩選器因「women’s」一詞懲罰女性,COMPAS 對黑人被告錯判高風險的比例是白人的兩倍;SageMaker Clarify 可在訓練前於原始數據計算偏見指標。歐盟《人工智能法案》要求高風險用途接受人類監督,罰款可達 EUR 35 million 或營業額的 7%。

正文

TL;DR: The limits of AI and ML sort into five checkable categories: data that decides what a model can know, engineering walls like overfitting and compute cost, trust failures in the black box and in ourselves, human limits the law now guards, and frontier failures like hallucination, with the speculative end labeled separately.

  • Bias is learned from data: Amazon’s 2015 resume screener penalized the word "women’s", and COMPAS wrongly labeled Black defendants high-risk at twice the white rate.
  • The bias fix is representativeness, not volume: SageMaker Clarify computes bias metrics on the raw dataset, before training, and they are model-agnostic.
  • Working AI is not automatically trustworthy. Radiologists caught 46 percent of cancers unaided but only 21 percent when a faulty automated aid stayed silent.
  • The EU AI Act, in force since 1 August 2024, phases in through 2026 and 2027 with fines reaching EUR 35 million or 7 percent of turnover and human oversight required for high-risk uses.
  • GPT-4 scored ahead of 99 percent of people on the Torrance Tests of Creative Thinking in 2023, so the honest creativity limit is narrow competence and reasoning gaps.

Between the do-everything pitch and the hype-collapse narrative sits a question neither side answers: where exactly do AI and machine learning stop, and with which tool? This companion to the Whiteboard Architect episode The Honest Limits of AI and ML walks the same ten chapters with its fact-check citations and the tables a video cannot hold. Every number traces to a checked reference; where the record cuts against a popular claim, the record wins.

Whiteboard still from the episode's cold open: the heading The honest limits underlined in red, the two claims AI will do everything and hype that folds crossed out, a red box asking what are the actual limits, and a blue list reading data, trust, humans, frontier underlined in green.

The cold open board: both extremes crossed out, and the four-part map that replaces them.

The data decides

No model has direct experience of the world; what it knows came in through its training data. Feed it incorrect, misleading, or unrepresentative examples and the output turns inaccurate, sometimes discriminatory. Wikipedia’s algorithmic-bias article tracks the mechanism: bias is learned, amplifies as training proceeds, and can leave a deployed system digitizing cultural prejudice. Poor-quality data in, poor-quality model out; imbalanced data is the quieter failure beside it.

Imbalance works by majority rule: the dominant class is served well, the minority poorly. Not one of the four cases was designed to discriminate; the data taught it.

Case Year What the data did
Amazon resume screener 2015 Penalized the word "women’s", excluded all-women’s colleges; Amazon switched it off
COMPAS recidivism scores 2016 analysis Black defendants wrongly labeled high-risk at twice the white rate (ProPublica)
Mortgage algorithms 2019 UC Berkeley found lending disparities hitting Latino and African American borrowers
Facial recognition arrests ongoing Misidentifying darker-skinned faces linked to wrongful arrests of Black men

Which named biases live inside machine learning?

The sourced taxonomy splits bias two ways. One level up, Wikipedia groups causes as pre-existing (bias that exists in society before any model touches it), technical (bias introduced by tooling and sampling), and emergent (bias that appears after deployment or in new societal contexts). Inside machine learning itself, the article names five: data bias, label bias, measurement bias, then algorithmic and deployment bias, with selection bias also named; a 2021 survey adds historical, representation, and measurement for overlapping phenomena.

The practical use of the list is diagnostic: when a system misfires, the fix differs depending on whether the damage entered through the data, the labels, the measurement, or the deployment.

Fixing bias without fooling yourself

Most checklists say collect more data. The checked sources say the dial is representativeness: larger biased datasets stay biased, and bigger samples can disregard underrepresented groups while looking more rigorous. What holds up is more representative, higher-quality data, audited for disparate error rates across every group the model will score.

Two habits keep the fix honest. Test before training: bias metrics need no trained model, they run on the raw dataset itself, so a representativeness hole is visible before any budget is spent. Then keep monitoring, because data evolves, drift appears, and retraining follows. AWS SageMaker Clarify implements the workflow end to end.

How do you measure bias before any model exists?

Clarify approaches bias as a dataset property first. Its metrics run on the rows and labels you already have, with no model in the loop, so whatever they say holds equally for every model you might build afterwards. The opening metric, facet imbalance, weighs how much training data each facet holds relative to the others; the rest look at how outcome labels spread across those same facets.

The age example shows the stakes: declare middle-aged and everyone else the two facets, and a lending dataset thin outside the middle-aged group handicaps those applicants no matter which algorithm trains on it. Read that way, a failed bias metric is cheap: you reject the dataset and have spent nothing on training.

The engineering limits

Even with clean data, the learning machinery fails in named ways; the four in the table cover most project post-mortems. Overfitting has the model learn the training set by heart, noise and all, then stumble on data it has never seen. Underfitting fails the other way, the model too simple for the structure. The one that never closes is generalization: nothing guarantees behavior outside the training distribution, and predictions on unseen data are the entire product.

Failure mode Symptom Standard counters
Overfitting Strong on training data, weak on new Cross-validation, regularization, early stopping, pruning, dropout
Underfitting Lacks capacity for the structure; high bias Added complexity, better features
Off-distribution inputs No guarantees beyond the training distribution Treat unseen data as the real test
Compute cost Exhaustive parameter search unaffordable Narrower search; budget GPU time

Compute is the fourth wall. GPUs displaced CPUs for training years ago, and the compute used by the largest deep learning runs grew about 300,000-fold from AlexNet in 2012 to AlphaZero in 2017. Applicability bounds the list from the other side: machine learning generalizes predictive patterns from examples, with competence that stays inside well-defined tasks; it is a tool for pattern-shaped problems, and a problem with no stable pattern to find is not an ML problem.

Where does the 300,000-fold compute number come from?

The figure comes from an OpenAI analysis of the field’s landmark runs: measuring AlexNet in 2012 against AlphaZero in 2017, it put the compute behind the biggest deep learning projects at 300,000 times more, doubling roughly every 3.4 months. Wikipedia’s machine learning article carries the estimate with links to the underlying analyses.

Two consequences follow for ordinary teams. Landmark training runs sit on a cost curve that doubles faster than hardware budgets, and the curve explains why exhaustive parameter search, the textbook’s tidy answer to tuning, is often the first thing a real project abandons. Specialized hardware, GPUs and the TPUs and NPUs built specifically for these workloads, exists because the arithmetic of training grew faster than general-purpose CPUs could follow.

The trust limits

Whiteboard still from board 8 of 18, titled The black box: a 3D black box with pipes in and an arrow out, the label black box in blue, a red note reading even its designers cannot audit, a blue note reading the pattern hides in the math, caliper tools probing the box for the interpretability methods feature importance, partial dependence, SHAP and LIME, and a red box listing medicine, defense, finance, law.

Board 8 of 18: the black box, with interpretability tools probing it from outside.

A working model can remain a black box: some systems output decisions that even the team that built them cannot audit, and deep learning is naturally opaque. The toolbox for opening it includes feature importance, partial dependence plots, SHAP, and LIME. The stakes are highest in medicine, defense, finance, and law.

The other trust limit sits with the operator. Automation bias is the documented human propensity to follow automated suggestions and screen out contradictory information, even correct information. The commission version has pilots shutting down a healthy engine on a false automated alert; the omission version appears in the mammography numbers, where unaided readers caught 46 percent of the study’s cancers and only 21 percent once a faulty aid flagged nothing. Add complacency, which NASA’s aviation safety reporting glossary defines as “self-satisfaction that may result in non-vigilance”, and systems so reliable they interfere with an operator’s ability to take back control.

What are commission and omission errors, concretely?

Commission errors are acts taken on an automated directive that contradicts other evidence. The aviation case on Wikipedia’s automation-bias page: crews received false alerts instructing them to shut down an engine, and they shut down the healthy one. The automation was wrong, the contradicting instruments were right, and the automated suggestion won anyway.

Omission errors are the mirror image: something real goes unflagged by the automation, and the human, trained into reliance on the flag, misses it. In the mammography study the page cites, unaided readers caught 46 percent of the cancers; when a faulty aid flagged none, detection fell to 21 percent. The two directions differ, commission acts wrongly and omission fails to act, and both grow with how reliable the aid usually is, which is why high reliability itself is listed as a contributing factor.

The human limits

The hardest limits concern who answers when a decision touches a person’s life. Microsoft’s Responsible AI standard organizes them as six principles, better read as obligations than slogans:

Principle What it asks in practice
Fairness Equitable benefits and harms, verified by measured outcomes
Reliability and safety Defined behavior under defined conditions, including failures
Privacy and security Transparent personal-data handling, consumer controls, differential privacy
Inclusiveness Designed for people the data underrepresents
Transparency Decisions legible to affected people; GDPR right to explanation
Accountability Designers and deployers answer for it

Accountability is where the principles meet enforcement. The standard’s line is that an AI system should never be the last word on a decision affecting someone’s life, and the EU AI Act gives that idea teeth: recruitment, health, justice, and law-enforcement uses are high-risk, with human-oversight obligations keeping a person in the loop by law. On jobs, three institutions span 47 percent of US employment at risk, 9 percent, and net creation.

The gravest human limit sits outside any commercial roadmap: autonomous weapons, which the US DoD Directive 3000.09 defines as systems that “select and engage targets without further intervention by a human operator” once activated. States have discussed them under the Convention on Certain Conventional Weapons since 2014, and there is still no binding prohibition, leaving blurred who answers for a particular killing.

How big is the AI jobs effect, really?

The spread across three credible institutions is the finding. Frey and Osborne’s Oxford study estimated 47 percent of US employment at high risk of computerization, and it set the terms of a decade of debate. The OECD’s 2016 study, counting across 21 member countries and measuring tasks rather than whole jobs, found on average 9 percent of jobs in foreseeable danger. The World Bank’s position is that technology destroys and creates jobs together and expects net creation over the period it models.

All three agree work changes substantially; they disagree about displacement, by a factor of five at the extremes, which is the honest number to quote when someone asks whether AI takes jobs.

The frontier limits

Three failure modes belong to this generation of large models. Hallucination: LLM chatbots embed plausible-sounding falsehoods in generated content, fabricated citations the signature case, and mitigation combines retrieval grounding, citation checking, and review by humans. The second mode is adversarial: classifiers read pixel patterns imperceptible to people; one adversarially chosen pixel flips a classification, and a planted backdoor can sit undetected. Model collapse: training on uncurated synthetic data degrades the outputs, nicknamed AI inbreeding, long-term extent unclear.

In 2023, GPT-4 scored ahead of 99 percent of people on the Torrance Tests of Creative Thinking, so the claim that machines cannot create is years stale. The verifiable limits: competence stays inside well-defined tasks, foundation models fall short on advanced reasoning and planning, and AGI itself remains hypothetical and contested.

Why can one changed pixel flip a classifier?

Image classifiers read statistics of pixel patterns, not objects the way humans see them. An adversarial example adds a perturbation tuned to those statistics, imperceptible or invisible to a person, that moves the input across a decision boundary; research cited on Wikipedia’s machine learning article includes cases where a single adversarially chosen pixel changes the predicted class.

Backdoors extend the idea during training: a model can learn a trigger pattern that forces a chosen output while behaving normally on ordinary test data, which is what makes them hard to detect. The engineering implication is that accuracy on a clean test set is an upper bound measured in the wrong environment; deployed vision systems get inputs that were never in the set, some of them crafted.

The law moves slower, but it moves

Whiteboard still from board 15 of 18, titled The law moves: a timeline with three ticks reading Apr 2021 proposed, Aug 2024 in force, and 2026-27 phases in, notes listing four risk tiers, bans on real-time public biometric ID and social scoring, red text reading fines up to 35 million euros or 7 percent of turnover, revised mid-course for generative AI, and copyright notes including US no human author no copyright.

Board 15 of 18: the EU AI Act timeline, and the copyright notes beneath it.

Regulation arrived on a schedule you can put on one line. The EU AI Act was proposed in April 2021, entered into force on 1 August 2024, and phases in through 2026 and 2027. It sorts uses into four risk tiers, bans social scoring and real-time biometric identification in public outright, and was revised mid-course to cover generative AI. Fines for the banned practices run to EUR 35,000,000, or to 7 percent of the offender’s total worldwide annual turnover when that is the larger figure.

Copyright runs in parallel. The US Copyright Office holds that works not created by a human author get no copyright, and a fully AI-generated image has been refused registration on that basis. 2025 guidance protects AI-assisted works where human creativity is evident, and training-data lawsuits are live: rules still being written while cases are heard.

What do the four EU AI Act risk tiers mean?

The unacceptable tier is banned outright: real-time remote biometric identification in public spaces and social scoring are the named prohibitions. The high-risk tier covers AI in health, law enforcement, justice, recruitment, education, and critical infrastructure; systems there face obligations on data quality, transparency, and human oversight, which is the clause that turns the ethics-principles discussion above into auditable requirements.

The limited-risk tier imposes transparency duties, mainly telling people when they are interacting with AI or seeing manipulated content. The minimal-risk tier covers everything else, from spam filters to game AI, and stays free of new obligations. The tier structure is why the same technology can be unregulated in one product and heavily regulated in another: the risk lives in the use, not only in the model.

The speculative end, clearly labeled

One entry belongs in its own box: the self-improving AI that becomes hard to stop. The category is hypothesized and contested, a debate about AGI-era capability, and nothing to engineer around today. A 2022 expert survey put the median extinction estimate at 5 to 10 percent. The researcher split is real and named: Hinton and Bengio worry publicly, LeCun and Ng dismiss the concern, and Gebru and Bender argue the fixation distracts from present-day harms.

Gotchas to avoid

If you hear The checked version
"More data fixes bias." Representativeness fixes it; a bigger biased dataset stays biased
"AI can’t be creative." Retired by the 2023 Torrance results; the accurate phrasing is narrow competence with reasoning gaps
"Self-improving AI is a current challenge." It is the contested, speculative end; survey medians 5 to 10 percent
"The AI failed." Audit the operator path too: reliable machine output, and a human who stopped verifying

In short

Whiteboard still from board 18 of 18, the recap, titled In short: a green-ticked checklist reading the data decides, representativeness not volume; overfitting, underfitting, no guarantees, compute; the black box plus automation bias, trust is separate; the law keeps humans in the loop; hallucination, adversarial, collapse; and a yellow sticky note titled IN SHORT reading speculative stays labeled and the law is moving.

Board 18 of 18: the recap checklist and the two-line sticky note.

Mapped honestly, the limits of AI and ML come down to this: what the training data carried decides what the model can do, and representativeness, collected group by group, is the bias repair that holds. Engineering bounds are just as real, from memorized training sets to compute bills that rule out exhaustive search, with opacity and operator drift layered on top of systems that technically work. EU oversight now keeps a person on high-stakes decisions, the frontier’s three failure modes are live, and the entry about self-improving AI stays where it belongs, in the labeled speculative box.

Continue learning


Disclosure: this article was drafted with AI from the script of my video lesson, then checked against its sources before publishing.

This lesson first appeared on iqraa.tech.

More whiteboard lessons on the Whiteboard Architect YouTube channel.

來源:DEV Community · dev.to