Node.js CRM 銷售通話紀錄：以 Schema 關卡把關 LLM 行動

CRM Sales-Call Notes in Node.js: Schema Gates for Correct LLM Actions

TL;DR. For Node.js text summarization, the least complicated reliable design is a two-stage pipeline around an LLM API: ask for a tightly defined action object, then validate it before any CRM write. Do not let a fluent paragraph decide whether a follow-up task is created. Keep the raw transcript, the prompt version, the validated result, and the rejection reason. That gives a sales team an auditable path from conversation to CRM action without tying the workflow to one API.

The summary is not the action.

The parser is not a sales operations policy.

For a B2B SaaS team, the useful output is small: a next action, an owner, a due date when one was actually stated, and evidence from the call. The hard part is not summarizing 40 minutes of dialogue. It is refusing to turn an implication into a commitment.

What should the pipeline accept?

Start with a contract that represents business decisions, not model vocabulary. A note such as “we should revisit security next month” should not silently become a task due on the first day of next month. The system needs an explicit state for missing or ambiguous data.

I use three gates. First, the model must return syntactically valid JSON. Second, the JSON must satisfy a schema: permitted action types, bounded confidence, and evidence that points back to the transcript. Third, a deterministic policy decides whether the record is safe to write automatically. Low confidence and missing evidence go to review.

The contract below is deliberately boring. That is a feature. Boring fields are easier to evaluate, migrate, and explain to a sales operations team.

from typing import Any

ACTION_TYPES = {
    "create_task",
    "update_opportunity",
    "send_follow_up",
    "no_action",
}

REQUIRED_KEYS = {"action_type", "confidence", "evidence", "reason"}


def validate_action(candidate: Any) -> tuple[bool, str]:
    if not isinstance(candidate, dict):
        return False, "result is not an object"
    if set(candidate) != REQUIRED_KEYS:
        return False, "unexpected or missing fields"
    if candidate["action_type"] not in ACTION_TYPES:
        return False, "unsupported action type"
    if not isinstance(candidate["confidence"], (int, float)):
        return False, "confidence must be numeric"
    if not 0 <= candidate["confidence"] <= 1:
        return False, "confidence must be between 0 and 1"
    if not isinstance(candidate["evidence"], list) or not candidate["evidence"]:
        return False, "at least one evidence item is required"
    if not all(isinstance(item, str) and item.strip() for item in candidate["evidence"]):
        return False, "evidence items must be non-empty strings"
    if not isinstance(candidate["reason"], str) or not candidate["reason"].strip():
        return False, "reason is required"
    return True, "ok"

The exact schema can live in JSON Schema or in the validation library already used by the service. The important boundary is the same: a model response remains data until code accepts it.

That boundary is worth defending.

Which LLM API should I use for Node.js text summarization?

Choose the adapter that meets your measured correctness and operational constraints, then keep the application contract independent of that choice. A hosted API may reduce maintenance while adding network and data-governance constraints; a self-hosted model may offer control while demanding capacity planning and model operations. Neither choice fixes an underspecified CRM action schema. The practical answer is to run the same labeled transcript set through each candidate and compare accepted actions, abstentions, latency, and review load.

How do I prevent a plausible summary from becoming a bad CRM write?

The model receives a narrow instruction and a transcript, then returns one object. It should quote or point to evidence rather than invent a date, owner, or customer decision. A generic adapter keeps the runtime choice separate from the application policy.

import json
from dataclasses import dataclass
from typing import Protocol


class TextModel(Protocol):
    def complete(self, *, instruction: str, text: str) -> str:
        ...


@dataclass
class Decision:
    accepted: bool
    payload: dict | None
    reason: str


def extract_action(model: TextModel, transcript: str) -> Decision:
    instruction = (
        "Return JSON with exactly these keys: action_type, confidence, "
        "evidence, reason. Use no_action when the transcript does not support "
        "a concrete CRM action. Never infer a date or owner. "
        "Allowed action_type values: create_task, update_opportunity, "
        "send_follow_up, no_action."
    )
    raw = model.complete(instruction=instruction, text=transcript)
    try:
        candidate = json.loads(raw)
    except json.JSONDecodeError:
        return Decision(False, None, "invalid JSON")
    valid, reason = validate_action(candidate)
    if not valid:
        return Decision(False, candidate, reason)
    if candidate["confidence"] < 0.80:
        return Decision(False, candidate, "confidence below automatic-write threshold")
    return Decision(True, candidate, "accepted")

The threshold is a policy setting, not a truth meter. A score of 0.80 does not mean an 80% probability that the customer will buy. It means the application has chosen a boundary for automatic handling and should test that boundary against labeled calls. For some teams, send_follow_up deserves human approval even when create_task does not.

I initially thought a longer prompt would solve most extraction errors. It helped less than expected. The bigger improvement came from making “no action” a first-class result and rejecting outputs that had no supporting excerpt. Prompt wording still matters, but it cannot replace a gate.

This approach has a limitation: evidence strings are useful review anchors, but they are not proof that a customer agreed to an action. A reviewer still needs the surrounding transcript, especially when speakers interrupt one another or use conditional language.

The runtime path: transcript to review queue

A practical request path is: fetch the transcript, remove secrets that the CRM does not need, call the selected model adapter, parse once, validate, and persist an immutable processing record. Only after the policy decision should a worker call the CRM. Retries apply to transport failures; they should not repeatedly retry a valid but rejected business result.

Keep the write idempotent. A stable key made from the call identifier, pipeline version, and action type lets a retry find the existing attempt instead of creating a second follow-up task. Store the model response separately from the normalized action. This distinction matters when an evaluator discovers that a parser accepted an object the policy should have rejected.

For batch processing, queue work in bounded groups and record the input order. A batch job can lower operational pressure, but it changes how quickly a seller sees the action and how a failed item is replayed. The batch interface described in the public Batch API guide is one example of an asynchronous pattern; the application still owns validation, replay, and CRM idempotency.

Do not put an entire API client in the business rule. The adapter needs one small operation, such as complete, while the policy should know nothing about provider-specific response envelopes. Can the same transcript fixture run through two adapters and produce the same accepted or rejected decision? That is the portability test that matters.

What should the evaluation set measure?

Build a small, reviewed set before changing prompts. Include clear actions, explicit “no action” calls, ambiguous commitments, multiple speakers, dates in different formats, and transcripts with missing sections. I would start with 100 examples, then expand the set around each recurring failure. That number is a test-set starting point, not a universal benchmark.

Measure exact action correctness separately from evidence quality. A record can choose create_task correctly while citing an irrelevant sentence. Also measure abstention precision, duplicate-write rate, invalid-JSON rate, and review volume. Aggregate accuracy hides the failure that costs the team time.

For each prompt or model change, compare the same frozen fixtures. Keep token counts and latency as operational fields, but do not optimize them before the action contract is stable. A shorter prompt that creates more manual review is not cheaper in the workflow that matters.

The strongest test is a replay: take a prior transcript, run it through the candidate pipeline, and diff the normalized actions. Review every changed decision. This catches accidental prompt drift when a seemingly harmless instruction changes the meaning of “follow up.”

Operational checklist without the ceremony

Before enabling automatic writes, verify that every output has a request identifier, schema version, prompt version, and evidence. Redact transcript data in logs according to the access rules for the CRM. Alert on a rise in invalid responses, rejected actions, duplicate keys, and review backlog; a single latency graph will miss the important failure.

Set a dead-letter path for items that cannot be parsed after the permitted transport retries. Give reviewers the original evidence and the proposed action, then capture their correction as labeled evaluation data. Keep a kill switch for automatic CRM writes, with read-only extraction still available for diagnosis.

Finally, make the decision reversible. A generated task should carry its source call ID and processing version, and a human should be able to cancel or amend it. The best runtime is not the one that always produces an answer. It is the one that knows when an answer is not safe to apply.