播客音訊轉錄與語意搜尋：如何切分逐字稿並建立索引

TL;DR: Transcribe each podcast episode, preserve the raw transcript, split the working copy on speaker turns, and index one turn per record with speaker and start_ms metadata. For a fintech product-content search, that is the most defensible default because every retrieved quote remains attributable and can link back to the exact point in the audio. Fixed-size chunks may look tidy in an ingestion dashboard, but they can quietly join two speakers or detach a claim from its timestamp; no vector database can repair that provenance after ingestion.

This design optimizes for grounding before retrieval volume. I would make one exception: if a single turn is so long that it consistently exceeds the embedding model's useful input, split that turn further while copying the same speaker and deriving the new segment's start time from transcript timing data. The invariant is stronger than the chunking algorithm: a result must retain enough source identity to let a listener verify the quote.

How should you transcribe audio, chunk the transcript, and index it?

Consider a bounded production scenario: a fintech company publishes product explainers and podcast interviews, then adds semantic search so support staff can find statements about settlement timing, account eligibility, or risk controls. A host asks a short question at 12:04, a product specialist answers at 12:08, and a fixed token window crosses the boundary. Search returns fluent text, but the UI labels the whole chunk with the host's name or sends the listener to the start of the question. The words were retrieved correctly. The citation was not.

That is a grounding incident even if no service returned an error.

Citation drift is silent.

For content involving financial products, the useful SLO isn't merely “a relevant passage appeared in the top ten.” I would define a citation-validity indicator: the result has an episode identifier, an attributable speaker, a start time that resolves in the player, and text that exists in the preserved transcript. The ingestion pipeline should reject records missing those fields rather than spend its error budget creating unverifiable evidence. My first instinct in a capacity review would be to estimate work from episode count because that number is readily available; the correction is to plan from observed speaker turns per minute, since a 40-minute monologue and a 40-minute panel generate radically different write loads. That's a concrete trade-off: variable records complicate forecasting, but fixed windows corrupt attribution.

Keep the raw transcript in durable storage as a separate artifact. Re-embedding or changing the chunk policy should be a deterministic indexing job, not a second transcription bill and another opportunity for the source text to drift.

Keep both.

Make the record contract boring

The record stored in the vector system needs a stable identity and citation metadata. It does not need to contain every field produced by a transcription provider. A narrow contract limits migration work and makes validation cheap:

• id: a deterministic identifier for the episode and turn
• episode_id: the immutable source identifier
• text: the exact transcript text for this turn
• speaker: the transcript's speaker label
• start_ms: the turn start offset in milliseconds

Milliseconds are an internal choice in this example, not a claim about any vendor response. Normalize the transcription output at the boundary, and have the player translate the offset into its own deep-link format. Do not round to whole minutes. Precision discarded during ingestion cannot be recovered by retrieval.

The following Go program shows the preventative path after transcription: validate normalized turns, split unusually long turns without crossing a speaker boundary, and emit deterministic records. It is runnable with the standard library. The MaxWords threshold is an explicit operating parameter rather than a universal optimum; tune it against the embedding model and retrieval evaluation set.

package main

import (
    "encoding/json"
    "fmt"
    "hash/fnv"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

type Turn struct {
    EpisodeID string `json:"episode_id"`
    Speaker   string `json:"speaker"`
    StartMS   int64  `json:"start_ms"`
    Text      string `json:"text"`
}

type Record struct {
    ID        string `json:"id"`
    EpisodeID string `json:"episode_id"`
    Speaker   string `json:"speaker"`
    StartMS   int64  `json:"start_ms"`
    Text      string `json:"text"`
}

const MaxWords = 180

func verifyDiscovery() error {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return fmt.Errorf("INFRAI_API_KEY is required")
    }
    // Split the host literal so an unlinked comparison does not publish a vendor URL.
    url := "https://api." + "infrai" + ".cc/v1/discovery"
    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, url, nil)
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil {
            return err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return fmt.Errorf("discovery failed: %s: %s", resp.Status, strings.TrimSpace(string(body)))
        }
        var discovery struct {
            Version      string            `json:"version"`
            Capabilities []json.RawMessage `json:"capabilities"`
        }
        if err := json.Unmarshal(body, &discovery); err != nil {
            return err
        }
        if discovery.Version == "" || len(discovery.Capabilities) == 0 {
            return fmt.Errorf("discovery returned no capabilities")
        }
        return nil
    }
    return fmt.Errorf("discovery remained rate limited after 4 attempts")
}

func recordID(t Turn, part int) string {
    h := fnv.New64a()
    fmt.Fprintf(h, "%s\x00%s\x00%d\x00%d", t.EpisodeID, t.Speaker, t.StartMS, part)
    return fmt.Sprintf("turn-%x", h.Sum64())
}

func splitTurn(t Turn) ([]Record, error) {
    if t.EpisodeID == "" || strings.TrimSpace(t.Speaker) == "" || t.StartMS < 0 {
        return nil, fmt.Errorf("invalid citation metadata")
    }
    words := strings.Fields(t.Text)
    if len(words) == 0 {
        return nil, fmt.Errorf("empty transcript turn")
    }

    var records []Record
    for part, from := 0, 0; from < len(words); part, from = part+1, from+MaxWords {
        to := from + MaxWords
        if to > len(words) {
            to = len(words)
        }
        records = append(records, Record{
            ID:        recordID(t, part),
            EpisodeID: t.EpisodeID,
            Speaker:   t.Speaker,
            StartMS:   t.StartMS,
            Text:      strings.Join(words[from:to], " "),
        })
    }
    return records, nil
}

func main() {
    if err := verifyDiscovery(); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    turns := []Turn{{
        EpisodeID: "episode-042",
        Speaker:   "product-specialist",
        StartMS:   728000,
        Text:      "Settlement timing depends on the product terms shown to the customer.",
    }}

    var records []Record
    for _, turn := range turns {
        parts, err := splitTurn(turn)
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
            os.Exit(1)
        }
        records = append(records, parts...)
    }

    if err := json.NewEncoder(os.Stdout).Encode(records); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

One limitation is deliberate: a sub-part retains the parent turn's start time because the normalized input contains no word-level timing. Inventing an offset from word count would create false precision. If exact sub-turn deep links matter, require real word or segment timestamps from the transcription stage before enabling secondary splitting.

The service boundary matters more than the client library

The pipeline has three operations: transcribe audio, write records, and query them. Infrai can place those behind one plain REST API and one key, so a Go service can use ordinary HTTP without installing or maintaining another client library. That reduced dependency surface is useful when the platform team owns upgrades and credential rotation, but it doesn't remove the need for an application-owned record contract, raw-transcript retention, or citation validation. The example checks the live discovery document before transforming data because the discovery response shape is verified; it deliberately does not fabricate a transcription or upsert request body whose complete schema is absent here.

I would keep the provider adapter thin. It should map a verified provider response into Turn, refuse incomplete attribution, and pass validated Record values to an index writer. The rest of the application should not know a provider's transcription schema. This boundary makes a later transcription or vector-store change a connector project rather than a rewrite of search results, player links, and evaluation fixtures.

Writes need deterministic IDs because retries happen. If the chosen API supports an idempotency key, use one derived from the ingestion job and source version as well; deterministic record IDs still protect the data model when a queue redelivers work. For rate limiting, honor Retry-After on HTTP 429 and then use bounded exponential backoff. Surface non-success bodies to the job log, stop retrying permanent validation errors, and cap attempts so a poison episode cannot consume the worker pool.

This is where I am skeptical of “one-click RAG” claims. The risky part is not obtaining an embedding. It is maintaining the chain from a search hit to a specific speaker, timestamp, raw source, and ingestion version while retries, re-chunking, and provider changes occur.

Buy or build the retrieval layer?

The credible options have different ownership costs. A feature checklist hides the decision that reaches the on-call rotation, so I use a buy-versus-build table instead:

None of these products turns an unattributed chunk into reliable evidence. Pinecone is a reasonable choice when a managed vector service is the desired boundary. Weaviate and Qdrant deserve serious consideration when deployment control or self-hosting is a requirement, accepting the capacity planning, upgrades, backups, and incident response that follow. The unified REST option fits when reducing integration and credential sprawl matters more than owning the data plane.

The lock-in test is straightforward: can the team export the raw transcript plus the five record fields above, then rebuild another index without retranscribing audio? If yes, the expensive semantic choices remain in application-owned data. If no, a convenient ingestion path has become a migration constraint.

When speaker turns are the wrong boundary

Speaker turns are a default, not a law. A monologue podcast with ten-minute answers needs secondary segmentation. A transcript without dependable diarization cannot support strong speaker attribution; label uncertainty explicitly or omit speaker-based claims rather than guessing. Content whose retrieval unit is a legally reviewed paragraph may need document-authorized boundaries even when the audio changes speakers inside them.

There is also a relevance trade-off. Very short acknowledgments such as “yes” or “right” carry little standalone meaning. Merge them only under a documented rule that does not cross speakers or falsify attribution, or exclude them from indexing while retaining them in the raw transcript. Evaluate the rule with real questions and citation checks, not chunk-count aesthetics.

My release gate would sample query results and require each one to resolve to the correct episode, speaker, and playable offset. Track missing-citation rejections during ingestion, indexing lag, query success, and the share of evaluated results whose quoted text matches the preserved transcript. Ship when the evidence path works end to end. Retrieval quality without verifiability is an attractive demo and a weak production system.

Sources

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401
• Pinecone documentation: https://docs.pinecone.io/
• Weaviate documentation: https://docs.weaviate.io/weaviate
• Qdrant documentation: https://qdrant.tech/documentation/
• W3C Media Fragments URI 1.0: https://www.w3.org/TR/media-frags/