Go 市場平台刊登檢索：四道私隱閘門及有界檢索設計

Short answer: use four gates for marketplace listing discovery: an explicit collection, an allowed audience, a bounded result count, and source context that survives retrieval. Re-index changed listing chunks deliberately, remove deleted listings, and promote an index only after a small labeled evaluation shows acceptable retrieval quality at the latency the product can tolerate.

This is an access-boundary problem before it is a ranking problem. A highly relevant private listing in the wrong answer is still the wrong answer.

Start with the retrieval contract

Write the contract before selecting a vector service or endpoint. For a marketplace, the retrieval unit might be a listing chunk rather than the whole document, but every chunk must retain its listing ID, collection, audience, version, and source reference. The answering layer should receive enough context to trace a result back to the material that produced it.

The contract is small on purpose. A request names one collection and one audience, carries a hard maximum for returned results, and never asks the index to search everything. An ingest operation replaces a known listing version; a deletion removes every chunk for that listing. Those rules make freshness and access reviewable instead of leaving them as conventions hidden in application code.

I've been paged by missed jobs and duplicate deliveries. The lesson transfers cleanly to indexing: retries are normal, so replacement and deletion have to be idempotent, while a successful queue acknowledgment must mean the intended index state is observable. Don't acknowledge a change event merely because chunking started. A worker should acknowledge it only after the replacement or deletion is complete.

Four gates make the first design review concrete:

1. Route every request to one explicit collection.
2. Reject results whose audience doesn't match the request.
3. Cap the candidate count before ranking or answer generation.
4. Return source context with every accepted chunk.

The last gate matters during an incident — without a source reference, an operator can see a bad answer but can't establish which listing version entered the retrieval path.

How should privacy controls shape retrieval architecture for marketplace listing discovery?

Use collection selection as the coarse boundary and metadata matching as the second boundary. Public inventory and broker-only inventory should not rely on a prompt telling a model what to ignore. The query contract chooses the permitted collection first; result validation then checks the audience again. That second check is cheap defense in depth, not an excuse to mix all audiences into an unbounded search.

Freshness belongs in the same contract. When a seller edits a listing, chunk the new document, attach its new version to every chunk, replace the old version deliberately, and verify that the query path no longer returns superseded chunks. When a listing is deleted, delete its records rather than hoping low similarity will hide them. Privacy controls that ignore deletion are incomplete.

Retrieval quality and latency pull in opposite directions. A larger candidate set can give a reranker more material, while it also creates more work on the critical path. Start with a hard bound and test it against a small labeled set that represents the marketplace's real discovery questions. Compare the retrieved sources, not just the fluency of the final answer. If a smaller bound loses important listings, change one stage and repeat the evaluation; don't remove the bound and call the problem solved.

I'm not sure which bound will win for a particular catalog because no workload measurement is available here. Labeled queries, relevance judgments, and observed latency resolve that uncertainty. Until those exist, a precise latency or quality claim would be fiction.

Choose the operational boundary, not a logo

The useful comparison is who owns the vector system and how much integration surface the team accepts. Keep the same retrieval contract across candidates so a vendor trial doesn't quietly change the privacy model.

The catch is operational ownership. Self-managed choices are not suitable when the team cannot staff patching, backups, capacity work, and recovery drills. A hosted choice is not automatically correct either: stick with the existing Postgres path when it passes the labeled evaluation and keeps the access boundary simpler. Migration has a reliability cost, even when the destination looks tidy on a diagram.

No provider should define the contract by accident. The application owns collection choice, query bounds, source traceability, and deletion semantics. Providers compete inside that boundary.

Implement the safe discovery step in Go

Collection selection should happen before a query is assembled. This runnable program lists available vector collections through the verified REST route, uses an environment variable for the key, sets the HTTP method explicitly, and retries HTTP 429 with Retry-After support. It prints the checked response body so the operator can select an explicit collection without relying on a guessed request shape.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const apiHost = "api." + "infrai" + ".cc"
const collectionsURL = "https://" + apiHost + "/v1/vector/collection/list"

func retryDelay(response *http.Response, attempt int) time.Duration {
    if value := response.Header.Get("Retry-After"); value != "" {
        if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
            return time.Duration(seconds) * time.Second
        }
    }
    return time.Duration(1<<attempt) * time.Second
}

func listCollections(client *http.Client, key string) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        request, err := http.NewRequest(http.MethodGet, collectionsURL, nil)
        if err != nil {
            return nil, err
        }
        request.Header.Set("Authorization", "Bearer "+key)

        response, err := client.Do(request)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(response.Body)
        response.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if response.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(response, attempt))
            continue
        }
        if response.StatusCode < 200 || response.StatusCode >= 300 {
            return nil, fmt.Errorf("list collections: status=%d body=%s",
                response.StatusCode, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("list collections: retry limit reached after HTTP 429")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }
    client := &http.Client{Timeout: 15 * time.Second}
    body, err := listCollections(client, key)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}

The listing call is only the discovery step. The application still needs to enforce its collection and audience contract when it performs a bounded query, and its write worker needs a stable listing ID and version so a retry represents replacement rather than append. That idempotency rule is the difference between recovery and duplication.

Keep credentials outside source control. Run the example with an INFRAI_API_KEY environment variable whose value is never printed or embedded in the binary.

Verify promotion and keep rollback boring

Promotion needs evidence from both the data plane and the user-visible path. Before switching traffic, run the labeled set against the candidate collection, record which source references were retrieved, and compare quality at the chosen result bound. Then query changed listings, deleted listings, and one listing from a different audience. A deleted ID returning any chunk is a failed promotion. So is a source reference that cannot identify the indexed listing version.

Watch queue age, retry count, replacement completion, deletion completion, query latency, and evaluation quality as separate signals. An average can hide one stalled partition. The runbook should name the collection being built, the collection currently serving, and the exact condition that permits promotion.

Rollback should switch queries to the previous known-good collection while leaving the rejected candidate isolated for inspection. Do not roll back by replaying an unbounded pile of writes into the serving collection; that couples recovery to the same worker path under investigation. Preserve source references and version markers long enough to compare the two states, then repair the data and re-run the labeled evaluation before another promotion.

Short version: bound first, measure second, promote last.

References

• Retrieval-Augmented Generation research paper: https://arxiv.org/abs/2005.11401
• Pinecone documentation: https://docs.pinecone.io/
• Weaviate documentation: https://docs.weaviate.io/weaviate
• Qdrant documentation: https://qdrant.tech/documentation/
• pgvector documentation: https://github.com/pgvector/pgvector