AlloyDB AI 如何在不中斷服務下升級嵌入模型

Imagine you've just launched a new semantic search feature for product reviews to help customers quickly research products on your site. Users love it. But the next day, you see an announcement for a new vector embedding model promising higher accuracy, better nuance, and multilingual search out of the box. After testing it in your dev environment, you confirm it's a perfect fit. But now you're left with a common dilemma: How do you upgrade the model without disrupting your users?

Upgrading embedding models in production requires re-embedding your entire dataset, rebuilding indexes, and coordinating schema migrations without interrupting user queries. You may encounter several potential problems:

• Embedding generation: All existing data must be re-embedded using the new embedding model, which typically requires updating complex pipelines and using dual write strategies while the new and old models run side-by-side for a period of time. 

• Mathematical incompatibility: Embeddings generated by different models inhabit distinct vector spaces and cannot be compared directly. Application query vectors and stored column vectors must switch models in unison to prevent severe search degradation.

• Index rebuild overhead: Vector indexes must be built on the new vectors before similarity search can perform efficiently at scale.

• Degraded user experience: Switching embedding models requires multiple DDL changes and other operations that acquire exclusive locks and can stall incoming read and write transactions if not carefully managed.

How AlloyDB AI simplifies vector lifecycle management

In this blog, we’ll highlight how the rich feature set in AlloyDB AI combines to enable zero-downtime embedding model upgrades, so that your users can benefit from the latest model advances without scheduled downtime maintenance. Model endpoint management gives you native in-database access to any embedding model (first party, third party, or even self-hosted), while auto embeddings rapidly generates new embeddings in the background (up to 3000/second) and keeps them up to date as source data changes. Using AlloyDB’s ScaNN index can also significantly reduce the wall clock time of your upgrade with index builds that are 10x faster than open source HNSW, and native PostgreSQL functionality like concurrent index creation, set-returning functions (SRF), and atomic column renames allow transparent blue/green and canary-style rollouts.

Step-by-step migration walkthrough

In this example, we demonstrate migrating a production table ecomm.product_reviews from text-embedding-005 (768-d) to gemini-embedding-001 (3072-d) while serving live traffic.

Step 0: Prerequisites and preflight checks

Verify that the google_ml_integration extension is updated to version 1.5.6 or later (tested on 1.6.1) and confirm model connectivity:

Step 1: Add the new vector column

Adding a nullable column without default values updates catalog metadata without creating new versions of each row. This makes the operation nearly instant. Note that the size of your vector column (3072 in this example) must match the number of dimensions output by your embedding model. We also set the lock_timeout to 2 seconds to prevent the operation from blocking critical production traffic while it waits to acquire an exclusive lock. If the operation fails due to a longer wait for an exclusive lock, simply retry.

Step 2: Initialize auto-embeddings and run backfill

Next, we initialize embeddings on our new embedding column using gemini-embedding-001.

Note that there are two modes available in auto embeddings to keep embeddings up to date as source data changes:  

• Transactional: When you set incremental_refresh_mode to transactional, embeddings are updated immediately when data changes. This keeps your embeddings transactionally consistent with your source data, but it also introduces additional write latency with an inline embedding model inference required for every INSERT/UPDATE operation. Use transactional mode when you want to keep your embedding data transactionally consistent with the source data and your rate of change is low.

• Manual: When you set incremental_refresh_mode to manual, AlloyDB tracks which rows have changed but does not immediately update the embedding. This allows you to quickly regenerate embeddings for only the changed rows (not the full dataset) on a schedule that you define using ai.refresh_embeddings(), which uses efficient batching techniques just like the initial backfill operation. Use manual mode when you have a high rate of change on your source data and you can tolerate embedding drift for the period of time defined by your scheduled refresh interval.

We use manual mode in this example to keep writes on the base table performant, while still having full control of how quickly changes are reflected in the embeddings by periodically running ai.refresh_embeddings(). 

Managing concurrent writer wait times
During backfill, AlloyDB Auto Embeddings processes rows in parallel groups (batch_size × max_parallel_requests). While the process for each micro batch is usually fast, concurrent updates or deletes have to wait for the batch to commit.

To cap the maximum wait times for writers on busy transactional tables, you can lower max_parallel_requests during the backfill session:

This change generally improves the commit time for each micro batch, while increasing the total wall clock time for the backfill operation. Our benchmark results on a 10,000-row table demonstrate the trade-off:

Monitoring auto embeddings progress

AlloyDB provides a view to track progress for auto embedding jobs, including state, rows processed, and estimated time remaining. To see how things are progressing, run the query below from a separate session:

Step 3: Catch up updated rows

Once the initial backfill operation completes, we can optionally regenerate embeddings for rows that were modified while it was running. When using manual mode, Auto Embeddings automatically tracks which rows were changed so that they can be quickly updated without re-embedding data that is already current. To catch those rows up, simply execute a quick refresh using ai.refresh_embeddings(). You should also run this refresh command on a schedule using pg_cron or an external scheduler to ensure the latest changes continue to be reflected in your embeddings .

Step 4: Validate vector completeness and retrieval quality

Now that the embeddings are generated and caught up, we’ll ensure completeness and correctness prior to traffic cutover. Note that in a live product environment, you will likely have active INSERT / UPDATE / DELETE traffic that modifies rows between steps 3 and 4. To prevent the results below from being skewed, we added a filter on google_ml_track_stale_embedding = 'false' to include only rows that haven’t been changed since the last invocation of ai.refresh_embeddings(). Note that your table may use a different name for this column, such as google_ml_track_stale_embedding_2, so be sure to update the query accordingly.

At this point, you should also validate your application logic using the new embedding model. In internal benchmarks comparing text-embedding-005 and gemini-embedding-001, the top-10 nearest neighbor results overlapped by only 31%. This is a good thing because the shift reflects the higher semantic fidelity of the 3072-d model. However, the new results for recall@10  may impact assumptions in your application, so be sure to conduct offline testing against a set of sample golden queries before routing production traffic to the new embeddings.

Step 5: Build a new ScaNN vector index 

Searching across a large number of vectors without an index is a computationally expensive and slow process. To avoid this, we’ll build a ScaNN index to enable industry leading filtered vector search performance, which is 10x faster than HNSW on open source PostgreSQL.

High dimensional models like gemini-embedding-001 require additional memory during index construction compared to lower dimensional models, so in this step we increase maintenance_work_mem to accommodate the additional overhead. We also build the ScaNN index using CONCURRENTLY to avoid blocking concurrent writes:

Building a ScaNN index is also up to 10x faster than HNSW on open source PostgreSQL. In our tests on 100,000 rows, a 3072-d ScaNN index built in approximately 50 seconds, whereas an HNSW index using halfvec took 7 to 13 minutes.

Be sure to also tune scann.num_leaves_to_search to balance recall and latency. In our tests, increasing num_leaves_to_search from the default value to 25 improved Recall@10 from 0.35 to 0.95 at a search latency of 8.8 ms:

Step 6: Cut over search traffic with zero downtime

At this stage, you have two primary options for your application cutover strategy:

• Pattern A: Atomic column rename swap — This is a simple approach that supports blue/green deployments with zero application code changes. It does not, however, support canary-style releases or auto embeddings in transactional mode (supports manual mode only).

• Pattern B: Custom GUC session variables + unified UNION ALL set-returning function (SRF) — This more advanced approach enables transparent query embedding model selection, live incremental traffic splitting, blue/green deployments, and canary A/B testing, while requiring only one application config change. This technique also supports auto embeddings in both transactional and manual modes.

Pattern A: Atomic column rename

First, we route application query embedding generation through a custom database function to abstract the model from the application. Application queries will use this function instead of directly using ai.embedding():

Then, when we’re ready to cut over, we simply execute a stored procedure like the example below to swap the column names and update the function to use the new model. 

Create the procedure:

Then execute the swap:

Because the column rename and model switch execute in the same transaction, queries never encounter a model mismatch. If you need to roll back, simply call the procedure with the column arguments reversed.

One advantage to this approach is that there are zero changes required on the application side. Your application code consistently uses the ecomm.product_reviews_query_embedding function to generate an embedding, and the column name used by the application query does not change.

While this column rename technique is simple, it has two limitations: First, it applies exclusively to manual mode for auto embeddings. In transactional mode, AlloyDB-managed triggers reference columns by hardcoded names, so renaming a column causes write operations to fail. Second, this pattern only supports blue/green style releases. You don’t have per-query control over which model you use, so you’re unable to use canary-style releases with, for example, 5% of traffic using the new model for testing while 95% of the traffic continues to use the existing model. 

Pattern B: Custom GUC + set-returning function for canary A/B testing

To execute canary rollouts, traffic splitting, or instant rollbacks, you can use custom PostgreSQL Custom Grand Unified Configuration (GUC) variables paired with a set-returning function abstraction.

1. Define the table-valued function abstraction layer
Create a database SRF that inspects a session-level GUC variable (app.embedding_model) to dynamically evaluate search queries against either the legacy or the new embedding column.

2. Standardized application queries
The application queries the unified SRF while generating query vectors via ai.embedding() using the active model parameter:

3. Execute canary traffic splitting and A/B testing
Application connections or middleware can dynamically control which model processes a query by setting the session variable:

Using this pattern, engineering teams can route a small percentage of application sessions to the new embedding model, validate accuracy and latency metrics, then gradually increase traffic to 100%. Or, they can revert instantly to the prior model by simply altering the session parameter.

A note on connection pooling: When using connection poolers such as PgBouncer in transaction-mode, a plain SET stays on the pooled server connection. This means it can leak to other clients, and it may not always apply to your own subsequent transactions. To mitigate this, you can use SET LOCAL inside an explicit transaction so the setting is cleared at COMMIT or ROLLBACK. You can also prevent this condition entirely by using PgBouncer in session mode, which clears session parameters when a connection is returned to the pool.

Step 7: Retire the legacy model and reclaim storage

Once you have completed the cutover to the new model and you are happy with its functionality and stability, you can decommission the legacy embedding column, index, and auto embedding configs to reclaim disk space.

Drop the auto embedding config and the old index:

Reclaiming disk space

Executing ALTER TABLE DROP COLUMN removes the column from PostgreSQL catalog tables, but the underlying TOAST space remains allocated on disk until data pages are rewritten. To make this disk space reusable for future operations, you can set the old vector column to NULL in batches before dropping it. 

Note that this operation is a trade-off: It creates new row versions for every row in the table in order to make disk space reusable, which requires a VACUUM to clean up the old row versions. You should determine whether it is a priority for you to immediately free up disk space for reuse on the same table versus allowing the old row versions to naturally age out through the course of normal operations.

In our tests on 100,000 rows, zeroing out the vectors and vacuuming before dropping the column reclaimed 381 MB of TOAST space for reuse by subsequent model upgrades.

Transactional mode considerations for auto embeddings

If your application uses auto embeddings with incremental_refresh_mode => 'transactional', vectors are generated synchronously inside each write transaction. Upgrading in transactional mode follows the same sequence shown above, with three key differences:

• Step 3 is skipped: You don’t need to run ai.refresh_embeddings(), because new writes update both columns automatically.

• Column swap technique is not supported: You must use the GUC + SRF abstraction method, which does not impact the triggers managed by auto embeddings.

• Write latency increases during migration: While both models are configured, each row write executes two remote embedding calls.

Because gemini-embedding-001 generates a more expressive 3072-dimensional embedding, its remote inference call takes longer. In our tests, we observed the following latency profiles for single-row UPDATE operations with transactional mode enabled for auto embeddings:

Be sure to review your application's write latency budget before choosing transactional mode for auto embeddings, especially with higher-dimension models.

Disk space and memory resource planning

Because old and new vector columns coexist during migration, your table storage will grow based on peak usage. Based on our testing, we observed the following average storage per row for each embedding type:

Based on these estimates, you can use the formula below to estimate your storage space:

• New vector column: Total Rows × ~13.3 KB (approx. 3x legacy storage).

• New ScaNN index: Total Rows × ~0.6 KB.

• Headroom Allocation: Add ~25% overhead for WAL generation and page updates during backfill and vacuum operations.

Example: A table with 10 million rows requires approximately 133 GB of disk space for the new vector column, 6 GB for the ScaNN index, plus headroom.

Summary

AlloyDB combines native PostgreSQL primitives with built-in AI capabilities to make upgrading vector embedding models in production faster and earlier, with no maintenance downtime or complex application-side changes. By combining AlloyDB AI auto embeddings and Model Endpoint Management with standard PostgreSQL operational controls, database administrators and developers can keep their semantic search models up to date with the latest embedding model capabilities as soon as they are released.

To get started, launch an AlloyDB trial cluster and consult the official AlloyDB AI documentation.

Posted in

• Databases