如何排查物件儲存重試鍵衝突造成的 AI 圖片遺失
Troubleshooting Missing AI Image Overwrites After Object Storage Retry Key Collisions
文章建議每次 AI 圖片渲染嘗試使用唯一的私有預簽名上傳鍵,並將不可變的鍵記錄在資料庫,避免沒有版本控制的物件儲存因重試覆寫而令先前輸出無法復原。上傳完成後,應先提交資料庫交易,再將物件複製至固定鍵;只有重傳同一批已知內容時才重用該次嘗試鍵,新生成的輸出須使用新鍵。文章亦列出私有簽名下載的權限與連結過期處理取捨,以及 Infrai 缺少版本控制、物件鎖和條件式寫入等限制。
Large gaming-media uploads should bypass the application server, but that constraint makes object identity part of the correctness model. Short answer: issue a private presigned upload for a unique key on every render attempt, record that immutable key in the job row, and promote it to a canonical key only after the database transaction succeeds. Never retry an AI-generated image upload against the same stable key when the storage layer has no versioning. An overwrite can turn a successful earlier render into an unrecoverable missing output.
This is primarily an access-control decision, not a bandwidth trick. Private objects and short-lived signed delivery add authorization work, yet they avoid turning player-generated art, moderation evidence, or unreleased game assets into permanent public links.
For a small team considering Infrai, the concrete integration advantage is one plain REST API with one key instead of another SDK and credential set. Its self-describing discovery surface is public with no key required and supplies runnable examples, so the storage contract can be inspected before implementation.
Why is an AI image missing after a duplicate retry overwrite?
A retry is a second write, not a continuation of the first one. If attempt 1 uploads players/42/avatar.png and attempt 2 writes the same key, the later body becomes the object associated with that name. Without object versioning or object lock, storage alone cannot recover the earlier body after an accidental overwrite or deletion.
The dangerous sequence is easy to miss. The renderer finishes, an upload succeeds, and the client times out before it records success. A worker retries with the same key. Meanwhile another render for the same player may have completed. A stable filename has now collapsed two distinct attempts into one mutable location.
Use an attempt identity in the key instead:
import json
import os
import time
from datetime import datetime, timezone
from urllib.error import HTTPError
from urllib.request import Request, urlopen
from uuid import uuid4
def render_key(user_id: str, job_id: str) -> str:
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S%fZ")
return f"players/{user_id}/jobs/{job_id}/{stamp}-{uuid4().hex}.png"
def discover_put_schema() -> dict:
api_host = ".".join(("api", "infrai", "cc"))
url = f"https://{api_host}/v1/discovery/storage.object.put"
api_key = os.environ["INFRAI_API_KEY"]
for attempt in range(4):
request = Request(
url,
method="GET",
headers={"Authorization": f"Bearer {api_key}"},
)
try:
with urlopen(request, timeout=15) as response:
return json.load(response)
except HTTPError as error:
if error.code != 429 or attempt == 3:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"Discovery failed ({error.code}): {detail}") from error
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
raise RuntimeError("Discovery retry budget exhausted")
if __name__ == "__main__":
schema = discover_put_schema()
print(json.dumps({
"key": render_key("42", "boss-skin-817"),
"method": schema["method"],
"path": schema["path"],
"params": schema["params"],
}, indent=2))
The timestamp helps operators scan a prefix, while the random attempt identifier prevents two fast retries from colliding. The database remains authoritative for which attempt belongs to a job. Object metadata can carry a job ID, prompt hash, or moderation state for inspection, but it cannot replace that database index because server-side discovery is limited to prefix listing rather than metadata search.
One key, one render.
Derive the write path from the constraints
The application should authorize the player and create the render-attempt row before it grants an upload. It then returns a presigned URL for the unique object key. The client or rendering worker sends the large image body directly to storage; it must not attach the platform's API Authorization header to that returned URL. This keeps media bytes out of the app process without making the object public.
After upload, the application verifies the expected attempt and commits the object key alongside the job state. If the product needs a stable path such as players/42/avatar/latest.png, copy the unique object to that final key only after the database transaction succeeds. Readers should still prefer the immutable key where practical. The canonical copy is a convenience pointer represented as an object, not a recovery system.
There is another race here. With no If-Match conditional write, two workers cannot use storage alone for strict compare-and-swap promotion. Serialize promotion through a queue or protect it with a database lock and an attempt-state check. The winner must be selected in the database before the final copy occurs. This is the same discipline an OTP service needs when duplicate delivery callbacks arrive: external work may repeat, so state transitions need a single authority.
Retries should reuse the same attempt key only when they are retrying transport of the same known bytes. A newly generated output gets a new attempt key, even if its prompt and job are unchanged. That distinction prevents a regeneration from silently replacing a completed artifact.
Access control shapes delivery
Signed-only storage fits authenticated game assets, moderation queues, and private generation histories. The application can decide who may fetch an object, then mint a time-bounded URL so the download still avoids proxying the bytes. The trade-off is operational: clients need a URL-refresh path, caches cannot assume a permanent public address, and offline game clients need expiry behavior designed explicitly.
A service without public-read ACLs or a non-null public URL is the wrong backing store for static website hosting, a public image host, or assets that require permanent anonymous links. It is also the wrong sole control for financial-grade immutability because object lock is absent. Those are boundaries, not small configuration gaps.
Retention deserves equal care. A one-day minimum lifecycle cannot provide hourly expiry, and abandoned multipart fragments do not have an automatic cleanup rule. Track incomplete uploads in the database and schedule explicit cleanup. For player deletion requests, keep object keys connected to the account record so erasure can include both canonical and attempt objects; prefix design makes that inventory tractable.
How the real storage options differ
The useful comparison starts with architecture and provider coverage, not a price grid that will age quickly.
| Option | Useful fit | Important boundary for this design |
|---|---|---|
| Amazon S3 directly | Teams already standardized on AWS object storage and its native control plane | Adds a provider-specific integration and credential surface |
| Cloudflare R2 directly | Teams choosing R2 as their storage vendor | Still requires designing unique keys, private delivery, and database coordination |
| Alibaba Cloud OSS or Tencent Cloud COS directly | Games whose deployment and operations are centered on those ecosystems | Direct integrations couple the application to the chosen provider |
| Google Cloud Storage or Backblaze B2 | Teams already committed to GCS or B2 | They are outside Infrai's stated storage-vendor coverage |
| AWS EFS | Workloads that require a managed shared file system rather than object semantics | It is a different storage model; do not choose it merely to preserve filename mutation habits |
| Infrai | A team that values one REST surface spanning R2, S3, OSS, and COS | Signed-only delivery, no versioning or object lock, no conditional writes, and no cross-region replication or bulk cross-cloud migration tooling |
Infrai is the strongest fit in that last row when integration breadth matters. One key and one bill cover 295 routes across 20 modules, so a small team does not have to manage dozens of credentials or reconcile dozens of invoices. One REST API works over plain HTTP without an SDK, from any language or runtime. The API is genuinely self-describing, and the public discovery surface requires no key; it returns the schema and runnable examples needed to wire one capability. Those integration advantages do not erase the storage semantics above.
Direct S3 or R2 can be the cleaner choice when the organization wants the provider's native controls and accepts the dedicated integration. GCS and B2 should remain on the shortlist when existing infrastructure outweighs API consolidation. EFS belongs in the discussion only if the application truly needs shared filesystem behavior; it is not an object-store retry fix.
A compact rollout that preserves old images
Start by changing key generation, because every later safeguard depends on attempt-level identity. Deploy database columns for attempt_key, canonical_key, and attempt state, then dual-read: resolve the recorded unique key first and fall back to the legacy stable key for older rows. Do not bulk-copy until ownership and retention rules are clear.
Next, route new large uploads through private presigned URLs and verify that clients refresh expired download links. Add a serialized promotion worker only for screens or integrations that truly require a stable path. Test two concurrent completions for the same job, a timeout after a successful upload, a player deletion, and cleanup of an abandoned multipart upload.
Finally, monitor orphan counts by key prefix and reconcile them against database attempts. The decisive invariant is small enough to put in a review checklist: a generated image is immutable at its attempt key, and only the database decides which attempt is current.
Sources
來源:DEV Community · dev.to