macOS Sequoia 翻譯快捷鍵失效的替代方案:本機 OCR 架構
macOS Sequoia broke your translation shortcut: an on-device OCR architecture instead
Zane 提出一種 macOS 翻譯架構,將用戶選取的螢幕區域交由 Apple Vision 在裝置上進行 OCR,只把辨識文字送往翻譯服務,再以浮動面板顯示結果。截圖只留在應用程式記憶體,離線時仍可進行 OCR,但翻譯需要網絡;這套流程亦不會改寫剪貼簿內容或搶走焦點。
If you ship a Mac app that translates text, macOS Sequoia has probably broken something for you. Two classes of breakage show up repeatedly in the field: accessibility-based shortcuts that go silent after an OS update, and clipboard-driven clients that overwrite whatever the developer had in the buffer.
This post walks through the failure modes and an architecture that sidesteps both: capture a screen region, OCR it on-device with Apple Vision, send only the recognized text out, and render the result in a floating panel.
Failure mode 1: the shortcut goes silent
Global shortcuts in translation clients commonly depend on either an event tap or accessibility (AX) permissions. Both are governed by TCC. After an OS update or a restore, the entry can desynchronize: the app still runs, the menu item still works, and the hotkey does nothing. There is no error dialog — just silence.
Practical checks:
- Does the app appear in System Settings → Privacy & Security → Accessibility, and is it still checked?
- Toggle it off, quit the app, toggle it back on, relaunch. Re-registration is often what fixes the desync.
- If the app uses an event tap, confirm
CGEvent.tapCreatesucceeded; a tap created before the permission was granted returns nil and stays nil until recreated.
Failure mode 2: the clipboard gets stolen
Clipboard-watching clients read NSPasteboard.general. Two properties matter:
- Every write increments
changeCount, so a poller can detect changes cheaply. - Some types carry intent markers —
org.nspasteboard.TransientType,org.nspasteboard.ConcealedType,org.nspasteboard.AutoGeneratedType. Clipboard managers skip those; naive pollers do not.
That is how an API key ends up being translated: it was copied, the poller read it, and the content was shipped to a translation endpoint. If your workflow depends on the clipboard, at minimum check the type markers before acting.
Failure mode 3: the governance question
Sending a screenshot of an internal log, a stack trace, or a customer record to a cloud OCR service is a decision someone has to sign off on. On many teams, that sign-off never arrives — which is why "just use a screenshot translator" quietly dies.
The architecture that avoids all three
Screen region (⌃S)
│
▼
CGImage in RAM (never written to disk)
│
▼
VNRecognizeTextRequest (on-device)
│
▼
Recognized text only
│
▼
Translation request (text, not pixels)
│
▼
Floating NSPanel next to the selection
On-device recognition
let request = VNRecognizeTextRequest()
request.recognitionLevel = .accurate
request.recognitionLanguages = ["en-US"]
request.usesLanguageCorrection = false
let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
try handler.perform([request])
let lines = (request.results ?? []).compactMap { $0.topCandidates(1).first?.string }
The image never leaves the process. If you keep usesLanguageCorrection = false, stack traces and identifiers survive intact; turning it on rewrites tokens into dictionary words and mangles paths.
Text-only egress
Only the recognized string is sent to the translation endpoint. That single design choice changes the security review conversation: the artifact crossing the network is text the user chose to read, not a screenshot of their desktop.
Be explicit about the trade-off: recognition is local, translation is a network call. Offline, you get OCR and no translation. Users deserve to know which half needs connectivity.
Non-activating overlay
An NSPanel with .nonactivatingPanel floats above the selection without stealing focus, so the editor keeps its caret and the user keeps their place. No app switching, no clipboard write.
What this does not fix
- Text too small to render legibly on screen
- Low contrast, overlays, watermarks
- Rotated or heavily italic text
- Math: sub/superscript association is lost in any linear read
- Network-loss translation: OCR still works, translation does not
Trying it
DeepL-style shortcut breakage on Sequoia, and the region-OCR alternative:
https://www.zipp-mac.com/solutions/deepl-shortcut-not-working-alternative/?utm_source=devto&utm_medium=article&utm_campaign=deepl_sequoiamacos
swift
ocr
translation
Distribution page (BOOTH):
https://zipp-mac.booth.pm/items/9082185
Demo (YouTube, ep.2):
https://www.youtube.com/watch?v=4_tQzj7VB_I
Takeaway
Two of the three failure modes are permission and buffer ownership problems, not translation problems. Moving the input from "whatever is in the clipboard" to "a rectangle the user selected" removes the shared mutable state entirely — and makes the privacy story easy to state in one sentence.
來源:DEV Community · dev.to