構建 AccessLearnAI：將講課內容轉換成文字稿、摘要、語音和點字
Building Access Learn AI: Turning Lectures into Text, Summaries, Audio and Braille
AccessLearnAI 以單一講課音訊為輸入，經 Whisper 轉錄、T5 摘要、edge-tts 轉換成語音，並生成點字輸出。Whisper 是開源模型；部署到 Render 免費方案時，Whisper 與 T5 同時載入記憶體，令後端超出記憶體限制。本機可運行的模型，仍可能因資源限制而無法在正式環境使用。
A lecture is one of the most common ways we learn, and one of the least accessible. A student who is deaf can't follow the audio. A student who is blind can't read the slides. A student with a cognitive or attention difficulty may struggle with a 60-minute recording and no summary.
Accessibility tools exist for each of these problems, but they are usually separate products. I wanted to see how far one pipeline could go, so I built AccessLearnAI: upload a lecture, and get back a transcript, a concise summary, spoken audio, and Braille output.
This post covers how it's built, the problems I ran into, and what I'd change.
The idea
Students need different formats of the same content. So instead of building one feature per disability, I built one pipeline that produces every format from a single input:
Whisper (speech to text) → T5 (summarization) → edge-tts (text to speech) → Braille generation
Each stage feeds the next:
- Whisper transcribes the lecture audio. I chose it because it is open source and holds up well on varied accents and recording quality, which matters for real classroom audio.
- T5 condenses the transcript into a short summary, so a student can review a lecture in minutes.
- edge-tts converts the summary back to natural-sounding speech for students who prefer to listen.
- Braille generation converts the text into Braille for blind and low-vision users.
Architecture
The system is split into three layers:
- Backend: Fast API with PostgreSQL for storage
- Frontend: React with TypeScript
- Deployment: Docker containers on GCP, with the frontend on Vercel
I used Fast API because the ML code is in Python, and it let me expose each pipeline stage as a clean API endpoint without a translation layer between the models and the web service.
The hard part: running four models on a free tier
The pipeline works fine on a development machine. Production was a different problem.
When I deployed the backend to Render's free tier, it kept running out of memory. The cause was that Whisper and T5 were both loaded into memory at the same time, and together they went over the free tier's limit.
This was a useful lesson
A model that works locally can still be unusable in production because of resource limits. Deploying an ML system means managing memory as carefully as you manage code.
Try it
來源：DEV Community · dev.to