Alexei Navalny 團隊分享以 AI 建立並維護數碼檔案庫的做法

One day, there will be a museum dedicated to Alexei Navalny in Russia. People will come to learn about his life, his work and the extraordinary legacy he left behind. Some will have followed him for years; others may know little more than his name. They will be able to watch his speeches, read his writings, explore photographs from different periods of his life and discover the Alexei that those who knew him remember: what made him laugh, what he loved and what he dreamed of.

We don't know when that museum will become a reality. In today's Russia, even supporting Alexei Navalny can lead to persecution. Many of those who worked with him or supported his cause have been forced to leave the country under threat of criminal prosecution.

But we cannot wait for Russia to change before we begin preserving his legacy.

There is an extraordinary wealth of material documenting Alexei's life and work. His speeches, interviews, blog posts and social media tell part of the story. So do the memories of those who knew him, the photographs taken over the years and the countless newspaper articles and television reports in which he appeared. Some of these materials are still available online. Others are scattered across private collections, hidden away in old newspapers, photo albums and videotapes. Some have already disappeared from public view.

The challenge is to find them, bring them together and preserve them before they are lost.

That is why Yulia Navalnaya, who is continuing her husband's work, decided to create a digital archive of his life and legacy at navalny.com. The Anti-Corruption Foundation (ACF) took on the task of building it, with an ambitious goal: to bring together as much of this material as possible and make it freely accessible to people around the world, in both Russian and English.

We launched the Archive on 4 June 2026, the day Alexei would have turned 50. By then, the collection contained more than 14,000 items, ranging from short social media posts and campaign leaflets to speeches lasting several hours.

We tracked down interviews that had seemed lost, scanned old newspapers and photographs, and translated thousands of texts, most of which had never been available in English. Some of the most exciting discoveries were materials that even those of us who had worked alongside Alexei for years had never seen before.

In this article, we'll explain how a small team managed such an enormous undertaking, where artificial intelligence helped us and what still had to be done by hand.

If you believe in preserving Alexei's legacy and making it accessible to people around the world, please consider supporting our work with a donation. Your contribution will help us expand the collection, cover the costs of storing and preserving the materials, and continue developing the website.

How We Found and Preserved the Materials

We began collecting materials more than a year before the Archive launched. At first, we worked from memory, compiling a Google spreadsheet of the best-known interviews, videos and other recordings featuring Alexei Navalny. Gradually, the collection began to take shape, with sections devoted to his speeches at rallies, his final statements in court, his interviews and much more.

Then came hundreds of hours of searching.

From the outside, this might sound like monotonous work. In reality, it was incredibly absorbing, especially when we managed to uncover something rare that everyone had assumed was lost forever.

There were considerable obstacles. Many Russian media outlets that operated in the 2000s and 2010s have since closed down, changed ownership or adopted a different editorial line. The Russian authorities persecute people for their connections to the Anti-Corruption Foundation, which they have designated an "extremist organisation" for its 15-year history of challenging Putin's regime. Faced with the threat of criminal prosecution, many publications have removed Alexei's interviews and comments from their websites.

Sometimes we would find a media outlet's website with a seemingly complete archive of its past publications, only to discover that the one article featuring Alexei had mysteriously vanished. This is what censorship looks like in practice.

For example, dozens of Alexei's interviews were missing from the archives of Echo of Moscow, the independent radio station. We searched YouTube, VKontakte, Odnoklassniki and other platforms, eventually recovering the vast majority of them.

We are particularly proud of these discoveries, as well as materials that can now be found nowhere else: recordings of the court hearings in the first Kirovles case, which captivated the country at the time, and previously unpublished archival photographs of Alexei.

Some of the materials we collected had never been available online at all. People shared magazines, newspapers and brochures from their personal collections. One of our colleagues spent days at a scanner, turning pages one by one so that visitors to the Archive could read them. Among the results were rare local campaign newspapers from Alexei's 2013 run for mayor of Moscow.

While our editors were tracking down and digitising materials, the development team was building the content management system and processing tools, and the designers were working out how best to present such a varied collection.

We chose to house the Archive on Alexei's own website, navalny.com. Unfortunately, the site is blocked in Russia, meaning people there need a VPN to access it. It was a limitation we had to accept: the Russian authorities would have immediately blocked any new website featuring material critical of the regime anyway.

Audience and Algorithms: Designing an Archive for Everyone

From the outset, Yulia Navalnaya wanted the Archive to serve two audiences: people who had followed Alexei for years and those discovering his work for the first time. We also knew it could become a valuable resource for historians, journalists and other researchers.

These different audiences shaped both the content of the Archive and the way we designed the website.

A researcher might need to find a particular speech, check its date, read a transcript or search for references to a specific person or event. Someone browsing out of curiosity, on the other hand, might not know what they're looking for until something catches their eye.

We wanted the Archive to work equally well for both.

That's why, alongside a comprehensive search function, we created themed collections and recommendations that encourage visitors to explore. Someone might start by exploring Smart Voting, the strategy Alexei developed to challenge Putin's ruling party after being barred from running for office himself, and end up discovering the books he read or the music he loved.

Our recommendation system takes into account both the subject and the date of each publication, helping visitors move naturally from one item to another.
And every day, the homepage features something Alexei said or wrote on that date in a previous year. It's a way of bringing his words back into the present, and giving people a reason to return, explore and discover something new.

Bringing It All Together: The Design Challenge

Over the years, Alexei Navalny's various projects had developed their own visual identities, created by different teams for different purposes. Before we could bring them together in a single Archive, our designers needed to establish a consistent visual language.

They went back to the foundations of Alexei's visual identity, restoring the original Swiss-style sans-serif typeface and signature colour, and resolving inconsistencies that had emerged over time.

We also had to think about the future. Once the Archive was launched, a small team would be responsible for maintaining and expanding it. The design therefore needed to be flexible enough to accommodate new materials and features without requiring a complete overhaul each time.

The design went through three major iterations. By the end, we had created 31 distinct screens, each adapted to at least four different screen sizes, using a shared library of components and design variables.

For inspiration, we looked to YouTube, Netflix and Apple TV. We loved the way these platforms make it easy to navigate enormous libraries of content and find something worth watching.

But our collection presented a rather different challenge. A scanned newspaper from twenty years ago might sit alongside a court document, a photo gallery or a three-hour interview. Each format needed to be presented in a way that suited the material itself.

Rather than force everything into a single template, we opted for a consistent design on the homepage while allowing different types of content to have their own layouts.

We developed standard previews for each type of content, while our designers created almost a thousand individual cover images for the most important publications. Our editors also selected 250 items and tagged them as “Top Content”. A random selection of these is displayed each time someone visits the homepage, keeping it fresh and varied, while all 250 receive higher priority in search results, helping visitors discover important materials regardless of when they were published.

What We Entrusted to AI

Every item in the Archive needed a title, a date and a link to its original source. We also had to identify the people, places and events mentioned, assign relevant tags and prepare an English translation. To make long speeches and interviews searchable, we needed transcripts, which meant extracting the automatically generated subtitles available on YouTube.

Of the roughly 60 people working at the Anti-Corruption Foundation, only about ten were dedicated to the Archive full-time. Processing thousands of items manually would have taken such a small team years.

Our developers therefore built an automated system to handle the initial processing. In the final two months before launch, we also brought in around 50 volunteers to help review the results.

Automation transformed the editorial workflow. Instead of spending hours on repetitive tasks, our editors and volunteers could concentrate on checking the AI-processed results, correcting mistakes and adding historical context.

We built a collaborative content management system that allowed us to assign materials to individual team members, track their progress and review their work. New volunteers initially received closer supervision, but as they became familiar with our editorial guidelines, they were given greater independence.

AI made it possible for a small nonprofit with a limited budget to process thousands of materials in time for launch.

For the initial classification, we used large language models. Gemini Flash handled metadata extraction, identifying topics, people and places, while GPT-5.4 was responsible for translation.

This freed our team to focus on tasks that required human knowledge and editorial judgement: recognising someone in an old photograph, for example, or understanding the historical context of a text Alexei had written from prison.

We decided against training our own classification model. At the start of the project, we simply didn't have enough labelled data, and creating a sufficiently large training dataset would have taken considerable time.

Instead, we developed detailed instructions and a set of around 10–20 carefully selected examples. We included several of these examples in each prompt to guide the model towards the results we needed.

How We Process Each Material

We needed a system that could take each material through a series of processing stages, running independent tasks in parallel wherever possible.

We chose a technology stack our developers already knew well: Python 3.13, Django, Pydantic for data transfer objects (DTOs), Django Q2 (a maintained fork of Django Q) for asynchronous tasks, PostgreSQL, Redis and Elasticsearch for data storage and search, Google Cloud Storage for files, OpenTelemetry for observability and GitLab for continuous integration and deployment (CI/CD).

Django allowed us to build a working prototype quickly, with a ready-made administration interface. But as the project grew, our processing pipeline became increasingly complex, and we began separating business logic from data access, following the principles of Clean Architecture.

Combining this approach with Django's Active Record pattern proved more time-consuming than expected. We had to move logic into repository classes, duplicate models in Pydantic and eventually rebuild the administration interface from scratch.

In other words, the time we saved at the beginning came at a cost later, in additional development work and substantial amounts of boilerplate code. Based on this experience, we will probably choose FastAPI and SQLAlchemy for our next project.

For every material, the system produces several outputs: a plain text representation of the content for searching and processing, a rich text version to be displayed on the website, and metadata, such as links to related topics, people and places. Some of this information comes directly from the original source; the rest is inferred during processing.

Take a video, for example. The system first extracts its subtitles and divides the resulting transcript into sections. It then launches four tasks simultaneously: generate a short description, assign tags, identify places and identify people mentioned in the recording.

A custom-built task runner coordinates the entire process, determining which operations need to happen in sequence and which can run in parallel.

By running independent tasks simultaneously, we significantly reduce the processing time of each material, reducing the time from initial upload to human review.

Where LLMs Got Things Wrong — and How We Fixed It

We quickly discovered that getting an LLM to produce technically correct results wasn't enough. The results also had to make sense to the people using the Archive.

Take tags, for example. Initially, we allowed the model to generate them freely. But it kept inventing slightly different labels for essentially the same subject: prison, penal colony, imprisonment. Before long, the collection was cluttered with overlapping tags.

We changed our approach, creating a fixed list of approved tags from which the model could choose. Only a human editor could add new ones, which could then be applied when existing materials were reprocessed.

The next challenge was determining how important a particular subject was to a material. A passing reference to a government official, for example, shouldn't carry the same weight as an entire investigation devoted to that person.

We then prompted the model to apply this scale to find the most relevant tags in each material’s transcript, obtaining a result similar to this:

`[
    {
        "reasoning": "The investigation is entirely centered around Dmitry Medvedev ('Dimon') and his hidden wealth, luxury real estate, and assets, making him and this theme the core subject of the material.",
        "tag": "dont-call-him-dimon",
        "score": 1.0
    },
    {
        "reasoning": "As the foundational focus of the film and Navalny's anti-corruption foundation, corruption is deeply analyzed throughout the entire investigation with extensive documentation.",
        "tag": "corruption",
        "score": 0.8
    },
    {
        "reasoning": "The investigation was produced, published, and driven by the Anti-Corruption Foundation (FBK), which conducted the research and released the film.",
        "tag": "fbk",
        "score": 0.8
    },
    {
        "reasoning": "One of the most famous specific properties exposed in the investigation is the elaborate estate colloquially referred to as the 'duck house' on the Milovka estate grounds.",
        "tag": "duck-house",
        "score": 0.5
    },
    {
        "reasoning": "The entire documentary is structured as a detailed, multi-part investigative report uncovering hidden ownership structures and slush funds.",
        "tag": "investigation",
        "score": 0.5
    },
    ...
]`

The next step was to decide which tags to keep. We used a simple 1D clustering algorithm to identify sharp drops in relevance scores and discard less relevant tags. If several tags scored highly and the next group received significantly lower scores, the algorithm placed the cutoff between them.

We also prompted the model to provide a short explanation for each score in a reasoning field. These explanations were never shown to visitors and were discarded during processing. But we found that forcing the model to explain its decisions made its assessments more consistent, a variation of Chain-of-Thought prompting.

We reinforced this approach via few-shot prompting—by including examples of correctly tagged texts in the beginning of the prompt.

Even with these improvements, mistakes remained. One of Alexei's posts, My Fear and Loathing, about the betrayal of democratic ideals by post-Soviet reformers, was classified as "Lifestyle". His prison writings were frequently tagged as "Humour". Alexei often joked, even in the bleakest circumstances, but behind the jokes were sometimes deeply painful stories that visitors would hardly expect to encounter in a collection of humorous posts. It also turned out to be surprisingly tricky to explain to an LLM that what it categorized as "Torture" was in fact just typical police behavior during a peaceful protest in Russia.

Our editors reviewed these cases with the project manager, and the developers used their feedback to refine the system instructions for the LLM.

Instagram posts proved particularly challenging. To keep costs down, we had decided not to use image recognition. But a caption alone often provided too little information to establish what was happening in a photograph, so editors had to check the assigned tags particularly carefully. For posts from 2011 and 2012, colleagues who remembered the events firsthand were invaluable. Younger staff members and volunteers often had no way of recognising the people or situations depicted in the photographs.

AI-generated descriptions presented a different problem. The model repeatedly fell back on formulaic phrases such as "The author argues that..." or "This article examines the issue of...".

No amount of system instructions and few-shot prompts entirely solved the problem. In the end, we treated AI-generated descriptions as first drafts, which editors would then revise to remove awkward, bureaucratic language and make the wording more precise.

Translations: Bringing Alexei's Words to the World

Translations were another enormous part of the project. We wanted people who don't speak Russian to be able to read Alexei's speeches, interviews and writings from different periods of his life, and form their own understanding of who he was and what he stood for.

By the time we launched, almost the entire Archive was available in English, with the vast majority of these materials translated for the first time. For us, this was one of the project's most important achievements, and we were genuinely impressed by the quality of the translations LLMs could produce.

To make this possible, we built our own translation service on top of GPT-5.4. It gave us control over how texts were processed and how translations were returned, and also allowed us to translate texts from other languages into Russian when needed.

The biggest challenge was dealing with long documents. When given too much text at once, the model would sometimes introduce details that weren't present in the original.

To minimise this problem, we divided texts into smaller sections, keeping paragraphs intact wherever possible. When that wasn't feasible, the system used punctuation and word boundaries to determine where to split the text, falling back on character limits only when necessary.

For each section, the model returned two fields: original, containing a copy of the source text, and translation, containing the English version.

We found that requiring the model to reproduce the original piece of text alongside its translation helped it stay more faithful to the source, a technique known as Context Anchoring.

Finding What You Need Among Thousands of Materials

Once the materials had been classified and translated, we needed to make them searchable, both by exact wording and by meaning.

Again, we had two types of visitors in mind. Someone might remember a particular phrase Alexei had used and want to find the original speech or article. Someone else might be interested in a broader subject but have no specific document in mind.

Our search system needed to work for both.

We used Elasticsearch to combine full-text search, filters and semantic search within a single engine.

For full-text search, we use BM25, an algorithm that ranks results based on how closely their wording matches the query. Alongside it, we use k-nearest neighbours (kNN) search with embeddings generated by OpenAI's text-embedding-3-small model. This allows the system to find materials that are similar in meaning, even when they don't contain the exact words entered by the user.

For example, someone searching for "corruption in government procurement" should be able to find investigations by RosPil, Alexei's project dedicated to exposing corruption in public procurement, even if the original text uses different expressions, such as "embezzlement of public funds through government tenders".

We chose OpenAI's embedding model because it supports multiple languages and was easy to integrate into our existing processing pipeline, which already used OpenAI.

The model represents each piece of text as a vector of 1,536 numbers, allowing the search engine to calculate how closely different texts and queries are related in meaning.

The cost is remarkably low. According to our developers' calculations, generating embeddings for the entire collection costs just a few cents, while processing a single search query costs approximately $0.0000005.

We also cache query embeddings in Redis for a week, so repeated searches don't require another request to the model.

Visitors can narrow their results using filters for source, content type, format and date. Anyone looking for an exact phrase can put it in quotation marks.

The search engine also accounts for the grammatical features of Russian and English, so different forms of the same word don't prevent relevant results from appearing.

There was, however, one technical compromise.

In our Elasticsearch configuration, Reciprocal Rank Fusion (RRF), a method for combining results from different search algorithms based on their rankings, was unavailable without a paid licence.

Instead, we simply add together the scores produced by full-text and semantic search, without normalising them to a common scale.

This makes the final ranking somewhat approximate. Nevertheless, in our testing, it has worked reliably for our collection.

Why We Had to Make Search Faster

Finding relevant items was only half the challenge. Visitors also needed to get their results quickly and see exactly where their search terms appeared in each document.

When we began load testing with k6, we discovered that Elasticsearch itself was remarkably fast, returning search results in milliseconds. The delays came later, during the fetch phase, when the system retrieved the matching documents and highlighted the relevant passages.

Each document's _source contained around 49 KB of vector data that wasn't needed to generate search results, since kNN search used a separate index. By removing this unnecessary data, we reduced the average document size from approximately 75 KB to 26 KB.

Highlighting proved more difficult to optimise.

With our texts, Elasticsearch took around 400 milliseconds to highlight matches on a single page of search results. We experimented with different highlighting engines, but none delivered a significant improvement. Under load, the Fast Vector Highlighter (fvh) consumed substantial CPU resources building term structures, even when there were no matches to highlight.

In our configuration, each Elasticsearch node had seven search threads. Under load, fetching and highlighting results took around a second, and our two-node setup reached its limit at approximately 13 requests per second.

We decided to move highlighting out of Elasticsearch and into our backend, which could be scaled horizontally by adding more instances. This gave us greater flexibility, but initially did little to reduce the time required for highlighting itself.

Our first implementation examined every word in a document, reduced it to its stem and compared it with the search query. The problem was that finding just a handful of matches could require processing thousands of words.
`
Replacing the Python stemmer with a C implementation made little difference.

So we changed the order of operations. Instead of stemming every word, we introduced an initial filtering step. A compiled regular expression searches for words with matching prefixes, and only those candidates are passed to the stemmer for verification.

The system also builds the snippets displayed in search results as it finds matches. Once it has collected enough relevant passages, it stops processing the document.

We retained stemming because matching prefixes alone can produce false positives. In Russian, for example, a search for words beginning with коррупц would match both коррупция (corruption) and коррупционеры (corrupt officials), even though they are different words. Stemming provides an additional check.

We also had to account for language-specific behaviour. The English stemmer, for instance, converts country to countri, so candidate selection must use a prefix shared by both forms. In Russian, the letters е and ё also need to be treated as equivalent for search purposes.

From 400 ms to 30 ms

The results were substantial. In our tests, highlighting time fell from approximately 400 milliseconds to just 30 milliseconds.

We also tested the new implementation against Elasticsearch's native highlighting on a corpus containing more than a million characters and found no discrepancies.

These figures refer specifically to highlighting performance. We measured the response time of the search service as a whole separately.

How We Manage Traffic

Once we had made the fetch phase more efficient, the next bottleneck was our backend.

In a separate test with highlighting disabled, the system handled approximately 107 requests per second, with a p95 response time of 1.8 seconds. In other words, 95% of requests were completed within that time.

A typical response containing around 20 search results was approximately 66 KB. This included full descriptions and lists of tags, people and places associated with each item. Reducing the amount of data returned became our next priority.

To make performance tuning easier, we moved the settings we needed to adjust regularly into the administration interface.

From there, we can enable or disable semantic search, adjust its weighting and similarity threshold, choose the highlighting engine and control how search snippets are generated: not at all, from descriptions or from the full text.

We can also configure response caching and cache expiration times.

These settings are stored in the database and accessed through a 30-second cache, allowing us to make changes without deploying new code. We later extended the same approach to adaptive throttling.

Other parameters, such as the number of Elasticsearch index shards, still have to be configured when an index is created.

To handle spikes in traffic, we introduced response caching in Redis and adaptive throttling. The system monitors p95 response times and, when it detects an overload, temporarily rejects some requests with an HTTP 429 ("Too Many Requests") response.

We also designed the system so that rebuilding the search index doesn't require taking the website offline. A new index is created alongside the existing one, and once it has been checked, we switch the alias to point to the new version.

This allows us to update the search infrastructure while keeping the Archive accessible to visitors.

The Archive Continues to Grow

From the beginning, we designed the Archive to keep growing long after its launch.

Every month, we add around 100–200 new items: videos, interviews, legal documents and other discoveries. Each goes through the same process as the original collection, from tracking down a digital file or paper document to processing, editorial review and, finally, publication on the website, where anyone can read, watch or search for it.

Some of our most valuable contributions come from Alexei's supporters. People discover old recordings and photographs in their personal collections and share them with us, helping us uncover parts of his life and work that even we knew little about.

There is still so much to find and preserve, and these contributions are invaluable. If you have any materials that might belong in the Archive, please write to us at archive@navalny.com.

We also invite you to explore the Archive for yourself. Browse a themed collection, revisit an old interview or discover what's featured on the homepage today. Even if you've followed Alexei for years, we hope you'll find something you've never seen before.

And if you believe this work matters, please consider supporting it.

Your donations help us preserve the collection, maintain the website and prepare new materials for publication. With your support, the Anti-Corruption Foundation can continue collecting and safeguarding the record of Alexei's life and work, making it accessible to people around the world — both those who remember him and those who are only just discovering his story.