跳到正文
DEV Community· ArshTechPro·· 4 小時前AI 評分60

Maestro 深入教學:使用 CI 為 SwiftUI 應用程式進行端到端測試

Maestro Deep Dive: Testing a SwiftUI App End to End (with CI)

AI 導讀

文章以約 120 行 Swift 的 SwiftUI 待辦事項示例,示範如何用 Maestro CLI 編寫端到端測試並在 CI 執行。五個 YAML 測試流程涵蓋登入、任務新增、完成與刪除,以及登出,並講解可重用子流程、測試報告和失敗排查。

正文

In Part 1 we covered what Maestro is, how it works and how it compares to Appium, XCUITest, Detox and the paid device clouds. Now it is time to write real tests.

In this article we will:

  1. Build a small SwiftUI app with a login screen and a to-do list.
  2. Make it testable with accessibility identifiers.
  3. Write a Maestro suite with reusable sub-flows, environment variables, conditional logic, JavaScript and tags.
  4. Run it locally, generate reports and debug failures.
  5. Look at how the same test would look in XCUITest, and where AI agents fit in.

Everything here uses the free, open-source Maestro CLI. The full app is about 120 lines of Swift, so you can copy it into a new Xcode project and follow along.


Prerequisites

  • macOS with Xcode 16 or newer and an iOS 17+ Simulator runtime
  • Java 17 or newer (java -version)
  • Maestro CLI:
curl -fsSL "https://get.maestro.mobile.dev" | bash
maestro --version

This article was written against Maestro CLI 2.10.0.


Part A: The sample app

Create a new Xcode project:

  • Template: iOS > App
  • Product name: TodoDemo
  • Organization identifier: com.example
  • Interface: SwiftUI
  • Minimum deployment: iOS 17

That gives the bundle ID com.example.TodoDemo, which Maestro uses as the appId.

The app has two screens:

  • Login: email, password, a "Log In" button and an error message for wrong credentials. The valid credentials are demo@example.com / password123.
  • My Tasks: add tasks, tap to complete them, swipe to delete, see a progress label, and log out. After login it asks for notification permission, which gives us a system dialog to deal with.

TodoDemoApp.swift

import SwiftUI

@main
struct TodoDemoApp: App {
    @AppStorage("isLoggedIn") private var isLoggedIn = false

    var body: some Scene {
        WindowGroup {
            if isLoggedIn {
                TodoListView()
            } else {
                LoginView()
            }
        }
    }
}

The login state is stored in UserDefaults through @AppStorage. That matters for testing: it survives an app restart, so we will need to clear app state between tests.

LoginView.swift

import SwiftUI

struct LoginView: View {
    @AppStorage("isLoggedIn") private var isLoggedIn = false
    @State private var email = ""
    @State private var password = ""
    @State private var errorMessage: String?

    var body: some View {
        NavigationStack {
            Form {
                Section {
                    TextField("Email", text: $email)
                        .keyboardType(.emailAddress)
                        .textInputAutocapitalization(.never)
                        .autocorrectionDisabled()
                        .accessibilityIdentifier("emailField")

                    SecureField("Password", text: $password)
                        .accessibilityIdentifier("passwordField")
                }

                if let errorMessage {
                    Text(errorMessage)
                        .foregroundStyle(.red)
                        .accessibilityIdentifier("loginError")
                }

                Button("Log In", action: logIn)
                    .accessibilityIdentifier("loginButton")
                    .disabled(email.isEmpty || password.isEmpty)
            }
            .navigationTitle("Welcome")
        }
    }

    private func logIn() {
        // Demo only. A real app would call an auth service here.
        if email.lowercased() == "demo@example.com" && password == "password123" {
            errorMessage = nil
            isLoggedIn = true
        } else {
            errorMessage = "Invalid email or password"
        }
    }
}

TodoListView.swift

import SwiftUI
import UserNotifications

struct TodoItem: Identifiable {
    let id = UUID()
    var title: String
    var isDone = false
}

struct TodoListView: View {
    @AppStorage("isLoggedIn") private var isLoggedIn = false
    @State private var items: [TodoItem] = []
    @State private var newTitle = ""

    private var doneCount: Int { items.filter(\.isDone).count }

    var body: some View {
        NavigationStack {
            VStack(spacing: 12) {
                HStack {
                    TextField("New task", text: $newTitle)
                        .textFieldStyle(.roundedBorder)
                        .onSubmit(addItem)
                        .accessibilityIdentifier("newTaskField")

                    Button("Add", action: addItem)
                        .accessibilityIdentifier("addTaskButton")
                        .disabled(newTitle.trimmingCharacters(in: .whitespaces).isEmpty)
                }
                .padding(.horizontal)

                if items.isEmpty {
                    ContentUnavailableView(
                        "No tasks yet",
                        systemImage: "checklist",
                        description: Text("Add your first task above.")
                    )
                } else {
                    List {
                        ForEach($items) { $item in
                            Button {
                                item.isDone.toggle()
                            } label: {
                                HStack {
                                    Image(systemName: item.isDone ? "checkmark.circle.fill" : "circle")
                                    Text(item.title)
                                        .strikethrough(item.isDone)
                                }
                            }
                        }
                        .onDelete { items.remove(atOffsets: $0) }
                    }

                    Text("\(doneCount) of \(items.count) done")
                        .font(.footnote)
                        .foregroundStyle(.secondary)
                        .accessibilityIdentifier("progressLabel")
                }
            }
            .navigationTitle("My Tasks")
            .toolbar {
                ToolbarItem(placement: .topBarTrailing) {
                    Button("Log Out") { isLoggedIn = false }
                        .accessibilityIdentifier("logoutButton")
                }
            }
            .task {
                _ = try? await UNUserNotificationCenter.current()
                    .requestAuthorization(options: [.alert, .sound])
            }
        }
    }

    private func addItem() {
        let title = newTitle.trimmingCharacters(in: .whitespaces)
        guard !title.isEmpty else { return }
        items.append(TodoItem(title: title))
        newTitle = ""
    }
}

Run it once in the Simulator from Xcode to make sure it works.

Why the accessibility identifiers?

Maestro can find elements by visible text (tapOn: "Log In"), and that is often fine. But text changes: a designer renames "Log In" to "Sign In", or you test in German. Identifiers do not change with copy or locale.

A practical rule:

  • Static controls (buttons, fields, labels you assert on): give them an accessibilityIdentifier and target them with id.
  • Dynamic content (task titles, user names, server data): target by text, because the text is the thing you are verifying.

You can see exactly what Maestro sees with:

maestro hierarchy

or by opening the Maestro Studio desktop app and clicking on elements in the live device view.


Part B: Build and install for the Simulator

Maestro tests the Simulator build of your app (.app). You can install it by running from Xcode, or from the command line, which is what CI will do:

# Boot a simulator (the device name depends on your installed runtimes)
xcrun simctl boot "iPhone 16" || true

# Build for the simulator
xcodebuild \
  -project TodoDemo.xcodeproj \
  -scheme TodoDemo \
  -configuration Debug \
  -sdk iphonesimulator \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  -derivedDataPath build \
  build

# Install the .app on the booted simulator
xcrun simctl install booted build/Build/Products/Debug-iphonesimulator/TodoDemo.app

Part C: The test suite

Here is the structure we are going to build:

.maestro/
├── config.yaml
├── subflows/
│   ├── login.yaml
│   └── dismiss_permission_dialog.yaml
└── flows/
    ├── 01_login_invalid.yaml
    ├── 02_login_success.yaml
    ├── 03_add_and_complete_tasks.yaml
    ├── 04_delete_task.yaml
    └── 05_session_and_logout.yaml

flows/ contains the tests. subflows/ contains reusable building blocks that are never run on their own.

config.yaml: the workspace configuration

# Only files matching these patterns are treated as tests.
# Sub-flows live outside flows/, so they are not run directly.
flows:
  - "flows/*"

When you run maestro test .maestro/, Maestro reads this file to decide which flows belong to the suite.

Sub-flow 1: login.yaml

appId: com.example.TodoDemo
---
- tapOn:
    id: "emailField"
- inputText: ${EMAIL || 'demo@example.com'}
- tapOn:
    id: "passwordField"
- inputText: ${PASSWORD || 'password123'}
- tapOn:
    id: "loginButton"

Two things to notice:

  • Everything inside ${...} is JavaScript. EMAIL || 'demo@example.com' means "use the EMAIL variable if someone passed one, otherwise fall back to the default". That lets the same sub-flow handle valid and invalid logins.
  • There are no waits. After tapOn: { id: "emailField" }, Maestro waits for the screen to settle before typing.

Sub-flow 2: dismiss_permission_dialog.yaml

appId: com.example.TodoDemo
---
- runFlow:
    when:
      visible: "Allow"
    commands:
      - tapOn: "Allow"

This is a conditional flow. If the system notification dialog is on screen, tap "Allow". If it is not, do nothing and continue. Because Maestro drives the whole device rather than just your app process, it can tap system alerts that in-process frameworks cannot reach directly.

We will mostly avoid the dialog entirely by pre-granting permissions in launchApp, but this sub-flow is a safety net for cases where you cannot.

Test 1: 01_login_invalid.yaml

appId: com.example.TodoDemo
name: Login fails with wrong credentials
tags:
  - auth
  - regression
---
- launchApp:
    clearState: true
- assertVisible: "Welcome"
- runFlow:
    file: ../subflows/login.yaml
    env:
      EMAIL: "wrong@example.com"
      PASSWORD: "not-the-password"
- assertVisible:
    id: "loginError"
    text: "Invalid email or password"
- assertNotVisible: "My Tasks"
  • clearState: true wipes the app's data before launching, so every test starts logged out. This is the single most important habit for a reliable suite: never let one test depend on what a previous test left behind.
  • runFlow with env overrides the variables for that sub-flow only.
  • The selector { id: "loginError", text: "..." } matches an element that has that identifier and that text.

Test 2: 02_login_success.yaml

appId: com.example.TodoDemo
name: Login succeeds with valid credentials
tags:
  - smoke
  - auth
---
- launchApp:
    clearState: true
    permissions:
      notifications: allow
- runFlow: ../subflows/login.yaml
- runFlow: ../subflows/dismiss_permission_dialog.yaml
- assertVisible: "My Tasks"
- assertVisible: "No tasks yet"
- takeScreenshot: after_login

permissions: { notifications: allow } grants the permission before the app starts, so the dialog should never appear. takeScreenshot saves a PNG, which is useful for reviewing UI in CI artifacts.

Test 3: 03_add_and_complete_tasks.yaml

appId: com.example.TodoDemo
name: Add tasks and mark one as done
tags:
  - smoke
  - tasks
---
- launchApp:
    clearState: true
    permissions:
      notifications: allow
- runFlow: ../subflows/login.yaml
- runFlow: ../subflows/dismiss_permission_dialog.yaml

- tapOn:
    id: "newTaskField"
- inputText: "Buy milk"
- tapOn:
    id: "addTaskButton"

- tapOn:
    id: "newTaskField"
- inputText: "Write Maestro article"
- tapOn:
    id: "addTaskButton"

- assertVisible: "Buy milk"
- assertVisible: "Write Maestro article"
- assertVisible:
    id: "progressLabel"
    text: "0 of 2 done"

- tapOn: "Buy milk"
- assertVisible:
    id: "progressLabel"
    text: "1 of 2 done"

Notice how the test reads almost exactly like a manual QA script. That readability is the main reason teams adopt Maestro.

Test 4: 04_delete_task.yaml (loops, JavaScript and swipes)

appId: com.example.TodoDemo
name: Delete a task with swipe
tags:
  - tasks
  - regression
---
- launchApp:
    clearState: true
    permissions:
      notifications: allow
- runFlow: ../subflows/login.yaml
- runFlow: ../subflows/dismiss_permission_dialog.yaml

# Create "Task 1", "Task 2" and "Task 3" using a loop and a JS counter
- evalScript: ${output.counter = 0}
- repeat:
    times: 3
    commands:
      - evalScript: ${output.counter = output.counter + 1}
      - tapOn:
          id: "newTaskField"
      - inputText: "Task ${output.counter}"
      - tapOn:
          id: "addTaskButton"

- assertVisible: "Task 3"

# Swipe left on the row to reveal the system Delete action
- swipe:
    from:
      text: "Task 2"
    direction: LEFT
- tapOn: "Delete"

- assertNotVisible: "Task 2"
- assertVisible: "Task 1"
- assertVisible: "Task 3"
- assertVisible:
    id: "progressLabel"
    text: "0 of 2 done"
  • output is a JavaScript object Maestro provides for storing values across steps.
  • repeat runs a block a fixed number of times (it also supports a while condition).
  • swipe with from starts the gesture on a specific element, which is how you trigger SwiftUI's .onDelete.

Use loops and scripts sparingly. If a flow starts to look like a program, consider seeding data through a test backend or launch arguments instead.

Test 5: 05_session_and_logout.yaml

appId: com.example.TodoDemo
name: Session persists across restarts until logout
tags:
  - auth
  - regression
---
- launchApp:
    clearState: true
    permissions:
      notifications: allow
- runFlow: ../subflows/login.yaml
- runFlow: ../subflows/dismiss_permission_dialog.yaml
- assertVisible: "My Tasks"

# Restart without clearing state: the user should still be logged in
- stopApp
- launchApp
- assertVisible: "My Tasks"

# Log out and confirm we are back on the login screen
- tapOn:
    id: "logoutButton"
- assertVisible: "Welcome"
- assertVisible:
    id: "loginButton"

This test checks behavior that crosses an app restart, something that is awkward to do in many in-process frameworks.


Part D: Running the suite

Make sure the Simulator is booted and the app is installed, then from the project root:

# Run the whole suite
maestro test .maestro/

# Run a single flow
maestro test .maestro/flows/02_login_success.yaml

# Run only smoke tests
maestro test --include-tags smoke .maestro/

# Exclude slow tests
maestro test --exclude-tags regression .maestro/

# Pass variables from the command line
maestro test -e EMAIL=qa@example.com -e PASSWORD=secret .maestro/flows/02_login_success.yaml

Faster authoring with continuous mode

While writing a flow, use continuous mode. Maestro reruns the flow every time you save the file:

maestro test -c .maestro/flows/03_add_and_complete_tasks.yaml

Combined with Maestro Studio for finding selectors, this gives you a feedback loop of a few seconds.

Validate syntax without a device

maestro check-syntax .maestro/flows/04_delete_task.yaml

Useful as a pre-commit hook or a quick CI step.

Reports and artifacts

# JUnit XML, understood by most CI systems
maestro test .maestro/ --format junit --output build/maestro-report.xml

# HTML report for humans
maestro test .maestro/ --format html --output build/maestro-report.html

# Put screenshots, logs and per-flow artifacts in a known folder
maestro test .maestro/ --test-output-dir build/maestro-artifacts

When a flow fails, the artifacts include a screenshot before each step, the view hierarchy at the moment of failure, device logs and crash reports. Start debugging there, not in the terminal output.

Running on several simulators in parallel

If you have more than one simulator booted, you can split the suite across them:

maestro test --shard-split 2 .maestro/

--shard-split divides the flows between devices. --shard-all runs every flow on every device, which is useful for testing several iOS versions or screen sizes at once.


Part E: The same test in XCUITest

To make the comparison from Part 1 concrete, here is roughly what Test 2 (successful login) looks like in XCUITest:

import XCTest

final class LoginUITests: XCTestCase {
    override func setUp() {
        continueAfterFailure = false
    }

    func testLoginSucceeds() {
        let app = XCUIApplication()
        app.launchArguments = ["-isLoggedIn", "NO"]   // reset @AppStorage
        app.launch()

        let email = app.textFields["emailField"]
        XCTAssertTrue(email.waitForExistence(timeout: 5))
        email.tap()
        email.typeText("demo@example.com")

        let password = app.secureTextFields["passwordField"]
        password.tap()
        password.typeText("password123")

        app.buttons["loginButton"].tap()

        // System alerts live in SpringBoard, outside the app under test
        let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
        let allow = springboard.buttons["Allow"]
        if allow.waitForExistence(timeout: 3) {
            allow.tap()
        }

        XCTAssertTrue(app.navigationBars["My Tasks"].waitForExistence(timeout: 5))
        XCTAssertTrue(app.staticTexts["No tasks yet"].exists)
    }
}

XCUITest is not bad. It is fast, it is Apple's own tooling and it runs on physical devices. But compare the two:

Maestro XCUITest
Lines for this test About 10 (plus a reusable sub-flow) About 30
Explicit waits None waitForExistence on most steps
System dialogs tapOn: "Allow" Separate SpringBoard app handle
Resetting state clearState: true Launch arguments or custom reset code
Rebuild after change No Yes, test target recompiles
Reusable on Android Often, with the same flow No, rewrite in Espresso
Physical iPhones Not officially, see Part 1 Yes

Many teams use both: XCUITest or unit tests for detailed component behavior, Maestro for end-to-end user journeys.


Part F: Writing flows with an AI coding agent

Maestro ships an MCP server inside the CLI. It lets an AI coding agent see the simulator, inspect the view hierarchy, run flows and read the results. To connect it, add this to your agent's MCP configuration:

{
  "mcpServers": {
    "maestro": {
      "command": "maestro",
      "args": ["mcp"]
    }
  }
}

Most agents also have a one-line command. For example, with the Codex CLI:

codex mcp add maestro -- maestro mcp

and with the Gemini CLI:

gemini mcp add maestro maestro mcp

Then you can ask something like: "Write a Maestro flow that logs in to TodoDemo, adds three tasks, completes the second one and checks the progress label. Run it and fix it until it passes."


Troubleshooting checklist

Symptom What to check
Element not found Run maestro hierarchy or open Studio. Is the identifier set on the element you think it is? Is the text an exact match?
Tap lands on the keyboard The keyboard may cover the target. Add - hideKeyboard or scroll first with scrollUntilVisible.
Test passes locally, fails in CI CI machines are slower. Use extendedWaitUntil with a longer timeout for slow screens, and increase MAESTRO_DRIVER_STARTUP_TIMEOUT.
Tests affect each other Use launchApp: { clearState: true } at the start of every flow.
Long animations break assertions Add - waitForAnimationToEnd before the assertion.
.ipa will not install Maestro needs a Simulator build (.app) built with -sdk iphonesimulator.
Flags stopped working after an upgrade Check the changelog. --ios-version is being replaced by --device-os.

Example of a longer explicit wait for a slow screen:

- extendedWaitUntil:
    visible:
      id: "progressLabel"
    timeout: 15000   # milliseconds

Best practices summary

  1. One user journey per flow. Short flows fail for one clear reason.
  2. Reset state at the start of every flow with clearState: true.
  3. Use identifiers for controls, text for content.
  4. Extract repeated steps into sub-flows (login, onboarding, navigation).
  5. Keep logic out of YAML where possible. Seed data through APIs or launch arguments.
  6. Tag your flows (smoke, regression, feature names) and run smoke tests on every PR, the full suite nightly.
  7. Pin the Maestro version in CI and upgrade on purpose.
  8. Always keep artifacts from failed runs.

Ending

With about 120 lines of Swift and five short YAML files, we have a test suite that covers login, error handling, adding, completing and deleting tasks, session persistence, logout and system permission dialogs. It runs locally in under a couple of minutes, runs in GitHub Actions without changes, and can move to a device cloud with one command.

That is Maestro's real value: the distance between "we should test this" and "it is tested in CI" is very short.

Resources

來源:DEV Community · dev.to