Revyl Documentation

Tests

Write tests as YAML

Create tests from YAML files, reuse modules, run code, use variables and control flow, and keep tests in sync with your team.

This guide covers the full test authoring lifecycle: YAML-first creation, reusable modules, code execution scripts, variables, control flow, workflows, and team sync patterns.

Choose a Workflow

  1. YAML-first CLI (recommended) -- from a project that already has a canonical .revyl/config.yaml, start with a local YAML file and create the remote test. The CLI checks the file with backend YAML validation before mutation; it never creates or migrates project configuration.
  2. Scaffold first -- create an empty or module-seeded remote test with revyl test create, sync the generated YAML into .revyl/tests/, then edit and push locally.
  3. Session to regression -- convert a completed exploratory device session with revyl test create --from-session <session-id>, then refine the synced YAML and push it back as a stable regression.

Prerequisites

  • Authenticated with revyl auth login
  • An app/build available for the target platform
  • The correct app name for test.build.name (check with revyl app list)
  • A canonical .revyl/config.yaml in the selected project. Use revyl config pull for an existing registered project, revyl init -y for a new local project, or revyl config migrate for a legacy file.

YAML-First CLI

1. Write a YAML file

test:
  metadata:
    name: smoke-login-ios
    platform: ios
    tags:
      - smoke
  build:
    name: ios-test
  blocks:
    - type: instructions
      step_description: Sign in with valid test credentials.
    - type: validation
      step_description: The inbox is visible.

2. Create the test

revyl test create smoke-login-ios --from-file ./smoke-login-ios.yaml

4. Iterate and push

# Edit .revyl/tests/smoke-login-ios.yaml, then:
revyl test push smoke-login-ios --force
revyl test run smoke-login-ios

Scaffold First

revyl test create smoke-login-ios --platform ios
revyl test create smoke-login-ios --platform ios --module login  # Seed with a module

Then edit .revyl/tests/smoke-login-ios.yaml and push.

Session to Regression

revyl device report --session-id <session-id> --json
revyl test create --from-session <session-id> smoke-login-ios --app <app-id>
revyl test run smoke-login-ios

If the session is linked to an app or the project has a default app configured, --app can be omitted. The test name can also be omitted; Revyl will use the compiled session title.

YAML Anatomy

test:
  metadata:
    name: my-test
    platform: ios
  build:
    name: ios-test
  blocks:
    - type: instructions
      step_description: Do something

Common block types:

TypeUsage
instructionsA single user action
validationAn assertion about expected state
manualFramework-level actions: wait, go_home, navigate, set_location, set_orientation, set_appearance, download_file, kill_app, open_app, end
module_importImport a reusable module
if / whileConditional logic and loops
extractionExtract data from the screen into a variable
code_executionRun Python/JS/TS/Bash code

Reusable Modules

Modules are shared groups of test blocks that can be imported into any test.

Create a module

# modules/login.yaml
blocks:
  - type: instructions
    step_description: Tap Sign In.
  - type: instructions
    step_description: Type "{{email}}" in the email field.
  - type: instructions
    step_description: Type "{{password}}" in the password field.
  - type: instructions
    step_description: Tap Continue.
  - type: validation
    step_description: The home screen is visible.
revyl module create login-flow \
  --from-file modules/login.yaml \
  --description "Standard email/password login"

Import into a test

revyl module insert login-flow    # Prints a ready-to-paste YAML snippet
- type: module_import
  module: "login-flow"

Manage modules

revyl module list                               # List all modules
revyl module list --search login                # Filter by name/description
revyl module get login-flow                     # View module blocks
revyl module update login-flow --from-file new.yaml
revyl module delete login-flow                  # Delete (fails if still imported)

Code Execution Scripts

Code execution blocks let you run Python, JavaScript, TypeScript, or Bash code as part of a test.

Create and register a script

# scripts/seed_user.py
import requests

response = requests.post("https://api.example.com/test-users", json={
    "email": "test@example.com",
    "password": "TestPass123!",
})

print(f"Created user: {response.json()['id']}")
revyl script create seed-user \
  --file scripts/seed_user.py \
  --runtime python \
  --description "Creates a test user via API"

Use in a test

- type: code_execution
  script: seed-user

Set an optional block-level timeout_seconds to an integer from 1 to 300 (default 60). For example:

- type: code_execution
  script: seed-user
  variable_name: seed_result
  timeout_seconds: 120

In the visual editor, set Timeout on the script step (1–300 seconds), or clear it to restore the 60-second default. The setting also works for steps in reusable modules and does not change other uses of the saved script.

Only successful stdout is captured into the variable. Reports display stdout followed by stderr in one output block, without preserving interleaving. A failure or timeout leaves the existing variable unchanged. With fail_fast enabled, a failed code step stops the test. Disabling it allows later steps to run, but the test still fails. Provisioning may retry transient failures; customer code is submitted at most once. See YAML test format for the timeout and output contract.

If stdout retrieval fails for a step with variable_name, the step fails and keeps the existing variable, while preserving the script's exit code. Without an output variable, that retrieval error is a warning. Stderr retrieval errors are diagnostic and do not fail the step or discard captured stdout.

Manage scripts

revyl script list                           # List all scripts
revyl script get seed-user                  # View details and code
revyl script update seed-user --file new.py # Update the code
revyl script usage seed-user                # List tests using this script
revyl script delete seed-user

Variables

Variables let you pass dynamic data between steps using Mustache-style {{variable}} templates.

Define variables up front

test:
  metadata:
    name: login-flow
    platform: ios
    variables:
      email: test@example.com
      password: TestPass123!
  build:
    name: my-ios-app
  blocks:
    - type: instructions
      step_description: Type "{{email}}" in the email field.
    - type: instructions
      step_description: Type "{{password}}" in the password field.

Extract values at runtime

- type: extraction
  step_description: Extract the displayed order number.
  variable_name: order_id

- type: validation
  step_description: The confirmation page shows order "{{order_id}}".

Manage variables via CLI

revyl test var list login-flow
revyl test var set login-flow email=test@new.com
revyl test var get login-flow email
revyl test var delete login-flow email

Control Flow

If / Else

- type: if
  condition: Is a cookie consent banner visible?
  then:
    - type: instructions
      step_description: Tap Accept All.
  else:
    - type: instructions
      step_description: Continue without dismissing.

The question phrase goes in condition:. The paths are then: and (optionally) else:.

While Loops

- type: while
  condition: Is there a "Load More" button visible?
  body:
    - type: instructions
      step_description: Tap Load More.
    - type: manual
      step_type: wait
      step_description: "2"

Loop body lives under body:.

Workflows

A workflow is a named collection of tests that run together.

Create and manage

revyl workflow create smoke-tests
revyl workflow add-tests smoke-tests login-flow checkout-flow search-flow
revyl workflow run smoke-tests --build         # Build, upload, and run
revyl workflow run smoke-tests                 # Run without rebuilding

Manage workflows

revyl workflow list
revyl workflow info smoke-tests
revyl workflow remove-tests smoke-tests logout
revyl workflow delete smoke-tests
revyl workflow config set smoke-tests --retries 2

Tags

revyl tag create smoke
revyl tag add login-flow smoke,regression
revyl tag list
revyl test create my-test --platform ios --tag smoke --tag ios

Authoring Best Practices

  1. Intent-level instruction steps. Use one free-form instruction for a meaningful user intent instead of splitting every tap and keystroke.
  2. Sparse separate validations. Keep assertions in their own validation blocks and add them only for important user-visible outcomes.
  3. Validate durable outcomes. Check user-visible state (e.g. "inbox is visible") not transient state (e.g. "loading spinner disappeared").
  4. Use variables for secrets. Never hardcode credentials in reusable tests.
  5. Put modules at the top. Shared setup flows belong at the beginning.

Full Example: E2E Checkout

test:
  metadata:
    name: full-checkout
    platform: ios
    tags:
      - e2e
      - checkout
    variables:
      email: checkout-user@example.com
      password: TestPass123!
  build:
    name: my-ios-app
  blocks:
    - type: code_execution
      script: seed-checkout-user

    - type: module_import
      module: login-flow

    - type: instructions
      step_description: Tap the Shop tab.
    - type: instructions
      step_description: Tap on "Orchid Mantis".
    - type: instructions
      step_description: Tap Add to Cart.

    - type: if
      condition: Is a promotional banner visible?
      then:
        - type: instructions
          step_description: Tap the X to dismiss the banner.

    - type: instructions
      step_description: Tap the Cart tab.
    - type: validation
      step_description: "Orchid Mantis" is listed in the cart.
    - type: instructions
      step_description: Tap Checkout.
    - type: instructions
      step_description: Fill in shipping details and tap Continue.
    - type: instructions
      step_description: Tap Place Order.

    - type: extraction
      step_description: Extract the order confirmation number.
      variable_name: order_number
    - type: validation
      step_description: The confirmation page shows order "{{order_number}}".

    - type: instructions
      step_description: Navigate to Order History.
    - type: validation
      step_description: Order "{{order_number}}" appears in the list.

Troubleshooting

ProblemFix
"A test with that name already exists"Choose a new name, or use revyl test remote to inspect existing tests
build.name does not resolveRun revyl app list --platform ios and update test.build.name to match
Test shows as staleRun revyl test sync --prune then revyl test push <name> --force

Your first test from the CLI

Previous Page

Run tests

Next Page

On this page

Choose a WorkflowPrerequisitesYAML-First CLI1. Write a YAML file2. Create the test4. Iterate and pushScaffold FirstSession to RegressionYAML AnatomyReusable ModulesCreate a moduleImport into a testManage modulesCode Execution ScriptsCreate and register a scriptUse in a testManage scriptsVariablesDefine variables up frontExtract values at runtimeManage variables via CLIControl FlowIf / ElseWhile LoopsWorkflowsCreate and manageManage workflowsTagsAuthoring Best PracticesFull Example: E2E CheckoutTroubleshooting