YAML test format

Tests can be exported as YAML from the test editor for version control and CI/CD use, or authored from scratch via the CLI (revyl test create --from-file my-test.yaml).

Every block uses an explicit type: field plus a step_description: payload. The shape is uniform across all step types, which keeps the schema parser-friendly and round-trippable through the editor and CLI.

SchemaCopy Anchor Link

test:
  metadata:
    name: "Test name"          # required
    platform: "android"        # required: "android", "ios", or "web"
    tags:                      # optional
      - smoke
  build:
    name: "My App"             # required, matches the app name in Revyl
    branch_id: "BRANCH_UUID"   # optional; omit for Default branch
    build_selection: inherit   # inherit (Automatic), latest, latest_all_branches, or pinned
    # pinned_build_id: "BUILD_UUID"  # required with pinned, with branch_id
  blocks:                      # required, array of steps
    - type: instructions
      step_description: "..."

The app launches automatically at the start of every test, so you don't need an initial open_app step.

Build SelectionCopy Anchor Link

A test's choice lives under test.build. See How a run picks its build for the model.

• No branch_id means Default branch: runs from the CLI, MCP, and the Revyl GitHub Actions try their Git branch first, then the app's default branch. YAML never records the Git branch.
• YAML takes a branch ID, not a name. Find IDs with revyl app branch list --app "My App" --json.
• On update, omitting branch_id, build_selection, and pinned_build_id keeps the saved choice; build_selection: inherit with no IDs saves Default branch.
• A new test from YAML with no choice starts on Default branch. revyl test pull omits latest_all_branches; add it back before creating a new test from pulled YAML.

Older files. The pinned_version key (a version label) is still accepted. A file with both pinned_version and the fields above must name the same build, branch, and version.

ExampleCopy Anchor Link

test:
  metadata:
    name: "Login flow"
    platform: android
  build:
    name: "My App"
  blocks:
    - type: instructions
      step_description: "Type '{{username}}' in the email field"
    - type: instructions
      step_description: "Type '{{password}}' in the password field"
    - type: instructions
      step_description: "Tap the Login button"
    - type: validation
      step_description: "The dashboard screen is visible"
    - type: extraction
      step_description: "Get the welcome message text"
      variable_name: "welcome_msg"

Block TypesCopy Anchor Link

Action and validation blocks use step_description; code-execution blocks can use script without a description.

Code-execution timeout_seconds accepts an integer from 1 to 300 and defaults to 60 when omitted. The deadline starts when the script is submitted. YAML/API configuration is preserved by visual-editor saves. You can also set Timeout on a script step in the test or reusable module editor; clearing the field restores the 60-second default. The limit applies to that step, not every use of the saved script. Only successful stdout is captured into variable_name. Reports display stdout followed by stderr in one output block, without preserving interleaving. Failed, timed-out, and empty output leaves an existing variable unchanged. A worker-enforced timeout reports exit code 124.

Output retrieval errors preserve the script's exit code. If stdout cannot be retrieved for a step with variable_name, the step fails and leaves the variable unchanged, even if the script exited successfully. Without an output variable, a stdout retrieval error is a warning. Stderr retrieval errors are diagnostic and do not fail the step or discard captured stdout.

The report's script details show the timeout recorded for that execution, separately from elapsed duration. Editing the test later does not change it. Reports without a captured limit show Timeout: not recorded. CLI report JSON and report APIs expose it as steps[].type_data.timeout_seconds; CLI run-summary JSON uses steps[].timeout_seconds and omits it when unavailable.

With fail_fast enabled, a failed code step stops the test before the next step. Disabling it allows continuation, but the test still fails. Transient provisioning failures may receive up to three creation attempts. Once submitted, customer code is never automatically resubmitted. Setup failures report that the script did not run.

Manual StepsCopy Anchor Link

The manual block runs framework-level actions. The step_type field picks which one:

# Wait for N seconds
- type: manual
  step_type: wait
  step_description: "3"

# Open a specific app (usually unnecessary — the app under test opens automatically)
- type: manual
  step_type: open_app
  step_description: "com.apple.mobilesafari"

# Kill the app under test
- type: manual
  step_type: kill_app

# Go to the home screen
- type: manual
  step_type: go_home

# Navigate to a URL or deep link
- type: manual
  step_type: navigate
  step_description: "myapp://settings"

# Set device GPS location
- type: manual
  step_type: set_location
  step_description: "37.7749,-122.4194"  # latitude,longitude

# Set device appearance
- type: manual
  step_type: set_appearance
  step_description: "dark"               # "light" or "dark"

# Download a file to the device
- type: manual
  step_type: download_file
  step_description: "https://example.com/cert.pem"

Control FlowCopy Anchor Link

# If / Else
- type: if
  condition: "A login prompt is visible"
  then:
    - type: instructions
      step_description: "Tap 'Sign In'"
  else:
    - type: instructions
      step_description: "Tap 'Continue'"

# While loop
- type: while
  condition: "A 'Load More' button is visible"
  body:
    - type: instructions
      step_description: "Tap 'Load More'"
    - type: manual
      step_type: wait
      step_description: "2"

ModulesCopy Anchor Link

Reusable groups of blocks live in modules. Import one by UUID:

- type: module_import
  step_description: "Login Flow"             # readable label
  module_id: "<module-id>"

Use revyl module list (CLI) to find module IDs, or revyl module insert <name> to print a ready-to-paste snippet.

VariablesCopy Anchor Link

Variables use Mustache-style {{name}} syntax. They can be defined up front, extracted at runtime, or pulled from org-level globals via {{global.name}}.

test:
  metadata:
    name: login-flow
    platform: ios
    variables:
      email: test@example.com
      password: TestPass123!
  build:
    name: my-ios-app
  blocks:
    - type: instructions
      step_description: 'Type "{{email}}" in the email field'
    - type: instructions
      step_description: 'Type "{{password}}" in the password field'

Variable names allow letters, numbers, hyphens, and underscores. They must not start or end with a hyphen or underscore.

ValidationCopy Anchor Link

The CLI validates YAML before it ever runs:

revyl test validate my-test.yaml

That same validator runs automatically during revyl test create --from-file and revyl test push. It checks: required fields, valid block types, valid manual step_type values, platform values, and variable definitions before use.