testml Read Docs

DOCS v0.4Updated weekly

The reference for one test syntax that runs in every language.

Read these docs front to back, or jump in. Each guide is short. The grammar is small. Most teams ship their first cross-runtime test in under an hour.

Start the quickstart $ open spec
Spec
v0.4 · stable
Runtimes
Python · JS · Ruby · Perl · Java
License
MIT · open source
Format
YAML-style · text
docs/quickstart.tml Pass
# Same file. Every language. test: "Adds two numbers" when: call: "add(a, b)" matrix: - a: 2, b: 3, expect: 5 - a: 7, b: 4, expect: 11 then: return: expect
testml run docs/quickstart.tml 2/2 rows · 11ms

Quickstart in three commands

~ 5 min read
~/shell Ready
# 1. Install for any host runtime $ pip install testml # 2. Write a single .tml file $ echo "test: hello" > hello.tml # 3. Run it. Same file in every language. $ testml run hello.tml
3 steps · zero config OK · suite green

No demo project to clone. No proprietary runtime. The same three commands work on a fresh laptop or a CI agent.

01

Test files (.tml)

A test file pairs a setup block with one or more assertion blocks. The syntax stays the same in every runtime.

Read the spec
02

Language bridges

Each bridge maps TestML primitives onto a host runtime. Pick the bridge for your language. The test file never changes.

See bridges
03

Data-driven blocks

Tables of inputs and expected outputs feed the same assertions. One block can drive hundreds of cases.

View examples
04

Assertions

Match by value, regex, JSON shape, or numeric range. Assertions read like documentation and run like code.

Assertion grammar
05

Fixtures & env

Pull values from the host environment with $ENV. Share fixtures across runtimes without rewriting them.

Fixture rules
06

CI integration

Run TestML inside your pipeline. Exit codes, JUnit XML, and a clean diff format slot into existing CI tools.

Pipeline guide

INSTALLpick any host

One package per language. Zero vendor lock-in.

TestML ships through the package manager your team already uses. Pick the host you build in. The test file stays the same.

There is no paid tier. There is no telemetry. The runtime is open source under the MIT licence.

  • Python
  • JavaScript
  • Ruby
  • Perl
  • Java
python · pip OK
$ pip install testml $ testml run tests/
javascript · npm OK
$ npm install --save-dev testml $ npx testml run tests/
ruby · gem OK
$ gem install testml $ testml run spec/
  1. STEP 01

    Name the case

    The test key holds a short, plain sentence. Reports and CI output use this name as-is. Write it for the next engineer.

  2. STEP 02

    Describe the action

    The when block lists what the host runtime does. Call a function. Hit an endpoint. Read a file. One block per case.

  3. STEP 03

    Feed it data

    The matrix block is a table. Each row becomes a separate run with the same assertions. Edge cases live next to happy paths.

  4. STEP 04

    State the result

    The then block lists assertions. Match a value, a regex, or a JSON shape. Failures point at the row that broke.

  5. STEP 05

    Run it anywhere

    The file does not change. Swap the runtime by swapping the bridge. The Python suite and the Java suite stay in sync by default.

tests/parser_round_trip.tml Pass
# step 01 — name test: "Parser round-trip preserves keys" # step 02 — action when: call: "parse_then_dump(input)" # step 03 — data table matrix: - input: "a: 1" - input: "nested:\n b: 2" # step 04 — assertion then: equals: input
5 blocks · 1 file 2/2 rows pass
test: Free-text name of the case · string
when: Action to perform before checks · block
then: Expected result after the action · block
matrix: Table of rows that re-run the case · table
$ENV.x Read x from the host environment · ref
/regex/ Match a value with a pattern · matcher
@file Inline a fixture stored on disk · loader
#tag Mark a case for filtered runs · selector

NEXTpick a path

Ready to run your first cross-runtime test?

Start with the quickstart for a five-minute green build. Or open the spec and read the grammar end to end. Both paths land in the same place.

Get Started Free $ browse examples