An advanced testing library and test runner for Standard ML, highly inspired by elm-test and Expecto.
This library provides a set of composable functions for writing tests, along with a built-in test runner.
$ git clone https://github.com/PerplexSystems/railroad.git $YOUR_PROJECT/vendor/railroadThen reference vendor/railroad/lib.mlb on your project's .mlb
file. Example:
$(SML_LIB)/basis/basis.mlb
vendor/railroad/lib.mlb
main.smlval tests =
describe "math operations"
[ test "sum 1 + 1" (fn _ =>
Expect.equal Int.compare 2 (1 + 1))
]
val _ = run testsCheck out the table of contents below for more information:
The Railroad module is the root module of this library.
The Railroad.Test module consists of functions that are involved in
creating and managing tests.
val concat: Test list -> Test
Concatenates a list of tests.
concat [ userTests, baggageTests ]val describe: string -> Test list -> Test
Describes a list of tests.
describe "math operators"
[ test "sum" (fn _ =>
Expect.equal Int.compare 2 (1 + 1))
, test "failing sum" (fn _ =>
Expect.equal Int.compare 3 (2 + 3))
]val focus: Test -> Test
Returns a Railroad.Test that causes other tests to be skipped, and
only runs the given one.
Calls to focus aren't meant to be committed to version
control. Instead, use them when you want to focus on getting a
particular subset of your tests to pass. If you use focus, your
entire test suite will fail, even if each of the individual tests
pass. This is to help avoid accidentally committing a focus to
version control.
If you you use focus on multiple tests, only those tests will run.
If you put a focus inside another focus, only the outermost only
will affect which tests gets run.
See also skip. Note that skip takes precedence over
focus; if you use a skip inside a focus, it will still get
skipped, and if you use a focus inside a skip, it will also get
skipped.
describe "math operators"
[ test "sum" (fn _ =>
Expect.equal Int.compare 2 (1 + 1))
, focus (test "this is the only test that will run" (fn _ =>
Expect.equal Int.compare 3 (2 + 3)))
]val run: Test -> unit
Runs the provided tests with default configuration and exits with success or failure based on the results.
run (test "sum" (fn _ => Expect.equal Int.compare 2 (1 + 1)))val runWithConfig: Setting list ->
Runs the provided tests with the provided Settings,
exits with success or failure based on the tests results.
val sumTest =
(test "sum" (fn _ => Expect.equal Int.compare 2 (1 + 1)))
runWithConfig [ Output TextIO.stdOut ] sumTestval skip: Test -> Test
Returns a Railroad.Test that gets skipped.
Calls to skip aren't meant to be committed to version
control. Instead, use it when you want to focus on getting a
particular subset of your tests to pass. If you use skip, your
entire test suite will fail, even if each of the individual tests
pass. This is to help avoid accidentally committing a skip to version
control.
See also focus. Note that skip takes precedence over
focus; if you use a skip inside a focus, it will still get
skipped, and if you use a focus inside a skip, it will also get
skipped.
describe "math operators"
[ test "this test will be the only one to run" (fn _ =>
Expect.equal Int.compare 2 (1 + 1))
, skip (test "this test is skipped" (fn _ =>
Expect.equal Int.compare 3 (2 + 3)))
]val test: string -> (unit -> Expectation) -> Test
Return a Railroad.Test that evaluates a single Expectation.
test "sum" (fn _ => Expect.equal Int.compare 2 (1 + 1))The Railroad.Test module consists of types and functions that are involved in
configuring the test runner.
The default Settings are the following:
{ output = TextIO.stdOut
, printPassed = true
, stopOnFirstFailure = false
, executionOrder = RandomOrder NONE
}datatype Setting =
Output of TextIO.outstream
| PrintPassed of bool
| StopOnFirstFailure of bool
| ExecutionOrder of ExecutionOrderRepresents the possible settings for the runner configuration.
Output: Where the output should be redirected to.PrintPassed: Whether to print passing tests (default:true).StopOnFirstFailure: Whether to stop running tests after the first failure (default:false).ExecutionOrder: The order in which tests should be executed (default:RandomOrder NONE).
datatype ExecutionOrder =
AlphabeticalOrder
| DeclarationOrder
| RandomOrder of int optionControls the order in which tests are executed:
AlphabeticalOrder: Runs tests in alphabetical order of their descriptions (full path including parent describes).DeclarationOrder: Runs tests in the order they were declared in the source code.RandomOrder of int option: Runs tests in a random order. This is the default. When usingRandomOrder NONE, a random seed is generated and printed to the output so you can reproduce the run. When usingRandomOrder (SOME seed), the provided seed is used for deterministic shuffling.
(* Run tests in alphabetical order *)
runWithConfig [ ExecutionOrder AlphabeticalOrder ] tests
(* Run tests in random order with auto-generated seed *)
runWithConfig [ ExecutionOrder (RandomOrder NONE) ] tests
(* Run tests in random order with a specific seed for reproducibility *)
runWithConfig [ ExecutionOrder (RandomOrder (SOME 12345)) ] testsWhen using RandomOrder, the seed will be printed at the end of the output:
...
=== PASS: Expect.atMost.greater value
Random seed: 1501112078
Passed: 60, failed: 0, skipped: 0, focused: 0
The Expect module consists of assertion functions that
describes a claim to be tested.
type 'a actual = 'a
Represents the actual value passed to an assertion function.
type 'a expected = 'a
Represents the expected value passed to an assertion function.
type 'a comparer = ('a expected * 'a actual) -> General.order
Represents a function that compares the expected
against the actual value.
type 'a tostring = 'a -> string
Represents a function that converts the given value to a string.
val pass: Expectation
Always passes.
test "this sum is always two" (fn _ =>
if (1 + 1) = 2 then
Expect.pass
else
Expect.fail "man, something is up...")val fail: string -> Expectation
Always fails.
test "this sum is always two" (fn _ =>
if (1 + 1) = 2 then
Expect.pass
else
Expect.fail "man, something is up...")val onFail: string -> Expectation -> Expectation
If the given expectation fails, replace its failure message with a custom one.
test "sum" (fn _ =>
Expect.onFail
"this shouldn't be failing"
(Expect.equal Int.compare 4 (2 + 2)))val all: Expectation list -> Expectation
Passes if all of the given expectations pass. If any expectation fails, returns the first failure.
test "value is within range" (fn _ =>
(* ... *)
Expect.all
[ Expect.atLeast Int.compare 0 value
, Expect.atMost Int.compare 100 value
])val any: Expectation list -> Expectation
Passes if at least one of the given expectations passes. If all expectations fail, the test fails.
test "value is either negative or greater than 10" (fn _ =>
(* ... *)
Expect.any
[ Expect.less Int.compare 0 value
, Expect.greater Int.compare 10 value
])val isTrue: bool actual -> Expectation
Passes if the provided value is true.
Expect.isTrue (2 > 1)val isFalse: bool actual -> Expectation
Passes if the provided value is false.
Expect.isTrue (2 < 1)val some: 'a option actual -> Expectation
Passes if the provided value is SOME.
val value = SOME 1
Expect.some valueval none: 'a option actual -> Expectation
Passes if the provided value is NONE.
val value = NONE
Expect.none valueval equal: 'a comparer -> 'a expected -> 'a actual -> Expectation
Passes if the arguments are equal.
Expect.equal Int.compare 2 (1 + 1)val equalFmt: 'a comparer -> 'a tostring -> 'a expected -> 'a actual -> Expectation
Passes if the arguments are equal, but receives a
tostring that encapsulates the values on the
Expectation.
Expect.equalFmt Int.compare Int.toString 2 (1 + 1)val notEqual: 'a comparer -> 'a expected -> 'a actual -> Expectation
Passes if the arguments are not equal.
Expect.notEqual Int.compare 3 (1 + 1)val notEqualFmt: 'a comparer -> 'a tostring -> 'a expected -> 'a actual -> Expectation
Passes if the arguments are not equal, but receives a
tostring that encapsulates the values on the
Expectation.
Expect.equalFmt Int.compare Int.toString 2 (1 + 1)val atMost: 'a comparer -> 'a expected -> 'a actual -> Expectation
Passes if the provide value is less or equal than the expected value.
Expect.atMost Int.compare 3 2
Expect.atMost Int.compare 2 2val atMostFmt: 'a comparer -> 'a tostring-> 'a expected-> 'a actual-> Expectation
Passes if the provided value is less or equal than the expeted value,
but receives a tostring that encapsulates the values on
the Expectation.
Expect.atMost Int.compare Int.toString 3 2
Expect.atMost Int.compare Int.toString 2 2val atLeast: 'a comparer -> 'a expected -> 'a actual -> Expectation
Passes if the provide value is greater or equal than the expected value.
Expect.atMost Int.compare 3 4
Expect.atMost Int.compare 3 3val atLeastFmt: 'a comparer -> 'a tostring -> 'a expected -> 'a actual -> Expectation
Passes if the provided value is greater or equal than the expeted
value, but receives a tostring that encapsulates the
values on the Expectation.
Expect.atMost Int.compare Int.toString 3 4
Expect.atMost Int.compare Int.toString 3 3val less: 'a comparer -> 'a expected -> 'a actual -> Expectation
Passes if the provided value is less than the expected value.
Expect.notEqual Int.compare 3 (1 + 1)val lessFmt: 'a comparer -> 'a tostring -> 'a expected -> 'a actual -> Expectation
Passes if the provided value is less than the expeted value, but
receives a tostring that encapsulates the values on the
Expectation.
Expect.atMost Int.compare Int.toString 3 2val greater: 'a comparer -> 'a expected -> 'a actual -> Expectation
Passes if the provided value is greater than the expected value.
Expect.notEqual Int.compare 3 4val greaterFmt: 'a comparer -> 'a tostring -> 'a expected -> 'a actual -> Expectation
Passes if the provided value is greater than the expeted value, but
receives a tostring that encapsulates the values on the
Expectation.
Expect.atMost Int.compare Int.toString 3 4