How to test your Lua
Test all the things!
After the first article of this series, setting up a project with Lua and LuaRocks, and a deep dive into managing Lua modules, it is time to look at how to test Lua code in this part of the series.
The article will start by showing how to quickly iterate over code and run small snippets — manually, without external dependencies. After that; unit testing, first without and then with an external module, LuaUnit; how to create mocks for tests; and finally using continuous integration.
All the code shown in this article is also available in the companion repository on GitHub, branch part-3.
About versions
The latest Lua version as of writing is 5.5 (released ), though everything in this article series will work with Lua 5.1 (released ) and up. If this deviates, a callout like the following will signal the required version.
An overview of all versions and main their features can be found on the official Lua version history page.
Quick setup
Besides Lua, LuaRocks will be used for the base test setup. I covered setting up both in the first article of this series, showing how to install and run Lua and LuaRocks for macOS, Linux, and Windows.
With both installed, running the following command inside a project folder will create a new LuaRocks projects.
$ luarocks init \
--lua-versions "5.1,5.2,5.3,5.4,5.5" \
my-project 1.0.0
$ is used to show a command will be entered.
This will set up the package "my-project" with LuaRocks. More about managing projects with LuaRocks in the second article of this series.
Running tests
With the project setup done, it is now possible to run
./luarocks test. Without further parameters, this will
run a test.lua file from the project root
if found.
Alternatively, the test property of the project
.rockspec file can be used to define a different
location or file for the test entry.
-- my-project-1.0.0-1.rockspec
-- other properties ...
test = {
type = "command",
script = "src/test.lua"
}
Besides using a Lua file as the test entry, a
shell command can be used by changing
script to command.
-- my-project-1.0.0-1.rockspec
-- other properties ...
test = {
type = "command",
command = "lua src/test.lua"
}
For now, the test entry file will only print that no tests are defined.
-- src/test.lua
print("No tests defined, yet.")
All options for setting up the test command can be found in the test command specification on the LuaRocks GitHub wiki.
Iterative work
Even with a way to run the test entry point via LuaRocks, there is also a simpler way to test code without LuaRocks or even tests — running and verifying the code manually.
I'm absolutely in favor of test automation, but for small scripts and snippets, manually testing, as with Lua's interactive mode, can be a viable way for quick iterations.
Running the lua CLI without any arguments
will enter interactive mode.
$ lua
Lua 5.4.8 Copyright (C) 1994-2023 Lua.org, PUC-Rio
>
To test how to iterate on code, the following small piece of code,
called
chunk
in Lua, will be defined inside a new file
factorial.lua.
-- factorial.lua
function factorial(n)
if (n == 0) then
return 1
else
return n * factorial(n - 1)
end
end
print(factorial(4)) --> 24
The idea is to use the dofile function for quick tests.
$ lua
Lua 5.4.8 Copyright (C) 1994-2023 Lua.org, PUC-Rio
> dofile("factorial.lua")
24
After making changes to the code, dofile can be
executed again from the console to re-run the code. Using this
approach will enable quick iterations on small chunks via the
command-line.
Unit testing via assert
Before diving into LuaUnit, for small tests, Lua's
assert
function can be used in combination with
pcall. To keep the setup minimal, all that is needed is a
test helper function.
-- src/examples/assert.lua
function test(fn)
local status, err = pcall(fn)
if not status then
print(err)
end
end
The function will take an anonymous function as an argument, call it
via
pcall, and in the case of an error, print the result.
Using this approach, tests for the factorial function
could look like the following.
-- src/examples/assert.lua
local factorial = require "examples/factorial"
test(function ()
assert(
factorial(1) == 1,
"Factor of 1 should be 1")
end)
test(function ()
assert(
factorial(4) == 24,
"Factor of 4 should be 24")
end)
A slightly more complex example would be to create a test runner table to handle how to exit the test on failure.
-- src/examples/test-runner.lua
local Runner = {
hasErrors = false
}
function Runner:test(fn)
local status, err = pcall(fn)
if not status then
self.hasErrors = true
print(err)
end
end
function Runner:evaluate()
if self.hasErrors then
print("Test suite exited with errors")
os.exit(1)
else
print("All tests successful")
end
end
return Runner
This will use the same test function but set a property
hasErrors when a test fails. Via the
evaluate function, called after all tests, a proper
exit code and status message can be emitted.
The same test suite for the factorial function with
this small runner could look like the following.
-- src/examples/factorial-test-2.lua
local tr = require "examples/test-runner"
local factorial = require "examples/factorial"
tr:test(function ()
assert(
factorial(1) == 1,
"Factor of 1 should be 1")
end)
tr:test(function ()
assert(
factorial(4) == 24,
"Factor of 4 should be 24")
end)
tr:evaluate()
Used as a base, one can already get quite far with this simple test runner, which can even be further extended.
For example, instead of directly calling defined test functions,
gather them all and run them when calling evaluate.
This would then enable one to
pretty print the test output or even
shuffle tests to ensure test isolation.
Unit testing via LuaUnit
The LuaUnit module installed via LuaRocks is a very popular unit testing framework for Lua that comes packed with features.
Setup LuaUnit
LuaUnit needs to be added to the test
dependencies of the project. Dependencies defined under the
test_dependencies property in the projects
.rockspec file will be
automatically installed when running
./luarocks test.
-- my-project-1.0.0-1.rockspec
-- other properties ...
test_dependencies = {
"luaunit >= 3.4"
}
The .rockspec test definition also needs some
adjustments, namely running the entry test file with Lua directly to
be able to pass command-line options. As
I really like TAP
I will use it as the output format for all tests.
-- my-project-1.0.0-1.rockspec
-- other properties ...
test = {
type = "command",
command = "lua src/test.lua -o TAP"
}
Test conventions
For LuaUnit
tests are functions or tables of functions that all
start with the word test or Test and get
executed through the root test file.
-- src/test.lua
local lu = require "luaunit"
-- Define tests ...
os.exit(lu.LuaUnit.run())
A single test
Taking the
example function from earlier, a
single test case can be created by defining a function starting with
Test.
-- src/examples/factorial.lua
local function factorial(n)
if (n == 0) then
return 1
else
return n * factorial(n - 1)
end
end
return factorial
-- src/test.lua
local lu = require "luaunit"
local factorial = require "examples/factorial"
function Test1()
lu.assertEquals(factorial(4), 24)
end
os.exit(lu.LuaUnit.run())
Group tests
Multiple test cases can be organized or grouped in a table. This will increase readability when running the tests.
-- src/test.lua
local lu = require "luaunit"
local factorial = require "examples/factorial"
TestFactorial = {}
function TestFactorial:test1()
lu.assertEquals(factorial(0), 1)
end
function TestFactorial:test2()
lu.assertEquals(factorial(1), 1)
end
function TestFactorial:test3()
lu.assertEquals(factorial(4), 24)
end
os.exit(lu.LuaUnit.run())
Multiple files
A growing test suite needs to be organized for better maintainability — splitting tests in multiple files can be one way. To enable this pattern, a test module needs to return the defined test table; the entry test file will then require all defined tests and execute them.
To showcase this, a new example function will be created.
-- src/examples/round.lua
local function round(num, numDecimalPlaces)
local mult = 10^(numDecimalPlaces or 0)
return math.floor(num * mult + 0.5) / mult
end
return round
The tests for this new example will be in a separate file, returning the local table with the test cases.
-- src/examples/round-test.lua
local lu = require "luaunit"
local round = require "examples/round"
local TestRound = {}
function TestRound:test1()
lu.assertEquals(round(3.44), 3)
end
function TestRound:test2()
lu.assertEquals(round(3.44, 1), 3.4)
end
return TestRound
Same for the
factorial tests, moving them
into a separate file and exporting the test table.
-- src/examples/factorial-test.lua
local lu = require "luaunit"
local factorial = require "examples/factorial"
local TestFactorial = {}
function TestFactorial:test1()
lu.assertEquals(factorial(0), 1)
end
-- other test cases ...
return TestFactorial
I like keeping tests close to the implementation,
which is why tests are in a file with the same name and the postfix
-test in the same folder — having a
factorial-test.lua test file for the
factorial.lua implementation. Mind you, this is my
personal preference. Another typical pattern is having a single test
folder containing all test files.
Now with dedicated test files, the entry file will require all tests and run LuaUnit.
-- src/test.lua
TestFactorial = require "examples/factorial-test"
TestRound = require "examples/round-test"
local lu = require "luaunit"
os.exit(lu.LuaUnit.run())
Running ./luarocks test will execute the test suite.
$ ./luarocks test
1..5
# Started on Fri Jul 12 20:15:00 2023
# Starting class: TestFactorial
ok 1 TestFactorial.test1
ok 2 TestFactorial.test2
ok 3 TestFactorial.test3
# Starting class: TestRound
ok 4 TestRound.test1
ok 5 TestRound.test2
# Ran 5 tests in 0.000 seconds, 5 successes, 0 failures
Setup and tear-down
LuaUnit provides
a way to call code before and after a set of grouped tests
— often called "setup" and "tear-down" or
"beforeAll" and "afterAll" — via the special
functions setUp and tearDown.
-- src/examples/logger-test.lua
local lu = require "luaunit"
local Logger = require "examples/logger"
local TestLogger = {}
function TestLogger:setUp()
self.fname = "mytmplog.log"
-- Ensure file does not already exist.
os.remove(self.fname)
end
function TestLogger:testLoggerCreatesFile()
Logger.setup(self.fname)
Logger.log("Hello?")
f = io.open(self.fname, "r")
lu.assertNotNil(f)
f:close()
end
function TestLogger:tearDown()
-- Cleanup log file.
os.remove(self.fname)
end
return TestLogger
Pass arguments to LuaUnit
Arguments passed to LuaUnit through
./luarocks test should be separated by --.
For example, using
LuaUnit's pattern option to select a subset of tests to run.
$ ./luarocks test -- -p TestFactorial
This will only execute tests matching the pattern
TestFactorial.
Mocking
There are many ways to isolate code and replace external dependencies for testing, called mocking. In my time using Lua, I found no test-mock library that won my heart; instead, I use a combination of different techniques to mock code for testing.
Testability of code
Increasing testability should always be the first step, avoiding any workarounds to mock code altogether. Instead of accessing a dependency inside a function, pass it as an argument; initialize an object with needed dependencies; or make default values accessible to override via a function or property.
Here is an example of moving dependencies outside a function.
function getEmail(name)
local user = UserService:query(name)
return user.email
end
getEmail("Martin")
UserService.function getEmail(service, name)
local user = service:query(name)
return user.email
end
getEmail(UserService, "Martin")
UserService as first argument.
Another example is having a default dependency with the option to override it on object creation.
-- Defining a default logger
Player = { logger = { log = print } }
function Player:new(o)
o = o or {}
setmetatable(o, self)
self.__index = self
return o
end
function Player:say(what)
self.logger.log(what)
end
-- Using the default logger
p1 = Player:new()
p1:say("Hello") --> "Hello"
-- Overriding the default logger
p2 = Player:new({
logger = {
log = function () print("Mocked") end
}
})
p2:say("Hello") --> "Mocked"
Basic mock
A basic mock can be anything that replaces a dependency in a chunk of code. What this value does or how it is shaped depends on the goal of the test; e.g., an empty function can be passed to skip or remove a call, check if a method actually got called, or simulate values on a table to fully isolate a test.
To showcase this, the following createUser function
will be tested; the callback will be mocked.
function createUser(name, callback)
-- User creation code ...
callback(user)
end
The goal is to test if the user gets created successfully; therefore, an empty callback function is of no use. Only if the callback is called does the user get created.
One way to test this is to create a mock function that toggles a boolean when called and asserts the value.
local lu = require "luaunit"
function Test1()
local hasBeenCalled = false
createUser("Martin", function ()
hasBeenCalled = true
end)
lu.assertTrue(hasBeenCalled)
end
os.exit(lu.LuaUnit.run())
Mock exported functions and values
Mocking a function or value exported by a module is best done inside
LuaUnit's
setUp and tearDown functions. The exported function or value gets replaced with a mock in
setUp, keeping a reference, and restoring the original
value in tearDown.
-- src/examples/mock-exported-test.lua
local lu = require "luaunit"
local someModule = require "some-module"
local TestSomeModule = {}
function TestSomeModule:setUp()
-- Backup the function that will be mocked
self.backupFn = someModule.fn
-- Mock function for tests
someModule.fn = function ()
return true
end
end
function TestSomeModule:test1()
lu.assertTrue(someModule.fn())
end
function TestSomeModule:tearDown()
-- Restore mocked function
someModule.fn = self.backupFn
end
os.exit(lu.LuaUnit.run())
Mock globals
As Lua keeps all
global variables in a table called _G, the
environment, global functions and values can be mocked for tests in the same
way as
exported values.
To demonstrate this, the goal for the next example is to mock the
print function and test if it has been called at least
once.
function log(text)
print(text)
end
Inside LuaUnit's setUp a reference to the global print
function will be kept and restored inside tearDown.
function TestGlobal:setUp()
self.backupPrint = _G.print
end
function TestGlobal:tearDown()
_G.print = self.backupPrint
end
The actual test unit will define the mock function with a boolean to check if it has been called.
function TestGlobal:test1()
local hasBeenCalled = false
_G.print = function ()
hasBeenCalled = true
end
log("Hello World")
lu.assertTrue(hasBeenCalled)
end
This is how the whole example looks.
-- src/examples/mock-global-test.lua
local lu = require "luaunit"
local function log(text)
print(text)
end
TestGlobal = {}
function TestGlobal:setUp()
self.backupPrint = _G.print
end
function TestGlobal:test1()
local hasBeenCalled = false
_G.print = function ()
hasBeenCalled = true
end
log("Hello World")
lu.assertTrue(hasBeenCalled)
end
function TestGlobal:tearDown()
_G.print = self.backupPrint
end
os.exit(lu.LuaUnit.run())
print.
Mock required modules
Let's take the following scenario: a function to be tested has a dependency to a module added via LuaRocks. To properly test the function, the required module needs to be mocked.
To better visualize this, a function to retrieve an asset path from the user's system depends on the module lua-path.
-- src/examples/mock-module/assets.lua
local path = require "path"
local Assets = {}
function Assets:getPath()
local userHome = path.user_home()
return userHome .. path.DIR_SEP .. "assets"
end
return Assets
The function getPath needs to be tested, and the values
path.user_home and path.DIR_SEP mocked, to
isolate the test from the user environment.
As require is a global function, it can be
mocked the same way as other global functions: by storing a reference inside LuaUnit's setUp and
restoring it in tearDown.
-- src/examples/mock-module/assets-test.lua
function TestModule:setUp()
self.actualRequire = _G.require
end
function TestModule:tearDown()
_G.require = self.actualRequire
end
Inside setUp the global require can be
overridden, listening for the dependency to be mocked, and
returning a mocked value in place. The
actualRequire is used to get any non-mocked module.
-- src/examples/mock-module/assets-test.lua
function TestModule:setUp()
self.actualRequire = _G.require
-- Override global require
_G.require = function (modname)
if modname == "path" then
-- Return mocked version of "path".
return {
user_home = function ()
return ""
end,
DIR_SEP = "/"
}
else
-- For anything else, return actual.
return self.actualRequire(modname)
end
end
end
The to-be-tested module will be required inside the test unit.
-- src/examples/mock-module/assets-test.lua
-- Other test code ...
function TestAssets:test1()
local assets = require "examples/mock-module/assets"
lu.assertEquals(assets.getPath(), "/assets")
end
Requiring the code for testing inside the actual test needs to be
done to ensure the require function is mocked first in
setUp.
Here is the full example how to mock
require.
local lu = require "luaunit"
local TestAssets = {}
function TestAssets:setUp()
self.actualRequire = _G.require
_G.require = function (modname)
if modname == "path" then
return {
user_home = function ()
return ""
end,
DIR_SEP = "/"
}
else
return self.actualRequire(modname)
end
end
end
function TestAssets:test1()
local assets = require "examples/mock-module/assets"
lu.assertEquals(assets.getPath(), "/assets")
end
function TestAssets:tearDown()
_G.require = self.actualRequire
end
return TestAssets
Continuous integration
Tests should be run often, and adding continuous integration will help spot issues early. With a repository on GitHub, GitHub Actions is a convenient way to enable this.
I will use two amazing actions created by Leafo, leafo/gh-actions-lua and leafo/gh-actions-luarocks.
Creating a new workflow called test, running on every
push.
# .github/workflows/test.yml
name: test
on: [push]
Defining the test job using the GitHub actions.
# .github/workflows/test.yml
# Other settings ...
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: leafo/gh-actions-lua@v10
- uses: leafo/gh-actions-luarocks@v4
Finally, installing all dependencies with LuaRocks
and running the test suite. To run all tests, the
environment variable LUA_INIT needs to be set with the
path to the setup file, letting Lua know where to find modules
installed with LuaRocks.
# .github/workflows/test.yml
# Other settings ...
- name: install
run: |
luarocks install \
--deps-only lua-series-1.1.0-1.rockspec
- name: test
run: luarocks test
env:
LUA_INIT: "@src/setup.lua"
The whole workflow file, enabling continuous execution of the test suite, looks as follows.
# .github/workflows/test.yml
name: test
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: leafo/gh-actions-lua@v10
- uses: leafo/gh-actions-luarocks@v4
- name: install
run: |
luarocks install \
--deps-only lua-series-1.1.0-1.rockspec
- name: test
run: luarocks test
env:
LUA_INIT: "@src/setup.lua"
What comes next
As said in the beginning, all the code shown in this article is also available in the companion repository on GitHub, branch part-3.
The next article will focus on debugging and profiling Lua code — really digging into fixing bugs, detecting performance bottlenecks, and even writing a small profiling tool.
Until then 👋🏻