Martin Helmut Fieber

Create, build, and publish modules for Lua

Lua rocks!

With the first article covering the basics of managing a project and adding dependencies with LuaRocks. A how-to for installing Lua and LuaRocks can be found in the first article as well.

This article will focus on how to create, build, and publish a Lua package, as well as documentation and version management. If you just want a quick summary you can jump down to the TL;DR at the end of the page.

Prerequisites

At least Lua and LuaRocks will be required for this article to follow along. The first article of this series shows how to install both for macOS, Linux, and Windows.

Create a new package

To create a new package the LuaRocks init command can be used. It is a convenient way of setting up a project, using locally installed dependencies. The init command will create some reasonable defaults, but it is recommended to at least specify the supported Lua versions.

If not defined, the project name will be taken from the folder name the command is called from, the default version will be dev-1. Providing at least those three options; the supported Lua versions, a project name, and a project start version; the command to initialise a package will look like the following.

$ luarocks init \
    --lua-versions "5.1,5.2,5.3,5.4" \
    my-package 1.0.0

This will create a .luarocks folder, mainly for config files, and the lua_modules folder, containing locally installed dependencies. The presence of both of those folder marks the root of the project directory.

Two executables are created as well, ./lua and ./luarocks, offering a way to run both commands for the local project, configured for the local dependency tree, and a default .gitignore file, containing both folders and the two local commands. If a .gitignore file is already present the four entries will be appended to it.

One file not to be ignored is the generated .rockspec file. It holistically describes a package and how it is handled.

Short note about the project structure

Any build artefacts that will be created, like .rock archive files by running luarocks pack, will be ignored through the .gitignore file.

/*.rock

Rockspec deep-dive

The .rockspec is a Lua file, though without access to any Lua functions. When editing the file it is good practice to check it being well-formed by running the LuaRocks lint command often.

$ ./luarocks lint

Even at the cost of repeating the official documentation, I think it is worth to look at the spec-file in more detail — specifically the general metadata, dependencies, and package source rules.

Package metadata

The project metadata is the base of the package description. There are only two mandatory fields for that section, package being the package name, and version. Though, being able to fill in more is better, providing solid help for package users — even if that is just future me.

Another crucial field is the rockspec_format, with many packages using the oldest format 1.0. To get the most out of a .rockspec file version 3.0 is recommended, giving the greatest range of options to describe a package.

Starting with those few fields, let's create a comprehensive .rockspec file.

-- my-package-1.0.0-1.rockspec
rockspec_format = "3.0"
package = "my-package"
version = "1.0.0-1"

Package source

When creating a package it is also mandatory to define how to actually retrieve the contents on build and/or install. This is done through the source.url property, taking a multitude of options, from different source control systems like Git or Mercurial, to archive files, and even FTP. All options can be found in the official LuaRocks documentation about build rules.

No matter the protocol or archive type used, important is that the source needs to be available for the used context. That means that if the package is published as archive to the official LuaRocks registry the URL needs to point to a publicly available source. If the package is only used locally, for example while development, the file:// protocol can be used, utilising the local filesystem.

A very common use-case is pointing to a Git repository combined with a specific branch or release tag. Assuming that releases are done via version tags, e.g. version 1.1.0 being the Git tag v1.1.0, this is how the configuration for this looks.

-- my-package-1.0.0-1.rockspec
-- earlier options ...

source = {
  url = "git+https://github.com/MartinHelmut/lua-series",
  tag = "v1.1.0"
}

Using a specific branch works similar.

source = {
  url = "https://github.com/MartinHelmut/lua-series.git",
  -- This is also the branch name for this article
  -- at the companion repository.
  branch = "part-2"
}

Supported platforms

In case the package is only supported on a few platform, the supported_platforms property can be used to signal this. It's an array of strings with the following full list of platform names: "unix", "bsd", "solaris", "netbsd", "openbsd", "freebsd", "dragonfly", "linux", "macosx", "cygwin", "msys", "haiku", "windows", "win32", "mingw", "mingw32", "msys2_mingw_w64".

With this, a platform can be excluded that is not supported, using the ! as prefix before the platform name.

supported_platforms = {
  "!macosx"
}

Alternatively a list of explicitly supported platforms can be given.

supported_platforms = {
  "linux",
  "macosx"
}

Package descriptions

The next larger chunk is the description table, containing a short and long description, the package license, links and maintainer information, and the option to add categories or labels to the package.

-- my-package-1.0.0-1.rockspec
-- earlier options ...

description = {
  summary = "This is a Lua series package.",
  detailed = "This package is for educational purposes ...",
  license = "MIT",
  homepage = "https://github.com/MartinHelmut/lua-series",
  issues_url
    = "https://github.com/MartinHelmut/lua-series/issues",
  maintainer
    = "Martin Helmut Fieber <info@martin-fieber.se>",
  labels = { "lua-series", "educational" }
}

If the description.homepage field is defined, a package user can open the homepage through LuaRocks by using the doc command.

$ ./luarocks doc --home

Dependencies

Declaring dependencies in the project, no matter what kind, is via a list of strings containing the dependency name and what version it uses. Those names will be looked up in the LuaRocks package registry on installation

Versions can be specified in a variety of ways, all documented in the official LuaRocks documentation about dependencies. The most common way to specify versions is via the operators ==, ~=, <, >, <=, and >=; and via a comma , separated range.

{
  -- The first two are the same, using a
  -- specific version.
  "package1 1.0",
  "package1 == 1.0",
  -- A version smaller 2.
  "package2 < 2",
  -- Multiple, exact version 2.1 and greater
  -- or equal 2.3, skipping 2.2.
  "package3 2.1, >= 2.3",
}

Let's look at the three properties most often used to declare dependencies, starting with dependencies. This is what you generally use to declare what dependencies the project will include and depends upon. Having run the init command this will already contain a special dependency, Lua, defining what versions of Lua the package supports.

dependencies = {
  "lua >= 5.1, < 5.5"
}

When setting up a project, all dependencies in that list can easily be installed via the LuaRocks install command.

$ ./luarocks install --deps-only my-package-1.0.0-1.rockspec

There are also build_dependencies and test_dependencies, used for dependencies at build and test time respectively. Both work the same as dependencies, but are not installed via the above command. The build_dependencies are only installed when running luarocks build, the test_dependencies when running luarocks test.

build_dependencies = {
  "ldoc >= 1.4"
}
test_dependencies = {
  "luaunit >= 3.4"
}

The external_dependencies field is specifically used to tell LuaRocks where to find e.g. a C-library. I will go into detail on how to use this in a later article abut C-rocks.

Platform specific dependencies

A lot of the .rockspec properties support platform specific options, and dependencies are no exception. With this it is possible to install specific packages or package versions only on certain systems. A list of all supported platform names can be found earlier in this article, under Supported Platforms.

dependencies = {
  "lua >= 5.1, < 5.5",
  "luaunit 3.2",
  platforms = {
    windows = { "luaunit 3.3" },
    unix = { "luaunit >= 3.4" }
  }
}

Install from git or url

With LuaRocks it is also possible to install package from source control or URL directly. The way this works is by pointing to a remote .rockspec file, for example in a Git repository. As seen in a later example.

$ ./luarocks install \
    https://raw.github.com/MartinHelmut/lua-series/v1.0.0/lua-series-1.0.0-1.rockspec

Be aware, dependencies installed like this can not be added to dependencies, build_dependencies, nor test_dependencies in the projects .rockspec file.

Cross-server dependency tracking

If it is planned to install dependencies from other registries than the default LuaRocks, a LuaRocks config file can be used to define multiple "rock servers".

Thanks to the local setup this can be done per project inside the .luarocks/config.lua file. Usually this file is generated with a specific version, like .luarocks/config-5.4.lua, when using Lua 5.4.

rocks_servers = {
  "http://luarocks.org/repositories/rocks"
}

Documentation

LuaRocks supports bundling documentation about the package inside the package itself. The idea is to use build.copy_directories in the .rockspec file to include a doc folder. The folder name is special, as it will be the one used for when calling luarocks doc.

-- my-package-1.0.0-1.rockspec

-- other options ...
build = {
  copy_directories = { "doc" }
}

With this setup, calling LuaRocks' doc command for the package will list the contained documentation files, in this example one Markdown file I created.

$ ./luarocks doc my-package

Documentation files for my-package 1.0.0-1
------------------------------------------

/Users/You/Projects/my-package/1.0.0-1/doc/
        Usage.md

The doc command has two more options, one is calling it with --home to open the projects home page defined through description.homepage in the default browser.

$ ./luarocks doc my-package --home

The other using the --porcelain flag to get a machine-readable output of the list of contained documentation files.

$ ./luarocks doc my-package --porcelain
/Users/You/Projects/my-package/1.0.0-1/doc/Usage.md

Pure vs. Source vs. Binary

When packing a project for distribution, there are three types of resulting archives that are referred to: pure-, source-, and binary-rock.

Pure-rock

The so called pure-rock is a package containing only Lua files. There is no need to compile anything on installation, and the package is most often platform independent. Though, usage can be restricted through the .rockspec by using the supported_platforms property.

Source-rock

A source-rock is a package that probably needs to be compiled on installation. It can contain a variety of files, build steps, and can even have platform dependent code. This is a common package type to be found.

Binary-rock

At last the binary-rock, most often containing platform specific compiled C-code, but can also contain Lua files. This package type also contains a manifest file, defining MD5 checksums for all contained executables.

Why?

One may ask, why distinguish between those types of packages? It is important to keep in mind what happens when distributing a package. Being able to skip a build step with a pure-rock can be desirable, and avoids adding specific code per platform.

Other times targeting specific platforms is a given, for example when using C-modules, so it needs to be clear how a package is installed on the target system. Those things are important for pure- and source-rocks.

Binary-rocks have some other requirements setting them apart, most notably containing at least one executable and a manifest file.

Building a module

Knowing what rock types exist, building a binary-rock will not be part of this article, and a later article in the series will cover C-rocks specifically.

To build a project written in Lua, LuaRocks needs to know where all Lua modules are located, and how to build them. The "builtin" build type will suffice. To configure this, the .rockspec file will get a build table.

-- my-package-1.0.0-1.rockspec
-- earlier options ...

build = {
  type = "builtin",
  modules = {}
}

Inside build.modules all available modules need to be listed. Let's say there is a src/main.lua file that should be the packages entry point. And a src/my-package/utils.lua as sub-module.

-- my-package-1.0.0-1.rockspec
-- earlier options ...

build = {
  type = "builtin",
  modules = {
    ["my-package"] = "src/main.lua",
    ["my-package.utils"] = "src/my-package/utils.lua"
  }
}

Alternatively, if a folder named lua is used in the package root, all containing files will be considered for build, as if every file had an entry in build.modules.

With the setup done, the project can be build.

$ ./luarocks make

Publish your rock

Usually there are two ways to publish a package for LuaRocks — one is where you provide a packaged source, for example built from a Git or Mercurial repository; the second option is publishing the code via an archive, like ZIP or Tarball, and having a URL point to it.

LuaRocks supports Git, CVS, Subversion, and Mercurial. I will show as example the usage via a Git repository, as the usage for the other source control management systems is similar.

Using a code repository

When using a Git repository it is important to use a tagged version, otherwise the package is pointing to the latest point in development. If this is the desired behaviour the .rockspec should be versioned as SCM (Source Code Management), resulting in a file name like my-package-scm-1.rockspec.

In any other case a proper version tag should be used, for example v1.0.0 for the first stable version of a package (assuming the usage of semantic versioning). As shown in the section about Package source, defining Git as a source looks as follows.

source = {
  url = "git+https://github.com/MartinHelmut/lua-series",
  tag = "v1.0.0"
}

To complete the example, here how to create and push a Git tag.

$ git tag v1.0.0 && git push --tags

When eventually packing and submitting the rock, this will be used to pack the sources, so that users won't need the respective source control system installed. Though, it is advised to use a stable URL nonetheless.

Using an archive

Archives use ZIP or Tarball, containing all sources in a folder with the same name and version. The only important structure to keep is the top level package name with the version in the archive, everything else can be custom to the packages structure, containing sources, documentations, tests, and so on.

$ tar czvpf my-package-1.0.0.tar.gz my-package-1.0.0/

source = {
  url = "https://stable.tld/my-package/my-package-1.0.0.tar.gz"
}

Build your rock

Before packing the project for upload, it needs to be built. The build command will build the rock, and install it into the project folder as dependency. When only one .rockspec is present no arguments are needed.

$ ./luarocks build

By default this will build the package provided by the .rockspec file. In case of a Git repository it will be cloned locally, and build from there. If this is not desired, a local only build can be created via the flag --pack-binary-rock.

$ ./luarocks build --pack-binary-rock

Ship it!

After the project is built, to actually publish the rock, an account at luarocks.org is needed. From the account settings page a new API key needs to be created, enabling the usage of the luarocks upload command.

What is left is packing the "rock" for publish.

$ ./luarocks pack my-package-1.0.0-1.rockspec

Even though nowhere mentioned in the official documentation, a JSON module is required for the LuaRocks upload command to work. I personally use lua-cjson for this, installed inside my project folder.

$ ./luarocks install lua-cjson

Having a JSON module available, it is now possible to publish the package for inclusion in the official LuaRocks registry.

$ ./luarocks upload my-package-1.1.0-1.rockspec \
    --api-key=<API_KEY_HERE>

The successful published package can than be found on LuaRocks.

Version management

Create a new version

After making changes to a project, creating a new version can be done via luarocks new_version. Running this command will create a new .rockspec file with the updated version number.

-- my-package-1.1.0-1.rockspec
rockspec_format = "3.0"
package = "my-package"
version = "1.1.0-1"
source = {
  url = "git+https://github.com/MartinHelmut/lua-series",
  tag = "v1.1.0"
}

Remark, this will not create a new Git tag. The newly created .rockspec file needs to be committed first, and a new Git tag created manually afterwards.

Having the new .rockspec file, and a pushed Git tag, the latest version can be published by using the pack and upload command.

$ ./luarocks pack my-package-1.1.0-1.rockspec
$ ./luarocks upload my-package-1.1.0-1.rockspec \
    --api-key=<API_KEY_HERE>

Old package versions

What about the old version? It is a common practice to keep older .rockspec version files around inside a .rockspec folder in the project. I do not do this — it is always possible to point to older versions with source control management systems, so I just remove the old .rockspec file.

For example pointing to a specific tag, with a package on GitHub, using the raw.github.com pathname.

$ ./luarocks install \
    https://raw.github.com/MartinHelmut/lua-series/v1.0.0/lua-series-1.0.0-1.rockspec

TL;DR

Alright, here a boiled down version of creating and publishing a new package to the official LuaRocks registry, using GitHub.

Create a new package.

$ luarocks init \
    --lua-versions "5.1,5.2,5.3,5.4" \
    my-package 1.0.0

Write your base .rockspec file.

-- my-package-1.0.0-1.rockspec
rockspec_format = "3.0"
package = "my-package"
version = "1.0.0-1"
source = {
  url = "git+https://github.com/YourName/my-package",
  tag = "v1.0.0"
}
description = {
  summary = "Amazing Lua package.",
  detailed = [[
    A long description of this amazing
    Lua package.
  ]],
  homepage = "https://github.com/YourName/my-package",
  license = "MIT",
  issues_url = "https://github.com/YourName/my-package/issues",
  labels = {
    "my-package",
    "another-label"
  },
  maintainer = "Your Name <your@email.tld>"
}
dependencies = {
  "lua >= 5.1, < 5.5",
  "good-package >= 1"
}
build = {
  type = "builtin",
  modules = {
    main = "src/main.lua"
  },
  copy_directories = { "doc" }
}
test_dependencies = { "luaunit >= 3.4" }

Have a quick look that you did that right.

$ ./luarocks lint my-package-1.0.0-1.rockspec

Create your project and commit your code to GitHub. Tag it!

$ git tag v1.0.0 && git push --tags

Get one of those sweet, sweet API keys, build, pack, and upload your package.

$ ./luarocks build my-package-1.0.0-1.rockspec
$ ./luarocks pack my-package-1.0.0-1.rockspec
$ ./luarocks upload my-package-1.0.0-1.rockspec \
    --api-key=<API_KEY_HERE>

Profit! 🎉

What comes next

The published package shown in this article can be found in the companion repository, branch "part-2" on GitHub.

The next article will be a deep-dive into testing Lua code — from first small iterations, unit testing, creating mocks, and running continuous integrating. Until then 👋🏻