LuaCov - Coverage analysis for Lua scripts

LuaCov
Coverage analysis for Lua scripts

## Overview

LuaCov is a simple coverage analyzer for Lua scripts. When a Lua script is run with the `luacov` module loaded, it generates a stats file with the number of executions of each line of the script and its loaded modules. The `luacov` command-line script then processes this file generating a report file which allows one to visualize which code paths were not traversed, which is useful for verifying the effectiveness of a test suite.

LuaCov is free software and uses the same license as Lua (MIT).

## Download

LuaCov can be downloaded via LuaRocks:

    luarocks install luacov

There are some C extensions LuaCov can use (if they are available) to improve performance and analysis accuracy. To install LuaCov with these extensions install CLuaCov package instead:

    luarocks install cluacov

LuaCov itself is written in pure Lua and has no external dependencies.

You can also get the code directly from the git repo.

## Instructions

Using LuaCov consists of two steps: running your script to collect coverage data, and then running `luacov` on the collected data to generate a report.

To collect coverage data, your script needs to load the `luacov` Lua module. This can be done from the command-line, without modifying your script, like this:

    lua -lluacov test.lua

or

    lua -erequire('luacov.runner')('myconfigfilename') test.lua

(Alternatively, you can add `require("luacov")` to the first line of your script.)

Once the script is run, a file called `luacov.stats.out` is generated. If the file already exists, statistics are added to it. This is useful, for example, for making a series of runs with different input parameters in a test suite. To start the accounting from scratch, just delete the stats file.

To generate a report, just run the `luacov` command-line script. It expects to find a file named `luacov.stats.out` in the current directory, and outputs a file named `luacov.report.out`.

This is an example output of the report file:

    ==============================================================================
    test.lua
    ==============================================================================
     1 if 10 > 100 then
    *0    print("I don't think this line will execute.")
       else
     1    print("Hello, LuaCov!")
       end

Note that to generate this report, `luacov` reads the source files. Therefore, it expects to find them in the same location they were when the `luacov` module ran (the stats file stores the filenames, but not the sources themselves).

To silence missed line reporting for a group of lines, place inline options `luacov: disable` and `luacov: enable` in short comments around them:

    if SOME_DEBUG_CONDITION_THAT_IS_ALWAYS_FALSE_IN_TESTS then
       -- luacov: disable

       -- Lines here are not marked as missed even though they are not covered.

       -- luacov: enable
    end

LuaCov saves its stats upon normal program termination. If your program is a daemon -- in other words, if it does not terminate normally -- you can use the `luacov.tick` module or the `tick` configuration option, which periodically saves the stats file. For example, to run (on Unix systems) LuaCov on Xavante, just modify the first line of `xavante_start.lua` so it reads:

    #!/usr/bin/env lua -lluacov.tick

or add this to `.luacov` config file:

    tick = true

LuaCov includes several configuration options, which have their defaults stored in luacov.defaults module. These are the global defaults. To use project specific configuration, create a Lua script setting options as globals or returning a table with some options and store it as `.luacov` in the project directory from where `luacov` is being run. For example, this config informs LuaCov that only `foo` module and its submodules should be covered and that they are located inside `src` directory:

    modules = {
       ["foo"] = "src/foo/init.lua",
       ["foo.*"] = "src"
    }

## History

0.14.0 [Jan 28, 2020]
  - Added option to include untested files in the report
  - Reduce probability of interrupt errors when running LuaCov in a subprocess
0.13.0 [May 5, 2018]
  - Added `luacov: disable` and `luacov: enable` inline options that mark source lines between them as impossible to hit.
  - Fixed error when reporing coverage for files with a shebang lines using CLuaCov.
0.12.0 [June 29, 2016]
  - Added support for experimental C extensions (CLuaCov).
  - Changed config format: options are now set by assigning to globals, old format (returning a table) is still supported.
  - Added `tick`config option, equivalent to using `luacov.tick` module.
  - Fixed coverage data being saved to wrong file when using relative `statsfile` path and the program running LuaCov changes directories.
  - Improved config loading error handling.
  - Added `:on_file_error()` stub method to base reporter class, used for reporting problems when analyzing coverage data related to a file.
0.11.0 [April 18, 2016]
  - Fixed several cases of lines falsely reported as missed.
  - Fixed `luacov.tick` module not working.
  - Improved default reporter output format.
  - Reduced coverage collection overhead.
  - Changed how coverage is saved, it's now possible to start a child Lua process with LuaCov enabled without wrapping the launch in `luacov.pause` and `luacov.resume` in the parent.
  - Several minor fixes and improvements.
0.10.0 [February 9, 2016]
  - Added debug_hook() function for use in custom debug hooks.
  - Fixed patterns passed as command-line arguments matching too much.
  - Fixed order in which module name translations are applied
0.9.1 [December 7, 2015]
  - Fixed error when running LuaCov using Lua 5.1.
0.9 [December 6, 2015]
  - with_luacov() function for covering coroutines created via the C API.
  - fix priorities in modules list
  - improve coverage analysis/exclusions list
  - improve handling of multiline function declarations
  - LDoc documentation
0.8 [September 30, 2015]
  - Improved lexer which reduces false positives
  - luacov.pause() and luacov.resume() functions
  - "modules" option for configuration
  - Plus several fixes and code cleanups.
0.7 [January 12, 2015]
  - Improvement in detection of long strings.
  - Added "savestepsize" option.
  - Fix handling "codefromstring" option.
0.6 [September 10, 2014]
  - Support for custom reporter objects
  - Configuration option for processing/skipping strings
  - Several fixes: behavior of on_exit, inclusion/exclusions lists, etc.
0.5 [February 8, 2014]
  - Improved performance in reporter module
  - More improvements in exclusions list
0.4 [December 3, 2013]
  - Lua 5.2 compatibility fixes
  - Several improvements in exclusions list
0.3 [October 10, 2012]
  - Added configuration options and files
  - Summary in report
  - Improved handling of long strings and comments
  - Support for coroutines and os.exit()
0.2 [April 30, 2009]
  - Ignore code loaded from strings.
0.1 [July 16, 2007]
  - Initial release.

## Credits

LuaCov was originally designed and implemented by Hisham Muhammad as a tool for testing LuaRocks. A number of people have improved it since: see the Git logs for the full list of contributors!