The Details: cover

CrossHair’s cover command finds inputs to your function that get coverage over it:

$ cat foo.py
from typing import List, Optional
def average(nums: List[float], default:Optional[float] = None) -> float:
    if len(nums) == 0:
        if default is None:
            raise ValueError
        return default
    return sum(nums) / len(nums)

$ crosshair cover foo.average
average([0.0, 0.0], None)
average([], 0.0)
average([], None)

CrossHair reports examples in order of added (opcode-level) coverage, descending.

You can even use the --example_output_format=pytest option to jumpstart your unit tests!:

$ crosshair cover --example_output_format=pytest foo.average
import pytest
from foo import average

def test_average():
    assert average([0.0, 0.0], None) == 0.0

def test_average_2():
    assert average([], 0.0) == 0.0

def test_average_3():
    with pytest.raises(ValueError):
        average([], None)

But don’t do this blindly! CrossHair only reports what your code does, not what it is supposed to do. Also note that CrossHair example data may not be minimal or very readable.

How do I try it?

$ pip install crosshair-tool
$ crosshair cover <module>.<function>

cover

crosshair cover --help

usage: crosshair cover [-h] [--verbose]
                       [--extra_plugin EXTRA_PLUGIN [EXTRA_PLUGIN ...]]
                       [--example_output_format FORMAT] [--coverage_type TYPE]
                       [--per_path_timeout FLOAT]
                       [--per_condition_timeout FLOAT]
                       FUNCTION

Generates inputs to a function, hopefully getting good line, branch, and path
coverage.
See https://crosshair.readthedocs.io/en/latest/cover.html

positional arguments:
  FUNCTION              A fully-qualified function to explore (e.g. "mymodule.myfunc")

options:
  -h, --help            show this help message and exit
  --verbose, -v         Output additional debugging information on stderr
  --extra_plugin EXTRA_PLUGIN [EXTRA_PLUGIN ...]
                        Plugin file(s) you wish to use during the current execution
  --example_output_format FORMAT
                        Determines how to output examples.
                            eval_expression     : [default] Output examples as expressions, suitable for
                                                  eval()
                            arg_dictionary      : Output arguments as repr'd, ordered dictionaries
                            pytest              : Output examples as stub pytest tests
                            argument_dictionary : Deprecated
  --coverage_type TYPE  Determines what kind of coverage to achieve.
                            opcode : [default] Cover as many opcodes of the function as possible.
                                     This is similar to "branch" coverage.
                            path   : Cover any possible execution path.
                                     There will usually be an infinite number of paths (e.g. loops are
                                     effectively unrolled). Use max_iterations and/or
                                     per_condition_timeout to bound results.
                                     Many path decisions are internal to CrossHair, so you may see more
                                     duplicative-ness in the output than you'd expect.
  --per_path_timeout FLOAT
                        Maximum seconds to spend checking one execution path.
                        If unspecified, CrossHair will timeout each path at the square root of the
                        `per_condition_timeout`.
  --per_condition_timeout FLOAT
                        Maximum seconds to spend checking execution paths for one condition

How does this work?

CrossHair uses an SMT solver (a kind of theorem prover) to explore execution paths and look for arguments. It uses the same engine as the crosshair check and crosshair watch commands which check code contracts.

Caveats

• This feature, as well as CrossHair generally, is a work in progress. If you are willing to try it out, thank you! Please file bugs or start discussions to let us know how it went.

• CrossHair likely won’t be able to fully explore complex code.

• Your arguments must have proper type annotations.

• Your arguments have to be deep-copyable and equality-comparable.

• CrossHair is supported only on Python 3.7+ and only on CPython (the most common Python implementation).

• Only deterministic behavior can be analyzed. (your code always does the same thing when starting with the same values)

• Be careful: CrossHair will actually run your code and may apply any arguments to it.