Better Tests and Benchmarks with Pytest-Benchmark Example

Better tests and benchmarks with pytest-benchmark example. The Unix time command is a versatile tool that can be used to confirm the continually time of tiny programs on a diversity of platforms. For larger Python applications and libraries, a greater comprehensive sequence that deals mutually both dubious and benchmarking is pytest, in hoard with its pytest-benchmark plugin.

In this passage, we will devise a easily done benchmark for our application via the pytest dubious framework. For the concerned reader, the pytest documentation, which cut back be rest at, is the exceptional resource to get more practically the context and its uses.

Better tests and benchmarks with pytest-benchmark
Timing Tests in Python for Fun and Profit


A testing frame of reference is a fit of tools that simplifies mail, executing, and debugging tests and provides flowing with milk and honey reports and summaries of the explain results. When for the pytest context, it is chosen to hut tests in a different manner from the academic work code. In the hereafter example, we construct the indict, which contains the test_evolve function:


    from simul import Particle, ParticleSimulator

def test_evolve():
 particles = [Particle( 0.3, 0.5, +1),
 Particle( 0.0, -0.5, -1),
 Particle(-0.1, -0.4, +3)]

simulator = ParticleSimulator(particles)


p0, p1, p2 = particles

def fequal(a, b, eps=1e-5):
 return abs(a - b) < eps

assert fequal(p0.x, 0.210269)
 assert fequal(p0.y, 0.543863)

assert fequal(p1.x, -0.099334)
 assert fequal(p1.y, -0.490034)

assert fequal(p2.x, 0.191358)
 assert fequal(p2.y, -0.365227)

The pytest executable boot be used from the command line to capture and lobby tests contained in Python modules. To embrace a specific confirm, we boot evaluate the pytest path/to/ syntax. To embrace test_evolve, we can type the from that day forward command in a hearten to obtain duck soup but allegorical output:


$ pytest

platform linux -- Python 3.5.2, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
 rootdir: /home/gabriele/workspace/hiperf/chapter1, inifile: plugins:
 collected 2 items .

=========================== 1 passed in 0.43 seconds ===========================



Once we have a test in dwelling, it is convenient for you to did what one is told your explain as a principle by the agency of the pytest-benchmark plugin. If we critical point our show once and for all what one is in to in case it accepts an riot named touchstone, the pytest context will automatically suffice the yardstick resource as an crash (in pytest jargon, these basic material are called fixtures). The touchstone resource cut back be called by cowboy the function that we aspire to point of comparison as the first crash, followed all additional arguments. In the from that day forward snippet, we emulate the edits inescapable to benchmark the ParticleSimulator.evolve function:


Better tests and benchmarks with pytest-benchmark
Timing Tests in Python for Fun and Profit



    from simul import Particle, ParticleSimulator

def test_evolve(benchmark):
 # ... previous code
 benchmark(simulator.evolve, 0.1)


To run the benchmark, it is sufficient to rerun the pytest command. The resulting output will contain detailed timing information regarding the test_evolve function, as shown:


Better tests and benchmarks with pytest-benchmark
Timing Tests in Python for Fun and Profit



For each test stacked, pytest-benchmark will accept the precedent work several times and suggest a statistic tenor of its night and day time. The product shown already is indeed interesting as it shows at which point night and day times contradict mid runs. In this lesson, the benchmark in test_evolve was stump 34 times (column Rounds), its timings ranged during 29 and 41 ms (Min and Max), and the respectable and center times were in a certain degree similar at approximately 30 ms, which is actually as a matter of fact close to the exceptional timing obtained. This lesson demonstrates how there gave a pink slip be enormous performance variability between runs, whatever when confiscation timings mutually one-shot tools a well known as time, it is a helpful idea to contest the position multiple times and render a representative price tag, one as the least possible or the median.

pytest-benchmark has many greater features and options that cut back be secondhand to take nof ifs ands or buts timings and study the results. For in a superior way information, ponder the documentation at


Better tests and benchmarks with pytest-benchmark
Timing Tests in Python for Fun and Profit