Skip to content

Pytest Allure

Pytest Allure Explanantion

Overview

  • Allure Report boosts collaboration and project quality by providing clear, detailed test reports that aid issue resolution for different team members
  • The goal is to set up all our tests to generate and publish this report
  • Here is the demo report from allure
  • The simplified flow for the Allure report is:
flowchart TD

subgraph A1[" "]
direction LR
    A[Run tests and generate\nthe Allure output] --> B[(Backup the Allure output\nto the AWS S3 bucket)]
    B --> C(Copy last history\nfrom the S3 bucket\nto the Allure output folder)
end
subgraph B1[" "]
direction LR
    D[Generate the HTML-report\nfrom the Allure output] --> E[(Publish the HTML-report\nto the S3 HTML bucket)]
end
A1 ==> B1 ==> C1[View the report in the browser]

Core features

Rich and Interactive Reports:

  • Allure generates visually appealing and interactive HTML reports, making it easy to analyze test results
  • Reports include detailed information about test cases, steps, attachments, and more

Annotations and Labels:

  • Allure uses annotations and labels to provide additional information about test methods, making it easier to understand and categorize test results
  • Annotations are used to mark and describe test methods, and labels help in categorizing and filtering tests
  • Allure maintains a history of test runs, allowing you to track changes in test results over time
  • Trends and statistics help identify patterns, improvements, or regressions in the application's behavior

Key Components

Pytest plugin for the Allure output generation

In order to generate the Allure output, we need to install the allure-pytest plugin. For the time and efforts saving reasons, we will install it on-the-fly in the container where we will run the tests. This feature is introduced as the allure_dir parameter to the run_<type>_tests tasks. If specified the allure-pytest plugin will be installed and the results will be stored in the specified directory.

Allure reporting tool

The CLI utility for generating Allure reports creates an HTML report from the Allure output. We'll install this utility using GitHub Actions workflow to generate the HTML report.

How it works

  • Allure Report is composed of a framework(pytest) adapter and the allure command-line utility
  • Run the tests with pytest options --alluredir=allure-results to store the results
  • The generated files include:
  • Test result files to describe execution of tests
  • Container files to describe test fixtures
  • If required, add additional files to the allure-results to display additional info in the report
  • Additional files includes:
  • Environment file to store some global data for all the tests in the report
  • History files to enable trend graphs and other history-related features
  • Categories file to group test results into custom categories
  • Use allure generate to generate the test report into the specified directory. For e.g.: allure generate allure-report

A test report generated by Allure can not only display data about the latest test launch, but also help you compare it with the data from previous reports. To do so, Allure can keep a history or previous reports.

In a tests report with the history included, you can:

  • See what statuses did a test have previously (see Details panel → History tab)
  • Find tests that changed status since last report (see Sorting and filtering → Filter tests by marks)
  • See how specific values change over time (see Graphs)