Introduction

Testing is an important component of the development process of Bash scripts that contributes to the final product’s quality.

In general, the test procedure consists of identifying core components of the target script and creating test cases that will verify the expected outcome is right. This process is iterative and must be executed every time the source code is changed.

Since creating and managing test cases can quickly become a challenge it’s recommended to adopt automated testing tools that can provide:

• bash functions for common test-patterns
• standard reporting formats
• integration with CI-CD tools
• integration with container engines
• test cases execution management

It’s also a best practice to separate test and dev environments by using purpose-built VMs or containers. This approach becomes crucial when working on scenarios that require the installation of additional OS packages and/or 3rd party applications, run privileged commands that can alter or destroy the environment, etc.

The “Testing bash scripts” document is a hands-on guide to exploring how to create, organize and run test cases using the bats-core and testmansh tools:

• bats-core: testing framework that provides bash functions for common test patterns and execution/reporting capabilities
• testmansh: test case manager that integrates shellcheck and bats-core with container-based testing

Organizing test cases

Let’s start with a simple folder structure used both for source and testing code:

1
2
3
4
5

mkdir 'tst_prj/'               # project base directory
mkdir 'tst_prj/bin/'           # base directory for development time tools
mkdir 'tst_prj/src/'           # base directory for source code
mkdir 'tst_prj/test/'          # base directory for testing code
mkdir 'tst_prj/test/batscore'  # test cases written in bats-core sintax

Additional subdirectories can be created in tst_prj/test/ to group test cases by modules, services, test types, etc.

Preparing the test environment

Although optional, it’s highly recommended to use containers to run tests. Verify that docker or podman are installed in your machine and ready to run.

Deploy the testmansh tool to tst_prj/bin/:

1
2
3
4

cd tst_prj/bin/
curl -O https://raw.githubusercontent.com/serdigital64/testmansh/main/testmansh
chmod 0755 testmansh
cd ..

Now create the test target bash script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

echo '#!/usr/bin/env bash

# Show package manager version
function show_package_manager() {
  /sbin/apk --version
}

# Convert strings to lowercase
function transform_to_lowercase() {
  local input="$1"

  [[ -z "$input" ]] && return 1
  echo "${input,,}"
}
' > src/example_script-functions

echo '#!/usr/bin/env bash

# Load project-wide functions
source "${BASH_SOURCE[0]%/*}/example_script-functions"

# Check parameters
[[ -z "$1" ]] && exit 1

# Main
transform_to_lowercase "$1"
' > src/example_script

chmod 755 src/example_script

Defining the test strategy

Before any test-case is created it’s advised to establish the testing scope. For example:

• scope: key script commands, key script functions, key functions from project-wide libraries
• out-of-scope: external bash libraries managed by their development process (e.g. OSS libraries, etc)

Once the general scope is defined identify what test cases are needed and the expected outcome:

• test-case-1: running the example script with no arguments should cancel execution and return exit status = 1
• test-case-2: running the example script with the required argument should complete with exit status = 0
• test-case-3: calling the transform_to_lowercase function with a mixed case parameter should print via STDOUT the same string converted to lowercase
• test-case-4: calling the show_package_manager function should print current package version via STDOUT

Understanding bats-core test-case syntax

The bats-core tool uses text files written in bash to define test cases.

Each file may contain one or more test cases with optional pre-test and post-test tasks:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

setup() {
  # (Optional) Predefined function name that is called by bats-core before test cases
}

@test 'test-case-name: test case description' {
  # @test is bats-core defined keyword used to declare the test-case
}

teardown(){
  # (Optional) Predefined function name that is called by bats-core after test cases
}

Additional keywords are available to include in the test-case definition:

• skip: use to skip test-case execution without raising an error
• run: use to execute external commands or functions and capture result in shell variables:
  • status: exit status of the executed command or function
  • output: combined content of the command’s or function’s standard-output (STDOUT) and standard-error (STDERR)

Creating test cases

Let’s implement the first test-case: probe that running the script with no args will raise an error:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

echo '
@test "run with no parameters" {

  # Use the keyword "run" to execute the shell script "src/example_script"
  run "${BATS_TEST_DIRNAME}/../../src/example_script"

  # Verify that the script rised an error (exist status != 0)
  ((status != 0))
}
' > test/batscore/test-case-1.bats

The second test-case will check that calling the script with the required argument runs successfully:

1
2
3
4
5
6
7
8
9

echo '
@test "run with required parameter" {

  run "${BATS_TEST_DIRNAME}/../../src/example_script" "Hello World"

  # Verify that the script finished successfully
  ((status == 0))
}
' > test/batscore/test-case-2.bats

For the thirst test-case verify that the function transform_to_lowercase successfully converts cases in 2 different scenarios:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

echo '
setup() {
  # Load project-wide functions
  source "${BATS_TEST_DIRNAME}/../../src/example_script-functions"
}

@test "convert all caps" {

  run transform_to_lowercase "HellO WorlD"

  # Verify that the result is right
  [[ "$output" == "hello world" ]]
}

@test "convert mixed caps" {

  run transform_to_lowercase "HELLO World"

  # Verify that the result is right
  [[ "$output" == "hello world" ]]
}
' > test/batscore/test-case-3.bats

Lastly, the fourth test-case will check that the function generates output (no matter what)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

echo '
setup() {
  # Load project-wide functions
  source "${BATS_TEST_DIRNAME}/../../src/example_script-functions"
}

@test "show package manager version" {

  run show_package_manager

  # Verify that the result is right
  (( status == 0 )) && [[ "$output" != "" ]]
}
' > test/batscore/test-case-4.bats

Running test cases

Now that all test cases are created let’s use the testmansh tool to run them in purpose-build containers:

1
2

# Run all test cases in Alpine3
./bin/testmansh -b -o -e alpine-3-bash-test:0.3.0

See below the execution report. All test cases passed successfully:

1
2
3
4
5
6
7
8
9

testmansh@phineas Process: [run bats-core tests] started
testmansh@phineas Task: run test cases on the container image: alpine-3-bash-test:0.3.0
1..5
ok 1 run with no parameters
ok 2 run with required parameter
ok 3 convert all caps
ok 4 convert mixed caps
ok 5 show package manager version
testmansh@phineas Process: [run bats-core tests] finished successfully

Since the function show_package_manager depends on the OS, let’s see what happens in a different OS:

1
2

# Run all test cases in Alpine3
./bin/testmansh -b -o -e debian-11-bash-test:0.5.0 -c test/batscore/test-case-4

As expected the test fails because the function is incompatible with the target OS:

1
2
3
4
5
6
7

testmansh@phineas Process: [run bats-core tests] started
testmansh@phineas Task: run test cases on the container image: debian-11-bash-test:0.5.0
1..1
not ok 1 show package manager version
# (in test file /test/test/batscore/test-case-4.bats, line 12)
#   `(( status == 0 ))' failed
testmansh@phineas Process: [run bats-core tests] finished with errors: exit-status-1

References

• Bats-Core Docs
• Bats-Core GIT Repo
• TestManSH Repo

Copyright information

This article is licensed under a Creative Commons Attribution 4.0 International License. For copyright information on the product or products mentioned inhere refer to their respective owner.

Disclaimer

Opinions presented in this article are personal and belong solely to me, and do not represent people or organizations associated with me in a professional or personal way. All the information on this site is provided “as is” with no guarantee of completeness, accuracy or the results obtained from the use of this information.