The test battery consists in a large number of simple simulations that should be run everytime changes are made to the Lagamine code to ensure these changes do not alter previous developments.
The goal of the test battery is to improve the robustness of the code and avoid mistakes.

The organisation chart of the procedure is presented in figure 1.

• The principal Python code, “Test_battery_launcher.py”, is used to run the test battery.
• The code reads the names of the tests to be run in the general file “Test_battery.txt”.
• For each test listed in “Test_battery.txt”, the code then runs Prepro and Lagamine.
• Afterwards, the code runs a python routine specific to each test to compare that the values obtained are the same as the reference values (values that were obtained using a previous stable version of Lagamine).
• The results of the comparison of each test are written in a report file.

Fig. 1: Test battery flowchart

For the moment, the battery of tests is installed in “\\obelix\c\Test Battery”.

• The principal python script to run the test battery: Test_battery_launcher.py
• The file defining the test battery path and the names of the tests in the same folder as the principal python script: Test_battery.txt. The structure of this file is shown in figure 3.

Fig. 2: The pricipal script working directory

Fig. 3: Test_battery.txt file

The folder 'Tests' (see figure 4) contains all the tests.

Each test is in a separate folder named with the same name as the test defined in “Test_battery.txt”. This folder contains:

• The simulation files (*.lag, *ex.dat, etc.)
• The reference simulation in a folder with the same name as the test +“_ref” (N.B. this folder can contain only the result files (*.ipe, *.ipn, *.rea, …) and not the totality of the simulation files, as they are the same as the test files.
• A comparator python script attached to each test. This script must have the same name as its corresponding test. In the script, the user defines the numerical results to compare and the cost function.

Fig. 4: Test_battery.txt file

Local report

The local report is attached to the test folder and has the same name as the test (name_test.rep). This local report contains the details of the error(s) obtained when comparing the new test results to the reference.

It is structured as follows:

• Local error
• Decision (Success/ Failure): depending on the value of the error, the test is considered as passed or failed. The recommanded acceptance value for the error is 0.1% for a Mini test and 1% for a Big test.

Fig. 5: Local report Global report

The global report is attached to the principal python script folder (Report_date.rep).

It contains the following information:

• Number of tests
• Global error
• List of tests
• Decision (Success/ Failure): once one test is failed, the whole test battery is failed
• List of failed tests (if the test battery is failed)

Fig. 6: Global report

To add a new test, the user should:

• Add the new test to “http://www.lagamine.uliege.be/dokuwiki/doku.php/battery/listtests”
• Modify the file Test_battery.txt: add the name of the new test
• Add the folder of the new test (to the test battery directory) containing:
  • The files of simulation (with the same name as the test)
  • The reference simulation folder (with the same name as the test + “_ref”)
  • The comparator python script (with the same name as the test): take an existing script and adapt it to the new test