Test and Validation Exercise
Demonstrating Test and Validation of Code Skills
This lab explores the fundamentals of software test and validation. In this exercise, we will design a testing plan for the software outlined in the previous lab.
Defining the Problem
For this exercise we will be using the same code problem as in the software design lab. Therefore, the description of the code is as follows:
A program is needed that processes temperature data from a CSV file. The file contains two columns: temperature values and their respective units ('Celsius', 'Fahrenheit', or 'Kelvin'). The program must provide a means read this data into memory, while also providing an interface that allows the programmer to convert and access the values stored in memory using any of the three supported units. For example, if one had 25.0 Celsius stored in an object named `TemperatureReading`, one should be able to access the temperature reading in Fahrenheit using something like `TemperatureReading.Fahrenheit()`, getting a return value of 77.0. The program must also provide a means to write the data back into a CSV file, using one of the defined temperature units for all readings.
If you are creating a Skills claim for this lab, you may wish to add a new section onto the skills page created in the previous lab, titled ‘Test and Validation of Code’.
In such a case, you will not need to repeat the code description again. Otherwise, add the description above to your skill claim document.
Creating the Test Specification
The Test and Validation skill explicitly states that we require a table to define the full range of tests that will be used when verifying. Our TemperatureReading class has several methods that must be tested. Beyond unique conditions and tests, each of the methods should, at least, be tested for three core conditions:
- Expected performance: Provide ideal inputs and check that the output is as expected.
- Edge-Cases and Failure States: Test the code performance in non-ideal conditions. For example, when inputs are incorrect, required files are missing or empty, or external function/method calls fail to return useful values.
- Malicious Use: Test the security and robustness of the code, assuming that the user has malicious intent. Are there ways a user may try to manipulate the operation of the function or method?
In this lab, we will identify each test as belonging to one of these three categories, using ‘E’, ‘F’, and ‘M’ as identifiers in the table. We will also use ‘T’ to classify any additional test that falls outside of these three categories - if needed.
Tests for the Read and Write functions
Our program has two functions to support working with files, read_temperature_csv() and write_temperature_csv().
In this section we will create a table to define tests for these functions.
| ID | Purpose | Steps | Conditions | Expected Result | Result |
|---|---|---|---|---|---|
| E1 | Verify CSV reads correctly. | run read_temperature_csv(‘testE1.csv’). | ‘testE1.csv’ contains two valid temperature reading entries in CSV format. | Returns a list of two valid temperatureReading objects. | TBD |
| E2 | Verify CSV single value read. | run read_temperature_csv(‘testE2.csv’) | ‘testE2.csv’ contains a single valid temperature reading in CSV format | Returns a list with a single valid temperatureReading object. | TBD |
| E3 | Verify CSV writes correctly. | run write_temperature_csv(‘testE3.csv’, testData) | testData = [TemperatureReading(20,’Celsius’), TemperatureReading(77, ‘Fahrenheit’)], delete testE3.csv before running test. | ‘testE3.csv’ should be created with the specified temperature readings stored and viewable in a text editor. | TBD |
| F1 | CSV read missing file. | run read_temperature_csv(‘testF1.csv’) | ‘testF1.csv’ should not exist | Raise a FileNotFoundError | TBD |
| F2 | File data not in correct format | run read_temperature_csv(‘testF2.csv’) | ‘testF2.csv’ contains the string ‘This is not CSV data’. | State the line where the error occurred and raise ValueError | TBD |
| F3 | CSV contains missing values. | run read_temperature_csv(‘testF3.csv’) | ‘testF3.csv’ has missing temperature or unit fields. | Raise ValueError or handle gracefully, returning only valid entries. | TBD |
| F4 | CSV contains invalid temperature values. | run read_temperature_csv(‘testF4.csv’) | ‘testF4.csv’ has temperatures as ‘abc’ | Raise ValueError or handle gracefully. | TBD |
| F5 | Reading an empty file. | run read_temperature_csv(‘testF5.csv’) | ‘testF6.csv’ exists but is empty. | Return an empty list and print a warning. | TBD |
| M1 | Command injection attempt. | run read_temperature_csv(‘testM1.csv’) | ‘testM1.csv’ contains “__import__(‘builtins’).print(‘Hacked!’)” as a temperature value. | Ensure the input is treated as text, not executed. Raise a ValueError if necessary. | TBD |
Tests for TemperatureReading Object
The table above gives an idea of how tests can be planned for code function. You will notice that there is no test to check that the temperature units or values make sense (e.g. not less than 0 kelvin, not using made up units) because that is not the responsibility of the read and write CSV functions. You will now define tests for the TemperatureReading object instantiation, and the three conversion functions. It is best to keep these in their own tables for clarity.
Create a similar table to the one above, defining tests to check for the following:
- Instantiating a TemperatureReading object correctly as Celsius (e.g. with 20 Celsius).
- Instantiating a TemperatureReading object correctly as Kelvin (e.g. with 200 Kelvin).
- Instantiating a TemperatureReading object correctly as Fahrenheit (e.g. with 77 Fahrenheit).
- Instantiating a TemperatureReading object with empty string as unit.
- Instantiating a TemperatureReading object with invalid unit (e.g. ‘Temperature’ instead of Kelvin).
- Instantiating a TemperatureReading object with invalid temperature (e.g. -10 Kelvin).
Specify suitable conditions and pass criteria/expected results for these tests.
Create a second table to test the conversion functions. Identify correct function, and failure state tests to consider all edge-cases and use-cases. You may need to specify tests that use all three conversion functions for full coverage.
Writing Code and Claiming Skills
Now that the tests have been defined, and software planned, you should be in a good place to write the TemperatureReading code. Doing this will allow you to run your tests and record the result, meaning you can claim the testing skill at Knowledge level. You should also be able to claim Python programming at application level by completing this task.
If you have designed the TemperatureAnalysis tool (used to claim Application level software design) then you could also plan tests for this, allowing you to claim application level on this Software testing skill.
This should mean that you have been able to claim all three skills at application level upon successful completion of this exercise.