How to change a test framework

Changing a test framework
Translate from one human language to another!
Lang to lang

Need a better translation or explanation?
Click on the image, on how AI can help you.
(Opens in a new window)

Introduction

The world changes faster and faster.

New frameworks comes out, old frameworks dies.

Sometimes we just need to change from one to another.

How do we do that?

And how can we keep up?

What is SpecFlow?

SpecFlow is an open-source tool that aids in Behavior-Driven Development (BDD) for .NET applications. It bridges the gap between technical and non-technical stakeholders by allowing developers to write executable specifications in a Gherkin language:

Given linkedin.com is opened in a browser
When a user logs in with incorrect password
Then the following error message is shown: "🤬"
🥒 Gherkin

A test engineer writes the needed glue code that can translate the Gherkin language into code, so a computer can execute it.

By leveraging Gherkin language, SpecFlow enables clear communication and collaboration among team members, ultimately enhancing the quality and reliability of software projects.

SpecFlow is dead?

Limited Updates: SpecFlow has seen a reduction in active development and fewer updates, as evidenced by the release history on GitHub, with the last stable release in May 2022 and the last beta version in February 2023:

Screenshot from https://github.com/SpecFlowOSS/SpecFlow/graphs/contributors

Forked Project: Although SpecFlow is open-source, the name “SpecFlow” is trademarked by Tricentis. Consequently, the SpecFlow project was forked on GitHub and renamed ReqnRoll (which amusingly reminds me of a “Rick Roll“)

This looks dire for SpecFlow.

To Upgrading or not to upgrade?

I asked ChatGPT 4o this question:

SpecFlow seems to be dead.
What would the reasons be to upgrade or not to upgrade to ReqnRoll?
Can you write it as short bullet points?
📝 AI prompt

and got the following answer:

Reasons to Upgrade to ReqnRoll
* Active Development: Regular updates and support.
* New Features: Enhanced functionality and tools.
* Active Community: More resources and community support.
* Better Compatibility: Improved integration with modern tools.
* Performance Boost: Faster and more efficient testing.

Reasons Not to Upgrade to ReqnRoll
* Migration Effort: Time-consuming and costly transition.
* Stability: SpecFlow might still meet current needs.
* Integration Issues: Potential disruptions with existing tools.
* Feature Gaps: ReqnRoll may lack some SpecFlow functionalities.
* Learning Curve: Need for retraining and adjustment.
🤖 AI response

I would like to add that:

  • Security Concerns: A test framework is not in production, so lack of security updates is not as big of a problem.
  • Proven Stability: The current version has been working for 2 years, so it will probably work for some time yet.
  • ReqnRoll is a new tool: Not all features have been migrated from SpecFlow to ReqnRoll. This is covered partially by “Feature Gaps” point.

So, an upgrade is not urgent.

An important question is: “Why fix it, when it ain’t broken?”

Answer: Because everything else is upgraded, like .NET and dependencies. When something finally breaks and we have waited too long, an upgrade might not be possible anymore, and a lot may need to be re-coded. This is a very high risk, even though it is not yet urgent.

How to upgrade an existing test setup

The following approach can be used for

  • SpecFlow → ReqnRoll
  • SpecFlow → something else
  • Something third →something forth

I asked ChatGPT 4o again:

any suggestions on how to upgrade an existing test suite from one framework
to another? Please answer in steps and each step must be short.
📝 AI prompt

It gave me 16 bullet points:

1. Evaluate New Framework:..., 2. Plan Migration: ..., 
3. Set Up Framework: ...,      4. Analyze Tests: ..., 
5. Map Features: ...,          6. Prioritize Tests: ..., 
7. Create Samples: ...,        8. Automate Migration: ..., 
9. Manual Migration: ...,      10. Validate: ..., 
11. Refactor: ...,             12. Update Docs: ..., 
13. Train Team: ...,           14. Integrate CI/CD: ..., 
15. Retire Old Framework: ..., 16.Monitor: ...
🤖 AI response

I asked:

other things to remember?
📝 AI prompt

And it gave me another list:

17. Backup Original Test Suite: ..., 18. Version Control: ..., 
19. Consistency: ...,                20. Dependencies: ..., 
21. Error Handling: ...,             22. Performance: ..., 
23. Parallel Execution: ...,         24. Reporting: ..., 
25. Stakeholder Communication: ...,  26. Feedback Loop: ..., 
27. Continuous Improvement: ...,     28. Documentation: ..., 
29. Compliance: ...,                 30. Cross-Platform Testing: ..., 
31. Resource Allocation: ...
🤖 AI response

I used it as a check list and wrote the next three parts:

  1. “How to select a new framework”
  2. “How to upgrade from one framework to another”
  3. “How to train the test team”

How to select a new framework

The best way to understand a new framework is to actually test it.

Make a test that pass, fails, throws an exception.

Make a data driven test, test with reporting, etc.

I have written an article on how to test a test-framework (it is regarding unit-tests, but it can be used for any other tests). https://bartek.dk/unit-testing-the-unit-testing/

The key point here is, the more we use something, the better we understand it.

So, start with the basics and make a pilot migration on the most critical test cases to experience how they perform.

How to upgrade from one framework to another

When we have an older working test suite, then please keep it and use it.

I have learned from experience that a larger upgrade can end up blocking new tasks. Why? Because all the time is used on making the test suite work again.

When a larger update needs to happen, then create a copy of the test suite (be it a new branch, fork, or a new completely new repository.)

Run both the legacy test suite and the new one.

When something breaks in the legacy test suite, then ask yourself: Can I fix it in max. 60 minutes? If yes, then fix it in the old test suite. If not, then migrate the test case to the new test suite and deprecate it in the legacy test suite.

When the test team has extra time (I know: imaginary thinking 😅), then migrate some test cases to the new test suite (start with the most critical ones).

Evaluate automatic migration vs. manual migration. Sometimes a script can migrate 1000’s of test cases in a short time. Sometimes it is simply cheaper to let people manually copy paste, than to make a script work.

Deprecate the old test suite, when it’s not needed anymore.

How to train the test team

Remember to train the team to use the new test suite.

1. Example test cases

Migrate yourself some of the test cases, so they can be inspired from your code.

2. Unit-test of helper classes as live-documentation

Remember to make unit-tests on your helper classes, to show exampled of how the helper classes are supposed to be used (have to already made unit-tests in the legacy test suite, then it will be easy to recreate the new helper classes in the new test suite).

Proper unit-tests can be used as documentation! And also make it easier to migrate from one framework to another.

3. AI to explain code

You can also use AI to help with the training (which I have written an article about: https://bartek.dk/#ai-learning )


Alternative explanation in specific context
Alt explain
Translating from one programming language to another
Code2Code
Translating human parts of code from one human language to another
Partial trans
Explain code with Given, when, and then
Gherkin2Code
Translate given, when, then to code
Code2Gherkin
Debugging your code with AI
AI debug

Need help?
Click anyone of them to get a guide, on how AI can help you.
(Opens in a new window)

It can translate code from one coding language to another (and from one test framework to another!)

It can help translate a piece of code to Gherkin, especially good for the short cryptic lines of code:

Please translate the following C# Code into Gherkin:
string result = $"{10:D5}";
📝 AI prompt

Translated into Gherkin:

Scenario: Formatting the integer 10 with leading zeros to ensure it is 5 digits long
  Given an integer value of 10
  When the integer is formatted using the pattern "D5" within an interpolated string
  Then the result should be a string "00010"
🤖 AI response

How cool is that?

Conclusion

Selecting a framework is done by testing it out
Changing a framework don’t have to be done at once
Training the test team can be done by examples, unit-tests and AI.

Our world is already changing fast.
AI will make it changer even faster.
AI (with AI-learning) will also help us adapt faster.

Music Intro

Congratulations – Lesson complete!

Red, green, refactor

Translate from one human language to another!
Lang to lang

Need a better translation or explanation?
Click on the image, on how AI can help you.
(Opens in a new window)

Introduction on how to write a test before code

It can be very difficult to grasp, on how to write a test case before writing code.
Many ask: “How can you test something that doesn’t exist yet?”

It is because we need to view development from a different angle.
Software is the logic that connects the expected inputs with the expected outputs.

In test driven development (TDD) we take one input and output and write them down as a test case. This test case will fail, since there is no code to support it (❌ RED)
Then we can write the needed code/logic to make the test pass (✅ GREEN)

Then we take another input and output and write it down as another test case.
This test case will most likely also fail (❌ RED).
We will then update the code/logic to make the test pass (✅ GREEN).
Then we need to make sure that all the previous test cases also pass. If not, then we need to refactor the code to make them pass (♻️ Refactor)

This is where
❌ RED → ✅ GREEN → ♻️ Refactor
comes from.

Music Intro

Exercise (Waterfall Method or Cowboy Coding)

Let’s try to implement a calendar.
A calendar is something everybody can use, but it is not as easy to specify.

Choose to either A or B:

A – Waterfall Method: Try to write down a specification for a calendar.
(use 5 and 15 min.)

B – Cowboy Coding: Try to code the calendar without any planning.
(use between 5 and 15 min.)

Then review your specification with the following questions:

  1. How easy was it?
  2. How do you know, if you have all the critical parts?
  3. What parts are non-critical?
  4. How do you know a part is critical or not?

Fun observations

I prefer to call the Waterfall method for “Waterfall Model Development (WMD)” because the acronym also means Weapons of Mass Destruction and it is not entirely false 😂)

But Cowboy Coding is not much better:
“Days of coding can save you minutes of planning” 😂😂

Waterfall method and Cowboy coding both bring their own value to the table.
TDD tries to combine them into tiny waterfalls, where each test case is a waterfall method. (we could call TDD for Minimum Viable Waterfalls 😂😂😂)

Minimum Viable Product

TDD is about making a minimum viable product, because we start with the most important test case and then add another test cases, then another.

Of course, we might learn under the development that something was more important than we thought previously, which we then write down as a test case, so it can be implemented.
We can always remake the priority list (if we wrote the priority down).

Our program will then grow, based on our experience and by each test case we will learn what is actually critical for the product. When all the critical parts are implemented, then we will have the minimum viable product.

Let’s try that.

TDD method: Alternative to the waterfall and cowboy coding

Write down the most critical input and output that a calendar must handle.
i.e. handling a date:

when we create the calendar with date "2024-07-12"
then we must be able to print it as "2024-07-12"
🥒 Gherkin

This test will fail, until we create the Calendar class:

Python
class Calendar:
    def __init__(self, dateString):
        self.dateString = dateString 

    def __str__(self):
        return self.dateString
        
💻 Code

We don’t need to think about how to store the date values or anything yet.
Currently the first test only needs to store a string value and be able to print it.

Adding Test Design Techniques

Then we add another critical input and output.
This is where test design techniques are handy.
Let start with boundary-value-analysis where we test for minimum and maximum value:

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description | input_date   | expected_data
  "min. date" | "2024-01-01" | "2024-01-01" 
  "max. date" | "2024-01-31" | "2024-01-31" 
🥒 Gherkin

This will pass, so the code doesn’t need to be updated.

Extend with a negative test

But with boundary-value-analysis we need to test below the minimum value, so we update the first test and add two more:

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
  "min. date"  | "2024-01-01" | "2024-01-01"  | null
  "max. date"  | "2024-01-31" | "2024-01-31"  | null
  "below min." | "2024-01-01" | null          | "day below minimum 01" 
🥒 Gherkin

The first two tests pass, while the 3rd test fails, until we update the class:

Python
class Calendar:
    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        self.dateString = dateString 

    def __str__(self):
        return self.dateString
💻 Code

Extend with another negative test

Then we Boundary-Value-Analysis requires us to test for above the max. value:

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
  "min. date"  | "2024-01-01" | "2024-01-01"  | null
  "max. date"  | "2024-01-31" | "2024-01-31"  | null
  "below min." | "2024-01-01" | null          | "day below minimum 01" 
  "above max." | "2024-01-32" | null          | "day above maximum 31" 
🥒 Gherkin

Which again fails and we need to update the class:

Python
class Calendar:
    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        
        elif(int(day)>31):
            raise ValueError(f"The day value '{day}' is above maximum value '31'")
        
        self.dateString = dateString 

    def __str__(self):
        return self.dateString
💻 Code

Extend until fail

We will skip February to begin with.
Then we will apply the boundary value analysis to March (which it will pass) and April (for which it will fail):

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
  "Jan below min." | "2024-01-01" | null          | "day below minimum 01" 
  "Jan min. date"  | "2024-01-01" | "2024-01-01"  | null
  "Jan max. date"  | "2024-01-31" | "2024-01-31"  | null
  "Jan above max." | "2024-01-32" | null          | "day above maximum 31" 
  "Mar max. date"  | "2024-03-31" | "2024-03-31"  | null
  "Mar above max." | "2024-03-32" | null          | "day above maximum 31" 
  "Apr max. date"  | "2024-04-30" | "2024-04-30"  | null
  "Apr above max." | "2024-04-31" | null          | "day above maximum 30" 
🥒 Gherkin

So, we update our Calendar class. We can also refactor the class with the method: “checkAboveMax()” to make it easier to understand:

Python
class Calendar:
    monthsWith31Days = ["01", "03"]

    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        if(month in self.monthsWith31Days):
            self.checkAboveMax(day, 31)
        else:
            self.checkAboveMax(day, 30)
        self.dateString = dateString 

    def checkAboveMax(self, day, max):
        if(int(day)>max):
            raise ValueError(f"The day value '{day}' is above maximum value '{max}'")

    def __str__(self):
        return self.dateString
💻 Code

Extending and grouping

Now we can extend this for each other month (except February) and group them into 31 and 30 day months:

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
  "Jan below min." | "2024-01-01" | null          | "day below minimum 01" 
  "Jan min. date"  | "2024-01-01" | "2024-01-01"  | null
  # 31 day months
  "Jan max. date"  | "2024-01-31" | "2024-01-31"  | null
  "Jan above max." | "2024-01-32" | null          | "day above maximum 31" 
  "Mar max. date"  | "2024-03-31" | "2024-03-31"  | null
  "Mar above max." | "2024-03-32" | null          | "day above maximum 31" 
  "May max. date"  | "2024-05-31" | "2024-05-31"  | null
  "May above max." | "2024-05-32" | null          | "day above maximum 31" 
  "Jul max. date"  | "2024-07-31" | "2024-07-31"  | null
  "Jul above max." | "2024-07-32" | null          | "day above maximum 31" 
  "Aug max. date"  | "2024-08-31" | "2024-08-31"  | null
  "Aug above max." | "2024-08-32" | null          | "day above maximum 31" 
  "Oct max. date"  | "2024-10-31" | "2024-10-31"  | null
  "Oct above max." | "2024-10-32" | null          | "day above maximum 31" 
  "Dec max. date"  | "2024-12-31" | "2024-12-31"  | null
  "Dec above max." | "2024-12-32" | null          | "day above maximum 31" 
  # 30 day months
  "Apr max. date"  | "2024-04-30" | "2024-04-30"  | null
  "Apr above max." | "2024-04-31" | null          | "day above maximum 30"
  "Jun max. date"  | "2024-06-30" | "2024-06-30"  | null
  "Jun above max." | "2024-06-31" | null          | "day above maximum 30"   
  "Sep max. date"  | "2024-09-30" | "2024-09-30"  | null
  "Sep above max." | "2024-09-31" | null          | "day above maximum 30" 
  "Nov max. date"  | "2024-11-30" | "2024-11-30"  | null
  "Nov above max." | "2024-11-31" | null          | "day above maximum 30" 
🥒 Gherkin

Where we need to update our class again (just a single line):

Python
class Calendar:
    monthsWith31Days = ["01", "03", "05", "07", "08", "10", "12"]

    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        if(month in self.monthsWith31Days):
            self.checkAboveMax(day, 31)
        else:
            self.checkAboveMax(day, 30)
        self.dateString = dateString 

    def checkAboveMax(self, day, max):
        if(int(day)>max):
            raise ValueError(f"The day value '{day}' is above maximum value '{max}'")

    def __str__(self):
        return self.dateString
💻 Code

Extend with exceptions

Then we can extend with other than 31 and 30 days (which is February):

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
  "Jan below min." | "2024-01-01" | null          | "day below minimum 01" 
  "Jan min. date"  | "2024-01-01" | "2024-01-01"  | null
  # 31 day months
  "Jan max. date"  | "2024-01-31" | "2024-01-31"  | null
  "Jan above max." | "2024-01-32" | null          | "day above maximum 31" 
  "Mar max. date"  | "2024-03-31" | "2024-03-31"  | null
  "Mar above max." | "2024-03-32" | null          | "day above maximum 31" 
  "May max. date"  | "2024-05-31" | "2024-05-31"  | null
  "May above max." | "2024-05-32" | null          | "day above maximum 31" 
  "Jul max. date"  | "2024-07-31" | "2024-07-31"  | null
  "Jul above max." | "2024-07-32" | null          | "day above maximum 31" 
  "Aug max. date"  | "2024-08-31" | "2024-08-31"  | null
  "Aug above max." | "2024-08-32" | null          | "day above maximum 31" 
  "Oct max. date"  | "2024-10-31" | "2024-10-31"  | null
  "Oct above max." | "2024-10-32" | null          | "day above maximum 31" 
  "Dec max. date"  | "2024-12-31" | "2024-12-31"  | null
  "Dec above max." | "2024-12-32" | null          | "day above maximum 31" 
  # 30 day months
  "Apr max. date"  | "2024-04-30" | "2024-04-30"  | null
  "Apr above max." | "2024-04-31" | null          | "day above maximum 30"
  "Jun max. date"  | "2024-06-30" | "2024-06-30"  | null
  "Jun above max." | "2024-06-31" | null          | "day above maximum 30"   
  "Sep max. date"  | "2024-09-30" | "2024-09-30"  | null
  "Sep above max." | "2024-09-31" | null          | "day above maximum 30" 
  "Nov max. date"  | "2024-11-30" | "2024-11-30"  | null
  "Nov above max." | "2024-11-31" | null          | "day above maximum 30"
  # other
  "Feb max. date"  | "2024-02-29" | "2024-02-29"  | null
  "Feb above max." | "2024-02-30" | null          | "day above maximum 29"
   
🥒 Gherkin

Which will fail and we need to update our class:

Python
class Calendar:
    monthsWith31Days = ["01", "03", "05", "07", "08", "10", "12"]
    monthsWith30Days = ["04", "06", "09", "11"]

    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        if(month in self.monthsWith31Days):
            self.checkAboveMax(day, 31)
        elif(month in self.monthsWith30Days):
            self.checkAboveMax(day, 30)
        else:
            self.checkAboveMax(day, 29)
        self.dateString = dateString 

    def checkAboveMax(self, day, max):
        if(int(day)>max):
            raise ValueError(f"The day value '{day}' is above maximum value '{max}'")

    def __str__(self):
        return self.dateString
💻 Code

More exceptions:

Then we can extend with leap year and not leap year:

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
...
  # other
  "Feb max. date (leap year)" | "2024-02-29" | "2024-02-29"  | null
  "Feb above max.(leap year)" | "2024-02-30" | null          | "day above maximum 29"
  "Feb max. date (not leap)"  | "2025-02-28" | "2025-02-28"  | null
  "Feb above max.(not leap)"  | "2025-02-29" | null          | "day above maximum 28"
   
🥒 Gherkin

And update the class to deal with the leap year:

Python
class Calendar:
    monthsWith31Days = ["01", "03", "05", "07", "08", "10", "12"]
    monthsWith30Days = ["04", "06", "09", "11"]

    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        if(month in self.monthsWith31Days):
            self.checkAboveMax(day, 31)
        elif(month in self.monthsWith30Days):
            self.checkAboveMax(day, 30)
        else:
            if(self.isLeapYear(year)):
                self.checkAboveMax(day, 29)
            else:
                self.checkAboveMax(day, 28)
        self.dateString = dateString 

    def isLeapYear(self, year):
        return int(year)%4==0

    def checkAboveMax(self, day, max):
        if(int(day)>max):
            raise ValueError(f"The day value '{day}' is above maximum value '{max}'")

    def __str__(self):
        return self.dateString
💻 Code

Is this critical? Then extend even more!

As we can see, the class has the minimum viable product in it and we can always ask: “what more are we missing?” and “is it critical?”

We could add the centurial leap year exception, when a leap year ends on “xx00”, i.e.: year 2100, 2200, 2300, then a leap year is not a leap year (yes, it is part of the Gregorian calendar).

The question is always: “Is the century leap year thing important for our application?”

If our app is not needed after the year 2099, then maybe this is not critical?
No need to add test cases that will never be needed or maintain the code for it them.

Let’s say we need them, so we add them to our tests:

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
  ...
  # other
  "Feb max. date (leap year)"                 | "2024-02-29" | "2024-02-29"  | null
  "Feb above max.(leap year)"                 | "2024-02-30" | null          | "day above maximum 29"
  "Feb max. date (not leap)"                  | "2025-02-28" | "2025-02-28"  | null
  "Feb above max.(not leap)"                  | "2025-02-29" | null          | "day above maximum 28"
  "Feb max. date (centurial year, not leap)"  | "2100-02-28" | "2100-02-28"  | null
  "Feb above max.(centurial year, not leap)"  | "2100-02-29" | null          | "day above maximum 28"
   
🥒 Gherkin

The “centurial year, not leap” test cases will fail again, and we can refactor our isLeapYear method in our Calendar class, to make it pass without breaking any test cases:

Python
class Calendar:
    monthsWith31Days = ["01", "03", "05", "07", "08", "10", "12"]
    monthsWith30Days = ["04", "06", "09", "11"]

    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        if(month in self.monthsWith31Days):
            self.checkAboveMax(day, 31)
        elif(month in self.monthsWith30Days):
            self.checkAboveMax(day, 30)
        else:
            if(self.isLeapYear(year)):
                self.checkAboveMax(day, 29)
            else:
                self.checkAboveMax(day, 28)
        self.dateString = dateString 

    def isLeapYear(self, year):
        output = False
        if(int(year)%4==0):
            if(int(year)%100!=0):
                output = True
        return output

    def checkAboveMax(self, day, max):
        if(int(day)>max):
            raise ValueError(f"The day value '{day}' is above maximum value '{max}'")

    def __str__(self):
💻 Code

How deep does the rabbit whole go?

The centural leap year has an exception, that when a year ends on x400, like 1200, 1600, 2000, 2400, then a leap year (which is not a leap year, because it ends on xx00) is a leap year anyway 😂😂😂).

Again we must ask us the question: “is this critical for our app?”
If our app is not needed for the year 2000 or 2400, then there is no need to implement it.

Let’s accept again that we need the app to work for the year 2000, so we implement the test cases:

when we create the calendar with date <input_date>
then we must be able to print it as <expected_data>

Examples
  description  | input_date   | expected_data | error_message
  ...
  # other
  "Feb max. date (leap year)"                 | "2024-02-29" | "2024-02-29"  | null
  "Feb above max.(leap year)"                 | "2024-02-30" | null          | "day above maximum 29"
  "Feb max. date (not leap)"                  | "2025-02-28" | "2025-02-28"  | null
  "Feb above max.(not leap)"                  | "2025-02-29" | null          | "day above maximum 28"
  "Feb max. date (centurial year, not leap)"  | "2100-02-28" | "2100-02-28"  | null
  "Feb above max.(centurial year, not leap)"  | "2100-02-29" | null          | "day above maximum 28"
  "Feb max. date (centurial year, leap)"    | "2000-02-29" | "2000-02-29"  | null
  "Feb above max.(centurial year, leap)"      | "2000-02-30" | null          | "day above maximum 29"
   
🥒 Gherkin

Which again will require refactoring the isLeapYear method:

Python
class Calendar:
    monthsWith31Days = ["01", "03", "05", "07", "08", "10", "12"]
    monthsWith30Days = ["04", "06", "09", "11"]

    def __init__(self, dateString):
        year, month, day = dateString.split("-")
        if(int(day)<1):
            raise ValueError(f"The day value '{day}' is below minimum value '01'")
        if(month in self.monthsWith31Days):
            self.checkAboveMax(day, 31)
        elif(month in self.monthsWith30Days):
            self.checkAboveMax(day, 30)
        else:
            if(self.isLeapYear(year)):
                self.checkAboveMax(day, 29)
            else:
                self.checkAboveMax(day, 28)
        self.dateString = dateString 

    def isLeapYear(self, year):
        output = False
        if(int(year)%4==0):
            if(int(year)%100!=0):
                output = True
            else:
                if(int(year)%400==0):
                    output = True

        return output

    def checkAboveMax(self, day, max):
        if(int(day)>max):
            raise ValueError(f"The day value '{day}' is above maximum value '{max}'")

    def __str__(self):
        return self.dateString
💻 Code

More features

  • Todays date: Method to get todays date.
  • Date Arithmetic: Methods for plusDays, plusMonths, plusYears, minusDays, minusMonths, and minusYears.
  • Refactor the date storage from string to ints: Instead of storing “2024-07-12” we would store year: 2024, month: 7, day: 12.
  • Support for adding/subtracting weeks: plusWeeks and minusWeeks.
  • Date Comparison: Methods to compare dates, such as isBefore, isAfter, isEqual.
  • Method to check if a date is within a certain range: isWithinRange(startDate, endDate).
  • Day of the Week: Method to get the day of the week for a given date: getDayOfWeek.
  • Date Formatting: Method to format dates in different styles: formatDate(style).
  • Flexible Date Parsing: Handle various date formats, such as “2024-7-01”, “2024-07-1”, “24-07-01”, “July 1, 2024”, “1st July 2024”, and “01/07/2024”.
  • Duration Calculation: Method to calculate the duration between two dates in days, months, years.
  • etc.

There are many possibilities, the question is: “what is critical?”

TDD is a risk based testing approach!

Rapid prototyping

We all thought we knew what a calendar was, but how many people were able to specify it in the first waterfall/cowboy coding exercise?

Knowledge is not a boolean (either true or false). Knowledge is more a taxonomy, where you can ask yourself the following questions:

  1. Can you recognize it, when you see it?
  2. Can you operate it?
  3. Can you repair it?
  4. Can you build it from scratch?

Most people can recognize a helicopter.
Few people can fly it.
Fewer can repair it.
Even less can build it from scratch.

The same goes for coding. Often we have no idea how something should be coded. Sometimes we have a feeling of how it should be coded. Sometimes we have no idea where to start.

TDD lets us experience something one test at a time, so it becomes more clear with each test we make.

It will not be perfect, but often “something” is better than “nothing”.

One could call it “Rapid prototyping,” where each new test case is a tiny prototype that needs to be tested. Or as we mentioned yearlier: “Minimun viable waterfalls” 😂😂😂

Conclusion

Let’s wrap up our journey into Test-Driven Development (TDD), shall we? 🚀

Introduction to Writing Tests Before Code
We have learned that writing test cases before writing code requires a different perspective, viewing development as the logic connecting expected inputs with expected outputs.

TDD Cycle
We have learned the iterative cycle of TDD: writing a test case (❌ RED), writing code to pass the test (✅ GREEN), and then refactoring the code (♻️ Refactor) to ensure all tests pass.

Comparison with Other Methods
We have learned that TDD offers a balanced approach compared to the Waterfall Method and Cowboy Coding, combining the structured planning of Waterfall with the flexibility of Cowboy Coding.

Creating a Minimum Viable Product (MVP)
We have learned that TDD focuses on creating a minimum viable product by starting with the most critical test cases and progressively adding more, ensuring the product evolves based on real requirements.

Example: Calendar Application
We have learned how to implement a calendar application using TDD, starting with simple date handling and progressively adding more complex requirements like boundary values, negative tests, and special cases like leap years.

Boundary-Value Analysis
We have learned to use boundary-value analysis to test minimum and maximum values, ensuring our application handles edge cases correctly.

Negative Testing
We have learned to extend our tests with negative cases, such as below minimum and above maximum values, to ensure the application correctly handles invalid inputs.

Extending with Exceptions
We have learned to handle special cases, like different month lengths and leap years, by extending our test cases and updating the implementation accordingly.

Critical Considerations
We have learned the importance of identifying and prioritizing critical functionalities, and considering the scope and longevity of the application when deciding which test cases to implement.

Refactoring
We have learned to refactor our code to make it more understandable and maintainable, ensuring all test cases pass after each change.

Ensuring Comprehensive Coverage
We have learned to ask critical questions about the application’s requirements and to extend our tests as necessary, ensuring comprehensive coverage of all important functionalities.

More Features
We have explored additional features, like getting today’s date, based on critical needs.
It’s a risk based testing approach.

Rapid Prototyping
We have learned that knowledge in coding is not binary; it’s a spectrum. TDD allows us to experience and refine our understanding one test at a time, making the development process clearer with each test. This approach can be seen as “Rapid Prototyping,” where each new test case is a tiny prototype that needs validation. Or, as humorously mentioned, “Minimum Viable Waterfalls” 😂😂😂.

Congratulations – Lesson complete!

Unit-testing the Unit-testing

About unit-testing
About unit-testing
Translate from one human language to another!
Lang to lang

Need a better translation or explanation?
Click on the image, on how AI can help you.
(Opens in a new window)

Introduction to Unit Testing

So, what exactly is Unit Testing?

When we need to develop code, we first determine the inputs our code will receive and the outputs it should produce. Then, we write the required logic to process these inputs into the desired outputs.

A unit test is essentially a description of these inputs and the expected outputs. By writing unit tests, we can verify that the piece of code we are building fulfills these expectations. It’s like setting up mini-experiments to ensure each part of our program behaves correctly.

Unit testing ensures our code is reliable and maintainable, providing a solid foundation for our software projects. It’s like having a safety net that catches errors before they cause trouble, giving us peace of mind and making our coding adventures much smoother!

Music Intro

Needed tools

With Python we can use PyTest or RobotFramework.
With Groovy we can use Spock.
With JavaScript we can use Mocha and Chai.
With C# I have no idea, but ChatGPT suggest: NUnit and xUnit.

The test framework is not that important, the most important part is to use it wisely.

Often it is better to use a simple test-framework and add new features others have developed or develop them yourself.

How to learn a test framework?

Testing is the basis for knowledge and one of the best ways to learn something.
We simply make small tests (experiments) to see what, what succeeds, what fails, and what can be learned from it.

In the next steps I will give you some of these tests, so you can learn the test-framework you want in any coding language you want.

We will take it one test case at a time, just like we would develop with Test Driven Development :-)

I will first show the first unit-test in Python with pytest, and then show how AI can translate it to JavaScript with Mocha and Chai, then into Groovy and Spock.

Test 1: to make it pass

The first test we are going to make is to make it pass.
We need to install the framework

pip install pytest
💻 Terminal input

and make a test.py file:

Python
import pytest


def test_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("
💻 Code

To run it we need to:

pytest test.py
💻 Terminal input

Which will give the following result:

test.py .                                                           [100%]
========================== 1 passed in 0.00s =============================
💻 Terminal result

Important: the method name needs to a prefix: text_
otherwise pytest will not run this method as a test.

Other languages and test frameworks

With a simple AI prompt:

Can you translate the following code into Javascript with Mocha and Chai?
(please use suite, test, and assert, instead of describe, it, and expect)
"""python
import pytest

def test_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("
"""
📝 AI prompt
const assert = require('chai').assert;

suite('Simple Addition Tests', function() {
    test('mustPass', function() {
        assert.equal(1 + 2, 3, "this test must pass, but it passed :(");
    });
});
🤖 AI response

Same can be done for Groovy with Spock:

import spock.lang.Specification

class SimpleAdditionSpec extends Specification {
    def "mustPass"() {
        expect:
        1 + 2 == 3
    }
}
🤖 AI response


Alternative explanation in specific context
Alt explain
Translating from one programming language to another
Code2Code
Translating human parts of code from one human language to another
Partial trans
Explain code with Given, when, and then
Gherkin2Code
Translate given, when, then to code
Code2Gherkin
Debugging your code with AI
AI debug

Need help?
Click anyone of them to get a guide, on how AI can help you.
(Opens in a new window)

Test 2: to make it fail

The second test needs to fail, so we can see how it fails!
So, we add another test case:

Python
import pytest


def test_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("


def test_mustFail():
    assert 1+2==4, "this test must fail, and it did :)"
💻 Code

Which will give the following result:

====================== short test summary info ===========================
FAILED test.py::test_mustFail - AssertionError: this test must fail, and it did :)
===================== 1 failed, 1 passed in 0.01s ========================
💻 Terminal result

This is of course a manual test.

Test 3: to make it crash (neither pass or fail)

The 3rd test needs to cast an exception, so it can’t complete.

Python
import pytest


def test_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("


def test_mustFail():
    assert 1+2==4, "this test must fail, and it did :)"


def test_mustCrash():
    raise RuntimeError("CRASH! ;-)")
    assert 1+2==3, "this must crash, but it failed :("
💻 Code

Which will give the following result:

====================== short test summary info ===========================
FAILED test.py::test_mustFail - AssertionError: this test must fail, and it did :)
FAILED test.py::test_mustCrash - RuntimeError: exception
===================== 2 failed, 1 passed in 0.01s ========================
💻 Terminal result

Pytest don’t show much of a difference between these two fails, but one is an AssertionError (test failed) and the other one RuntimeError (test not completed / crashed / stopped). Some frameworks give these a different colors / icon like

🏋️ Exercise

Take the programming language of your choice.
Select a test framework for it (ask AI like ChatGPT about it, if you don’t know any).
Make 3 test cases one that passes, one that fails, and one that crashes.
Be inspired by

Test 4: Let’s automate the passed, failed, and crashed test

Testing how something works is a great way to learn.
We test how things work, and how they fail so we can recover better from the fails.

To remember or share our knowledge, we can write it down as documentation.
A great way is to do documentation as unit-tests, because we can run them.

When all the unit-tests passes, then the documentation is up-to-date.
When a unit-test fails/crashes, then the documentation needs to be updated.

This is called live-documentation, because the documentation is alive and evolving with the system under development.

Let’s try with the 1st test (pass):

In order to automate our test cases, we first need to rename the prefix test_, so pytest won’t run them automatically.

Python
import pytest

def toBeTested_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("
💻 Code

Then we can add a new unit-test that we want pytest to run, so it must start with test_ prefix:

Python
import pytest

def toBeTested_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("

def test_toBeTested_mustPass():
    toBeTested_must_pass()
    assert True
💻 Code

This will of course pass:

test.py .                                                           [100%]
========================== 1 passed in 0.00s =============================
💻 Terminal result

We write assert True in the end, because if the toBeTested_must_pass would fail, then the test would stop there.

Let’s try to to make it fail, by replacing toBeTested_mustPass with toBeTested_mustFail and experience what happens:

Python
import pytest

def toBeTested_mustFail():
    assert 1+2==4, "this test must fail, and it did :)"

def test_toBeTested_mustPass():
    toBeTested_mustFail() # we changed this line from pass to fail
    assert True
💻 Code
====================== short test summary info ===========================
FAILED test.py::test_toBeTested_mustPass - AssertionError: this test must fail, and it did :)
========================== 1 failed in 0.01s =============================
💻 Terminal result

Let’s try to contain the 2nd test (fail):

We need to use try and except (in other languages it is called try and catch):

Python
import pytest

def toBeTested_mustFail():
    assert 1+2==4, "this test must fail, and it did :)"

def test_toBeTested_mustFail():
    # Given 
    errorMessage = None

    # When 
    try:
        toBeTested_must_fail()
    except AssertionError as e:
        errorMessage = str(e)

    # Then
    assert errorMessage == "this test must fail, and it did :)\nassert (1 + 2) == 4"

💻 Code
test.py .                                                           [100%]
========================== 1 passed in 0.00s =============================
💻 Terminal result

Many test frameworks contain error and exception handlers that can be used instead:

Python
import pytest

def toBeTested_mustFail():
    assert 1+2==4, "this test must fail, and it did :)"

def test_toBeTested_mustFail():
    with pytest.raises(AssertionError, match=r"this test must fail, and it did :\)\nassert \(1 \+ 2\) == 4"):
        toBeTested_must_fail()
💻 Code

But I really dislike it, because the readability is horrible.

I often set this rule: when code can’t be split into given when then parts, then the readability becomes harder. It is like with regular language:

When the function is called and expected to fail with an AssertionError
Then the error message must match the expected message
🥒 Gherkin
The function is called and expected to fail with an AssertionError, and the exception message should match the expected regular expression
🥒 Gherkin

Both are readable, but I will let you decide which one is easier to read.

I know that some programmers will disagree with me and that is fine.
We all have preferences and different contexts to work in.

Let’s try to contain the 3nd test (crash):

It is almost the same, except that we try to contain the RuntimeError instead of the AssertionError

Python
def toBeTested_mustCrash():
    raise RuntimeError("CRASH! ;-)")
    assert 1+2==3, "this must crash, but it failed :("

def test_toBeTested_mustCrash():
    # given
    errorMessage = None

    # when
    try:
        toBeTested_must_crash()
    except RuntimeError as e:
        errorMessage = str(e)
        
    # then
    assert errorMessage == "CRASH! ;-)"
💻 Code
test.py .                                                           [100%]
========================== 1 passed in 0.00s =============================
💻 Terminal result

It’s very straight forward.

🏋️ Exercise

Make a unit test for each of your 3 test cases that passed, failed and crashed.

Data driven testing

Sometimes it is better to have a single test that is data driven, than to have multiple tests.

The balance between them is the readability of the test.

Let’s try to make the 3 first test (pass, fail, crash) data driven.

The original tests looked like this:

Python
import pytest

def test_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("

def test_mustFail():
    assert 1+2==4, "this test must fail, and it did :)"

def test__mustCrash():
    raise RuntimeError("CRASH! ;-)")
    assert 1+2==3, "this must crash, but it failed :("
💻 Code

Which can be made into:

Python
import pytest
from dataclasses import dataclass, field

@dataclass
class TestCase:
    name: str
    input: int
    expected: int
    raisesException: bool

testCases = [
    TestCase(name="1 must_pass", input=1+2, expected=3, raisesException=False),
    TestCase(name="2 must_fail", input=1+2, expected=4, raisesException=False),
    TestCase(name="3 must_crash", input=1+2, expected=4, raisesException=True),
]

ids = [testCase.name for testCase in testCases]
params = [testCase for testCase in testCases]

@pytest.mark.parametrize("testData", params, ids=ids)
def test_cases(testData:TestCase):
     # when
    if testData.raisesException:
        raise RuntimeError("CRASH! ;-)")               
    else:
        assert testData.input==testData.expected, "this test must fail, and it did :)"

💻 Code
====================== short test summary info ===========================
FAILED test.py::test_cases[2 must_fail] - AssertionError: this test must fail, and it did :)
FAILED test.py::test_cases[3 must_crash] - RuntimeError: CRASH! ;-)
===================== 2 failed, 1 passed in 0.02s ========================
💻 Terminal result

I like to use something called @dataclass that we can build test cases from and is supported by autocomplete!

Python
...
from dataclasses import dataclass, field

@dataclass
class TestCase:
    name: str
    input: int
    expected: int
    raisesException: bool
...
💻 Code

Then I can define my test cases as a list of TestCase objects:

Python
...
testCases = [
    TestCase(name="1 must_pass", input=1+2, expected=3, raisesException=False),
    TestCase(name="2 must_fail", input=1+2, expected=4, raisesException=False),
    TestCase(name="3 must_crash", input=1+2, expected=4, raisesException=True),
]
...
💻 Code

Then it transforms the testCases into something pytest understands:

Python
...
ids = [testCase.name for testCase in testCases]
params = [testCase for testCase in testCases]

@pytest.mark.parametrize("testData", params, ids=ids)
def test_cases(testData:TestCase):
    if testData.raisesException:
        raise RuntimeError("CRASH! ;-)")               
    else:
        assert testData.input==testData.expected, "this test must fail, and it did :)"
💻 Code

🏋️ Exercise

Make you 3 original test cases (that passed, failed, and crashed) to be data driven, so you can experience how a data driven test does all 3 things. (some test frameworks stops at the first fail, which is not good).

Making unit-tests understandable

It is really important to try to form the test cases, so it is easy to understand them.

A test case that is not understandable is valueless 
and we would be better without it.

If we take the example from previous chapter:

Python
import pytest
from dataclasses import dataclass, field

@dataclass
class TestCase:
    name: str
    input: int
    expected: int
    raisesException: bool

testCases = [
    TestCase(name="1 must_pass", input=1+2, expected=3, raisesException=False),
    TestCase(name="2 must_fail", input=1+2, expected=4, raisesException=False),
    TestCase(name="3 must_crash", input=1+2, expected=4, raisesException=True),
]

ids = [testCase.name for testCase in testCases]
params = [testCase for testCase in testCases]

@pytest.mark.parametrize("testData", params, ids=ids)
def test_cases(testData:TestCase):
     # when
    if testData.raisesException:
        raise RuntimeError("CRASH! ;-)")               
    else:
        assert testData.input==testData.expected, "this test must fail, and it did :)"

💻 Code

Then it is longer than the following example:

Python
import pytest

@pytest.mark.parametrize(
   "i, e, rte", [
   (1+2, 3, False),  # this test must pass
   (1+2, 4, False),  # this test must fail
   (1+2, 3, True)  # this test must crash
    ],
    ids=["must_pass", "must_fail", "must_crash"]
)
def test_cases(i, e, rte):
    if rte:
        raise RuntimeError("CRASH! ;-)")               
    else:
        assert i==e, "this test must fail, and it did :)"
💻 Code

Except, it is much harder to understand and maintain:

  • What does i, e, and rte mean?
  • the ids and the comments needs to be paired and updated manually.
  • The parameters can figured out, but with 7+ parameters, different value lengths, and 10+ test cases would make this hell to maintain!

So, please use:

  • Readable parameter names like: number1 + number2 == result and not n1+n2==r
  • Use group parameters into Inputs and Expected, so it is easy to understand what transformation needs to be done (not necessary how it is done) (In Python we can use @dataclasses
  • Use test id’s/description to easier navigate which tests has failed.
  • Try to create the context the test case needs to work within. Then it will be easier to understand, why something works the way it works.
  • Additionally use Skipped/Ignored category in case you want a test case skipped (described in Skipping tests)

🏋️ Exercise

Go through your previous exercises and evaluate if they are understandable – if not, then please improve them.

Skipping tests

Sometimes we can find a bug, and a test will fail.

There is a dangerous question to ask, that many have opinions about.

Imagine a tests starts to fail, because of a bug. What should we do?

  • Fix the bug as soon as possible!
  • Let the test fail, until it is fixed.?
  • Mark the test with @Ignore and a Jira-bug, so we can fix it soon.

I have tried multiple approaches and all of them has a price.

  • When it is not possible to fix all bugs, then we fix only the most critical ones.
  • When we are not in production yet, then a bug might be critical, but not urgent.
  • When we see a red test report, then we can get used to the red color. Non-critical bugs can make it impossible to see the critical ones.

So, a @Ignore of @skip function can be a good thing, as long as we remember to give it a comment or a link to a jira-story/bug.

In Pytest we can skip tests with:

Python
import pytest


def test_mustPass():
    assert 1+2==3, "this test must pass, but it passed :("

@pytest.mark.skip(reason="Jira: bug-101")
def test_mustFail():
    assert 1+2==4, "this test must fail, and it did :)"

@pytest.mark.skip(reason="Jira: bug-102")
def test_mustCrash():
    raise RuntimeError("CRASH! ;-)")
    assert 1+2==3, "this must crash, but it failed :("
💻 Code
test.py .                                                           [100%]
===================== 1 passed, 2 skipped in 0.01s =======================
💻 Terminal result

Then we can always read about the bug and status in the Jira bug.

A data driven test can be skipped a little differently, by
adding a property to the TestCase called skip (line 10),
assign it to each TestCase (line 13-15)
and then add the skip option to the implementation (line 23-24):

Python
import pytest
from dataclasses import dataclass, field

@dataclass
class TestCase:
    name: str
    input: int
    expected: int
    raisesException: bool
    skip: str

testCases = [
    TestCase(name="1 must_pass", input=1+2, expected=3, raisesException=False, skip=None),
    TestCase(name="2 must_fail", input=1+2, expected=4, raisesException=False, skip="Jira: bug-101"),
    TestCase(name="3 must_crash", input=1+2, expected=4, raisesException=True, skip="Jira: bug-102"),
]

ids = [testCase.name for testCase in testCases]
params = [testCase for testCase in testCases]

@pytest.mark.parametrize("testData", params, ids=ids)
def test_cases(testData:TestCase):
    if(testData.skip):
        pytest.skip(testData.skip)

    if testData.raisesException:
        raise RuntimeError("CRASH! ;-)")               
    else:
        assert testData.input==testData.expected, "this test must fail, and it did :)"

💻 Code
test.py .                                                           [100%]
===================== 1 passed, 2 skipped in 0.02s =======================
💻 Terminal result

🏋️ Exercise

Try to make a skip in one of your regular tests and one for the test-driven tests (you may only skip one of the sub-tests in the data-driven tests).

Reporting

Sometimes it can be a great idea to make a test report in html.
It can be easier to get an quick overview or navigate through the test cases easier.
For more complex tests it can also create a better overview in a visual day.

A table like this:

Python
TestCase(test=1, product="A", validFrom=today(days=-2), validTo=today(days=-1), expectedValidity=TRUE)
TestCase(test=2, product="B", validFrom=today(days=-2), validTo=today(),        expectedValidity=FALSE)
TestCase(test=3, product="C", validFrom=today(),        validTo=today(),        expectedValidity=FALSE)
TestCase(test=4, product="D", validFrom=today(),        validTo=today(days=2),  expectedValidity=TRUE)
TestCase(test=5, product="E", validFrom=today(days=1),  validTo=today(days=2),  expectedValidity=FALSE)
💻 Code

Can be reformed into a graph like this:

Which is much more readable.
Especially when the complexity grows and we add i.e. timezones.

I will not go much into reporting in this chapter, but will write a separate one, which will contain all kind of good ideas, incl. testing of report templates.

Conclusion

Let’s wrap up our journey into Test-Driven Development (TDD), shall we? 🚀

Needed tools
There are many programming languages and even more test-frameworks. To compare them better, it is recommended to test them out.

How to Learn a Test Framework
We have learned that creating small, incremental tests helps us understand the outcomes and build knowledge of the test framework, similar to the principles of Test-Driven Development (TDD).

Test 1: To Make It Pass
We have learned how to set up a basic test that is designed to pass, involving the installation of the framework, writing a simple test, and ensuring it runs successfully.

Test 2: To Make It Fail
We have learned the importance of including a test case meant to fail, as it helps us understand how the test framework handles and reports failures.

Test 3: To Make It Crash
We have learned to create a test that raises an exception to simulate a crash, which helps distinguish between assertion errors and runtime errors in the test results.

Automating Tests
We have learned to automate tests by renaming methods to avoid automatic execution, using wrapper tests to verify behavior, and ensuring non-crashing tests pass successfully.

Containing Failures and Crashes
We have learned to handle expected errors using try-except blocks (or try-catch in other languages) to manage assertion errors and runtime exceptions effectively.

Data-Driven Testing
We have learned to consolidate multiple tests into a single parameterized test using @dataclass to define test cases, which enhances readability and maintainability.

Making Tests Understandable
We have learned the importance of clear and maintainable tests by using descriptive parameter names, grouping inputs and expected results, and avoiding cryptic variable names.

Skipping Tests
We have learned to mark tests that should not run due to known issues with @skip or @Ignore, and to provide reasons or links to bug tracking systems for reference.

Reporting
We have learned the value of HTML reports for better visual representation and navigation of test results, especially useful in more complex scenarios.

Congratulations – Lesson complete!

Spice it up with TDD

Learn TDD
Learn TDD
Translate from one human language to another!
Lang to lang

Need a better translation or explanation?
Click on the image, on how AI can help you.
(Opens in a new window)

Intro

Let’s dive into the world of Test Driven Development (TDD) together, shall we? 🚀

So, what exactly is TDD? Well, it’s a way to describe things in a programming language.. Imagine it’s like telling a story, but instead of using words, we’re using code!

Here’s how it works: when we want to describe something using English, we focus on describing one thing at a time.

It’s like building with Lego bricks – we start with one piece and add more as we go along!

But here’s the cool part: just like in science like physics, we can even test our code, to see if it behaves the way we expect it to. It’s all about making sure our code is usable and understandable, just like in a science experiment!

So, think of TDD as our own little computer-science experiment. We write down our specifications and then make sure the computer behaves the way we want it to 🤖✨

Music Intro

Lets do some coding!

First we split our code into 3 parts: “given”, “when”, and “then”

Python
# Given (where we set our preconditions)
number1 = 1
number2 = 2

# When (where the action happens)
result = number1 + number2

# Then (where we assert/return our output)
print(result)
💻 Code

With the print in the end, we can verify the result manually:

Python3 2024-06-03 on Linux x86_64)
----
3
💻 Terminal result

You can always use our web editors: 💻 Python / 💻 JavaScript / 💻 Groovy / 💻 C#

The “given”, “when”, and “then” is usually known in TDD as the triple A’s: “Arrange”, “Act”, and “Assert” and they do the same, except given, when, then is easier to read and arrange, act, assert is more technical.’

Python
# Given two numbers are defined
# When the numbers are added together into the result
# Then the result must be verified to be the sum

# Arrange: two numbers are defined
# Act: the numbers are added together to get the result
# Assert: verify the result is the sum of the numbers
💻 Code

Assert instead of print

What we can do instead, is to assert it automatically, by replacing print with assert:

Python
# Given
number1 = 1
number2 = 2

# When 
result = number1 + number2

# Then
assert(result == 3)
💻 Code

The == means that we want the computer to verify if the equal is actually equal.
It responds with either true (the equal is equal) or false (the equal is not equal).
The assert consumes the true or false and if it’s true, then it does nothing, because everything is fine:

Python3 2024-06-03 on Linux x86_64)
----
💻 Terminal result

If it’s true, then it the codes passes and everything is fine (even though we can’t see it, but no news is good news!).


Alternative explanation in specific context
Alt explain
Translating from one programming language to another
Code2Code
Translating human parts of code from one human language to another
Partial trans
Explain code with Given, when, and then
Gherkin2Code
Translate given, when, then to code
Code2Gherkin
Debugging your code with AI
AI debug

Need help?
Click anyone of them to get a guide, on how AI can help you.
(Opens in a new window)

Assert & print

If we want a result we can print it:

Python
# Given
number1 = 1
number2 = 2

# When 
result = number1 + number2

# Then
print(f"result: {result}")
assert(result == 3)
💻 Code

The f in the f”…” means it is a text with parameters, where we can take the value of result.
It has different names in different programming languages:

LanguageExampleName
Pythonf"text {parameter] text"f-string
JavaScript`text ${parameter} text`template string
C#$"text {parameter} text"interpolated string
Groovy"text {parameter} text"g-string

Negative test

To see a negative result, we need to replace the 3 with a 4 (because 1+2 is not 4):

Python
# Given
number1 = 1
number2 = 2

# When 
result = number1 + number2

# Then
assert(result == 4)
💻 Code

Then we will see an assertion error, where the result is not 4:

Python3 2024-06-03 on Linux x86_64)
----
Traceback (most recent call last):
  File "apps/console-python/", line 9, in <module>
    assert(result == 4)
AssertionError
💻 Terminal result

Better error messages

To give it a better error message, we could add an actual error message:

Python
# Given
number1 = 1
number2 = 2

# When 
result = number1 + number2

# Then
assert(result == 4), f"(actual) {result} != 4 (expected)"
💻 Code

Which gives:

Python3 2024-06-03 on Linux x86_64)
----
Traceback (most recent call last):
  File "apps/console-python/", line 9, in <module>
    assert(result == 4), f"(actual) {result} != 4 (expected)"
AssertionError: (actual) 3 != 4 (expected)
💻 Terminal result

and put it inside the text. So, f”{result} =! 4″ becomes: “3 != 4”

You can see, some languages are more fun than others 😂

Why use an assert, when print makes it much easier?

In a simple program like that, we can use a print.

A more complex program may have 20 kinds of outputs, depending on the inputs.
It is much easier, faster and more precise to automate all the outputs, instead of testing them manually:

Python
# Imagine we have already a complex program called buyTicket(numberOfZones, travelerType)

# here would the tests be:
assert (buyTicket(1, "adult")=="2 €")
assert (buyTicket(8, "adult")=="16 €")
assert (buyTicket(1, "child")=="1 €")
assert (buyTicket(8, "child")=="8 €")
# etc... 
💻 Code

No need to run the buyTicket manually 20 times and checking all the values are correct.
A machine can run a 1000 test cases per minute, and we humans can’t.
And there is also no need to remember all the results, when the machine can do it for us.

Testing before coding

TDD is not only about testing the code.
It’s more about writing down our assumptions that we have in our head.

It’s about getting your idea out of your head and define where we start and where we need to end.

So, often we need to start with the given’s (where we start) and the then’s (where we need to end):

Python
# Given
# write your code before writing the when code

# When 
# write your code after writing the given and then code

# Then
# write your code before writing the when code
💻 Code

The when code is written last, for the given and then are the specification of the wanted system behavior. The when code is the required logic that makes the given and then to connect.

And whenever we want to change the specification, we just need to change the given and the then code. Then we will see the existing when code will fail, and therefore needs to be refactored (updated).

Exercise:

Write the needed when code, so all the asserts pass. Please don’t change the code in the given and then.

Python
# Given (writen before when)
number1 = 1
number2 = 2
number3 = 3
number4 = 5

# When (written after given and then)
# your code goes here

# Then (written before when)
assert(result1 == 3), f"{result} =! 3"
assert(result2 == 4), f"{result} =! 4"
assert(result3 == 5), f"{result} =! 5"
assert(result4 == 6), f"{result} =! 6"
assert(result5 == 7), f"{result} =! 7"
assert(result6 == 8), f"{result} =! 8"
💻 Code
Hint or Check if your result is correct)
Python
# Given
number1 = 1
number2 = 2
number3 = 3
number4 = 5

# When we add 2 numbers together to get the result1-6
result1 = number1 + number2
result2 = number1 + number3
result3 = number2 + number3
result4 = number1 + number4
result5 = number2 + number4
result6 = number3 + number4

# Then
assert(result1 == 3), f"{result} =! 3"
assert(result2 == 4), f"{result} =! 4"
assert(result3 == 5), f"{result} =! 5"
assert(result4 == 6), f"{result} =! 6"
assert(result5 == 7), f"{result} =! 7"
assert(result6 == 8), f"{result} =! 8"

What’s next?

In the next lessons, we will do some coding exercises.

I would like you to use the assert, so you don’t have to remember the results from all the exercises.

You are welcome to add a print, if you want to. It can be very useful, to use a print, get a view inside to see what the code is doing.

Saving and loading

In the web editor (in case you want to use that to begin with), there is no option to save your code, but what you can do is to copy & paste your code into a notepad for future use.

Then you can always copy & paste it back into a web editor, if you want to use it again.
It is also possible to have multiple web editors open, without them interfering with each other.

Exercise:

Add an assert after the print. And make sure the whole code passes.

Python
# Given name and a greeting is defined (you can use any name)
name = "Bartek"
greeting = "Hello"

#When the greeting, a space, and the name is combined into the text.
text = greeting + " " + name

# Then the text can be printed as output and be verified by us
print(text)
💻 Code
Hint or Check if your result is correct)

assert(text == “Hello Bartek”)

Then save your code into your favorite notepad, and save it.

Then update the code, to make it fail with an AssertionError.

Extra: if you want more challenge, you can try to add an improved error message.

Spilting:

Sometimes it is not possible to write code in the given, when, then format.
This is a sign, that we should split the code into smaller parts (like methods/functions or classes).

A piece of code should often only do a single thing, even though the outputs can be different.

Conclusion

Let’s wrap up our journey into Test-Driven Development (TDD), shall we? 🚀

So, here’s the scoop: Test-Driven Development (TDD) offers a structured approach akin to the scientific method, providing a framework for software development that prioritizes clarity, predictability, and reliability.

Picture this: before we even start writing the actual code, we write tests to describe how we want our software to behave. It’s like drawing a map before going on a big adventure – it helps us know where we’re going!

And here’s where things get really cool: we use something called assertions to check if our code works the way we want it to. It’s like having a super fast and precise robot double-checking our work to make sure everything’s OK! 🤖✅

TDD not only makes testing super fast and easy, but it also helps us understand our coding problems better. It’s like putting together a puzzle – each piece fits together perfectly to create something awesome!

So, as we dive deeper into coding and keep exploring TDD, let’s make assertions our best friend. They’ll help us build super strong, super reliable software that can handle anything life throws our way! 💪🌟

Congratulations – Lesson complete!