Learn about Boundary-value-analysis (BVA) with AI

Boundary-Value-Analysis in Minecraft

Musical Intro

Introduction

Imagine you’re playing Minecraft, and you need to move as close to the edge as possible, to set the next block, without falling off the platform. Boundary-Value Analysis (BVA) in software testing is like that, where we find out where the edge of not-falling and falling is.

Example:

Scenario Outline: Withdraw boundary-adjacent values
    Given the user has an active bank account
    When the user tries to withdraw <amount> dollars
    Then the withdrawal should be <result>

    Examples:
      | amount | result     | comment
      | 499    | successful |  # Just below maximum boundary
      | 500    | successful |  # Maximum boundary
      | 501    | rejected   |  # Just above maximum boundary
πŸ₯’ Gherkin

Using AI

All this have been created with the cooperation with AI.

Often it can be more useful just to learn, what is needed in the moment, than to learn everything about a topic. This is where AI is perfect.

We can use the free ChatGPT 3.5 or 4o to learn about operators. You are welcome to try other and if you do, please write a comment!

Lets look at essential prompts!

Learn from examples (prompt):

I want to learn about "Boundary-value-analysis".
Can you give me an example written with BDD examples? 
πŸ“ AI prompt

Boundary Value Analysis (BVA) is a testing technique used to identify errors at the boundaries of input domains rather than within the ranges. The idea is to focus on the edges of the input ranges where errors are most likely to occur. This is particularly useful in situations where values are input as a range or limit.

To demonstrate Boundary Value Analysis using Behavior-Driven Development (BDD), let's take an example of a banking system that has the following constraints for account balance

Scenario Outline: Withdraw boundary-adjacent values
    Given the user has an active bank account
    When the user tries to withdraw <amount> dollars
    Then the withdrawal should be <result>

    Examples:
      | amount | result     |
      | 9      | rejected   |  # Just below minimum boundary
      | 10     | successful |  # Minimum boundary
      | 11     | successful |  # Just above minimum boundary
      | 499    | successful |  # Just below maximum boundary
      | 500    | successful |  # Maximum boundary
      | 501    | rejected   |  # Just above maximum boundary
πŸ€– AI response

It is understandable, why we need 9 (below minimum), 10 (minimum), 500 (maximum), and 501 (above maximum). It is not as intuitive to test for 11 (above minimum) and 499 (below maximum).

The ChatGPT 4o, 3.5, and even 4 can’t answer it properly. It just give a lot of explanation that it needs to be, but can’t make a proper why.

An experienced tester would explain it as following:

Example 1, working:
When a code is supposed to be x >= 10, and is coded that way.
Then the test results will be:
Test 1: When x is set to 9, then it must be invalid (passed βœ”οΈ)
Test 2: When x is set to 10, then it must be valid (passed βœ”οΈ)

Example 2, bug:
When a code is supposed to be x >= 10, but is coded as x>10
Then the test results will be:
Test 1: When x is set to 9, then it must be invalid (passed βœ”οΈ)
Test 2: When x is set to 10, then it must be valid (failed ❌)

Example 3, bug:
When a code is supposed to be x >= 10, but is coded as x==10
Then the test results will be:
Test 1: When x is set to 9, then it must be invalid (passed βœ”οΈ)
Test 2: When x is set to 10, then it must be valid (passed βœ”οΈ)
Test 3: When x is set to 11, then it must be invalid (failed ❌)
Which means we need the 3rd test, in order to catch this kind of bug

More examples? (prompt):

(I have picked the best out)

Can you give more examples?
πŸ“ AI prompt
Scenario Outline: Validate username length
    Given the system requires a username length between 5 and 15 characters inclusive
    When the user enters a username with <userName> characters
    Then the validation should be <result>

Examples:
  | userName           | result     | comment
  | "use4"             | rejected   |  # Just below minimum boundary
  | "user5"            | successful |  # Minimum boundary
  | "userN6"           | successful |  # Just above minimum boundary
  | "userNameoooo14"   | successful |  # Just below maximum boundary
  | "userNameooooo15"  | successful |  # Maximum boundary
  | "userNameoooooo16" | rejected   |  # Just above maximum boundary
  | null               | rejected   |  # outside of string length
πŸ€– AI response

Sometimes you might need to start a new session.

Sometimes the AI makes a mistake like writing Given, Then, When (wrong order) and sometimes starting a new session makes it go away.

What usages are there? (prompt):

ChatGPT 3.5, 4, and 4o can have issues, when we ask too much of it.
So, sometimes it is better to ask something more specific (in this case we avoid using the BDD examples, because it loses it’s creativity and most of the examples would be similar about values and not topics.

What can be tested with boundary-value-analysis?
πŸ“ AI prompt
Boundary Value Analysis (BVA) is a robust testing technique used to identify errors at the boundaries of input ranges, where most errors tend to occur. It is particularly effective in scenarios where inputs fall into defined ranges or thresholds. Here are some areas where BVA can be applied:

1. Input Fields Validation
Numeric Input Fields: Minimum and maximum allowed values, such as age, quantity, price, or score.
Text Input Fields: Minimum and maximum character lengths for fields like passwords, usernames, descriptions, etc.

2. String lengths...
3. Arrays...
4. File sizes...
5. Date and Time Validations...
6. Special characters...
7. Colors...
8. User Interface Elements...
9. Form Validations...
10. Repeats...
11. Multi-threading... 
πŸ€– AI response

As you can see, this answer gave many more different applications, that the one before.
I took the description from the first one and only the titles from the rest, because I wanted to transform some of them into BDD features.

Making a BDD scenario from a specific test case (prompt):

Can you make a BDD feature of a boundary-value-analysis of Disk Space Utilization: Validating file uploads of files? The max size is 4MB.
πŸ“ AI prompt
Scenario Outline: Uploading boundary-adjacent file sizes
    Given the user tries to upload a file of size <size> MB
    When the file is uploaded
    Then the upload should be <result>

    Examples:
      | size | result     | comment
      | 3.99 | successful | # Just below maximum valid size
      | 4.00 | successful | # Maximum valid size
      | 4.01 | rejected   | # Just above maximum valid size
πŸ€– AI response

It wrote more, but it is up to us to filter out, what is important and what isn’t.

Common mistakes (prompt):

What are the common mistakes are there with boundary-value-analysis?
πŸ“ AI prompt
Ignoring Edge Cases Outside the Boundaries:
Testing only the boundary values themselves but not the values immediately outside the boundaries.
Example Mistake: Testing only the values 1 and 30 but not testing 0 or 31 for an age input range of 1 to 30.

Not Testing Both Sides of a Boundary:
Focusing only on the lower or upper boundary, but not both.
Example Mistake: Testing the maximum value of 30 but forgetting to test the minimum value of 1.

Overlooking Off-by-One Errors:
Missing off-by-one errors which are common in boundary-related bugs.
Example Mistake: If the boundary condition is <= 30, failing to test with the value 31.

Misinterpreting Boundary Conditions:
Misunderstanding whether the boundary is inclusive or exclusive.
Example Mistake: Assuming a boundary condition is inclusive when it is actually exclusive, leading to incorrect test cases.

Assuming Boundaries Without Verification:
Assuming the boundary values without checking the actual requirements or specifications.
Example Mistake: Assuming an age range is 1 to 30 without confirming with the requirements document.

Not Testing the Exact Boundary Value:
Forgetting to test the exact boundary value itself.
Example Mistake: Testing 29 and 31, but forgetting to test the exact boundary value of 30.

Focusing Solely on Numerical Inputs:
Only applying BVA to numerical inputs and ignoring other types like strings or dates.
Example Mistake: Focusing only on numeric ranges and not testing boundaries for string lengths or date ranges.

Overlooking Interaction Between Multiple Boundaries:
Ignoring the interaction between multiple boundaries when they exist.
Example Mistake: If both age and income need to be tested together, only testing them separately and not considering combined boundary scenarios.

Inadequate Test Data Coverage:
Not providing enough test data to cover all the boundary scenarios adequately.
Example Mistake: Only testing the lower boundary and missing the upper boundary entirely.

Ignoring the Context of the Application:
Not considering the specific context or environment where the boundaries are applied.
Example Mistake: Testing file upload sizes without considering different file formats that might affect the boundary limits.
πŸ€– AI response

Training exercises

Can you give me some training excesses, so I can train boundary-valuse-analysis and avoid these mistakes? (please write the answers at the end only)
πŸ“ AI prompt
Exercise 1: Age Verification

Scenario:
An online registration system only accepts users aged between 18 and 60 inclusive.

Tasks:
1. Identify the boundary values for this input range.
2. Write test cases for:
   - The exact boundary values.
   - Values just below and above the boundaries.
3. Execute the test cases and note the expected outcomes.

Answers:
- Test cases:
  - Age 17 (just below minimum boundary) - Expected result: Rejected
  - Age 18 (minimum boundary) - Expected result: Accepted
  - Age 19 (just above minimum boundary) - Expected result: Accepted
  - Age 59 (just below maximum boundary) - Expected result: Accepted
  - Age 60 (maximum boundary) - Expected result: Accepted
  - Age 61 (just above maximum boundary) - Expected result: Rejected
πŸ€– AI response
Exercise 2: Password Length Validation

Scenario:
A system requires passwords to be between 8 and 20 characters inclusive.

Tasks:
1. Determine the boundary values for password length.
2. Create test cases for:
   - The minimum and maximum length.
   - Values just below and above the minimum and maximum length.
3. Execute the test cases and note the expected outcomes.

- Test cases:
  - Password length 7 (just below minimum) - Expected result: Rejected
  - Password length 8 (minimum) - Expected result: Accepted
  - Password length 9 (just above minimum) - Expected result: Accepted
  - Password length 19 (just below maximum) - Expected result: Accepted
  - Password length 20 (maximum) - Expected result: Accepted
  - Password length 21 (just above maximum) - Expected result: Rejected
πŸ€– AI response
Exercise 3: File Upload Size

Scenario:
A web application allows file uploads with sizes between 1MB and 4MB inclusive.

Tasks:
1. Identify the boundary values for the file size.
2. Write test cases for:
   - The exact boundary values.
   - Values just below and above the boundaries.
3. Execute the test cases and note the expected outcomes.

- Test cases:
  - File size 0.9MB (just below minimum) - Expected result: Rejected
  - File size 1MB (minimum) - Expected result: Accepted
  - File size 1.1MB (just above minimum) - Expected result: Accepted
  - File size 3.9MB (just below maximum) - Expected result: Accepted
  - File size 4MB (maximum) - Expected result: Accepted
  - File size 4.1MB (just above maximum) - Expected result: Rejected
πŸ€– AI response
Exercise 4: Loan Approval Based on Credit Score

Scenario:
A bank approves loans based on credit scores:
- Scores below 600 are rejected.
- Scores between 600 and 749 are conditionally approved.
- Scores 750 and above are fully approved.

Tasks:
1. Determine the boundary values for the credit scores.
2. Create test cases for:
   - Values just below and above each boundary.
   - The exact boundary values.
3. Execute the test cases and note the expected outcomes.

- Test cases:
  - Credit score 599 (just below minimum for conditional approval) - Expected result: Rejected
  - Credit score 600 (minimum for conditional approval) - Expected result: Conditionally Approved
  - Credit score 601 (just above minimum for conditional approval) - Expected result: Conditionally Approved
  - Credit score 749 (just below maximum for conditional approval) - Expected result: Conditionally Approved
  - Credit score 750 (minimum for full approval) - Expected result: Fully Approved
  - Credit score 751 (just above minimum for full approval) - Expected result: Fully Approved
πŸ€– AI response
Exercise 5: Temperature Control

Scenario:
A thermostat allows setting temperatures between 60Β°F and 90Β°F inclusive.

Tasks:
1. Identify the boundary values for the temperature setting.
2. Write test cases for:
   - The exact boundary values.
   - Values just below and above the boundaries.
3. Execute the test cases and note the expected outcomes.

- Test cases:
  - Temperature 59Β°F (just below minimum) - Expected result: Rejected
  - Temperature 60Β°F (minimum) - Expected result: Accepted
  - Temperature 61Β°F (just above minimum) - Expected result: Accepted
  - Temperature 89Β°F (just below maximum) - Expected result: Accepted
  - Temperature 90Β°F (maximum) - Expected result: Accepted
  - Temperature 91Β°F (just above maximum) - Expected result: Rejected
πŸ€– AI response
Exercise 6: Discount Application Based on Purchase Amount

Scenario:
A store offers discounts based on purchase amount:
- Purchases below $50 get no discount.
- Purchases between $50 and $100 get a 10% discount.
- Purchases above $100 get a 20% discount.

Tasks:
1. Determine the boundary values for the purchase amounts.
2. Create test cases for:
   - Values just below and above each boundary.
   - The exact boundary values.
3. Execute the test cases and note the expected outcomes.

- Test cases:
  - Purchase amount $49 (just below minimum for discount) - Expected result: No discount
  - Purchase amount $50 (minimum for 10% discount) - Expected result: 10% discount
  - Purchase amount $51 (just above minimum for 10% discount) - Expected result: 10% discount
  - Purchase amount $99 (just below maximum for 10% discount) - Expected result: 10% discount
  - Purchase amount $100 (maximum for 10% discount) - Expected result: 10% discount
  - Purchase amount $101 (just above maximum for 10% discount) - Expected result: 20% discount
πŸ€– AI response
Reflection Questions:

1. Did you miss any edge cases outside the boundaries? How can you ensure they are always included in your tests?
2. Were both sides of each boundary adequately tested? Why is it important to test both?
3. Did you encounter any off-by-one errors? How can they be prevented in future tests?
4. Were the boundary conditions interpreted correctly (inclusive vs. exclusive)? How can misinterpretations be avoided?
5. Did you verify the boundary values with the requirements or specifications before testing?
πŸ€– AI response

Advances analysis (prompt):

Can you make an boundary-value-analysis, for:
We have an app that must show products that have date-ranges.
We may only see products, which date-range include today.
How would we test this? 
πŸ“ AI prompt
Boundary-Value Test Cases:
1. Exact Match with Today:
Start Date: 2024-05-17, End Date: 2024-05-17
Expected result: Product is displayed.

2. Starts Before, Ends After:
Start Date: 2024-05-16, End Date: 2024-05-18
Expected result: Product is displayed.

3. Starts Today, Ends After:
Start Date: 2024-05-17, End Date: 2024-05-18
Expected result: Product is displayed.

4. Starts Before, Ends Today:
Start Date: 2024-05-16, End Date: 2024-05-17
Expected result: Product is displayed.

5. Starts Before, Ends Before:
Start Date: 2024-05-15, End Date: 2024-05-16
Expected result: Product is not displayed.

6. Starts After, Ends After:
Start Date: 2024-05-18, End Date: 2024-05-19
Expected result: Product is not displayed.

7. Starts Before, Ends Before:
Start Date: 2024-05-15, End Date: 2024-05-16
Expected result: Product is not displayed.
πŸ€– AI response
arent 5. and 6. the same?
πŸ“ AI prompt
You are right, cases 5 and 7 are the same in the initial list. Let's refine the scenarios to ensure all boundary cases are covered distinctly and effectively.
...

7. Product Date Range Completely Before Today:
Start Date: 2024-05-14
End Date: 2024-05-15
Expected result: Product is not displayed.

8. Product Date Range Completely After Today:
Start Date: 2024-05-19
End Date: 2024-05-20
Expected result: Product is not displayed.
πŸ€– AI response

But it didn’t get the refinement correct, because instead of removing number 7, it changed it and added number 8 (which are covered by 5 and 6)!

Can you rewrite it as a BDD example?
And instead of dates use, today, today +1 day, today -1 day, etc.?
πŸ“ AI prompt
Feature: Display Products Based on Date Ranges
  As a user of the app
  I want to see only the products that are available today
  So that I can make purchasing decisions based on current availability

  Background:
    Given today's date is "2024-05-17"

  Scenario Outline: Product date range includes today
    Given a product with a start date of "<start_date>" and an end date of "<end_date>"
    When I view the list of available products
    Then the product should be "<result>"

Examples:
start_date     | end_date       | result        | Comment
today          | today          | displayed     | # Exact match with today
today – 1 day  | today + 1 day  | displayed     | # Starts before, ends after
today          | today + 1 day  | displayed     | # Starts today, ends after
today – 1 day  | today          | displayed     | # Starts before, ends today
today – 2 days | today – 1 day  | not displayed | # Ends just before today
today + 1 day  | today + 2 days | not displayed | # Starts just after today
today – 3 days | today – 2 days | not displayed | # Completely before today
today + 2 days | today + 3 days | not displayed | # Completely after today
πŸ€– AI response

And here we can see easier the problem with 7. and 8., since it just added “Completely”, even though it gives no extra data.

Building examples with the AI

AI can do good things, but it is better to build them together, so I did do some examples:

Example 1: Date ranges

Date ranges are even more fun.

Scenario Outline: Product date range includes today
    Given a product with a start date of "<start_date>" and an end date of "<end_date>"
    When I view the list of available products
    Then the product should be "<result>"

Examples:
start_date    | end_date       | result        | Comment
today         | today          | displayed     | # Exact
today –1 day  | today + 1 day  | displayed     | # Before & After
today         | today + 1 day  | displayed     | # Exact & After
today –1 day  | today          | displayed     | # Before & Exact
today –2 days | today – 1 day  | not displayed | # Only after
today +1 day  | today + 2 days | not displayed | # Only before
πŸ§‘πŸ€– Human + AI

It is much easier to understand it as a graph:

Example 2: Time (regular, sommer time)

Time ranges are crucial in many applications, such as booking systems. Office meeting rooms, for instance, have specific hours during which they can be booked.

Scenario Outline: An IT-system can only process one event per hour.
  Given events are sat in CET/CEST time zone
  When an event is tried to be assigned at <event_datetime>
  Then it must be <result>
    
Examples:
event_datetime | result      | comment
2024-03-31 01:59:59 | succesful | # Before DST spring forward
2024-03-31 02:00:00 | rejected  | # Just during the missing hour
2024-03-31 02:00:01 | rejected  | # During the missing hour
2024-03-31 02:59:59 | rejected  | # During the missing hour
2024-03-31 03:00:00 | succesful | # Just after the DST change
2024-03-31 03:00:01 | succesful | # After the DST change
πŸ§‘πŸ€– Human + AI

Example 3: Special characters (formatting)

Character encoding can affect boundaries.

UTF-8 encoding is a variable-length character encoding for Unicode that uses one to four bytes to encode characters.

Symbols such as A-Z, a-z, 0-9 takes 1 byte per character.
Including a single special character such as Γ±, will upgrade the text to 2 bytes per character.
Including a single special character such as 汉, will upgrade the text to 3 bytes per character.
Including a single special character such as πŸ˜€, will upgrade the text to 4 bytes per character.

Scenario Outline: Validate input length and encoding boundary-adjacent values
    Given the system allows a maximum text size of 12 bytes
    When I enter a text <input_text> characters including <character_type>
    Then the input should be <result>
    
Examples:
input_text      | result     | comment
"a........11"   | successful | # Just below maximum byte size
"1.........12"  | successful | # maximum byte size
"1..........13" | rejected   | # Just above maximum byte size
"Γ±...5"         | successful | # Just below maximum byte size
"Γ±....6"        | successful | # maximum byte size
"Γ±.....7"       | rejected   | # Just above maximum byte size
"汉.3"          | successful | # Just below maximum byte size
"汉..4"         | successful | # maximum byte size
"汉...5"        | rejected   | # Just above maximum byte size
"πŸ˜€2"          | successful | # Just below maximum byte size
"πŸ˜€.3"         | successful | # maximum byte size
"πŸ˜€..4"        | rejected   | # Just above maximum byte size
πŸ§‘πŸ€– Human + AI

So, remember that when ever you write a :-) instead of πŸ˜€, you actually save 1 byte of data per character in the text you send! Think about the amount of CO2 you might save, by being old school ;-)

Example 4: Color profiles

In 2020, there was a specific wallpaper image that, when set as the background on certain Android phones (especially Samsung devices), caused the phone to crash and get stuck in a boot loop. This issue was related to how the Android system processed the color profile of the image.

The problematic image had a color profile that included colors outside the standard sRGB gamut. When the Android system attempted to convert this color profile to its default display profile (usually sRGB), it encountered colors that were out of range.
Read more

The system should handle out-of-range values by clipping them to the nearest valid value (e.g., setting any value above 255 to 255)

It might be a good idea to test converting images from one profile to another, while giving colors extreme values.

Scenario Outline: Validate handling of out-of-range color values during color profile conversion
    Given an image with a <color_profile> color profile
    When the image is converted to sRGB
    Then any color value above 255 should be rounded down to 255
    And the image processing should be <result>

    
Examples:
color_profile	| initial_value |	converted_value	| comment
Adobe RGB     |	259	          | 254             | # not rounded
Adobe RGB     |	260	          | 255             | # not rounded 
Adobe RGB     |	261	          | 255             | # rounded down 
ProPhoto RGB  |	299           | 254             | # not rounded
ProPhoto RGB  |	300	          | 255             | # not rounded 
ProPhoto RGB  |	301	          | 255             | # rounded down 
πŸ§‘πŸ€– Human + AI

Example 5: Repeats (maximum)

Login attempts can also be as a boundary-value-analysis, since there is a boundary between “allowed attempt” and “too many attempts”

Scenario Outline: Validate maximum password retry attempts
    Given a user is attempting to log in
    When the user enters an incorrect password <retry_count> times
    Then the system should <result>

    
Examples:
retry_count	| result	      | comment
2           |	retry_allowed	| just below max.
3           |	retry_allowed	| at max.
4           |	retry_denied	| just above max.
πŸ§‘πŸ€– Human + AI

Example 6: Multi-threading (minimum) for performance testing

Boundary-Value-Analysis can also be applied to a minimum simultaneous threads.

Scenario Outline: Validate minimum thread handling for concurrent requests
    Given the server can handle a minimum of 10 threads
    When <thread_count> threads are used to handle concurrent requests
    Then the server performance should be <result>

    
Examples:
threads	| result	            | #comment
9       |	good                | just below max.
10      |	good	              | at max.
11      |	slow but acceptable | just above max.
πŸ§‘πŸ€– Human + AI

So, BVA is even applied to with Performance testing!

Conclusion

From this lesson, we’ve learned several key aspects of using the Boundary-Value-Analysis (BVA) effectively through structured lessons and examples:

  1. What BVA is: Like playing Minecraft and move to the edge as close as possible without falling down
  2. Learn from examples: That we can learn from actual examples.
  3. Get more examples: That multiple examples can show us more usages.
  4. Sometimes you might need to start a new session: This point underlines the iterative nature of working with AI.
  5. What usages are there: That often it is better to ask AI about usages, and not form it into a specific format like BDD scenarios. Then we can get a broader scope like: Input Fields Validation, Range-Based Functionality, Resource Allocation, Financial Transactions, Date and Time Validations, Threshold-Based Alerts, Performance Testing, User Interface Elements, Form Validations, and Configurations and Settings.
  6. Making a BDD scenario from a specific test case: When we have the broad input, then it is much easier to transform it into a specific format. And that we might have to manually filter the answer, because it can present too much data.
  7. Common mistakes: Negative scenarios can also give a broader scope, so we also learn of not to do. With BVA it was: Ignoring Edge Cases Outside the Boundaries, Not Testing Both Sides of a Boundary, Overlooking Off-by-One Errors, Misinterpreting Boundary Conditions, Assuming Boundaries Without Verification, Not Testing the Exact Boundary Value, Focusing Solely on Numerical Inputs, Overlooking Interaction Between Multiple Boundaries, Inadequate Test Data Coverage, and Ignoring the Context of the Application.
  8. Training exercises: To actually create scenarios that have to be solved for Age Verification, Password Length Validation, File Upload Size, Loan Approval Based on Credit Score, Temperature Control, Discount Application Based on Purchase Amount, and Reflection Questions!
  9. Advances analysis: To let the AI create a boundary-value-analysis for an app, where we only may see products, that are applicable only for today. The AI created one too many scenarios, and even though it admitted it’s fault, it added another one instead of removing the wrong one. The AI could transform it into a BDD scenario, to make it easier for us to filter it ourselves.

AI is indeed a great tool to learn about Boundary-Value-Analysis.
It’s not perfect, but better than some of the free tutorials on the net.

Congratulations – Lesson complete!

Want more?

Check out the comments on Linked.

Learn about BDD (Behavior Driven Development) with AI

Given, When and Then in BDD

Musical Intro

Introduction

A way to view BDD is to see it in 3 parts:

  1. Describing systems, by describing their behavior.
  2. Describing behavior with given, when, and then steps.
  3. Automating the description with a glue framework.

This lesson will focus on the first two steps. If you want the third, then I can recommend my coding course.

In BDD we use words such as “Given”, “When”, and “Then” to describe a process/behavior.

It’s like building with Lego bricks:

Given you have these blocks (don't start without them).

When specific set of blocks are combined in a specific way.
Then a specific structure must be achieved.

When more specific blocks are added in a specific way.
Then another specific step in the structure must be achieved.

...

When the final blocks are added in a specific way.
Then the final structure must be achieved.
πŸ₯’ Gherkin

This approach had worked very well for Lego and Ikea to describe the processes of assembling things (an excellent manual).

BDD can also be used to describe your product and used as an manual.
This is called live-documentation, because the documentation is always up-to-date, otherwise the BDD test cases won’t pass.


Using AI

All this have been created with the cooperation with AI.

Often it can be more useful just to learn, what is needed in the moment, than to learn everything about a topic. This is where AI is perfect.

We can use the free ChatGPT 3.5 to learn about operators. You are welcome to try other and if you do, please write a comment!

Lets look at essential prompts!

Learn from examples (prompt):

I want to learn about writing a test case as a BDD scenario (given, when and then).
Can you show me an example on how to write such a scenario on "the process of making a sandwich?'

Given must be the preconditions, before it can start.
When must be the needed actions.
Then must be the "must" verifications.

Don't use "I", but a proper user.
πŸ“ AI prompt
Absolutely, here's a BDD scenario for making a sandwich without using "I":

Boundary Value Analysis (BVA) is a testing technique used to identify errors at the boundaries of input domains rather than within the ranges. The idea is to focus on the edges of the input ranges where errors are most likely to occur. This is particularly useful in situations where values are input as a range or limit.

To demonstrate Boundary Value Analysis using Behavior-Driven Development (BDD), let's take an example of a banking system that has the following constraints for account balance

Scenario: Making a Sandwich
  Given a user has slices of bread available
    And there is a selection of fillings
  When the user selects two slices of bread
    And chooses a filling from the selection
    And places the filling between the slices of bread
  Then the user should have a sandwich
    And the sandwich should contain the chosen filling
    And the sandwich should be ready to eat

In this scenario:
- Given: Describes the initial conditions or preconditions required for the scenario to start. In this case, the availability of bread slices and a selection of fillings.
- When: Describes the actions or events that occur. Here, it's a user selecting two slices of bread, choosing a filling, and assembling the sandwich.
- Then: Describes the expected outcomes or verifications that should be true after the actions have been performed. In this case, verifying that a sandwich exists, it contains the chosen filling, and it's ready to eat.
πŸ€– AI response

Get more examples, by varying the subject:

I want to learn about writing a test case as a BDD scenario (given, when and then).
Can you show me an example on how to write such a scenario on "the process of changing a bike wheel"?

Given must be the preconditions, before it can start.
When must be the needed actions.
Then must be the "must" verifications.

Don't use "I", but a proper user.
πŸ“ AI prompt
Certainly, here's a BDD scenario for changing a bike wheel:

Scenario: Changing a Bike Wheel
  Given a cyclist has a spare tire and necessary tools
    And the bike has a flat tire
  When the cyclist removes the wheel with the flat tire
    And replaces it with the spare tire
    And properly secures the new wheel in place
  Then the cyclist should have a bike with a functional wheel
    And the bike should be ready to ride
    And the flat tire should be safely stored for repair or disposal


In this scenario:
- Given: Describes the initial conditions or preconditions required for the scenario to start. In this case, the cyclist having a spare tire and necessary tools, and the bike having a flat tire.
- When: Describes the actions or events that occur. Here, it's the cyclist removing the flat tire, replacing it with a spare tire, and properly securing the new wheel.
- Then: Describes the expected outcomes or verifications that should be true after the actions have been performed. In this case, verifying that the bike has a functional wheel, is ready to ride, and that the flat tire is appropriately handled.
πŸ€– AI response

Sometimes you might need to start a new session.

Sometimes the AI makes a mistake like writing Given, Then, When (wrong order) and sometimes starting a new session makes it go away.

Improving a scenario (prompt):

I have written the following scenario, can analyze it on how it can be improved?
--
Scenario: Changing a Bike Wheel
  Given a cyclist has a spare tire and necessary tools
    And the bike has a flat tire
  When the cyclist removes the wheel with the flat tire
    And replaces it with the spare tire
    And properly secures the new wheel in place
  Then the cyclist should have a bike with a functional wheel
    And the bike should be ready to ride
    And the flat tire should be safely stored for repair or disposal
--
πŸ“ AI prompt
Your scenario is well-structured and covers the main steps involved in changing a bike wheel. However, here are some suggestions for improvement:

1. Clarify Preconditions: While you mention that the cyclist has a spare tire and necessary tools, it might be helpful to specify what tools are needed. For example, a wrench or tire lever. Additionally, you could mention the condition of the spare tire - is it in good condition and properly inflated?

2. Add Clarity to Actions: The steps under the "When" section are clear, but you could add more detail to make them more explicit. For example, specify how the cyclist removes the wheel (e.g., using a wrench), and how they secure the new wheel in place (e.g., tightening the bolts to the recommended torque).

3. Check for Completion: The "Then" section mentions that the cyclist should have a bike with a functional wheel and that the bike should be ready to ride. Consider adding verification steps to ensure these conditions are met. For example, checking that the tire pressure is correct and that the wheel spins freely without rubbing against the frame or brakes.

4. Address Safety Concerns: While you mention that the flat tire should be safely stored for repair or disposal, you could provide more guidance on how to do this. For example, emphasizing the importance of properly disposing of old tires to minimize environmental impact.

5. Consider Error Handling: It might be beneficial to include steps for handling potential errors or complications that could arise during the wheel-changing process. For example, what should the cyclist do if they encounter difficulties removing the wheel or if the spare tire is also damaged?

By addressing these points, you can provide a more comprehensive and detailed scenario for changing a bike wheel, ensuring that users have clear guidance on each step of the process and any associated requirements or considerations.
πŸ€– AI response

Implement improvements and ask again or let the AI help you (prompt):

(not all it suggests are good improvements, so pick what you agree with)

Can you implement your improvements?
Certainly! Here's the improved version of the scenario:

Scenario: Changing a Bike Wheel
  Given a cyclist has a spare tire, a wrench, and tire levers
    And the spare tire is in good condition and properly inflated
    And the bike has a flat tire
  When the cyclist uses the wrench to loosen the bolts securing the wheel with the flat tire
    And uses tire levers to pry the tire off the rim
    And removes the wheel from the bike
    And places the flat tire in a secure location for repair or disposal
    And positions the spare tire onto the wheel rim
    And inflates the spare tire to the recommended pressure
    And tightens the bolts to secure the new wheel in place
  Then the cyclist should have a bike with a functional wheel
    And the tire pressure of the new wheel should be checked and adjusted if necessary
    And the bike should be ready to ride
    And the flat tire should be safely stored for repair or disposal
πŸ€– AI response

Split a scenario into multiple more detailed ones (prompt):

Can we split the following scenario into more BDD scenarios (given when then)?
--
Scenario: Changing a Bike Wheel
  Given a cyclist has a spare tire, a wrench, and tire levers
    And the spare tire is in good condition and properly inflated
    And the bike has a flat tire
  When the cyclist uses the wrench to loosen the bolts securing the wheel with the flat tire
    And uses tire levers to pry the tire off the rim
    And removes the wheel from the bike
    And places the flat tire in a secure location for repair or disposal
    And positions the spare tire onto the wheel rim
    And inflates the spare tire to the recommended pressure
    And tightens the bolts to secure the new wheel in place
  Then the cyclist should have a bike with a functional wheel
    And the tire pressure of the new wheel should be checked and adjusted if necessary
    And the bike should be ready to ride
    And the flat tire should be safely stored for repair or disposal
--
πŸ“ AI prompt
Sure, we can break down the scenario into more specific BDD scenarios. Here are some suggestions:

1. Preparing for Wheel Change:

Given a cyclist has a spare tire, a wrench, and tire levers
And the spare tire is in good condition and properly inflated
And the bike has a flat tire
When the cyclist gathers all necessary tools and materials
Then the cyclist is ready to change the bike wheel


2. Removing the Flat Tire:

Given the cyclist has gathered all necessary tools and materials
When the cyclist uses the wrench to loosen the bolts securing the wheel with the flat tire
And uses tire levers to pry the tire off the rim
And removes the wheel from the bike
Then the flat tire is detached from the bike


3. Preparing the Spare Tire:

Given the cyclist has gathered all necessary tools and materials
And the flat tire has been removed
When the cyclist prepares the spare tire for installation
Then the spare tire is ready to be mounted onto the wheel rim
Mounting the Spare Tire:

Given the cyclist has prepared the spare tire for installation
And the flat tire has been removed
When the cyclist positions the spare tire onto the wheel rim
And inflates the spare tire to the recommended pressure
Then the spare tire is properly mounted and inflated
Securing the New Wheel:

Given the spare tire has been properly mounted and inflated
When the cyclist tightens the bolts to secure the new wheel in place
Then the new wheel is securely attached to the bike frame
Final Checks and Cleanup:

Given the new wheel is securely attached to the bike frame
When the cyclist checks the tire pressure of the new wheel
And adjusts the tire pressure if necessary
And ensures the bike is ready to ride
Then the bike is ready for use with the functional wheel
And the flat tire is safely stored for repair or disposal
Breaking down the scenario into smaller, more focused steps like these can help clarify the behavior expected at each stage of the process.
πŸ€– AI response

Adding a examples, to make it data driven (prompt):

I have written the following scenario, can you add "Examples" , to make it data driven?
Please use these example variables in the steps.
--
Scenario: Changing a Bike Wheel
  Given a cyclist has a spare tire and necessary tools
    And the bike has a flat tire
  When the cyclist removes the wheel with the flat tire
    And replaces it with the spare tire
    And properly secures the new wheel in place
  Then the cyclist should have a bike with a functional wheel
    And the bike should be ready to ride
    And the flat tire should be safely stored for repair or disposal
--
πŸ“ AI prompt
Certainly! Here's how you can add an "Examples" section to make the scenario data-driven using the given variables:

Scenario: Changing a Bike Wheel
  Given a cyclist has the necessary <tools>
    And the bike has a <flat_tire>
  When the cyclist removes the wheel with the flat tire
    And replaces it with the spare tire
    And properly secures the new wheel in place
  Then the cyclist should have a bike with a functional wheel
    And the bike should be ready to ride
    And the flat tire should be safely stored for <disposal_or_repair>
    
  Examples
    tools	         | flat_tire | disposal_or_repair
    wrench, levers | punctured | repair
    pump, levers   | worn      | disposal
    wrench, pump   | deflated  | repair
πŸ€– AI response

The AI is not yet good enough to make advanced scenarios (prompt):

When humans learn about scenarios (faster with AI)
And improve their written scenarios on the internet
Then AI will learn from them
and get better at teaching us humans!

Scenario: Changing a Bike Wheel
  Given a cyclist has a flat tire and <tools>
    And the bike has a puncture
  When the cyclist removes the wheel with the flat tire
    And <action_with_spare_tire> the punctured tire
  Then the bike should be <ready_state> to ride
  
  Examples
    tools	         | action_with_spare_tire                    | ready_state
    wrench, levers | uses the spare tire to replace            | ready
    wrench         | missing the levers, and can't replace     | not ready
    levers         | missing the wrench, and can't replace     | not ready
    wrench, levers | missing the spare tire, and can't replace | not ready
πŸ§‘πŸ€– Human + AI

Conclusion

From this lesson, we’ve learned several key aspects of using the Behavior-Driven Development (BDD) methodology effectively through structured lessons and examples:

  1. What is BDD: A way to document a process/behavior of a system with “Given,” “When,” and “Then”, and it has worked well for Lego and Ikea. This is called live-documentation, because it is always up-to-date, as long as the BDD test cases pass.
  2. Learn from examples: We explored how BDD scenarios are structured into “Given,” “When,” and “Then” to articulate test cases clearly. This format helps define the prerequisites, actions, and expected outcomes clearly.
  3. Get more examples, by varying the subject: We saw how applying BDD to different contexts, such as making a sandwich or changing a bike wheel, helps in understanding the versatile application of BDD in various real-world scenarios.
  4. Sometimes you might need to start a new session: This point underlines the iterative nature of developing BDD scenarios, where revising and refining through multiple iterations can lead to clearer and more effective test cases.
  5. Implement improvements and ask again or let the AI help you: We discussed refining BDD scenarios by incorporating feedback to enhance clarity and detail, demonstrating the iterative improvement and the role of tools like AI in assisting these refinements.
  6. Split a scenario into multiple more detailed ones: Breaking down complex scenarios into simpler, focused parts was shown as an effective strategy to manage detailed testing processes, making each step more manageable and comprehensible.
  7. Adding examples, to make it data driven: We learned how to make BDD scenarios data-driven by adding “Examples” tables, which specify multiple sets of conditions and outcomes, enabling comprehensive testing across various scenarios.
  8. The AI is not yet good enough to make advanced scenarios: This reflects on the current limitations of AI in generating complex BDD scenarios and emphasizes the need for human oversight in creating sophisticated test cases and interpreting nuanced scenarios.

Together, these lessons underscore the importance of a systematic approach in BDD to ensure thorough understanding and effective testing of user behaviors and system functionalities.

Limitless learning with a 24/7 tutor!!!

Congratulations – Lesson complete!

Want more?

Check out the comments on Linked.