On properties, part 1

I was sick the past few days, hence the absence of post

Let’s talk about testing. More specifically, let’s talk about property based testing, as I understand it after one and a half days of playing around with it as I follow the Haskell book. But first, a definition in my own words:

Given a rule, property based testing will test it with random values of an appropriate type and see if it holds true for all them. A rule in this case, is a statement that can be evaluated to a boolean value of true or false.

A rule (or property) can be absolutely anything you can think of that can be proven in some way. And the inputs will be random inside of the defined group of possible ones. So, for example, if we have a function that will take any string in the world and turn it into a valid url, with https protocol, it could have some properties like:

const propertyStartsWithHttps = str => {
	return createUrl(str).indexOf(“https://“) !== -1
}

const propertyContainsAtLeastOneDot = str => {
	return createUrl(str).indexOf(“.”) !== -1
}
/*
.
.
. and so on
*/

And, when we run our tests on these functions, our framework will call the test functions hundreds of times with random strings and see if the properties hold true for every case.

Value

Part of the value of TDD with Unit testing is to incrementally build features, or drive them using small tests and going in small steps. We can't really do that with Property Based Testing. What we can do, is a heighten the confidence that our code fulfils a set of rules that we define for it. Of course, in order to use effectively, we first need to define those rules, and that in itself is an exercise in thinking about our code, much like the TDD cycle says we should do. There are 3 things we need to think about when we use PBT:

• What should always be true about our code? ➡ Keep in mind, it will fail if it's not true for any one input.
• What inputs can our code receive? ➡ Most of the time, we won't need to test functions against all the possible Integer values out there.
• Is it really necessary? ➡ Perhaps the most important part. It makes us think about wether it will really add any value to our codebase.

But, is there a way to know for sure that we need it?

Rule of no thumb

The truth is, I don’t really know. Yet. What I do know, is that PBT and Unit Testing are not mutually exclusive. Unit tests tell us what our code should do and Property Based Tests tell us what the results of our code look like. They’re different things, and they can co-habitate the same codebase. The way I see it right now, is that we can always start with Unit tests and once our code is implemented we can define some properties to test if necessary.

But then again, my experience with PBT is really limited right now. Stay tuned for more though, as it’s a very interesting topic.