-

Automated Exploratory Test Fixtures

This is part 3 in my series about  BDD and MicroServices. You may want to read part 1, and part 2 first.  

As I have blogged about earlier, I have been spending some time exploring BDD/executable examples in a MicroServices environment, and how we can benefit from executable examples (SBE, Specification By Example) here at SpeedLedger. It has been a really good learning experience, and fun too.
Thanks to the initial progress, I have managed to persuade my boss to let me work on this full time! We are doing it as an time-limited experiment, but hopefully this will be extended.

In this blog post, I would like to share one of our greatest takeaway so far: That End-to-End Exploratory Testing is more valuable than End-to-End Regression Testing.
Both categories are valuable of course, but if I had to choose to have a really good setup for exploratory tests OR only a great suite of regression tests, I would definitely choose the former. Having automated fixtures (setup procedures) for interesting “moments” in the lives of our customers when using our product is super valuable, and I will list the reasons below.

 

When all this started, we named the experiment “BDD across team borders”. This was a good name at the time, but we later realised that the benefits of BDD/SBE can be appreciated by all co-workers, even those outside development teams. We have identified three main areas where we – as a company – can benefit from automated environments and automated test data:

Learn

If you are teaching yourself a new app/system/service, wouldn’t it be great if you could experiment and explore all aspects of it without being afraid to destroy anything? And in a simple way set up situations that the typical customers face every day?
Similarly, if you are training a new hire about your system; Wouldn’t the training be more efficient and fun if you could go through these scenarios interactively, setting them up with the touch of a button?

Discuss

When discussing existing or new features with you peers, what would it be like if you could throw up a the system on a wide-screen monitor, demoing the current feature in question and the proposed solution respectively. While being able to repeat all setup step automatically and indefinitely?
Compare this to discussing features with the help of only stale sketches or obscure user stories.

Verify

This is the obvious one, of course. Most people think that verification of current features is the only benefit of automated test. But as we have seen above, there are several nice bonus effects.
In addition, there are other – maybe more subtle – positive side effects that come from using the test-first mindset. But that is a different story…

How about you? Can you automatically setup the preconditions for your customers’ most interesting interactions with your product?

 

 

 

Comments 0
-

Outside-in — To the max! (BDD and Microservices part 2)

This is part 2 in a series about my 20% project BDD across team borders. Part 1 can be found here.  

When writing any piece of software, it’s very easy to start working on the details before you have enough knowledge about the targeted context. I think most developers recognize this pattern:
You start with some innocent (often unintentional) guessing about one or a few unknowns. Pretty soon the guesses turn into assumptions and before you know it they have morphed into ‘truths’. You happily code away with these truths as your guiding light. Later, when you try to integrate your code into a bigger context, the assumptions and truths fall apart and you need to rethink your work.

– Oh, is that how the incoming data looks like!
– Why didn’t anybody tell me I have to ask the database for that value?

Personally, I have made these kinds of mistakes countless of times, and I will probably make many similar ones in the future. I think this is due to human nature; We really like making assumptions for some reason. To minimize the impact of this phenomenon, we have to try really hard to be aware of it at all times. We also have to find strategies and techniques that help us avoid making these false assumptions.

spiralTest Driven Development (TDD) is one excellent tool for this purpose. It forces us to step outside of the current context and look at the problem from the outside, or from “one level above” if you wish. More importantly, the TDD approach fits on all levels of software development! No matter if your context is a oneliner method in a class or the public API of a complete service, you always benefit from taking one symbolic step out of your current context.

In my 20% project BDD across team borders, I wanted to take this line of thinking to the extreme, and work on the highest level possible, thus making sure I was taking the “outside-in” approach to the max, maximizing our helicopter view in the process.
I set as a constraint on my work going forward that I was only allowed to write the BDD tests against our publicly available APIs and web pages. No matter how tempting, I would not “cheat” by calling internal APIs or access anything not available to a user in production. If I did, I would risk getting my assumptions wrong, as described above.

Working against public APIs also provides another great bonus;  It forces you to write user-centric stories that are sliced vertically. A good rule is that “Every story must have a noticable effect on the public API level”. If you write a story that has no noticable effect for any user, you have not succeeded in writing a vertical story.

My claim to test on the “highest possible level” is not entirely true, of course. There will always be third party dependencies and other integration points that we cannot incorporate in the system under test. We will have to fake or mock things like partner APIs, for instance. As long as we don’t mock or fake our own code or apps, we should be good to go!

Comments 0
-

I want my helicopter view back! (BDD and Microservices, part 1)

502377266

Here at SpeedLedger Engineering, we took the plunge into the world of microservices some time ago. We have gradually phased out our old monolithic style applications in favor of many small and independent micro services/apps.

Overall, the micro services paradigm aligns very well with our organization (small independent teams) and we are getting pretty good at it, I must say! We are constantly improving our infrastructure and deployment tools, enabling us to develop and deploy quicker and more often. As a result, the general time-to-market has decreased and our customers are getting more new features, sooner.

However, one big drawback of working in isolated islands of people and code is that you risk losing the big picture, the “helicopter view”. Unsurprisingly, this is what happened to me and my teammates. We often found ourselves asking questions like:

  • What’s supposed to happen in this business flow?
  • What are the possible inputs to this API?
  • Which apps will call this endpoint?

The answers to these and similar questions are not impossible to find but it always takes valuable time to track them down and to get them clarified.  

I felt something had to be done about this situation. For a long time, I have been a fan of BDD (Behavior Driven Development) but I never really got my feet wet in a real world situation. I suspected our current situation could benefit a lot from a successful BDD effort, and that it provided a perfect opportunity for me to experiment with and learn more about BDD.

For someone who doesn’t know what BDD  is, it can be described as “executable documentation” or “executable examples” that a non-programmer can understand and edit. More about this later…

After some initial discussion, we decided that I would do this in the context of a “personal 20 percent project”, famously invented and used by Google.  The main reason being that part of the work had to be done outside the team boundary, touching all the other teams in some way. In line with this, we actually named the project “BDD across team borders”. 

This was in February, and since then I have spent most of my Fridays exploring the possibilities of introducing BDD at SpeedLedger, and – more generally – automated testing on the highest possible level in a microservices environment.  In a series of coming posts, I will blog about BDD and the progress of the project. Hopefully someone will find it interesting. Also, I would love to get some feedback from you blog readers along the way!

 

Comments 0
-

Counting Stories

A few weeks ago, we started measuring our development performance in a new way. We simply count the number of completed stories (stories with business value only) and track this number over time. We are firm believers in the psychological effect called “You get what you measure”, i.e. when you start to measure some metric and make it very visible, it will converge to the desired value over time. To achieve this effect and to help us focus on the story pace, we have installed monitors in our team room that display our current pace at all times.

One of our team monitors showing the number of stories closed so far the current week (week 38). The number 5 is the current weekly goal.

One of our team monitors showing the number of stories closed so far the current week (week 38). The number 5 is the current weekly goal.

Just counting the number of stories without using story points or any kind of estimation may seem like a crude strategy, but we think it will have some really nice benefits and we are excited and anxious to evaluate the experiment over time.

A sceptic may react to this strategy by saying: That’s silly, you just have to make less work in each story and you will get a higher score!

This is very true. And also very advantageous! It creates a win-win situation in which we get the story pace metric for tracking purposes, and a mechanism/incentive to decrease the size of our stories. Smaller stories provide a better flow, more frequent feedback, and less waste. (To name just a few benefits)

So, how long can we rely on this mechanism? Well, if we are successful at decreasing the story size continually, the number of stories per unit of time will increase. Eventually, the overhead of switching between stories will become too costly (percentage-wise). To achieve an even higher story pace at this point, we will have to focus on our tools and processes to reduce the overhead. As a result, we go from win-win to win-win-win: By introducing the story pace metric, we get the metric itself, an incentive to create smaller stories, and finally a mechanism that incentivises us to streamline our process and our tools.

This advantageous spiral may – in theory – go on forever. However, some kind of practical equilibrium will probably settle after a few cycles. At this point we will have become experts at slicing our stories into super thin slices and and also at streamlining/automating our process. We will have a nice high throughput of quality work, but we will also have something that may be even more desirable than a high pace: A predictable pace. Our product owners are going to love us! (Even more…)

All of the above is of course only our theory/hypothesis. We are looking forward to the coming months to see if we can pull it off in practice!

Comments 4