microCPH – The Microservice Conference in Copenhagen

Copenhagen

2018-05-15 08.18.08

I had the unique opportunity to attend µCPH 2018 in the wonderful city of Copenhagen this year.
The relatively small community conference was a great mix of high-level and hands-on talks.
Hosting a single track of talks did not jeopardised me to pick from a variety of presentation titles. Instead it felt like the order of talks was perfectly aligned to guide you from the eagle’s perspective (keynote on coupling by Michael Nygard) to Microservices and patterns of Domain Driven Design, down to the hands-on experiences of an actual move to a dockerized system of Financial Times, closing with some great remarks of a framework for breaking up the UI monolith (mosaic from Zalando).

One of my favorite presentations: _Microservices: Patterns and Antipatterns_ by Stefan Tilkov (slide deck, video). Many talks take the motivation for microservices either for granted or put scalablity, fault-tolerance and moving agile through independent release cycles upfront. In this talk however, Tilkov focuses on actual problems and how they have been solved in the past without the label micro service and how you shoot yourself in the foot by blindly applying conference-motivated architecture.

Another great talk by Dmitriy Kubyshkin explained how Zalando deals with a micro-service infrastructure and their front-end (slides, video). Many times micro-services talks are about back-end systems, and left the front-end out of discussion. At speedledger we also have the problem, that we somehow have to combine UI components with their back-end counterparts that are served through micro-services. Mosaic – the framework that Zalando developed has some great inspirations on that topic.

Ben Stopford with his talk on Kafka Streams (video, slides) gave some great ideas how you can turn services inside out and use advanced Kafka features to distribute the state of your system to avoid letting services turning into a database with a REST interface.

µCPH was a very well organised conference with a great schedule that gave plenty of time between the sessions to exchange with other attendants and to network.

I highly recommend to attend µCPH, if you have the chance to do so 🙂
– Tobi

Outside-in — To the max! (BDD and Microservices part 2)

This is part 2 in a series about my 20% project BDD across team bordersPart 1 can be found here.  

When writing any piece of software, it’s very easy to start working on the details before you have enough knowledge about the targeted context. I think most developers recognize this pattern:
You start with some innocent (often unintentional) guessing about one or a few unknowns. Pretty soon the guesses turn into assumptions and before you know it they have morphed into ‘truths’. You happily code away with these truths as your guiding light. Later, when you try to integrate your code into a bigger context, the assumptions and truths fall apart and you need to rethink your work.

– Oh, is that how the incoming data looks like!
– Why didn’t anybody tell me I have to ask the database for that value?


Personally, I have made these kinds of mistakes countless of times, and I will probably make many similar ones in the future. I think this is due to human nature; We really like making assumptions for some reason. To minimize the impact of this phenomenon, we have to try really hard to be aware of it at all times. We also have to find strategies and techniques that help us avoid making these false assumptions.

spiralTest Driven Development (TDD) is one excellent tool for this purpose. It forces us to step outside of the current context and look at the problem from the outside, or from “one level above” if you wish. More importantly, the TDD approach fits on all levels of software development! No matter if your context is a oneliner method in a class or the public API of a complete service, you always benefit from taking one symbolic step out of your current context.

In my 20% project BDD across team borders, I wanted to take this line of thinking to the extreme, and work on the highest level possible, thus making sure I was taking the “outside-in” approach to the max, maximizing our helicopter view in the process.
I set as a constraint on my work going forward that I was only allowed to write the BDD tests against our publicly available APIs and web pages. No matter how tempting, I would not “cheat” by calling internal APIs or access anything not available to a user in production. If I did, I would risk getting my assumptions wrong, as described above.

Working against public APIs also provides another great bonus;  It forces you to write user-centric stories that are sliced vertically. A good rule is that “Every story must have a noticable effect on the public API level”. If you write a story that has no noticable effect for any user, you have not succeeded in writing a vertical story.

My claim to test on the “highest possible level” is not entirely true, of course. There will always be third party dependencies and other integration points that we cannot incorporate in the system under test. We will have to fake or mock things like partner APIs, for instance. As long as we don’t mock or fake our own code or apps, we should be good to go!

I want my helicopter view back! (BDD and Microservices, part 1)

502377266

Here at SpeedLedger Engineering, we took the plunge into the world of microservices some time ago. We have gradually phased out our old monolithic style applications in favor of many small and independent micro services/apps.

Overall, the micro services paradigm aligns very well with our organization (small independent teams) and we are getting pretty good at it, I must say! We are constantly improving our infrastructure and deployment tools, enabling us to develop and deploy quicker and more often. As a result, the general time-to-market has decreased and our customers are getting more new features, sooner.

However, one big drawback of working in isolated islands of people and code is that you risk losing the big picture, the “helicopter view”. Unsurprisingly, this is what happened to me and my teammates. We often found ourselves asking questions like:

  • What’s supposed to happen in this business flow?
  • What are the possible inputs to this API?
  • Which apps will call this endpoint?

The answers to these and similar questions are not impossible to find but it always takes valuable time to track them down and to get them clarified.  

I felt something had to be done about this situation. For a long time, I have been a fan of BDD (Behavior Driven Development) but I never really got my feet wet in a real world situation. I suspected our current situation could benefit a lot from a successful BDD effort, and that it provided a perfect opportunity for me to experiment with and learn more about BDD.

For someone who doesn’t know what BDD  is, it can be described as “executable documentation” or “executable examples” that a non-programmer can understand and edit. More about this later…

After some initial discussion, we decided that I would do this in the context of a “personal 20 percent project”, famously invented and used by Google.  The main reason being that part of the work had to be done outside the team boundary, touching all the other teams in some way. In line with this, we actually named the project “BDD across team borders”. 

This was in February, and since then I have spent most of my Fridays exploring the possibilities of introducing BDD at SpeedLedger, and – more generally – automated testing on the highest possible level in a microservices environment.  In a series of coming posts, I will blog about BDD and the progress of the project. Hopefully someone will find it interesting. Also, I would love to get some feedback from you blog readers along the way!