DevSum 2018 and the next steps on my journey

I really like conference driven development, where you choose an interesting subject, submit it to a conference and you are forced (in a good way) to get a much deeper understanding of the topic to make it into a interesting session. I presented ‘Scala for Java Developers’ at DevSum 2018 which is based on my journey into Scala from C# that started when I joined SpeedLedger about one and a half years ago. Now when I’ve gotten more Scala experience I’m thinking about new interesting topics that I can share. Before, I’ve always had purely technical sessions but I’m thinking of changing that. We are starting a new initiative where we are going to work more like a startup. This is to create a much faster delivery pace of many small MVP’s that we will try out with our customers and then keep developing the ideas that show potential. This will affect both technology choices and process. If you went to a conference would you have visited a session about this?

I’m already trying to get into the right mindset by gaging interest in idea before investing more time in making an abstract and preparing a conference talk. I believe we’re about to engage in a very interesting and rewarding journey over the coming years.

microCPH – The Microservice Conference in Copenhagen

Copenhagen

2018-05-15 08.18.08

I had the unique opportunity to attend µCPH 2018 in the wonderful city of Copenhagen this year.
The relatively small community conference was a great mix of high-level and hands-on talks.
Hosting a single track of talks did not jeopardised me to pick from a variety of presentation titles. Instead it felt like the order of talks was perfectly aligned to guide you from the eagle’s perspective (keynote on coupling by Michael Nygard) to Microservices and patterns of Domain Driven Design, down to the hands-on experiences of an actual move to a dockerized system of Financial Times, closing with some great remarks of a framework for breaking up the UI monolith (mosaic from Zalando).

One of my favorite presentations: _Microservices: Patterns and Antipatterns_ by Stefan Tilkov (slide deck, video). Many talks take the motivation for microservices either for granted or put scalablity, fault-tolerance and moving agile through independent release cycles upfront. In this talk however, Tilkov focuses on actual problems and how they have been solved in the past without the label micro service and how you shoot yourself in the foot by blindly applying conference-motivated architecture.

Another great talk by Dmitriy Kubyshkin explained how Zalando deals with a micro-service infrastructure and their front-end (slides, video). Many times micro-services talks are about back-end systems, and left the front-end out of discussion. At speedledger we also have the problem, that we somehow have to combine UI components with their back-end counterparts that are served through micro-services. Mosaic – the framework that Zalando developed has some great inspirations on that topic.

Ben Stopford with his talk on Kafka Streams (video, slides) gave some great ideas how you can turn services inside out and use advanced Kafka features to distribute the state of your system to avoid letting services turning into a database with a REST interface.

µCPH was a very well organised conference with a great schedule that gave plenty of time between the sessions to exchange with other attendants and to network.

I highly recommend to attend µCPH, if you have the chance to do so 🙂
– Tobi

Speakers dinner

flatMap Oslo 2018

The first day most of the sessions were on a very high level and there was a lot of talk about Free Monads, Tagless final and category theory. Upcoming news about Scalaz 8 and cats and what they have learnt along the way there. It was obvious that you should already be familiar with monads, monoids, applicatives and the monadic laws to get the most out of the advanced sessions.

The second day the sessions were more focused on how you can apply beautiful functional concepts to improve your code with state monads and cats validation.

Good discussions on what can be learnt and applied from other languages. It feels like the Scala community has matured and started experimenting with more advanced things now that there are more people familiar with the language.

I felt that my workshop on Scala for Java developers was well received. The attendees had a little or no experience of Scala since before but all of them saw opportunities to use it at work. I get the feeling that Scala is becoming more popular and accepted even in enterprise environments where Java has been the only choice before.

It’s great to see that my feeling that functional programming languages is getting more popular is also reflected in surveys. The industry is slowly going towards less null reference exceptions and code that is easier to parallelise.

Speaking at flatMap Oslo again this year

Putting the finishing touches on my “Scala for Java developers” workshop for the FlatMap Oslo conference. It will be my second year speaking there. Last year the conference was fantastic. Good speakers, well organised but I was still a little bit nervous when heading off for the conference. I’ve been speaking at developer conferences for close to 10 years but it’s been .Net focused conferences where I have known many of the attending speakers. Going to a conference around functional programming and not knowing anyone there, using Scala that is fairly new to me felt like a big challenge. I got a very warm welcoming by my fellow speakers and the organisers and felt right at home. My presentation went well and I felt really encouraged. I submitted a few more abstracts and got to speak at Scala.IO in Lyon later that year. This time around I know many of the speakers. I’m looking forward to meeting all my new friends and getting an inspiration boost from the conference.

Are you curious about Scala but you haven’t had the chance to play around with it yet or have you experienced all the benefits and you want to sell it to your colleagues than you should join the workshop . It will do a quick comparison between Scala and Java and also get you familiar with the Scala syntax and some basic functional concepts.

I started at SpeedLedger about a year ago and I feel that I have grown by learning so many new things, getting to know new people and speaking at conferences. I started here because I wanted to work with functional programming and I’ve had a great journey. We are hiring and if you are interested in Scala, live and Gothenburg and want to know more, give me a shout.

ScalaIO

ScalaIO 2017 in Lyon

At ScalaIO 2017 in Lyon I gave a presentation about “Scala for Java developers”. The slides can be found at https://mharlin.github.io/ScalaForJavaDevelopers/. I got the feeling that the presentation was well received by the audience. When all the videos from this years conference are ready they will be published on ScalaIO FR’s youtube channel. At the time of writing this they have not yet been published. Last year they published the videos about a week after the conference.

It’s been very interesting to think about my own Scala journey when preparing for this talk. I realized how much I’ve learned in the past year since I started working with Scala. It’s good with some reflection to see what you have accomplished and to set new and exciting goals for yourself.

I had my talk early the first day and after that I could relax more and enjoy the rest of the conference. There were many takes on how to use the type system effectively. One of the ones I liked the most was by Markus Hauck, Let the compiler help you: How to make the most of Scala’s type system. It’s great to get inspiration from all the presentations. For me the most valuable part is sharing experiences with all the people you meet. It’s extra interesting to share war stories with your programming heroes.

Functional programming is getting more popular. More people are using it in their day to day work and several new conferences around functional programming have been popping up during the last years. There are plenty of opportunities to get accepted as a speaker at conferences and share the message of how functional programming has solved problems for you. I’m very happy that SpeedLedger encourages me to go out and speak. I want more colleagues that love functional programming and if you like presenting it’s even better. Than we can share the great message of functional programming
together.

Automated Exploratory Test Fixtures

This is part 3 in my series about  BDD and MicroServices. You may want to read part 1, and part 2 first.  

As I have blogged about earlier, I have been spending some time exploring BDD/executable examples in a MicroServices environment, and how we can benefit from executable examples (SBE, Specification By Example) here at SpeedLedger. It has been a really good learning experience, and fun too.
Thanks to the initial progress, I have managed to persuade my boss to let me work on this full time! We are doing it as an time-limited experiment, but hopefully this will be extended.

In this blog post, I would like to share one of our greatest takeaway so far: That End-to-End Exploratory Testing is more valuable than End-to-End Regression Testing.
Both categories are valuable of course, but if I had to choose to have a really good setup for exploratory tests OR only a great suite of regression tests, I would definitely choose the former. Having automated fixtures (setup procedures) for interesting “moments” in the lives of our customers when using our product is super valuable, and I will list the reasons below.

 

When all this started, we named the experiment “BDD across team borders”. This was a good name at the time, but we later realised that the benefits of BDD/SBE can be appreciated by all co-workers, even those outside development teams. We have identified three main areas where we – as a company – can benefit from automated environments and automated test data:

Learn

If you are teaching yourself a new app/system/service, wouldn’t it be great if you could experiment and explore all aspects of it without being afraid to destroy anything? And in a simple way set up situations that the typical customers face every day?
Similarly, if you are training a new hire about your system; Wouldn’t the training be more efficient and fun if you could go through these scenarios interactively, setting them up with the touch of a button?

Discuss

When discussing existing or new features with you peers, what would it be like if you could throw up a the system on a wide-screen monitor, demoing the current feature in question and the proposed solution respectively. While being able to repeat all setup step automatically and indefinitely?
Compare this to discussing features with the help of only stale sketches or obscure user stories.

Verify

This is the obvious one, of course. Most people think that verification of current features is the only benefit of automated test. But as we have seen above, there are several nice bonus effects.
In addition, there are other – maybe more subtle – positive side effects that come from using the test-first mindset. But that is a different story…

How about you? Can you automatically setup the preconditions for your customers’ most interesting interactions with your product?

 

 

 

Speaking at FlatMap Oslo about Property Based Testing

In the beginning of May I’m heading to FlatMap in Oslo to give a 90 minute workshop about Better testing with less code.  I will go through the basics of property based testing with ScalaCheck. About half the time will be hands on exercises to get a feel for the framework a get comfortable enough to start using it in your own projects. The exercises and presentation can be found in my github repo.

I find it very rewarding to speak at conferences and agree with everyone that says that the best way to get a deeper understanding of a subject is to start teaching it to others.

Outside-in — To the max! (BDD and Microservices part 2)

This is part 2 in a series about my 20% project BDD across team bordersPart 1 can be found here.  

When writing any piece of software, it’s very easy to start working on the details before you have enough knowledge about the targeted context. I think most developers recognize this pattern:
You start with some innocent (often unintentional) guessing about one or a few unknowns. Pretty soon the guesses turn into assumptions and before you know it they have morphed into ‘truths’. You happily code away with these truths as your guiding light. Later, when you try to integrate your code into a bigger context, the assumptions and truths fall apart and you need to rethink your work.

– Oh, is that how the incoming data looks like!
– Why didn’t anybody tell me I have to ask the database for that value?


Personally, I have made these kinds of mistakes countless of times, and I will probably make many similar ones in the future. I think this is due to human nature; We really like making assumptions for some reason. To minimize the impact of this phenomenon, we have to try really hard to be aware of it at all times. We also have to find strategies and techniques that help us avoid making these false assumptions.

spiralTest Driven Development (TDD) is one excellent tool for this purpose. It forces us to step outside of the current context and look at the problem from the outside, or from “one level above” if you wish. More importantly, the TDD approach fits on all levels of software development! No matter if your context is a oneliner method in a class or the public API of a complete service, you always benefit from taking one symbolic step out of your current context.

In my 20% project BDD across team borders, I wanted to take this line of thinking to the extreme, and work on the highest level possible, thus making sure I was taking the “outside-in” approach to the max, maximizing our helicopter view in the process.
I set as a constraint on my work going forward that I was only allowed to write the BDD tests against our publicly available APIs and web pages. No matter how tempting, I would not “cheat” by calling internal APIs or access anything not available to a user in production. If I did, I would risk getting my assumptions wrong, as described above.

Working against public APIs also provides another great bonus;  It forces you to write user-centric stories that are sliced vertically. A good rule is that “Every story must have a noticable effect on the public API level”. If you write a story that has no noticable effect for any user, you have not succeeded in writing a vertical story.

My claim to test on the “highest possible level” is not entirely true, of course. There will always be third party dependencies and other integration points that we cannot incorporate in the system under test. We will have to fake or mock things like partner APIs, for instance. As long as we don’t mock or fake our own code or apps, we should be good to go!

I want my helicopter view back! (BDD and Microservices, part 1)

502377266

Here at SpeedLedger Engineering, we took the plunge into the world of microservices some time ago. We have gradually phased out our old monolithic style applications in favor of many small and independent micro services/apps.

Overall, the micro services paradigm aligns very well with our organization (small independent teams) and we are getting pretty good at it, I must say! We are constantly improving our infrastructure and deployment tools, enabling us to develop and deploy quicker and more often. As a result, the general time-to-market has decreased and our customers are getting more new features, sooner.

However, one big drawback of working in isolated islands of people and code is that you risk losing the big picture, the “helicopter view”. Unsurprisingly, this is what happened to me and my teammates. We often found ourselves asking questions like:

  • What’s supposed to happen in this business flow?
  • What are the possible inputs to this API?
  • Which apps will call this endpoint?

The answers to these and similar questions are not impossible to find but it always takes valuable time to track them down and to get them clarified.  

I felt something had to be done about this situation. For a long time, I have been a fan of BDD (Behavior Driven Development) but I never really got my feet wet in a real world situation. I suspected our current situation could benefit a lot from a successful BDD effort, and that it provided a perfect opportunity for me to experiment with and learn more about BDD.

For someone who doesn’t know what BDD  is, it can be described as “executable documentation” or “executable examples” that a non-programmer can understand and edit. More about this later…

After some initial discussion, we decided that I would do this in the context of a “personal 20 percent project”, famously invented and used by Google.  The main reason being that part of the work had to be done outside the team boundary, touching all the other teams in some way. In line with this, we actually named the project “BDD across team borders”. 

This was in February, and since then I have spent most of my Fridays exploring the possibilities of introducing BDD at SpeedLedger, and – more generally – automated testing on the highest possible level in a microservices environment.  In a series of coming posts, I will blog about BDD and the progress of the project. Hopefully someone will find it interesting. Also, I would love to get some feedback from you blog readers along the way!

 

Maker Days 2016 Q1

Wait, what?

Maker Days is what you get when you take a team of 20 or so developers, feed them sugar and a pseudo-scientifically regulated amount of beer and sit them down at a computer for an extended period of time: An in-house hackathon arranged a couple of times a year here at SpeedLedger.

We form small teams around interesting or novel challenges and go at it. Some times the tasks are technically advanced dares on the outer rims of known code-space; other times, more banging rocks together in the hopes of having them throw beautiful sparks.

The execution

Anton and Lis looking dapper in their MakerDays tees

Heroes.

To kick things off we met for lunch at John Scott’s stable: A local pub with free access to a projector for presentations. As we ate we shared our ideas and formed loosely knit teams to take them on.
The actual event started Wednesday, two days after our lunch meet. With an array of snacks and caffeinated beverages at our disposal, we donned our mandatory MakerDays 2016-shirts and set to work. We were heroes.

Two days later we emerged from our makeshift cave to present the result of our labour. Each team got 5 minutes to give a brief recap of the original idea and the end-result. Encouragingly, all participants had something to demo and presentations proceeded with only the briefest technical hiccup.

The projects

Das blinken lights lets us know that while some of us are at work, others aren’t.

Das blinken lights lets us know that while some of us are at work, others aren’t.

So what wondrous creations did we come up with this time?
Thanks for asking!

  • Das blinken lights
    This mysterious LED-adorned box houses a Raspberry Pi programmed to detect mobile devices on the local network. It doubles as nerdy christmas decoration.
  • The SpeedLedger Button™
    Ah, the button. So succinct, so tantalisingly clickable.
    “Why don’t we make a few for our partners’ admin websites, so that they may synchronise stuff like their member registry and what not?” mused Per, and so he did.
  • Frikkin’ Flags
    Dynamic and highly configurable feature toggles to let us try things out for a select group of users — like A/B testing — or simply activate and deactivate a service-feature without having to redeploy. Implemented using LaunchDarkly. (Note: No flags were harmed in the process of development.)
  • Listen2Speedledger
    Based on the theory that everyone loves geometric shapes and random intermittent noise, one brave team set out to visualise user interaction events in almost-real time. The events were dirty polled from Elasticsearch and linearized so that they played one after another, rather than all at once much like a grand piano hitting concrete after being dropped from a tall building.
  • Async-duh!
    A lot of our intra-service communication is done synchronously even when it would be simpler to use an asynchronous data flow. This project experimented with exposing streams of logging events to consumers, using Kafka so that interested parties wouldn’t have to resort to dirty polling.
  • Hipstereport
    Finding Clear Reports’ PDF generation capabilities “slightly cumbersome” (“… holy crap, just look at all those boxes crammed on top of each other!”), another Per came up with the idea to replace it all with a custom-built report tool using HTML5 and CSS3 for layout.

Lastly a team of talented designers managed to freshen up the careers section of our website.

Awesome work everybody!

The wrap-up

Presentation of The SpeedLedger button™.

Per, demonstrating the undeniable affability of buttons.

Last year we had some issues with screen sharing between teams; each team were responsible to share their presentation over the network so that remote location participants could partake. This year we opted for a simpler solution by having the teams connect directly to the projector and offering a video stream to remotes. Much simpler.

All in all, the event was a triumph (I’m making a note here, huge success). Most of the projects initiated were completed and the overall experience was rewarding.

Two days of hacking feels like a reasonable timeframe for hackathon projects; any less and we would be rushed to produce a viable candidate. It would be interesting to see what we could do with three days.