microCPH – The Microservice Conference in Copenhagen

Copenhagen

2018-05-15 08.18.08

I had the unique opportunity to attend µCPH 2018 in the wonderful city of Copenhagen this year.
The relatively small community conference was a great mix of high-level and hands-on talks.
Hosting a single track of talks did not jeopardised me to pick from a variety of presentation titles. Instead it felt like the order of talks was perfectly aligned to guide you from the eagle’s perspective (keynote on coupling by Michael Nygard) to Microservices and patterns of Domain Driven Design, down to the hands-on experiences of an actual move to a dockerized system of Financial Times, closing with some great remarks of a framework for breaking up the UI monolith (mosaic from Zalando).

One of my favorite presentations: _Microservices: Patterns and Antipatterns_ by Stefan Tilkov (slide deck, video). Many talks take the motivation for microservices either for granted or put scalablity, fault-tolerance and moving agile through independent release cycles upfront. In this talk however, Tilkov focuses on actual problems and how they have been solved in the past without the label micro service and how you shoot yourself in the foot by blindly applying conference-motivated architecture.

Another great talk by Dmitriy Kubyshkin explained how Zalando deals with a micro-service infrastructure and their front-end (slides, video). Many times micro-services talks are about back-end systems, and left the front-end out of discussion. At speedledger we also have the problem, that we somehow have to combine UI components with their back-end counterparts that are served through micro-services. Mosaic – the framework that Zalando developed has some great inspirations on that topic.

Ben Stopford with his talk on Kafka Streams (video, slides) gave some great ideas how you can turn services inside out and use advanced Kafka features to distribute the state of your system to avoid letting services turning into a database with a REST interface.

µCPH was a very well organised conference with a great schedule that gave plenty of time between the sessions to exchange with other attendants and to network.

I highly recommend to attend µCPH, if you have the chance to do so 🙂
– Tobi

QCon London venue

QCon London 2016

 

I’ll try to summarize three great days at QCon in London, not going into detail about all sessions that I attended but high-lighting some of the ones I really liked.
Most sessions were recorded and will be available for the public audience over time, make sure to check them out!

 

QCon offers a wide variety of tracks ranging from low level “Close to the metal” to more “soft” skills like “Optimizing You”.  There were also a track for the main sponsors of the conference. The wide variety of content and speakers made choosing what session to attend somewhat of a problem; I had made a schedule before traveling to the conference, and it broke down during the first presentation of the tracks before the first keynote…
Kicking of QCon was a great keynote “Unevenly Distributed” by Adrian Colyer. Adrian reads a paper a day, summarises it and publishes it on his blog “The Morning Paper”. This was a very inspiring and well presented keynote that raised my interest in reading papers and as Adrian said “5 reasons to love papers”:

  • Great thinking tools
  • Raise your expectations
  • Applied lessons
  •  Great conversation
  • Unevenly distributed

The only problem with reading more papers and learning more is that:

“The more I learn, the more I realize how much I don’t know.” – Einstein

After the keynote we listened to Gavin Stevenson, Engineering Lead at WilliamHill, who talked about WilliamHills betting engine and how they are transitioning from a large database centric solution to a micro service based architecture (this was a common theme during the conference). They were building a “production ready” betting engine in 2 week sprints, testing it with real production data. The most interesting take away was how important it is to really try to break your system. When the system fails, that’s when you learn. The old saying “If it ain’t broken, don’t fix it” just doesn’t apply anymore.

“If it ain’t broken, try harder!” – Gavin on testing

One of the few really non-software related talks was held by long distance runner Simon Wheatcroft.
When losing his sight the age of 17 due to a genetic degenerative eye condition, he began a journey of adapting tech to achive the impossible.
Through the aid of the Runkeeper application, he started running solo outdoors. Simon will be running his first solo race in May 2016; the Four Deserts Series Sahara Race in Namibia. There, he’ll use GPS coordinates and a mobile app to navigate across the 250 kilometer distance.
A very inspirational and humbling talk!

“You just have to tell yourself that pain doesn’t last forever”- Simon on running 250k

From non-technical to real in-depth low level Netty implementation details. Norman Maurer from Apple described how Apple is using Netty as a web service delivery platform for most of Apple services. Apple are running 550000+ Netty based services, handling 10s of Petabytes of data every day and millions of request per second.
Norman guided us in different aspects of the Netty framework and how the default JDK implemtations just aren’t good enough for this kind of load and how they’ve commited several improvements to the Netty open source code. Very interesting and down to the metal of how to hack the JDK and using JNI to get better performance with; for example memory allocation and SSL.

Martin Kleppmann held one of the best presentations of the whole conference where he talked about keeping data sources in sync, moving away from (distributed) transactions to streams. The content was not very in-depth, but Martin had deep knowledge of the subject, excellent slide and a lot of energy when presenting. This is a talk everyone should watch and learn from.

“Stupidly simple solutions are the best” – Martin Kleppman

Day two started with Linda Northrops keynote “Reflections on Software Architecture”. Linda looked at software architecture and how it’s importance and acceptance have changed over the last 20 years. To summarize: architecture is important, and it’s a way to manage technical debt.

Josh Evans from Netflix presented how Netflix have expanded there streaming services to almost the entire globe (Netflix#Everywhere).
Netflix have had some major outages and failures, both in their own software and the underlying cloud AWS-platform. Josh concludes that “Failure is inevitable” and that one really have to embrace the failure and not fail in the same way twice. This had lead Netflix to embrace a “Failure-driven architecture” approach when building their platform.
Netflix’s architecture is really impressive, although not applicable for most comapanies/services, so it’s always interesting to hear what they are doing to actually run the platform at that scale.
Josh presented Netflix four architecture pillars; data, caching, traffic and micro services, and how they use (among other techs) EVCache, Cassandra and DNS to keep their services up and running in case of total failures of an AWS datacenter/region. He also showed how they test failure in different regions and route trafic to another region to minimize customer impact.
If infrastructure and architecture at scale is of any interest, watch this talk when it comes online!

“Never fail in the same way twice” – Josh Evans

Mitchell Hashimoto (founder of Hashicorp) gave his talk “Observe, Enhance, Control: From VMs to Containers”. In his talk he takes us back to 2006 and the age of VMs and how the datacenters and the problems to solve are driving the architecture of the software for Monitoring, Configuration and Deployment. Jumping to 2016 and the age of containers, Mitchell argues that the “state-of-the-art” tools from the age of VMs are not really suited to handle the tasks anymore. Even though the tools are extremely good, they do solve a completely different problem. The content of the talk was nothing new, but it is really inspiring to listen to Mitchell talk.

Gil Tene talked about Hardware Transactional Memory. Really low-level stuff about CPU pipeline and cache optimization. HTM in the JVM is not new, Azul has been delivering both hardware and a customized JVM with JVM for 10 years. What’s interesting is that it will become mainstream now when Intel is shipping CPUs with support for HTM. Gil succeeded in a very educational way describe the complexity of HTM and how it can be implemented in for example the standard JVM. In the end Gil talked about how the developers must reason about locking and synchronization to make the most of HTM in their code.

One of my most anticipated talks during the week was Dan North‘s “Making a sandwich”. I hade very high expectations for this talk, and Dan managed to exceeded them (as usual). Dan talked about giving feedback, how feedback in itself is a system and why we should do it. Giving and receiving feedback (which is really just to say ‘Thank you’) is, in my opinion, one of the hardest skills to master and we should really practice a lot! Dan presented some useful techniques and tricks, but you should really watch this yourself!

Last day started with a very entertaining and inspiring keynote delivered by Kolton Andrus (Netflix) and Peter Alvaro (University of California). Peter and Kolton shared their expereience of a very successful collaboration between industry and academia. Peter had a “big idea”, Lineage-driven fault injection, and together with Kolton this evolved from a theoretical model into an automated failure testing system that leverages Netflix’s state-of-the-art fault injection and tracing infrastructures.

“My code is now actually running live on Netflix…” “…well, minus all of the println statements”

Vikki Read and Alex Wilson from Unruly described how they are using the extreme programming (XP) ideas to deliver high quality software and how it can be made to work in a very agile environment. Since agile and XP focus a lot on collaboration and knowledge sharing, they shared some problems that they’ve had with getting new employees up to speed and how growing teams make for example (too long) stand-ups being a problem.

The final session I attended before heading to the airport was Tammer Saleh at Pivotal talking about the mistakes people make when building a microservices architecture, i.e. Microservice anti-patterns. When are microservices appropriate as an alternative to the monolithic app? The problem with monolithic apps is not about the code, it’s about the teams! Large teams (or multiple teams) can’t work effectively in the same codebase.

Its not about code, its about teams

Its not about code, its about teams

Tammer stressed that “the most common mistake is to start with microservices”. Start monolithic and extract as needed, because microservices are complex and impose a constant tax to your development.
Tammer explores how to draw the lines between services. dealing with performance issues, testing and debugging techniques, managing a polyglot landscape and the explosion of platforms, managing failure and graceful degradation.

“Boring is beautiful” – Tammer Saleh

Most of the interesting sessions I attended during the week were about failure, and how to handle failures. Quotes like “Failure is inevitable”, “Failure is an opportunity to learn” and the importance of building an architecture that can manage failures were common topics. Migrating from a monolithic application to a more micro service oriented architecture were also popular.
Overall QCon London was a great conference, I will most likely try to get back next year! All tracks had great speakers, which is problematic since you have to choose between sessions – on the other hand most of the sessions are recorded so I know what I will be doing the coming weeks.

 

Ship your Scala code often and easy with Docker

At SpeedLedger we have during the last year been working with gradually containerizing our services using Docker. At the moment we have 20 different services containerized that are running in production. During this period we have learned a lot and made a lot of mistakes. We are using a number of tools and developed some of our own. For example our deployment tool is built in-house and is responsible for starting and configuring our Docker containers in a standardized way to handle logging and service registration for our applications.

At the end of April it’s time for this years edition of the flatMap conference in Oslo. I (Marcus Lönnberg) will be hosting a talk there about how to deploy Scala applications using Docker. Some of the talk will be general about Docker and various tips from my journey with it at SpeedLedger. I will also talk a bit about how one can use sbt-docker, which I recently released version 1.0.0 of, to easily build Docker images using sbt. There will also be a few other tools that we will look into, for example the deployment tool we have developed at SpeedLedger.

SpeedLedger at the DataTjej conference

Earlier, in February, I participated in the DataTjej Conference held in Gothenburg. “DataTjej” roughly translates to “ComputerGirl” and DataTjej is a non profit organization aiming to increase women’s interest in computer science and IT, disregarding age or experience. Their long term goal is to increase the number of women in computer science and software development. Each year peaks in a conference held at different universities in Sweden. As of this year it was held at Chalmers University of Technology in cooperation with Gothenburg University here in Gothenburg. The conference aims to enhance the student-business relationship within computer science, IT and information systems. You can find out more about the organization and the conference here.

I have attended the conference several times as a student, but this was the first time representing my company. The very first time I came in contact with DataTjej was during my first year at Chalmers University of Technology. Some of my seniors organized the conference in Gothenburg 2009 and they encouraged me to go.

This year my co-worker, Marcus Lönnberg, and I attended the conference in order to promote SpeedLedger as an awesome employer and to recruit more women to SpeedLedger’s engineering department. We took part in three different activities during the conference; a presentation, a dinner and a fair.

So I had a few ideas when I put together the presentation. I knew we were going to have a booth at the fair, so I didn’t feel the need to promote SpeedLedger during the presentation, instead I wanted to give the audience something useful. Something code related. I also remembered my first time at DataTjej. We had just finished the second course in object oriented programming, which basically was Java with Swing. I hate Swing. During this period my mind was set on surviving the programming courses and then graduate to work with development processes and not touch code for the rest of my life.

But over time though, I came to realize that there are a different types of programming. Just because you don’t like one specific type, doesn’t necessarily mean that you dislike programming. You need to find the type of programming you like. As I see it, it’s always easier to learn something that you like and feel joy doing. For me it was datastructures and algorithms that changed everything.

So I let my presentation revolve around algorithms and how it awoke my interest for programming. Algorithms can be used to solve everyday problems and I gave the audience a demonstration of how two algorithms find the fastest path between two points, plus the theory behind it. To do this I hacked together a small program showing how the algorithm thinks in every step of the problem solving process.

A* solving a maze

A* solving a maze

I really wanted the presentation to have a technical aspect, rather than the typical business-connecting-with-students type. I hope the audience appreciated the technical connection and if anyone of them felt insecure I hope they were given a motivation to why coding can be fun. They have already chosen to study computer science, now we need to convince them to stay and enjoy the ride.

Back at the office I got the question: Why is this so important and why do we need girls in our teams?
This is a very relevant question. We actually don’t need women in our programming teams because they are just that, women that is. We need women in our team to create a heterogeneous group. Just as we need male nurses. There are numerous studies showing that a heterogeneous group is more efficient and drives each other forward at a much higher pace than a homogeneous group.

DataTjej’s most important task is to create an environment free from prejudice and to be a place where women, who otherwise might feel alone in a classroom full of dudes, can meet and share experiences.

So, to summarize: More diversity in software development! That is what we need to be stronger, better and faster as a team!