Continuous delivery best practices, global challenges & orbs

Continuous delivery is a very hot topic as we enter 2020, and here at JAXenter, we sat down for a good long talk to CI/CD platform provider CircleCI‘s Chief Executive Officer Jim Rose.

A bit about CircleCI

CircleCI was founded in 2011, and their product first went live in 2012. They offer an automation solution for software delivery teams, automating all the steps between an upstream change occurring and it being built, tested, validated and ultimately released. The company’s idea was originally to build this as a service for small, fast-moving, venture-backed startups.

CircleCI is about 300 people all focused on the automation pipeline and CI/CD, making it as smooth, easy and scalable as possible for a wide variety of customers. Things have really taken off for them over the last couple of years. Until the end of 2017, CI/CD was pretty much just seen as the area of Facebook, Amazon, Netflix and Google, and some really technically sophisticated startups.

Whereas today, continuous delivery, instead of being the glimmer in somebody’s eye, is really the stated aspirational endpoint for the majority of software development shops. And now, folks are trying to figure out, what’s necessary to get there.

Why open offices in Japan and London?

CircleCI is trying to get continuous delivery to as many customers round the world as possible. They have very diverse businesses and a very global spread; about half of their customers are in North America, with the rest sort of spread out across the world. And over the last year and a half they’ve been trying to get teams set up on the ground so they can do a good job supporting the market and their customers. This is especially important once you start thinking about Europe or Asia as it’s difficult to keep up with those customers’ needs if all your staff are in the US.

Last summer they announced and opened their first such office in Japan to take care of Japan and the greater APAC region. The most recent office was opened in late 2019 in London to support EMEA customers. When we asked him about why they picked London for their next office, Jim said that the EMEA region accounts for somewhere between 16 and 25% of their overall business, depending on the month – the markets vary and change but they’ve done very strong business in the UK, Ireland, Germany and other European markets. That’s why they want to get closer to those customers and make sure they’re getting exactly what they want and need out of the product.

What about Brexit?

Of course, it’s difficult to talk about anything happening in the UK without wondering if Brexit will have an effect on it. So of course I asked what impact Brexit has on CircleCI’s decision to put a team on the ground in London. Jim said:

It was definitely something we kept an eye on. You know, it’s hard to find anybody who actually knows what’s going to happen there. At the end of the day for us, we have a really big business in the UK generally, but specifically in London as well. We were always going to have a presence in London; from a hiring perspective and from a talent perspective, it’s still an amazing place to set up, and so we decided to start there. I think as we get into some of the other parts of Europe, the thinking for us is that as those markets get to critical mass, we will likely figure out a way of getting on the ground in some of those markets.

Since CircleCI is already a globally distributed team – one of the co-founders is Irish – they do have a number of employees located all across Europe, so they have a foot in the door in those developing communities.

In the end, it seems like Brexit or no, London is too key to their business as well as too tempting to want to resist for all of the talent and skills available in somewhere like London that simply isn’t available elsewhere.

What’s it like working with different cultures?

The idea of test now, test often, automate anything that a developer has to do more than twice gets you to a much better, much higher quality end product in a far faster manner.

I was very intrigued as to how different it is conducting business in the UK as opposed to North America, or if there’s a big difference in mindset they have to consider when working with Japan, so I asked Jim a couple of questions about CircleCI’s experiences bridging different cultures. He explained that the end result is more or less the same no matter where you are, but the motivation for starting down the road of continuous delivery with CircleCI tends to be where the differences are to be found.

Depending on which market you are in, there tends to be two different motivations for adopting continuous delivery. In the US, it’s all about speed. It’s about trying to build faster, release faster, and get in front of opportunities and problems and release as many times as you can to take advantage of that. Jim referenced Facebook’s famous mantra, “Move fast and break things” as still being relevant; the focus on delivery speed is what drives the process. To go faster means you have to release in smaller chunks, and means that you have to have an incredibly fast test process. So as a side-effect of increased speed, you get increased quality.

In Japan, their continuous delivery goals are more oriented towards high quality; people want to make sure the thing they put in front of their customers has been properly tested and isn’t going to break down. To get there, they release very quickly and in small chunks. So they then get speed as an after-effect of the quality.

In Europe, they’ve found a bit more of a mix. In London and Ireland in particular, there are companies driven by speed, but in mainland Europe, most of the market is driven by the desire for quality, with speed coming as an after-effect. Jim said:

It really differs by company. If you’re the disruptor, and you’re trying to move quickly against some big incumbent company, speed is one of your primary differentiators. Whereas if you’re the one who is the incumbent and someone’s trying to come in and grab the market away from you, just being smart and really focusing on quality and making sure that you get good, well-considered, tested solutions out to the market at a reasonable pace becomes a competitive differentiator for you.

He went on to say that in the past people thought that moving faster often meant cutting corners from a quality perspective, and while that might have been true in the past, with CI/CD is really about a change in process and mentality, where by moving faster and testing in smaller units, you end up building a much better, much higher quality applications than if you made a bunch of changes in one merge after several weeks. It also has the benefit of not turning into a tangled ball of string where you have to try and tease out the problems and solve them while everything else is being changed at the same time.

Respecting tradition

Jim also talked about how there is also a strata of companies that, no matter where they are in the world – Japan, London, the US – they all behave the same. These are the companies that are so far ahead of the curve, they really all want the same things. But on the other hand, there are companies that have a much more corporate and traditional approach. Jim explained that with these companies the business culture changes and the way they adopt technology changes. And CircleCI’s job is to try and accommodate all of these different and very specific cultural norms that need to be considered in some markets.

SEE ALSO: 2020 Digital Innovation Benchmark report – Innovate fast or die trying

It seems, however, that actually in Japan and London what’s happening is that it’s the startups taking the lead in advanced development practices and impacting the big companies in how things work. And that’s why they don’t just send a couple of sales people into the offices that they open, but what Jim calls a “full stack”, from developer relations all the way up to sales reps. He explained that actually the only way for them to succeed is to become deeply involved in the community. Once they’ve managed that, he said, they typically see that changing positions in open source development often leads them into some of the biggest accounts out there.

Best practices for continuous delivery

Once we were done talking about how different cultures and different companies approach continuous delivery, we got to talking about whether or not the continuous delivery model we use now is the pinnacle of software development, or if there’s still room for improvement. Jim said, “We can be opinionated, but not prescriptive.” Their job is to give companies what they do want, not tell them what they should want.

Although every company has their own unique set of processes and ways of doing things, CircleCI does have what Jim called a “grab bag of best practices” to try and encourage customers that aren’t spending too much time thinking about continuous delivery to go down a particular route. He thinks that the goals and expectations of continuous delivery are becoming much more universal; building and testing in small chunks, focusing on the stability of your main line, ensuring that the default branch gets fixed as soon as possible if something does go wrong, and getting your application into the hands of customers as fast as possible. Those are the four main values of continuous delivery that are universal.

How does a high-functioning development team actually look?

We moved on to talk about a report published by CircleCI analyzing 30 million pipelines in which they looked for patterns and trends in how pipelines are being used. They found that the top performers build their software often, and they build their software quickly. They also found that actually the top 50% of CircleCI customers have a lead time of less than 10 minutes between clicking commit and the change arriving in the data center. And if it does break, 50% of the time it can be fixed in one try in less than an hour.

SEE ALSO: The trendy five: Ring in the new year with the coolest GitHub repos of December 2019

Shift left and shift right

Next I asked Jim what comes next for his company. He told me that they operate on a pull model, meaning they go where the demand is, they don’t decide where they would like to expand. They’re constantly monitoring the global communities to see where their next base of operations could be, but right now there’s nothing fixed. However, he did explain that they are being pulled closer to the developer:

What is software delivery is really evolving. In the past, earlier renditions or conceptions of what a software delivery process looked like was: software developer makes a change on their desktop and they commit their change to the remote repository or where the code is hosted, then your CI/CD system kicks off and builds and tests that change and then it deploys into the data stack. And what we’re finding now is that this core desire to validate change is pulling our platform in really different ways. One is, as the individual software developer, usually my most expensive set of tests is when I push to the remote repository and it kicks off all these automated pipelines. So for example, if I’m building a big AI application – instead of a 5 or 6 minute build, that might be a 5 or 6 hour build. So what I want to be able to do is test as much of that on my desktop as I possibly can.

To help support this, they have created agents that developers can put onto their desktop to be able to test and validate what happens there before pushing into the main repository. Jim also brought up two other big trends: shifting left, pulling testing and security higher up in the development toolchain, and shifting right, which is where testing and validation is happening in production. As a result, CircleCI’s platform is being pulled into production validation as well:

In the past when you had a monolithic model, all you had to really do was make sure that your test environment mirrored exactly what your production environment was, so when you ran your tests and if everything was green, you knew once you deployed into production that everything was going to work great. But now what’s happening as customers are taking these monolithic applications and breaking them down into microservice architectures, so taking one application and breaking it into dozens if not hundreds of smaller services, is that when you want to test the change you have for your service, you can’t really guarantee what the state of all the other services are around your service because they’re all moving at the same time. So it’s like all the chairs are moving, and they’re all moving at different times and different paces, and the only way that you can then fully validate that the change is good is to push it into the data center and continue testing it against production traffic.

In a world where you’re trying to change a lot, and you’re trying to change faster, the tooling, platforms and processes necessary to do that get more and more important, and that is another big area that we’re focused on: Giving developers that confidence, reliability and speed that regardless of where that final validation happens, they know they can look in one place and see that everything is good.

Monitoring and orbs

Then towards the end of our conversation we moved onto the topic of monitoring these deep and complex systems. Jim said that most places already have a monitoring solution such as Datadog or Sentry, so they’re not trying to replace any of those. What they’re doing instead is something I found really interesting. He described a framework called orbs that allows the CircleCI platform to communicate with other software:

We wanted to make sure the two systems can communicate so we created this framework that we call orbs. The best way to describe it is like RubyGems but for software delivery, where you can create these very flexible but highly validated integrations between CircleCI and hundreds of other platforms out there.

I wanted to know more about these orbs so I asked Jim to talk me through them in a bit more detail. Here’s what he said:

Basically, it’s like a standardized set of code and commands that creates an integration point between our system and the third-party system. Everyone’s software delivery system is different and everyone’s tooling is different, and the way that CircleCI works and one of our core values is that if you can script it, if you can code it, you can run it on CircleCI. That allows for an incredible amount of power and flexibility for our customers.

However, if you’re a vendor – for example if you’re Datadog – and you want to make sure that every customer of yours that is building software on CircleCI is using your API endpoints or is using your service correctly to make sure that you pass the data back and forth, you want to make sure that you have ONE valid connection as opposed to 5,000 different cobbled-together connections. And that’s what orbs allows for; it allows for one commonly validated process and package between our platform and a multitude of others. And then you don’t have to worry about breaking things.

Orbs is a way for us to integrate with other third parties to ensure that you know that connection is always tested, that it will work the same way every time and that the vendor is going to be able to support that well.

Essentially, they want to meet developers on the command line, because that’s where they’re working. He also mentioned that they’ve seen orbs used in bigger companies to standardize the business processes between departments or different ways of working within teams, so really they’re a very powerful tool to have in your arsenal.

Closing thoughts

Finally, I asked Jim if there was anything more that he wanted to say before we parted ways.

Obviously, on the go to market side, we’re really excited to be directly involved in the EMEA market. We see incredible innovation happening globally and incredible innovation happening in London. As a vendor it’s really fun to see what people are building on the platform and what they’re doing, how they’re doing it and how best practices are being adopted and changed and moved forward. In general the market itself is moving very quickly; I’ve been here for about five and a half years and the amount of change that has happened in the last two is really mind boggling. And that’s a fun change to watch and be a part of – and we expect that in the next year it’s only going to accelerate. It’s an exciting challenge, so we’ll see how it goes!

So that was that. Thanks once again to Jim Rose for a fascinating and illuminating conversation, and thank you for reading!

The post Continuous delivery best practices, global challenges & orbs appeared first on JAXenter.