Continuous Delivery Expert Check 2020 – Kubernetes and the future of CI/CD (Part 2)

JAXenter: The rise of containers and Kubernetes has changed software development significantly. How has it changed the way that software is delivered?

Tracy Miranda: It has made it more complex. In a distributed system, with multiple environments, if you haven’t set up CI/CD you can end up in big trouble if something goes wrong. Things you might have gotten away with before are quite difficult to get away with in a cloud native world. Combine that with the proliferation of new tools in the industry, tools that don’t necessarily interoperate well and it is very difficult for users to know what is best. This is one of the key reasons many in the industry worked together to launch the Continuous Delivery Foundation which aims to tackle challenges like this and help people deliver software in whatever their chosen environment is.

Containers have become the expected packaging format for software.

Priyanka Sharma: The rise of containers and Kubernetes or the advent of cloud native has changed software development and delivery alike. Developers can build faster with microservices. However, that is not useful if the code cannot be tested and rolled out quickly as well. So, the age of Kubernetes has ushered in the age of continuous delivery and deployment.

Clark Boylan: Containers have become the expected packaging format for software. Potential users expect prebuilt container images that they can start and immediately begin evaluating your software. From a CI/CD perspective they present a convenient format to ensure the software you tested and is known to work is the same software you deploy into production. Kubernetes builds on this and makes managing those containers in production simple and approachable, and encourages developers to think in terms of composable microservices. But this focus on microservices introduces new testing and integration challenges on the CI side.

Baruch Sadogursky: I am stepping on the minefield here because I represent JFrog, so full disclosure. We have a continuous integration/continuous delivery tool called JFrog Pipelines. It’s based on Shippable, which we acquired. We are going to release this tool in the beginning of January. I know from within that this is going to be an amazing tool. JFrog aside, there are other great tools in the market. If we’re talking about continuous delivery and continuous deployment, obviously, Spinnaker comes to mind. Although it has some years under its belt, it’s getting better. It’s a great tool that I recommend hardily for continuous delivery and deployment. For continuous integration, very interesting aspects are in development.

SEE ALSO: DevOps in 2020 – our big DevOps survey

Tekton, an open source cloud native engine for continuous integration is being worked on with different contributors. There are already continuous integration tools that use Tekton, such as Jenkins X. These are part of a new foundation under the umbrella of the Linux Foundation: The Continuous Delivery Foundation. I recommend keeping an eye on what’s going on there. Outside of the foundation, there are a lot of tools which embrace the cloud native way of doing things. They build in the cloud using containers and Docker and use a configuration that is very dear to the heart of the cloud native world. CircleCI comes to mind, but also many others.

Joonas Lehtimäki: It has made software deployment a lot easier for sure. Kubernetes helps teams to deploy software in scale, but the actual benefit came from containers itself. If you think about how people deployed software before that, the actual process was slow, and it was hard to keep all systems identical. Containers tackled that issue by packaging all the necessary ingredients to a single package. That made every single deployment identical. Kubernetes then just made it easy to deploy on a bigger scale, solved networking issues, and so on. But Kubernetes is not perfect for every single software, for example, it doesn’t make sense to deploy a static web application to a Kubernetes cluster. It would make more sense to use in this situation, something like Google Cloud’s App Engine.

Nir Koren: In almost all aspects. I think the main benefit we experience is the fact that the image is almost identical in dev and in prod so we have no last-minute chaos after production deployments.

Christian Uhl: The reproducibility of containers and the rise of Kubernetes have simplified the delivery process of software immensely. With these technologies, we can perform the same clever deployment strategies with one team of software engineers what needed whole departments of people before. This allows small players to play in the big league, so to speak. It also amounts to some really hard to debug problems 🙂

JAXenter: Which CI/CD tools do you recommend? Do you know of any hidden treasures you’d like to share?

Tracy Miranda: The open source tools that are part of the CD.Foundation, and in particular Jenkins X especially if you are looking at modern app development. Don’t be fooled by the name – Jenkins X is a completely new tool that provides best-in-class CI/CD support for modern applications running on Kubernetes. It achieves this by incorporating other great open source tools such as Tekton [5] and Kubernetes. Jenkins X provides pipeline automation, built-in GitOps, and preview environments to help teams collaborate and accelerate their software delivery at any scale.

Priyanka Sharma: As a company and builds and uses its own CI/CD tools, I can’t help but recommend GitLab CI and CD. There is a lot of innovation in the space and we can expect to see a lot of standardization coming from the continuous delivery foundation as well.

The rise of edge computing makes the future of software delivery very, very exciting.

Clark Boylan: We are proud users and maintainers of Zuul, a purpose-built CI/CD tool that focuses on project gating. Zuul ran more than two million jobs over six months to produce the OpenStack Train release. It is self-hosting (tests itself), and performs CI/CD for our OpenDev services. BMW and Volvo are major users that show Zuul has a wide range of applicability. From open source software development to self-driving cars, Zuul helps to make your software better.

One major benefit to using Zuul for CI/CD, particularly in a containerized world, is its ability to test complex change trees across multiple projects. As microservices become more prevalent you need to be able to test the combination of many different pieces of software all working in concert. Zuul is able to test these change combinations and does so before a single change merges. This enables complex integration testing, and ensures that your application is always functional.

Baruch Sadogursky: It’s obviously very hard to tell what the future will bring but I think the rise of edge computing is upon us. More and more, computers are getting out of data centers and into places where the computing is needed, such solar panel arrays, wind turbines, and cellular stations. In retail, everything is getting better by adding computing power in the form of edge devices. Obviously, then there are IoT cars, which are edge computers on wheels. The rise of edge computing makes the future of software delivery very, very exciting because we are upon a completely new set of challenges that cannot be solved yet with continuous delivery and continuous deployment. Then there’s the prediction of continuous updates, which is a vision designed for delivering software to the edge. That is the next big thing.

Joonas Lehtimäki: That really depends on the needs, to be honest. I do prefer open-source ones like Drone. Because those projects actually listen to the needs of the community and don’t force you to follow any specific type of development strategy. I’m looking forward to what Argo and Flux merger since they are both really powerful CDs.

Christian Uhl: The team from GitLab have built an amazing toolbox, and I think they currently lead the market in terms of innovation. Also, Kubernetes and Helm are currently under fire for their complexity, but I value these tools immensely and they provide so much value, it’s worth learning about them.

JAXenter: For the last question, you should take out your crystal ball: How will the world of CI/CD change in the coming year(s)? What does the future have in store for us?

Tracy Miranda: I only have to look at the working groups of the CD.Foundation to see where things are headed and there are two areas which are particularly exciting to me: The CDF Interoperability SIG is focussed on making all CI/CD tools work better together. There are many ideas in play: standardized deployment metadata, standardized pipelines, common terminology/vocabulary, etc. This could be huge for the industry because once we stop folks reinventing the wheel and standardize on common components that frees us up to innovate at higher levels up the stack. So more and faster innovation in the CI/ CD place, driven by open source collaboration is something to look forward to.

Developers can build faster with microservices. However, that is not useful if the code cannot be tested and rolled out quickly as well.

The second area is the CDF MLOps SIG which focuses on applying DevOps principles to machine learning applications. Machine learning adoption is rising fast yet the extra dimensions of model, data and code present extra challenges for CI/CD tooling. The sooner we can help machine learning practitioners with best practices and fit-for-purpose tools the faster things will progress here in a sustainable way, i.e. not just shoving Jupyter notebooks into production. I’m excited for the developments in this area too and the role the CD.Foundation can play here. Both working groups are open to everyone to participate, so follow the links and join us driving the future of continuous delivery!

Priyanka Sharma: I believe there will be standardization in the CI/CD space so that organizations can port pipelines from one CI provider to another. Vendors will differentiate on value they provide rather than the specific way their pipelines are designed. I also believe that the GitLab concept of Runners will catch on to other providers. With the help of runners, CI pipelines become composable and decoupled from the rest of the infrastructure.

Clark Boylan: I think we’ll see a greater focus on automated developer assistance tools using machine learning. Amazon’s CodeGuru is a notable example of this, but expect to see machine learning impact more than just code review. Automated failure analysis with bug triage and fix suggestions, security audits, and more are likely to become expected tools in the software quality world.

SEE ALSO: The new DevOps Magazine – now available as a free download

Joonas Lehtimäki: It will get more and more automated. In the future, developers will probably just need to select what sort of deployment strategy they want, and then the intelligence behind the pipeline will do all the heavy lifting.

Nir Koren: I think the CI/CD will become more and more standardized so the entire system (infrastructure + pipeline) will be developed as a single package from different companies. We already see such products today. Therefore, the entire CI/CD will be more like a black box for developers.

Christian Uhl: As the industry catches up, and tooling further improves, continuous deployment will soon become the absolute standard of how software gets released and everybody not on board will face a very tough time because they will be outperformed by their competition. This also means we have to find new ways to differentiate ourselves. If you look at how seamless the developer experience of https://zeit.co/ is, we see that less and less configuration will be necessary to continuously deploy applications to the “cloud”. I hope in the future we won’t even notice deployments, it will feel as natural as changing the code “live” – but with all the benefits we get from our pipelines.

The post Continuous Delivery Expert Check 2020 – Kubernetes and the future of CI/CD (Part 2) appeared first on JAXenter.