3 Musketeers for an epic Developer Experience

Sat, 28 Aug 2021 14:18:09 +0000

We all want to reduce toil and improve the developer experience (when starting new applications or joining the team). Developers have had to invest significant time in creating production-ready pipelines when building new products and configuring their workstations when joining teams. Their workstations may not run all the same tests as their pipeline and thus they’re left with #slowfeedback and an inconsistent experience.

The larger the organization (ie Enterprise with 10’s-100’s of teams and applications) the worse the problem gets.

GSD is built on the principle of reusability through a pattern called 3 Musketeers (test, build, and deploy your apps from anywhere, the same way). It gives consistency, control, and confidence to development teams.

Docker is a key part of this pattern, but the use of docker, docker-compose and make together really unlock the powers of consistency, control and confidence in your application build, test and deploy steps. The 3musketeers Docs have some great examples of these patterns.

Here's what the most simple usage looks like:

While the pattern itself is simple, the power (and benefit) comes from the widespread usage across your organization. Here’s what usage might look like:

Agree on a standard interface for all pipeline actions (much like API’s have an agreed interface, all pipelines in the org use the same interface, ie `make build`, `make test`, `make deploy`)
Keep the interface simple (avoid custom pipeline logic, one simple Makefile calls Compose, the container handles all the logic)
Create docker containers for each pipeline step, make them stateless, reuse them where it makes sense

Wins: Modern pipelines already support running steps in docker containers. This pattern extends portability across almost any pipeline tool or development workstation and runs the same way in every single environment. Pipeline steps can be tested independently and reused. This pattern makes it easy to go one step further and generate pipelines on-demand based on boilerplate starter templates. Oh yeah, and #fastfeedback (we’re talking pre-commit).

Info

Also posted on medium as 3 Musketeers for an epic Developer Experience.

How Cloud Transformation at Scale can enable Good Software Delivery

Wed, 04 Aug 2021 22:50:33 +0000

Why should you transform?

Each person, team and organization is going to be on their own journey to cloud, writing good software and attempting to delight customers, or increase profitability. Finding your why can take some time and deep thinking (and deserves a whole blog to itself).

For now here are some common why’s for transformation and good software delivery that might also apply to you:

To enable product (development) teams to ship software with less dependencies on other teams/humans
To speed up delivery
Allow the business to scale products quickly
To build new products quickly
To support the business in getting fast feedback through frequent delivery
To achieve consistency in infrastructure and software deployment, so we can reduce the risk of issues due to manual changes
To reduce the manual but repeatable steps close to production (security and compliance checks) that often slow down or delay releases

A picture

If a picture tells a thousands words, then I present to you Cloud Transformation at Scale:

Cloud Landing Zone

In purple we have the “Cloud Landing Zone” which is the starting point for key cross-cutting concerns that span the entire organization:

Automated Account-Provisioning for shared Cloud Accounts
Account Guardrails
Security & Logging Accounts (SIEM & IAM)
Networking Foundations
Shared-Services Account

This is the foundation of most modernization efforts that depend on the Cloud. Automation of Accounts & Infrastructure unlocks the ability to “move fast” in a way that traditional on-prem data center or operation teams aren’t able to match.

Most organization on their cloud journey are building or buying some variation of this or started off with some manual steps and a few scripts, and are maturing their operations to look at lot like this as they scale.

Services for Developers

In red we have the next set of shared services focused on supporting product (and thus development) teams. This includes:

Automated Infrastructure-Provisioning for Shared Services (think shared k8s clusters)
Common Shared Services (build agents, artifact storage etc)
Self-Service Cloud Accounts for Product teams (self-service being the key)

It’s important that developers can manage their own accounts via a self-service mechanism (if you’re an organization with more than 10 developers). This means automated pipelines for infrastructure and account provisioning that have a well thought out developer-experience that make it easy for them to manage their own accounts, securely (ie with Account Guardrails built into the pipelines and Cloud Accounts), and sans human intervention.

Product Teams — Trust and Verify

In blue we have application and app-related infrastructure pipeline(s). Each commit should result in a single artifact built and tested then promoted into production if it passes the automated tests.

Developers should have access to reference pipelines that show exactly how to pass all the organizational, security and compliance requirements (which should all be automated within the pipeline). It should be easy to start a new project that can quickly pass through the pipeline into production.

Developers should have the freedom (and trust) to do whatever they need to do before they reach production. They should have access to every automated security and compliance check that will be run in production so they can quickly assess how ready they are for production on the very first commit.

The organization should automatically verify every production deployment with the same rules they supplied the developers on day-one, commit-one. No surprises, no extra work to do on production-day.

This seems like a simple concept, and the name I give it is “Good Software Delivery” because it’s simply how we all should be delivering the software we want to ship.

So how does one transform? In practice it’s a journey that individuals, teams and organizations need to be prepared to go down. And that’s a completely different rabbit hole I welcome you to explore: https://faun.pub/one-devops-please-part-1-df7a2787fde8

Appendix

We could go deeper into each area, but here’s a quick teaser that goes into more detail, in particular for the blue Product Team’s Pipeline(s):