software development on Andrew Khoury

How to implement Good Software Delivery in 30 seconds

Thu, 11 Nov 2021 19:22:11 +0000

Good Software Delivery (GSD) is the term we use for the set of practices that help deliver, well, good software. There’s a focus on short feedback loops, a consistent developer experience, with more time spent delivering and less time spent on chores like workstation setup and debug.

For context, I’ve previously written the following blogs that explain a lot of the “why” you’d want to implement the app I’m about to demo:

Assumptions

Before we dive in I want to share my assumptions and expectations. This blog/demo is aimed at developers, who are also familiar with pipelines, local development environments, and tools like docker.

While you can clone the repo and run the commands quickly (even with zero knowledge or experience in the underlying Golang app), it will take you longer than 30 seconds to read this blog, and even longer to look through the repo’s README.md and code in the repo to see what’s happening when you type each command. I’ve tried to add as much detail and context for those that’d like to deep dive, both in this blog and in the repo itself.

The purpose isn’t to copy and paste what you see here into production, but I do hope I’m able to demonstrate some ways of setting up your repo and local development environment in a way that could save you and your team time for your next product!

What we’ll do in this demo

In this post, I wanted to share a hands-on implementation of some of these concepts that you can try for yourself. We’ll be using a hello-world app written in Golang, and by the end of the demo I’m hoping:

You’ll see how wrapping our pipeline in 3 Musketeers can greatly reduce the amount of toil spent getting a new project started (thus the 30 seconds claim for this simple app)
You’ll see that you can adapt the concepts in this demo to just about any app (Java, .Net etc) — More complex apps may take longer to compile, but no extra effort should be required in terms of desktop setup (toil)
You’ll be able to extend the logic for other pipeline tasks like testing or security

If this demo takes you longer than 30 seconds to run this demo, I suspect:

We need to get you a better Internet connection (most of the 30 seconds for me is spent downloading the initial Golang image on an empty docker cache)
We need to help you with the base requirements that this blog assumes you already have (make,dockerand docker-compose)

In any case, reach out on LinkedIn if you have suggestions to improve this demo, or set up a virtual meeting with me if you’d like a 1:1 workshop/chat!

The demo (build and run a Go app in 30 seconds)

At a minimum, you’ll need _make_, _docker_ and _docker-compose_ on your workstation. See requirements for more info or continue on if you’ve already got these.

Not sure about running this demo on your computer? Never fear! You can use this online environment from the safety of your browser: GSD HelloWorld katacoda (repo will already be cloned in this environment, with required software ready to go)

You can see all of my Katacoda courses at https://www.katacoda.com/drewkhoury/.

Clone the repo: Start by cloning the GSD HelloWorld repo:

git clone git@github.com:contino/gsd-hello-world.git && cd gsd-hello-world

2. Build the app: Once you have the codebase, build the app using:

make build

This step will create a build artifact (in our case, a docker image) by doing the following:

downloads the appropriate docker image onto your workstation
copies the source for our app into the image
builds the application in the image
configures the port/cmd to be used at runtime

3. Run the app: Now that you have your build artifact handy, you can run the application locally with:

make run

This step will run/deploy the artifact on your workstation by doing the following:

uses docker to create a container based on the image from the build step
exposes port 8080 locally
the docker image runs the cmd ./main which starts the application for us

That’s it, you’re running the GSD HelloWorld application written in Golang and containerized so that it can quickly be deployed in Docker.

Bonus: You can run make test to execute the test suite (note this may legitimately fail if it doesn’t meet the testing threshold), andmake down to clean up once you’re done.

The beauty of this model is in its simplicity. We’re using make as the interface and we’re running all the tasks in containers to ensure consistency. It’s easy enough to peak under the hood to see what either of those tools is doing, and this runs just as easily on your workstation as it does on your CI tool of choice.

Digging into the code

To keep this section brief I’m going to assume some prior knowledge around development, docker, and make.

What if I want to make my own changes or do something similar in my own repo?

For the curious: Checkout the Makefile, docker-compose.yml and Dockerfile in the repo. These drive our pipeline and app build etc, and while they abstract the complexity away from day-to-day usage, they’re intended to be developer-friendly in terms of understanding exactly what each step is doing.

Building the app: For example make build is what builds our application, and in the Makefile you’ll see it does the following:

docker build -t ${FULL_TAG} .

Replacing the app: If you wanted to replace our Golang application with your own Java application, you’d drop in your own Dockerfile , replace the /src, and leave everything else as-is. The beauty is that any developer using the repo still runs make build and they don’t need to worry about Go vs Java dependencies (or in theory even know that you completely rewrote the application) — it just works™.

Note: This repo is a testing ground for GSD practices — simple but powerful ways to supercharge your development.

If you’d like to see what else you can do, have a look at the rest of the https://github.com/contino/gsd-hello-world — we have plenty for you to explore.

build, test and run actions are driven by a simple Makefile — with underlying execution via docker containers
Build badge in README.md
Repo Visualization in README.md
Sonar quality, reliability, and security badges in README.md
Golang security in-pipeline
Local tests via make test
Artifact storage via Github Packages
Github Actions
Deployment to k8s in GCP

Further reading:

May your software be good, and your delivery be continuous — drew

Info

Also posted on medium as How to implement Good Software Delivery in 30 seconds.

The cost of failure is education

Tue, 12 Feb 2019 09:40:54 +0000

wow, so cool.

If you’ve ever been responsible for software running in production, you should already be well aware of failure. But it’s not only traditional “operations” roles that are affected by failure.

Consider some of the changes we’ve seen in IT in the last decade:

2003: SRE - Site Reliability Engineer, a discipline created at Google that incorporates aspects of software engineering and applies them to IT operations problems.
2006: AWS EC2 - The birth of the cloud “Amazon Elastic Compute Cloud (EC2)” which made Infrastructure-as-a-Service available to developers around the globe via APIs. It paved the way for CloudNative, Serverless, PaaS and the hundreds of services we have available to us today.
2009: DevOps - The combination of software development (Dev) with information technology operations (Ops)”.

It’s clear there have been changes in the people and teams who need to be involved in running software for customers. At the same time there have been significant changes to how we build, test and deploy software.

Culture is a common topic that comes up when we talk about things like DevOps, Lean Software Development, and SRE but what do we mean when we say culture? Do we just give developers pizza and beer until they produce good software and hope they don’t leave for another company offering gourmet pizza and craft beer?

Origins of lean in manufacturing

To understand the origins that paved the way for DevOps, SRE, and the “Culture/Transformation” we’re hearing so much about in IT we need to go back, back to 1990 and back to car manufacturing. Consider the following quote from https://en.wikipedia.org/wiki/Lean_manufacturing

Lean manufacturing makes obvious what adds value, by reducing everything else (which is not adding value). This management philosophy is derived mostly from the Toyota Production System (TPS) and identified as “lean” only in the 1990s

Steps to achieve lean systems

The following steps should be implemented to create the ideal lean manufacturing system:

Design a simple manufacturing system
Recognise that there is always room for improvement
Continuously improve the lean manufacturing system design

Sound familiar? These themes are present in Lean, Agile and DevOps.

The andon cord — Game-changing rope

Back in the early days of the Toyota Production System (TPS), Toyota implemented many unique tools and processes. The Andon Cord was born, and while it was just a rope it was the most powerful rope you’ve ever seen.

Anyone had the right to pull the cord at any time and pulling it would instantly stop all work on the assembly line. The technology behind this wasn’t revolutionary, but the thinking, culture and trust behind this system is something we can learn from today. The “Safety Culture” around this process was reinforced when the team leader arrived at the workstation, thanking the team member who pulled the Cord.

The team member did not, and never would be, in a position of feeling fear or retribution for stopping the production line

Toyota has so much to teach us about working together an respecting each other. Even if the team leader knew a better answer, Toyota principally felt that a solution from the team member was a better outcome.

Postmortums & lean software development

Leaving cars behind and getting back to software, Mary and Tom Poppendeick’s work on Lean Software development in the early 2000s paved the way for the sort of thinking coming out of Google (SRE), and one great example of that is postmortem culture.

Writing a postmortem is not punishment — it is a learning opportunity for the entire company.

As an IT professional with a decade of experience I’ve seen how the culture of the company can significantly impact how people, teams and the organisation grow and improve. The great opportunity that exists when failure happens is learning. The companies and teams that truely believe this are the ones that will continuously improve, both their software and their culture.

Info

Also posted on medium as The cost of failure is education.