Docker on Andrew Khoury

How to implement Good Software Delivery in 30 seconds

Thu, 11 Nov 2021 19:22:11 +0000

Good Software Delivery (GSD) is the term we use for the set of practices that help deliver, well, good software. There’s a focus on short feedback loops, a consistent developer experience, with more time spent delivering and less time spent on chores like workstation setup and debug.

For context, I’ve previously written the following blogs that explain a lot of the “why” you’d want to implement the app I’m about to demo:

Assumptions

Before we dive in I want to share my assumptions and expectations. This blog/demo is aimed at developers, who are also familiar with pipelines, local development environments, and tools like docker.

While you can clone the repo and run the commands quickly (even with zero knowledge or experience in the underlying Golang app), it will take you longer than 30 seconds to read this blog, and even longer to look through the repo’s README.md and code in the repo to see what’s happening when you type each command. I’ve tried to add as much detail and context for those that’d like to deep dive, both in this blog and in the repo itself.

The purpose isn’t to copy and paste what you see here into production, but I do hope I’m able to demonstrate some ways of setting up your repo and local development environment in a way that could save you and your team time for your next product!

What we’ll do in this demo

In this post, I wanted to share a hands-on implementation of some of these concepts that you can try for yourself. We’ll be using a hello-world app written in Golang, and by the end of the demo I’m hoping:

You’ll see how wrapping our pipeline in 3 Musketeers can greatly reduce the amount of toil spent getting a new project started (thus the 30 seconds claim for this simple app)
You’ll see that you can adapt the concepts in this demo to just about any app (Java, .Net etc) — More complex apps may take longer to compile, but no extra effort should be required in terms of desktop setup (toil)
You’ll be able to extend the logic for other pipeline tasks like testing or security

If this demo takes you longer than 30 seconds to run this demo, I suspect:

We need to get you a better Internet connection (most of the 30 seconds for me is spent downloading the initial Golang image on an empty docker cache)
We need to help you with the base requirements that this blog assumes you already have (make,dockerand docker-compose)

In any case, reach out on LinkedIn if you have suggestions to improve this demo, or set up a virtual meeting with me if you’d like a 1:1 workshop/chat!

The demo (build and run a Go app in 30 seconds)

At a minimum, you’ll need _make_, _docker_ and _docker-compose_ on your workstation. See requirements for more info or continue on if you’ve already got these.

Not sure about running this demo on your computer? Never fear! You can use this online environment from the safety of your browser: GSD HelloWorld katacoda (repo will already be cloned in this environment, with required software ready to go)

You can see all of my Katacoda courses at https://www.katacoda.com/drewkhoury/.

Clone the repo: Start by cloning the GSD HelloWorld repo:

git clone git@github.com:contino/gsd-hello-world.git && cd gsd-hello-world

2. Build the app: Once you have the codebase, build the app using:

make build

This step will create a build artifact (in our case, a docker image) by doing the following:

downloads the appropriate docker image onto your workstation
copies the source for our app into the image
builds the application in the image
configures the port/cmd to be used at runtime

3. Run the app: Now that you have your build artifact handy, you can run the application locally with:

make run

This step will run/deploy the artifact on your workstation by doing the following:

uses docker to create a container based on the image from the build step
exposes port 8080 locally
the docker image runs the cmd ./main which starts the application for us

That’s it, you’re running the GSD HelloWorld application written in Golang and containerized so that it can quickly be deployed in Docker.

Bonus: You can run make test to execute the test suite (note this may legitimately fail if it doesn’t meet the testing threshold), andmake down to clean up once you’re done.

The beauty of this model is in its simplicity. We’re using make as the interface and we’re running all the tasks in containers to ensure consistency. It’s easy enough to peak under the hood to see what either of those tools is doing, and this runs just as easily on your workstation as it does on your CI tool of choice.

Digging into the code

To keep this section brief I’m going to assume some prior knowledge around development, docker, and make.

What if I want to make my own changes or do something similar in my own repo?

For the curious: Checkout the Makefile, docker-compose.yml and Dockerfile in the repo. These drive our pipeline and app build etc, and while they abstract the complexity away from day-to-day usage, they’re intended to be developer-friendly in terms of understanding exactly what each step is doing.

Building the app: For example make build is what builds our application, and in the Makefile you’ll see it does the following:

docker build -t ${FULL_TAG} .

Replacing the app: If you wanted to replace our Golang application with your own Java application, you’d drop in your own Dockerfile , replace the /src, and leave everything else as-is. The beauty is that any developer using the repo still runs make build and they don’t need to worry about Go vs Java dependencies (or in theory even know that you completely rewrote the application) — it just works™.

Note: This repo is a testing ground for GSD practices — simple but powerful ways to supercharge your development.

If you’d like to see what else you can do, have a look at the rest of the https://github.com/contino/gsd-hello-world — we have plenty for you to explore.

build, test and run actions are driven by a simple Makefile — with underlying execution via docker containers
Build badge in README.md
Repo Visualization in README.md
Sonar quality, reliability, and security badges in README.md
Golang security in-pipeline
Local tests via make test
Artifact storage via Github Packages
Github Actions
Deployment to k8s in GCP

Further reading:

May your software be good, and your delivery be continuous — drew

Info

Also posted on medium as How to implement Good Software Delivery in 30 seconds.

3 Musketeers for an epic Developer Experience

Sat, 28 Aug 2021 14:18:09 +0000

We all want to reduce toil and improve the developer experience (when starting new applications or joining the team). Developers have had to invest significant time in creating production-ready pipelines when building new products and configuring their workstations when joining teams. Their workstations may not run all the same tests as their pipeline and thus they’re left with #slowfeedback and an inconsistent experience.

The larger the organization (ie Enterprise with 10’s-100’s of teams and applications) the worse the problem gets.

GSD is built on the principle of reusability through a pattern called 3 Musketeers (test, build, and deploy your apps from anywhere, the same way). It gives consistency, control, and confidence to development teams.

Docker is a key part of this pattern, but the use of docker, docker-compose and make together really unlock the powers of consistency, control and confidence in your application build, test and deploy steps. The 3musketeers Docs have some great examples of these patterns.

Here's what the most simple usage looks like:

While the pattern itself is simple, the power (and benefit) comes from the widespread usage across your organization. Here’s what usage might look like:

Agree on a standard interface for all pipeline actions (much like API’s have an agreed interface, all pipelines in the org use the same interface, ie `make build`, `make test`, `make deploy`)
Keep the interface simple (avoid custom pipeline logic, one simple Makefile calls Compose, the container handles all the logic)
Create docker containers for each pipeline step, make them stateless, reuse them where it makes sense

Wins: Modern pipelines already support running steps in docker containers. This pattern extends portability across almost any pipeline tool or development workstation and runs the same way in every single environment. Pipeline steps can be tested independently and reused. This pattern makes it easy to go one step further and generate pipelines on-demand based on boilerplate starter templates. Oh yeah, and #fastfeedback (we’re talking pre-commit).

Info

Also posted on medium as 3 Musketeers for an epic Developer Experience.

Automating Google App Engine

Mon, 14 Oct 2019 13:46:01 +0000

Google App Engine is a pretty amazing service. You write code, and Google will make sure it runs, and scales well. This service was available way back in 2008, well before everyone was getting excited over lambdas and serverless.

In my opinion, AppEngine is one of the most underrated services I’ve seen. It’s beautifully simple, and it has planet-level scaling (Google words). You don’t have to configure load balancers, tweak an OS or install/configure a Runtime. The focus is on optimizing for the developer experience, and having your spend more time writing code.

Even though Google makes it easy to get started with App Engine I wanted to reduce the steps even further. I’ve created a GAE demo that offers a local development and deployment environment via docker containers and simple to use commands.

Google App Engine Demo

Local Development

Creating a local development environment can be done with one command:

1~/code/gae-demo $ docker-compose up local

This is what the console output should look like:

This is what the app looks running on your machine:

The demo app comes with instructions and sample code that adds entries to a local datastore.

One of the amazing features of Google App Engine is the local admin environment that offers similar functionality to the online admin console from Google. The demo comes with everything pre-configured and ready to use.

Local Datastore Viewer:

Interactive Console:

Not bad for one line of code!

Deploying

Google has a wonderful feature called “ Projects” that allow you to group related resources to make them easier to manage. When you run the `new-deploy` command the demo will create a new project and related application, then deploy it for you.

1~/code/gae-demo $ docker-compose up new-deploy

Deploying updates to your application is quick and easy too. First specify your project id from the previous step, then run re-deploy.

1~/code/gae-demo $ export CLOUDSDK\_CORE\_PROJECT=devops-demo-xxx ~/code/gae-demo $ docker-compose up re-deploy

That’s it, why are you still here? You could be spending this time deploying your new application.

You can find the full demo and instructions here: https://github.com/drewkhoury/gae-demo

Info

Also posted on medium as Automating Google App Engine.