Developer Experience on Andrew Khoury

Tips for how to use 3 Musketeers to supercharge your Developer Experince

Fri, 01 Sep 2023 19:22:11 +0000

3 Musketeers is a pattern popularized by Frederic L where you can Test, build, and deploy your apps from anywhere, the same way.

Note: Frederic's website was preivously available via 3musketeers.io but is now https://3musketeersdev.netlify.app/

The key benifits include being able to have a mix of Linux, Mac, and Windows workstations. You also get consistency with pipeline/CI tools that also run the same steps in the same way.

More Info:

https://3musketeersdev.netlify.app/guide/patterns.html
https://github.com/liatrio/3musketeers (brief intro, and install guide)
https://github.com/drewkhoury/katacoda-scenarios-contino - interactive lessons (before katacoda got taken down) - but you can look at the readme's for each lesson to see the steps
https://github.com/drewkhoury/minimum-viable-delivery/blob/main/cicd-spec.md#portability-directive-runtime-must-be-independent-of-pipelinebuilddeploy - CI/CD spec I wrote a while back to explain the philosophy/reason behind it

Examples that also include docs in their READMEs:

https://github.com/drewkhoury/gsd-hello-world - demo (Golang) and docs (and link to detailed blog)
https://github.com/drewkhoury/3-musketeers-base - setup, env, aws profile, python and docs
https://github.com/drewkhoury/static-site - Demo of a static site using AWS CDK with Python

More Examples:

https://github.com/liatrio/3musketeers/tree/main/examples/cloud-in-a-box (LocalStack w/EKS cluster)
https://github.com/drewkhoury/pact-5-minute-getting-started-guide/blob/main/README-demo.md (PACT dockerized)
https://github.com/drewkhoury/cucumber-declarative-gherkin (shows chrome headless testing of apps, w/screenshots through selenium web driver, in containers - also happens to be a full BDD example with more than just some trivial tests)
https://github.com/drewkhoury/backstage-3m
https://github.com/drewkhoury/communities - static site with hugo

When to use 3 musketeers

It's important to understand the goals of Consistancy, Control and Confidence and the part each tool plays in the pattern.

Can't I just skip this and use my tools directly?

The benfitis of using these tools together might not be obvious if you've only worked in a single project, or in a small team or if you haven't seen a project grow in complexity over time.

You might also be new to tools such as make, compose and docker and as such the idea of introducing them into your pipeline might seem unessasary. This might be especially true if your team is working with legacy applications and you don't have any container presense in your organization or team.

Only your organization and your team can decide if the goals of Consistancy, Control and Confidence are worth the investment in pipeline standarization.

These patterns and tools truely shine when:

Developers start to seek "fast feedback" by "shifting left" - ie wanting to run some tests early (before they commit code)
Working in larger teams
With different underlying development machines - or with developers who install different versions of software (how much time did a new developer spend getting a working build env?)
Across multiple teams in large oragnizations
With CI/CD agents managed by other teams where changes to agents are not easy and you don't have permission/flexibility (did someone upgrade a build agent and break how java works for you? how would you know?)
When having to share code between vendors
The more complex the application gets, or as more changes are made to the application and it's build steps (having to refactor or share build steps either within your team or between teams)
Testing your pipeline code (how many times have you done 20 check-ins of code just to test one bit of functionaility "on the build server")

The 3 tools

As the name implies 3 musketeers is made up of 3 tools: https://3musketeersdev.netlify.app/about/tools.html

Docker

Docker is the most important musketeer of the three.

Many tasks such as testing, building, running, and deploying can all be done inside a lightweight Docker container.

The portability of Docker ensures you can execute the same tasks, the same way, on different environments like MacOS, Linux, Windows, and CI/CD tools.

Example: Docker can be invoked directly from make, or can be invoked from a docker-compose command in make.

Make

Make is a cross-platform build tool to test and build software and it is used as an interface between the CI/CD server and the application code.

A single Makefile per application defines and encapsulates all the steps for testing, building, and deploying that application.

Of course other tools like rake or ant can be used to achieve the same goal, but having Makefile pre-installed in many OS distributions makes it a convenient choice.

1# Makefile
2echo:
3 docker-compose run --rm alpine echo 'Hello, World!'

Compose

Docker Compose, or simply Compose, manages Docker containers in a very neat way.

It allows multiple Docker commands to be written as a single one, which allows our Makefile to be a lot cleaner and easier to maintain.

Testing also often involves container dependencies, such as a database, which is an area where Compose really shines. No need to create the database container and link it to your application code container manually — Compose takes care of this for you.

1# docker-compose.yml
2version: '3'
3services:
4 alpine:
5 image: alpine

Can't I choose my own mukseteers?

Who says make, compose, docker are the perfect combo?

The key to Consistancy, Control and Confidence is not only in the specific combination of make, compose, docker but actually the agreement of your team and your organization to use a standard that's easy for everyone to follow.

Consider:

Do most of your development teams have easy access to make, compose, docker - or can you agree/reach a state where this is possible?
Do you want to replace make with something else? (zeus, python, shell) Sure thing, just agree as a team what would be more easily to install and maintain
Do you want to start with Make and Docker and introduce Compose only when the complexity demands it?

Q & A

Are there pre-reqs? What if my workload or app isn't docker based?

This pattern doesn't operate at your workload or application/deployment architecture level, it's all about the CI steps (like build, test, publish, deploy etc).

The workload could be anything, that scope is outside of the pattern. It could be the case that your workload is containerized but that's unrelated to the higher-level pattern and how the local/CI steps are run.

You need to agree on having Make, Compose and Docker available on developer workstations, and for the CI tool.

Make is difficult to use, and hard to learn

make is a powerful tool that can do a lot of things. However, the intent is to use make as a simple interface between your build tool and your build steps.

As you should only be using the most basic features of make it shouldn't be difficult to implement. If you're having a lot of difficulties, consider if you're over complicating your task.

Consider some real-life simple make files (easy to read, maintain and run):

You could consider using multiple makefiles or consider another language if you have complex logic that demands it.

I really don't want to use docker compose - I'm not even deploying anything / I'm using k8s

Don't confuse using docker-compose during your build/test steps with deploying your application in production. With 3musketers we're looking at the patterns we use to inteface with our build tool (like building and testing your application in your pipeline).

People often use compose files (with 3musketeers) to avoid having long and messy make files.

To Compose or Not to Compose?

If your use of docker is simple for a particular task, feel free to skip the compose file.

https://3musketeersdev.netlify.app/guide/patterns.html#docker

Make calls directly Docker instead of Compose. Everything that is done with Compose can be done with Docker. Using Compose helps to keep the Makefile clean.

When you have a multi-line docker invocation, or 5 lines of docker commands in a single task, consider if compose could simplify things for you.

https://3musketeersdev.netlify.app/guide/patterns.html#task-management-tool shows an example of how to implement an "npm" command "cleanly".

Notice that the compose file does all the volume mounting, workdir (and would do all the env injection etc).

Consider an even more complex senerio that invovled multiple services, logs, naming of containers, starting and stopping containers, cleanup when done.

How will I manage container versions/instances

Without container images in your CI pipeline you may have managed binaries and software through an artifact store like Artifactory. You would use a similar approach for containers too.

Your organization will need to decide how teams can pull images (directly from the Internet, or through Artifactory as a proxy etc), and have a model for developers to be able to maintain and update docker images. Often docker images that need customization will need their own pipelines and independent lifecycle management (assuming you need to extend the features of a docker image maintained somewhere like Docker Hub).

This concern also exists without the use of 3 Muskteers, if your organization uses docker images in anywhere in any capacity (ie for application deployment or in any other part of the organization). As long as docker images aren't banned in your organization, and you have any need to modify or update a docker image anywhere, you'll have to solve for this.

Won't I have to do more scripting to do things like export the results of scan details and test results out to another location?

Yes, there's an extra layer with docker that introduces volume mounts, a more careful look at permissions and ownership, and files need to be stored/saved outside the container run. That's an overhead and a bit of a learning journey which can be painful to begin with, especially without docker knowledge. Generally once you get those patterns working well, you have less issues moving forward (and you can repeat/share the patterns), and imprpoving your docker skills may be useful if you need them for your app development too.

This question ends up being one about tradeoffs as you get more consistency and reuse across Linux/Mac/Windows and your CI tool, you make life easier if you ever move CI tools (think about moves to GitLab, Azure, GitHub etc), and setup is a breeze for new devs joining the company or the team.

The key advice here is to use small containers that have a single binary and a single purpose, when they do an operation that produces an output it should be available via a volume mount that allows easy access either via your workstation or CI tool, after that it's functionally the same as having run it without docker.

Alternative: In theory you could cheat the pattern a little here by using native container image steps in a tool like GitLab/GitHub to reduce the work you need to do to mount files and export them, but it does make it less portable for local usage or if you ever move CI tools.

Alternative: If you were to not use containers at all you wouldn't be able to run everywhere as easily and as consistently, which would arguably shift some toil back to developers.

Who's responsibility is it?

3musketeers will not stop you from writing bad pipelines or bad code.

3musketeers will not write your build steps for you.

3musketeers will not replace your build process or your build tool.

3musketeers will not deploy your application for you.

3musketeers will not store metadata or artifacts during your build process.

For example, if your project, team or organization creates a docker container to provide some sort of functionality (e.g a sonar scanner image) this docker image should have it's own:

documentation
development lifecycle
versioning / latest tags in git and in it's artifact store
versions should be immutable - never override a version with new code
if the container is run without the approriate environment variables, it should set sensible defaults where it can, and fail with explict error messages where it can't set a defaults
requirements about connectitity should be well documented (and properly testing with proper errors when there's lack of connecivity)
auth should be clearly documented and variablized
if volume mounts are required and if files are outputted, this should be clearly articulated

Making changes:

scripts/containers should allow you to override config files and/or supply additional config files or variables to override defaults (thus reducing the need to change the container/script itself each time)
if your container is really a packaging mechinisum for a script consider how the script will be used and if it's better to be baked into the container (ie each change to the script should cause a new version of the container) or if you'd like users to maintain the script themselves and simply call the container with the script mounted into it

Optimizing for a Cloud-native Developer Experience

Wed, 23 Mar 2022 19:22:11 +0000

Chris and Drew share their combined knowledge around developer experience, why (micro) feedback loops matter a whole lot, and present a live demo of a Cloud Native application working locally - complete with unit tests, end-to-end tests and smoke tests.

Originally presented by Chris and Drew Optimizing for Developer Experience in a Cloud Native World - AWS Meetup Group
Concepts based on the work from Tim Cochran via Martin Fowler Maximizing Developer Effectiveness
Cloud Native DX miro board for presentation content
Following the 3 Musketeers pattern

This talk is packed with real patterns used by Chris and Drew working with teams to help them increase their effectiveness. The demo was designed so you can follow along on your own workstation - and be up and running within minutes without the need to install and configure the underlying tech (Node, AWS SAM, Python, Playwright, etc).

Micro feedback loops

"From what I have observed, you have to nail the basics, the things that developers do 10, 100 or 200 times a day. I call them micro-feedback loops. This could be running a unit test while fixing a bug. It could be seeing a code change reflected in your local environment or development environments." - Tim Cochran

Demo

Demo Repo: GSD-AWS-CDK-Serverless-Example

If you have make, docker-compose and docker installed you should be able to have a local env running in a few minutes.

1git clone git@github.com:chrishart0/gsd-aws-cdk-serverless-example.git && cd gsd-aws-cdk-serverless-example
2make install && make run

Hint: If you run make after cloning the repo, it will show you the help menu and let you know what commands to run to build, test and deploy the application.

Micro feedback loops demonstrated in this repo:

Unit tests (front-end, back-end, infra)
Local environment - available in the browser
End-to-end tests
Direct deployment to AWS from your workstation
CI/CD runs the same as local, with additional security checks

Giving us your feedback (loop)

We'd love to hear what you think about these patterns, and get some feedback on your own developer experience using the demo repo.

Contact Chris
Contact Drew
Raise an issue in the repo

3 Musketeers for an epic Developer Experience

Sat, 28 Aug 2021 14:18:09 +0000

We all want to reduce toil and improve the developer experience (when starting new applications or joining the team). Developers have had to invest significant time in creating production-ready pipelines when building new products and configuring their workstations when joining teams. Their workstations may not run all the same tests as their pipeline and thus they’re left with #slowfeedback and an inconsistent experience.

The larger the organization (ie Enterprise with 10’s-100’s of teams and applications) the worse the problem gets.

GSD is built on the principle of reusability through a pattern called 3 Musketeers (test, build, and deploy your apps from anywhere, the same way). It gives consistency, control, and confidence to development teams.

Docker is a key part of this pattern, but the use of docker, docker-compose and make together really unlock the powers of consistency, control and confidence in your application build, test and deploy steps. The 3musketeers Docs have some great examples of these patterns.

Here's what the most simple usage looks like:

While the pattern itself is simple, the power (and benefit) comes from the widespread usage across your organization. Here’s what usage might look like:

Agree on a standard interface for all pipeline actions (much like API’s have an agreed interface, all pipelines in the org use the same interface, ie `make build`, `make test`, `make deploy`)
Keep the interface simple (avoid custom pipeline logic, one simple Makefile calls Compose, the container handles all the logic)
Create docker containers for each pipeline step, make them stateless, reuse them where it makes sense

Wins: Modern pipelines already support running steps in docker containers. This pattern extends portability across almost any pipeline tool or development workstation and runs the same way in every single environment. Pipeline steps can be tested independently and reused. This pattern makes it easy to go one step further and generate pipelines on-demand based on boilerplate starter templates. Oh yeah, and #fastfeedback (we’re talking pre-commit).

Info

Also posted on medium as 3 Musketeers for an epic Developer Experience.

Optimizing for DX — The Developer Experience

Sun, 19 Jan 2020 19:18:39 +0000

One thing I’m very much obssessed with is something I’m calling the Developer Experience (DX). I’m literally on a global & lifelong mission to optimize DX.

Hint: Once you’re sold on the benefits of a great DX you can skip ahead to the “3 Steps for success” section.

What is DX and why should you care?

If you’re part of a team/organization that does development then you’re likely going to run into one or more of the following:

Having to keep README’s up to date
Building & testing your code
Bringing new people onto the team & showing them how your code works
Building pipelines & debugging them when they don’t behave
Fixing your dev environment when it breaks
Supporting that one person who wants to use <insert whatever OS the rest of the team isn’t using>
Starting new projects and re-doing the basic building blocks to get started
Trying to support 10 teams all working on their own software but all needing to be able to work closely together (and not re-invent the wheel)

My perfect world!

In a perfect world the mundane parts of development would be simple:

Creating a new project should be fast (I shouldn’t worry about any of the ‘boilerplate’ steps that would be the same between projects)
As a consumer of code: I shouldn’t have to worry about how code is built, tested & deployed
As a developer of code: I should be able to easily update how my code is built, tested & deployed without affecting other developers or build servers wanting to consume my code

If I have to talk to a developer, install some special software dependencies, read a confluence page or README.md, figure out how to get things working on my OS or spend more than 5 minutes getting your software built or a new project setup then you’ve failed to streamline the DX.

3 Steps for success

The’s a concept called 3 Musketeers which is really just an agreed pattern for standardization when building software. It’s most common form uses Make as a standard interface to Compose & Docker Containers but you can adapt the concept to suit your needs.

3 Musketeers
_Test, build, and deploy your apps from anywhere, the same way. Get Started → Run the same commands no matter where you…_3musketeers.io

The net result is a consistent experience that works the same on Windows, Mac & Linux. It’s portable across development machines, build servers and provides consistency that scales from 1 team to 100 teams.

If you’re already using containers in your build server steps or as part of your development then you’re one step closer to a utopia of standardization!

Containers, and the 3 Musketeers concept can be applied any time you need to simplify a process or some instructions. You can consider it “self-documenting”. A good way to check if you’re giving the best DX is to look at what you need to put in your README to explain your process.

No Musketeers

Just

…

Install node version 4 but definitely not version 3, and I have no idea about version 5. I’ve you’re on Windows then I can’t help you at all. There are also a few dependencies that you’ll need but I can’t remember what they were. The following 12 page document is incomplete and you’re about to waste half a day that you’ll never get back.

1 Musketeer — Docker

Just run the following docker command:

docker run --rm -v foo:bar -v foo:bar -e FOO=BAR container bash -c ’echo hello world’

2 Musketeers — Docker & Docker Compose

Just run the following compose command:

docker-compose run --rm alpine echo ‘foo’

3 Musketeers — Docker, Docker Compose & Make

Just run the following make command:

make echo

And it’s as simple as that. Developers typically implement steps like “make build test validate deploy” but you can be as creative as you like and write the steps that make sense for you!

Supercharge your development

Using an agreed pattern like 3 Musketeers is a great way to standardize among your teams. As the number of teams you have in your organization grows so too will the benefits. Humans (and those robot automation scripts) will be able to consume your repos more easily and that’s a great thing.

But why stop there?

What if we could template what a good project looks like and deliver that to development teams in a self-documenting and self-updating fashion?

The most basic implementation of this would be writing a “hello world” application and asking your development copy/paste the code when starting new projects.

But there’s a tool called https://yeoman.io/ and it provides scaffolding for modern web apps. It’s another simple yet impressive concept executed well. What it provides is a templating mechanism with a really slick user-input feature and enough glue in-between to great that amazing DX we’re all chasing.

With yeoman development teams can setup and maintain examples of good software delivery with teams benefiting from the learnings of others. You can create a generator for just about anything, so the only thing left is your imagination!

Yeoman has it’s own generator to help you create your first generator (how meta, right?). The only thing missing for me was a reliable docker container to do some development in. I created a repo with a yeoman development container to make life easier. Hopefully you find it useful and make some steps to improve your own DX!

drewkhoury/devyo
_yeoman container - for local development of yeoman generators - drewkhoury/devyo_github.com

Want more?

Presentation (PDF): Optimizing for DX in a Cloud Native world
Presentation (Video): https://www.youtube.com/watch?v=ZYlbNnaoseI

Info

Also posted on medium as Optimizing for DX — The Developer Experience.