DevOps on Andrew Khoury

How to implement Good Software Delivery in 30 seconds

Thu, 11 Nov 2021 19:22:11 +0000

Good Software Delivery (GSD) is the term we use for the set of practices that help deliver, well, good software. There’s a focus on short feedback loops, a consistent developer experience, with more time spent delivering and less time spent on chores like workstation setup and debug.

For context, I’ve previously written the following blogs that explain a lot of the “why” you’d want to implement the app I’m about to demo:

Assumptions

Before we dive in I want to share my assumptions and expectations. This blog/demo is aimed at developers, who are also familiar with pipelines, local development environments, and tools like docker.

While you can clone the repo and run the commands quickly (even with zero knowledge or experience in the underlying Golang app), it will take you longer than 30 seconds to read this blog, and even longer to look through the repo’s README.md and code in the repo to see what’s happening when you type each command. I’ve tried to add as much detail and context for those that’d like to deep dive, both in this blog and in the repo itself.

The purpose isn’t to copy and paste what you see here into production, but I do hope I’m able to demonstrate some ways of setting up your repo and local development environment in a way that could save you and your team time for your next product!

What we’ll do in this demo

In this post, I wanted to share a hands-on implementation of some of these concepts that you can try for yourself. We’ll be using a hello-world app written in Golang, and by the end of the demo I’m hoping:

You’ll see how wrapping our pipeline in 3 Musketeers can greatly reduce the amount of toil spent getting a new project started (thus the 30 seconds claim for this simple app)
You’ll see that you can adapt the concepts in this demo to just about any app (Java, .Net etc) — More complex apps may take longer to compile, but no extra effort should be required in terms of desktop setup (toil)
You’ll be able to extend the logic for other pipeline tasks like testing or security

If this demo takes you longer than 30 seconds to run this demo, I suspect:

We need to get you a better Internet connection (most of the 30 seconds for me is spent downloading the initial Golang image on an empty docker cache)
We need to help you with the base requirements that this blog assumes you already have (make,dockerand docker-compose)

In any case, reach out on LinkedIn if you have suggestions to improve this demo, or set up a virtual meeting with me if you’d like a 1:1 workshop/chat!

The demo (build and run a Go app in 30 seconds)

At a minimum, you’ll need _make_, _docker_ and _docker-compose_ on your workstation. See requirements for more info or continue on if you’ve already got these.

Not sure about running this demo on your computer? Never fear! You can use this online environment from the safety of your browser: GSD HelloWorld katacoda (repo will already be cloned in this environment, with required software ready to go)

You can see all of my Katacoda courses at https://www.katacoda.com/drewkhoury/.

Clone the repo: Start by cloning the GSD HelloWorld repo:

git clone git@github.com:contino/gsd-hello-world.git && cd gsd-hello-world

2. Build the app: Once you have the codebase, build the app using:

make build

This step will create a build artifact (in our case, a docker image) by doing the following:

downloads the appropriate docker image onto your workstation
copies the source for our app into the image
builds the application in the image
configures the port/cmd to be used at runtime

3. Run the app: Now that you have your build artifact handy, you can run the application locally with:

make run

This step will run/deploy the artifact on your workstation by doing the following:

uses docker to create a container based on the image from the build step
exposes port 8080 locally
the docker image runs the cmd ./main which starts the application for us

That’s it, you’re running the GSD HelloWorld application written in Golang and containerized so that it can quickly be deployed in Docker.

Bonus: You can run make test to execute the test suite (note this may legitimately fail if it doesn’t meet the testing threshold), andmake down to clean up once you’re done.

The beauty of this model is in its simplicity. We’re using make as the interface and we’re running all the tasks in containers to ensure consistency. It’s easy enough to peak under the hood to see what either of those tools is doing, and this runs just as easily on your workstation as it does on your CI tool of choice.

Digging into the code

To keep this section brief I’m going to assume some prior knowledge around development, docker, and make.

What if I want to make my own changes or do something similar in my own repo?

For the curious: Checkout the Makefile, docker-compose.yml and Dockerfile in the repo. These drive our pipeline and app build etc, and while they abstract the complexity away from day-to-day usage, they’re intended to be developer-friendly in terms of understanding exactly what each step is doing.

Building the app: For example make build is what builds our application, and in the Makefile you’ll see it does the following:

docker build -t ${FULL_TAG} .

Replacing the app: If you wanted to replace our Golang application with your own Java application, you’d drop in your own Dockerfile , replace the /src, and leave everything else as-is. The beauty is that any developer using the repo still runs make build and they don’t need to worry about Go vs Java dependencies (or in theory even know that you completely rewrote the application) — it just works™.

Note: This repo is a testing ground for GSD practices — simple but powerful ways to supercharge your development.

If you’d like to see what else you can do, have a look at the rest of the https://github.com/contino/gsd-hello-world — we have plenty for you to explore.

build, test and run actions are driven by a simple Makefile — with underlying execution via docker containers
Build badge in README.md
Repo Visualization in README.md
Sonar quality, reliability, and security badges in README.md
Golang security in-pipeline
Local tests via make test
Artifact storage via Github Packages
Github Actions
Deployment to k8s in GCP

Further reading:

May your software be good, and your delivery be continuous — drew

Info

Also posted on medium as How to implement Good Software Delivery in 30 seconds.

How Cloud Transformation at Scale can enable Good Software Delivery

Wed, 04 Aug 2021 22:50:33 +0000

Why should you transform?

Each person, team and organization is going to be on their own journey to cloud, writing good software and attempting to delight customers, or increase profitability. Finding your why can take some time and deep thinking (and deserves a whole blog to itself).

For now here are some common why’s for transformation and good software delivery that might also apply to you:

To enable product (development) teams to ship software with less dependencies on other teams/humans
To speed up delivery
Allow the business to scale products quickly
To build new products quickly
To support the business in getting fast feedback through frequent delivery
To achieve consistency in infrastructure and software deployment, so we can reduce the risk of issues due to manual changes
To reduce the manual but repeatable steps close to production (security and compliance checks) that often slow down or delay releases

A picture

If a picture tells a thousands words, then I present to you Cloud Transformation at Scale:

Cloud Landing Zone

In purple we have the “Cloud Landing Zone” which is the starting point for key cross-cutting concerns that span the entire organization:

Automated Account-Provisioning for shared Cloud Accounts
Account Guardrails
Security & Logging Accounts (SIEM & IAM)
Networking Foundations
Shared-Services Account

This is the foundation of most modernization efforts that depend on the Cloud. Automation of Accounts & Infrastructure unlocks the ability to “move fast” in a way that traditional on-prem data center or operation teams aren’t able to match.

Most organization on their cloud journey are building or buying some variation of this or started off with some manual steps and a few scripts, and are maturing their operations to look at lot like this as they scale.

Services for Developers

In red we have the next set of shared services focused on supporting product (and thus development) teams. This includes:

Automated Infrastructure-Provisioning for Shared Services (think shared k8s clusters)
Common Shared Services (build agents, artifact storage etc)
Self-Service Cloud Accounts for Product teams (self-service being the key)

It’s important that developers can manage their own accounts via a self-service mechanism (if you’re an organization with more than 10 developers). This means automated pipelines for infrastructure and account provisioning that have a well thought out developer-experience that make it easy for them to manage their own accounts, securely (ie with Account Guardrails built into the pipelines and Cloud Accounts), and sans human intervention.

Product Teams — Trust and Verify

In blue we have application and app-related infrastructure pipeline(s). Each commit should result in a single artifact built and tested then promoted into production if it passes the automated tests.

Developers should have access to reference pipelines that show exactly how to pass all the organizational, security and compliance requirements (which should all be automated within the pipeline). It should be easy to start a new project that can quickly pass through the pipeline into production.

Developers should have the freedom (and trust) to do whatever they need to do before they reach production. They should have access to every automated security and compliance check that will be run in production so they can quickly assess how ready they are for production on the very first commit.

The organization should automatically verify every production deployment with the same rules they supplied the developers on day-one, commit-one. No surprises, no extra work to do on production-day.

This seems like a simple concept, and the name I give it is “Good Software Delivery” because it’s simply how we all should be delivering the software we want to ship.

So how does one transform? In practice it’s a journey that individuals, teams and organizations need to be prepared to go down. And that’s a completely different rabbit hole I welcome you to explore: https://faun.pub/one-devops-please-part-1-df7a2787fde8

Appendix

We could go deeper into each area, but here’s a quick teaser that goes into more detail, in particular for the blue Product Team’s Pipeline(s):

Info

Also posted on medium as How Cloud Transformation at Scale can enable Good Software Delivery.

Creating a Vision

Sat, 04 Apr 2020 19:10:37 +0000

Creating a vision isn’t always an easy thing — but it can be rewarding to see one grow.

Disclaimer: There’s no “perfect” vision, and there isn’t a perfect method to share that vision with a team.

A technique I picked up from my good friend Ralph Bou Mechrek is to always be running experiments (in this context, an experiment is simply the testing of a theory in a real-life scenario). I can tell you from first hand experience it’s the fastest way to get feedback!

So with all that out of the way I want to share with you the experiment of running a vision workshop with your team…

A Wild Vision Appears

A vision can help guide your team’s decision making and help them hone in on what matters. Once you have a shared understanding of your vision you can more easily create roadmap and plan within your teams. Your vision should also motivate your team and provide clarity and focus while describing the future/target state. For this reason, explaining the “why” behind your vision is crucial.

If you already have a strong vision you‘re going to have plans you want to share with your team. As a technical leader in consulting I draw from my previous experience and deep understanding of the power of Business Value to form a strong vision I can share with my team.

It can take weeks, or months to formulate a vision and in this time you might be talking to likeminded people, exploring what matters most, where to focus efforts, what problem you’re trying to solve (and why). Usually you end up with one or more diagrams to help explain your vision — or something you can draw on a whiteboard.

The other thing you’ll need is passion. You need to believe in your vision before you can share it with others.

At the end of the day, you’re telling a story and you’re asking others to join you on a journey.

Workshop Part 1 — Seeding a Vision

So let’s assume you have a vision and you’re ready to share it with your team. The first part of your workshop should focus on sharing your vision with the team and gauging their response/interest while answering their questions.

Here are the sorts of things you might include in your diagrams, whiteboard or your passionate speech to the team:

What will the organization/team look like in 6 months
Teams and streams of work (What they’ll do, Success when…)
How teams will work together
What it looks like when we’re done (artifacts, technicals, logical diagrams)
What else is important? (Training, Metrics, Comms back to the business)

The key here is to keep things high-level while leaving the slide-ware at home. After sharing what you have your team should be able to articulate which parts resonate with them, and where they have questions, concerns or just other ideas!

Depending on how familiar your team is with the vision and how involved they’ve been thus fair — there may be lots of “why” questions. Why is this vision necessary? Why is there a big focus on X? Why are teams changing? It’s important to make sure the “why” has been effectively communicated, and that you’ve explored this with the team.

Assuming things have gone well, and it sounds like the team is on-board (think lots of head nodding and hearing the same things from multiple people) we can move onto the playback.

Tip 1: If your team isn’t on-board or there’s more confusion than there is support, now’s the perfect time to either continue that conversation and take notes of the feedback — or break so that the team has time to consume the information and re-group with more of a plan. Don’t be disheartened at this stage — you’ve just got some honest feedback and that’s great!

Tip 2: If you haven’t got any feedback or engagement from your team at this stage (silent room, or one word responses) either your vision isn’t very exciting, you caught them at the end of a long day, or your team isn’t ready to contribute to your vision. It’s important to make sure you team understands what you expect from them at the start of the workshop to avoid this situation. Again if you need to regroup — now is a good time.

Part 2 — Vision Playback

Okay so let’s assume you’ve shared your vision. You have an energized team that are engaged and contributing to the discussion. What’s next?

Here’s where the fun happens. What we want to do next is have each team member “play back” the vision in their own words. It’s critical that this part happens so each person in the team demonstrates buy-in, so that they have a voice, and so that you (as a leader) can understand where each of your team members is at in terms of how much they believe-in and buy-into the shared vision.

One simple way to do this playback is by way of a group role play session. Have one person in the group stand up facing the rest of the team (they’re going to play themselves).

Tip: For your first person pick someone who gets the vision and is confident — to show the rest of the team that this is a safe-place and this exercise is more fun than it is scary

Now have someone else in the group ask them a question (as if they’re the someone outside the group that’s just heard about this new “initiative”). Let’s work through an example.

Drew:

Ralph: (playing the role of outsider)

Hi I heard you’re part of this new initiative, can you tell me more?
I heard you don’t like puns, is that true? (in-joke / icebreaker question)
What is your role in all of this?
What’s the most important part of the initiative to you?
What will we have at the end of it?
How will this change our company?
Will my team have to use it? What if we don’t want to?
I’ve heard your using k8s — isn’t that more trouble than it’s worth?
What makes you think your group will succeed where others have failed?
Will it be hard for me to use your new product?
What do I do if I want to join your initiative?
I heard this won’t last very long as nobody wants it?

Those were just sample questions, the key here is to prepare the team for the sorts of questions others might ask, or the questions they may be asking each other. During this session you’re not focusing on right vs wrong questions but you might want to comment on what you observed after each person is done (in particular something you’d like to praise or see more of). Did one of their answers teach us something new? Are we seeing a theme with the answers? Are there some gaps in our vision?

If you heard some things have could have been answered better — consider holding onto that feedback to see how the next person tackles it. If you hear something that’s way off base you might want to suggest a different response they could have used to set expectations with the team (ie focus less on the negative and make sure you’re providing a solution/alternative response they could use).

Tip: Try not to prepare the questions ahead of time, you should be able to seed the whole activity with just two confident team members who are already bought-in. Let your team have a bit of fun with it by encouraging a few jokes. This should lead to a more natural conversation which will help your team think on their feet.

Wrap Up — A Vision is Born

There are a few key things that go into forming a high-performing team — and sharing your vision is one of them!

Once your team has a shared vision or goals you’ll notice all sorts of wonderful things:

People are happier
Planning work is easier
Your team communicate with each more
The messaging outside of your initiative is more cohesive
You’ll create a buzz around your initiative — and others will want to join

Oh and I guess I forgot to mention…

You’ll have a much better chance of achieving your goals and making your vision a reality.

Info

Also posted on medium as Creating a Vision.

Driving change and building a high-performance DevOps culture

Wed, 22 Jan 2020 18:31:01 +0000

I had the pleasure of listening to Mark Schwartz live at ReInvent 2019 and I was amazed by his wealth of knowledge and insights when it comes to organizational transformation.

His presentation was Driving change and building a high-performance DevOps culture and it caught my interest!

Mark Schwartz: Enterprise Strategist, Amazon Web Services.

If you’re not sure who Mark is, you may have heard of some of his books:

The Art of Business Value
A Seat at the Table: (IT Leadership in the Age of Agility)
War and Peace and IT: (Business Leadership, Technology, and Success in the Digital Age)

If you haven’t heard of these books, you now have a new reading list :)

What I learnt (My Notes)

Communication between leaders, middle management (“frozen middle”) and builders
What is the job of QA?
How to do skunk works projects (experiment at fast pace with low risk)
What decisions makers want

How to sell & what execs want
Making better use of data

Driving change and building a high-performance DevOps culture (DOP207)

You can watch the same presentation I saw live — https://youtu.be/o9XgEpOj3f0.

Below is an overview of the slides/content from Mark’s presentation.

Transformational change can be driven from anywhere in an organization.

What’s essential are passion, commitment, vision, and a willingness to take on challenges.

You can categorize transformation in three ways, each with their own pros and cons:

Top-down
Middle
Bottom-up

Leading from the top down

Set a strong vision and goals aligned to it
Remove impediments
Change incentives
Connect and coordinate
Mange relationships up and across
Assume all risk

Mitigating the problem of distance

Set up feedback loops
Walk around
Encourage fast escalation … nuclear option
Manage through Big Hairy Audacious Goals

Leading from the middle

Do what is possible within scope of influence
Free up resource from prototyping/POCs/reference architectures
Create a skunkworks (don’t be sneaky)
Get small wins constantly
Forge alliances
Sell up and across

Leading from the bottom up

Go ahead and make changes
Start small and work incrementally
Use changes to improve your own productivity
Create concrete benefits (e.g., reduce deployment errors)
Lower risk my banking it clear you are invested in POCs
Find collaborators, and share good practices

How to sell

Know what you’re asking for
Frame your idea in terms of how they will lead to successes
Provide actual evidence (like a POC)
Mitigate risk for the “buyer”
Think carefully about the language you use

What do executives want?

They do not care about sprints, backlogs, microservices, technical debt, or even the cloud.

They do care about revenues, costs, risks, competitive positioning.

Today’s big concerns:

Have a growth story (revenue)
Be future-ready to avoid disruption
Mange today’s risks (compliance, security, etc)
Unlock the value in their databases

Wrap Up

You can transform and you can do it today — stop making excuses. Change can be driven from anywhere in your organization. It always involves influencing other people.

Info

Also posted on medium as Driving change and building a high-performance DevOps culture.

Optimizing for DX — The Developer Experience

Sun, 19 Jan 2020 19:18:39 +0000

One thing I’m very much obssessed with is something I’m calling the Developer Experience (DX). I’m literally on a global & lifelong mission to optimize DX.

Hint: Once you’re sold on the benefits of a great DX you can skip ahead to the “3 Steps for success” section.

What is DX and why should you care?

If you’re part of a team/organization that does development then you’re likely going to run into one or more of the following:

Having to keep README’s up to date
Building & testing your code
Bringing new people onto the team & showing them how your code works
Building pipelines & debugging them when they don’t behave
Fixing your dev environment when it breaks
Supporting that one person who wants to use <insert whatever OS the rest of the team isn’t using>
Starting new projects and re-doing the basic building blocks to get started
Trying to support 10 teams all working on their own software but all needing to be able to work closely together (and not re-invent the wheel)

My perfect world!

In a perfect world the mundane parts of development would be simple:

Creating a new project should be fast (I shouldn’t worry about any of the ‘boilerplate’ steps that would be the same between projects)
As a consumer of code: I shouldn’t have to worry about how code is built, tested & deployed
As a developer of code: I should be able to easily update how my code is built, tested & deployed without affecting other developers or build servers wanting to consume my code

If I have to talk to a developer, install some special software dependencies, read a confluence page or README.md, figure out how to get things working on my OS or spend more than 5 minutes getting your software built or a new project setup then you’ve failed to streamline the DX.

3 Steps for success

The’s a concept called 3 Musketeers which is really just an agreed pattern for standardization when building software. It’s most common form uses Make as a standard interface to Compose & Docker Containers but you can adapt the concept to suit your needs.

3 Musketeers
_Test, build, and deploy your apps from anywhere, the same way. Get Started → Run the same commands no matter where you…_3musketeers.io

The net result is a consistent experience that works the same on Windows, Mac & Linux. It’s portable across development machines, build servers and provides consistency that scales from 1 team to 100 teams.

If you’re already using containers in your build server steps or as part of your development then you’re one step closer to a utopia of standardization!

Containers, and the 3 Musketeers concept can be applied any time you need to simplify a process or some instructions. You can consider it “self-documenting”. A good way to check if you’re giving the best DX is to look at what you need to put in your README to explain your process.

No Musketeers

Just

…

Install node version 4 but definitely not version 3, and I have no idea about version 5. I’ve you’re on Windows then I can’t help you at all. There are also a few dependencies that you’ll need but I can’t remember what they were. The following 12 page document is incomplete and you’re about to waste half a day that you’ll never get back.

1 Musketeer — Docker

Just run the following docker command:

docker run --rm -v foo:bar -v foo:bar -e FOO=BAR container bash -c ’echo hello world’

2 Musketeers — Docker & Docker Compose

Just run the following compose command:

docker-compose run --rm alpine echo ‘foo’

3 Musketeers — Docker, Docker Compose & Make

Just run the following make command:

make echo

And it’s as simple as that. Developers typically implement steps like “make build test validate deploy” but you can be as creative as you like and write the steps that make sense for you!

Supercharge your development

Using an agreed pattern like 3 Musketeers is a great way to standardize among your teams. As the number of teams you have in your organization grows so too will the benefits. Humans (and those robot automation scripts) will be able to consume your repos more easily and that’s a great thing.

But why stop there?

What if we could template what a good project looks like and deliver that to development teams in a self-documenting and self-updating fashion?

The most basic implementation of this would be writing a “hello world” application and asking your development copy/paste the code when starting new projects.

But there’s a tool called https://yeoman.io/ and it provides scaffolding for modern web apps. It’s another simple yet impressive concept executed well. What it provides is a templating mechanism with a really slick user-input feature and enough glue in-between to great that amazing DX we’re all chasing.

With yeoman development teams can setup and maintain examples of good software delivery with teams benefiting from the learnings of others. You can create a generator for just about anything, so the only thing left is your imagination!

Yeoman has it’s own generator to help you create your first generator (how meta, right?). The only thing missing for me was a reliable docker container to do some development in. I created a repo with a yeoman development container to make life easier. Hopefully you find it useful and make some steps to improve your own DX!

drewkhoury/devyo
_yeoman container - for local development of yeoman generators - drewkhoury/devyo_github.com

Want more?

Presentation (PDF): Optimizing for DX in a Cloud Native world
Presentation (Video): https://www.youtube.com/watch?v=ZYlbNnaoseI

Info

Also posted on medium as Optimizing for DX — The Developer Experience.

Why Business Value eats DevOps for breakfast

Sat, 11 Jan 2020 22:18:26 +0000

It’s 2020 and my new years resolution is to keep a list of words that deserve to be on the naughty list:

DevOps
Agile
Requirements

It’s not that these words are inherently bad — but my industry has twisted, abused and polluted them leaving behind a trail of over-engineered solutions and misguided teams.

For an in-depth look at “how to do the DevOps” see my post One DevOps Please which talks about how to do transformation and discusses the unicorns that are “DevOps teams”.

First, a little warm up…

I’d like to interrupt this blog to do a warm up exercise. Sorry-not-sorry for anyone I’m about to offend.

Repeat after me:

I am not a DevOps Engineer

Let’s try another one:

I don’t belong to a DevOps team, because I only work with things that are real

One more:

Doing standups every day at 9am doesn’t make me agile

You say these every day along with burpees, planking or whatever your daily routine is. I promise you they’re good for your health.

Why — should I care?

You might have heard the following:

We’ve always done it this way

If “why” is the super hero of this story, then this phrase is the villain.

Like many youngsters I spent most of my childhood asking “why”. Ironically I’m sure my parents asked themselves “why” they had to have one more kid as I pestered them with endless questions about science, the universe, or whatever else was on my mind.

My questions didn’t stop at childhood. I’ve always had an obsessively curious mind and a determination to understand everything inside-out. I did it with helpdesk support, then when I stepped into development, and again when I learnt about cloud and automation.

Helpdesk me:

Why do clients always call with the same MSSQL replication errors

Developer me:

Why does my code work differently in production

“DevOps” me:

Why doesn’t this DevOps thing solve my client’s problems

An important part of what I do today is about helping foster the same kind of curiosity in others, and helping hone in the skills to make the best use of that curiousity.

Examples of times you should ask — Why

Hint: If you’re not sure — always ask why.

You’ve been invited to a meeting, with no description or context, and no idea what the outcome is.

You’re creating a presentation as a team but you haven’t agreed on the audience, the intention or the outcome.

You’ve been told to create a kubernetes cluster.

You’ve been told stand-ups happen at 9am (and they happen to go for 45 minutes, nobody seems to have an understand of what to do for the rest of the day even after the meeting).

You’ve been asked to join a new project, team or a recruiter has told you they think you’re “perfect for Job X”.

Your company has decided they’re moving to the Cloud.

Why we ask why

Okay so that heading was very meta. The intention isn’t to challenge or be confrontational. It’s not a power play or a fun party trick.

Asking why is about having a shared understanding.

In each of the previous scenarios and indeed in many interactions there’s always an underlying intention. Sometimes it’s just about communicating with each other to gain that shared understanding.

Warning: Here be dragons. Sometimes (e.g when people aren’t sure why) asking them can lead to defensiveness and well, make them angry. Other’s don’t like being interrupted or challenged. Some people have values and principals that favor hierarchy over creativity and innovation. Asking why isn’t always a happy path to making friends.

What does all this have to do with Business Value?

Business Value should be well known and clearly articulated in every team of your organization. Everyone should know the why (and how what they’re doing ties back to a very specific business value). Your business should have a way to measure the business value (so you know when you’re done). Everyone should be contributing to the business value metrics, and collaborating to get it done together.

Assuming you’ve done the hard work of figuring out what your business is trying to do (you understand the business value and have some metrics to measure against) the next part can be fun and rewarding!

Why haven’t you talked about DevOps yet?

Let’s assume (like many companies today) you “develop software as part of an effort to provide products & features, to give you a competitive advantage”.

Where I think “DevOps” has failed us is the misconception that it’s “half dev” and “half ops” and that problem lies in the very core of DevOps — It’s name! But in reality without software there’s nothing for “Ops” to do.

Customer leads to Business Value leads to Software. How we build, test, deploy and operate our software is important but we can’t call it DevTestSecQADeployOps so let’s just call it Good Software Delivery Practices.

Good Software Delivery Practices

You’ll likely want to incorporate automation and optimizations that allow you to “produce working (aka releasable) software every sprint” and reduce manual steps in favor of automated ones where rework is likely to occur.

You’ll want to “drive value through the system” and this usually translates into working on new products and features that benefit your customer (hopefully identified by your product team’s close relationship with it’s customers).

Your product/development team might also decide to operate in some flavor of “you build it, you run it” team which encourages them to build good quality software with resilience and alerting that’s actually helpful.

If I know I’m going to be woken up at 3am when things go wrong it’s going to fundamentally affect my choices when designing and operating my software (hint: for the better).

Ultimately when you combine “Business Value” with “Good Software Delivery Practices” every choice you make has clarity. You know “why” you’re doing something and you can trace it back to a specific metric that has a positive impact on your business (and by that I mean improving the lives of your customers in some way).

So when you go to that next meeting, or pickup your next Jira ticket from the “todo” column ask yourself:

Do I know why am I doing this?

Info

Also posted on medium as Business Value eats DevOps for breakfast.

Teaching DevOps in one afternoon

Sun, 05 Jan 2020 18:09:37 +0000

A non-technical friend of mine approached me wanting to know everything they could about DevOps. He asked if he could have a few hours of my to pick my brain and ask me some questions. If only he know how much joy that brings me.

So we used One DevOps Please as our springboard for the conversation. It worked well enough and we got to spider out into other conversations where he had questions. There were many times I had to pause and say “that’s a conversation for another day” so I wanted to capture some of those topics that we didn’t get to delve into. I figured it would be about the same effort to turn it into a blog post and share it with the world.

Existing Blogs

It’s no surprise that I already have a lot of blog posts within this theme. Feel free to explore those, or skip ahead to the new stuff!

What is DevOps
Build your first Python App using Google App Engine
Learn Ansible
Things I learned working for IT Consulting companies (as it relates to DevOps, Agile, Cloud and team building)
The best guide for Agile that you'll ever find
Talking about building a Learning culture

Umbrella Themes

For this post I’d like to break down “DevOps” into three main themes. I believe that the “people” side of things greatly influences how we do the technical parts so in some ways it’s one of the most difficult and most import parts to get right.

People, Culture & Ways of Working

Continuously improve (Growth mindset)
Encourage innovation
Foster collaboration

Measurable

Measuring everything
Make decisions based on data
Make data easily available

Continuous Delivery

Higher quality software through repeatable, predictable delivery
Reduction of defects
Reduce cycle time of ideation (I have an idea) to production (released in prod)

People

Customer Focused — app teams or cross-functional teams with your org may be your customer.

How is your organization structured? https://web.devopstopologies.com/

Agile — no need to be overly prescriptive. Understand where “Agile” came from and what they indended to do with it: https://agilemanifesto.org/principles.html

Extreme Programming: https://www.agilealliance.org/glossary/xp/

The Planning Game
Small Releases
Metaphor
Simple Design
Testing
Refactoring
Pair Programming
Collective Ownership
Continuous Integration
40-hour week
On-site Customer
Coding Standard
Sit Together
Whole Team
Informative Workspace
Energized Work
Pair Programming
Stories
Weekly Cycle
Quarterly Cycle
Slack
Ten-Minute Build
Continuous Integration
Test-First Programming
Incremental Design

Measure Everything

Let’s say it together “measure everything”.

In particular having metrics that are focused on Business Value Drivers are key.

Metrics should be captured and key metrics displayed on dashboards & TVs so they’re easily accessible.

Teams should produce APIs to be consumed internally — https://cachethq.io/.

Site Reliability Engineering.

Particularly for large organizations it can help to understand where each teams is at on their journey. Usually I’ll see teams try and go through some sort of journey of maturity in “DevOps”, “Cloud”, “Agile” or just “Good Software Development Practices”.

Maturity Assessments:

Agile
CI/CD
12 Factor — https://12factor.net/
Data
6R’s Application Treatment Plan — https://aws.amazon.com/blogs/enterprise-strategy/6-strategies-for-migrating-applications-to-the-cloud/

Continuous Delivery

This is really the art of getting code from laptop to production, and yes it can be part art, part science and a little bit of magic sometimes.

https://continuousdelivery.com/

Low risk releases. blue-green deployments
Faster time to market automate the build and deployment, environment provisioning, and regression testing processes
Higher quality automated tools that discover regressions within minutes
Lower costs investing in build, test, deployment and environment automation, we substantially reduce the cost of making and delivering incremental changes to software by eliminating many of the fixed costs associated with the release process
Better products work in small batches. This means we can get feedback from users throughout the delivery lifecycle based on working software
A/B testing & hypothesis-driven approach to product development
Happier teams continuous delivery makes releases less painful and reduces team burnout

https://continuousdelivery.com/implementing/patterns/

Only build packages once
Deploy the same way to every environment
Smoke test your deployments
Keep your environments similar

Trunk Based Development

A branching model to aspire to (or at least understand and learn from): https://trunkbaseddevelopment.com/

short iterations
less than few day story size
short build times
feature flags
humans and CI use same scripts
Compile > Unit Test > Integration > Functional
Quick build notifications
Shift Left — Undesirable to find out about breakages AFTER the commit

Containers

Containers are an amazing innovation that have many every day uses. They can be used locally, in your build servers, for development and production deployments. You can also use them to test new services and keep your laptop free of clutter!

How to build, test and deploy modern code in a repeatable fashion: https://3musketeers.io/

Containers 101…

https://en.m.wikipedia.org/wiki/Kubernetes

https://en.m.wikipedia.org/wiki/Docker_(software)

https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/

https://danlebrero.com/2018/07/09/kubernetes-explained-in-pictures-the-theme-park-analogy/

(Don’t) Build it yourself

The phrase “undifferentiated heavy lifting”: https://aws.amazon.com/blogs/aws/we_build_muck_s/

When it comes to investing time you should consider the implications of “building and managing infrastrcuture yourself” vs “letting specalisits do the heavy lifting”. This might mean different things for different people, but here are some examples:

Configuring AWS Accounts & setting up guardrails: https://aws.amazon.com/controltower/features/
The quickest and best way (imo) to run k8s: https://cloud.google.com/kubernetes-engine/
Do you even need a k8s cluster? Sometimes you just need someone to host your app for you: https://cloud.google.com/appengine/

And that’s (a little more than) one afternoon teaching DevOps.

Info

Also posted on medium as Teaching DevOps in one afternoon.

One DevOps Please — Part 2

Sat, 23 Nov 2019 20:09:40 +0000

“One DevOps Please” — Part 1 recap:

DevOps is part of a learning journey for people
We can categorize how people learn in 4 Stages
The hardest part of the learning process can be a willingness to learn
Companies don’t transform just because we teach a handful of people in isolation (BottomUp)

Organizational Transformation — The path to “DevOps”

DevOps is an overloaded term these days. A DevOps journey might touch on agile, lean, learning, metrics, business value, pipelines, CI/CD, shifting left, 12 factor apps, Cloud, serverless and kubernetes. It can be exhausting! Why is DevOps so vague and buzzwordy? Because DevOps covers the people, process and technology aspects of modern software development. Let’s expand on our original definition from Part 1:

“DevOps is a culture shift or a movement that encourages great communication and collaboration to foster building better-quality software more quickly with more reliability.” — Mike Kavis

But what does transformation look like for an entire company? While teaching somebody how to master automated pipelines could take a few weeks or months, it could take years to see changes that span the entire company. This is because companies are made up of people — each with their own contribution to it’s culture.

Let’s go on the same learning journey we did for people but through the lens of the entire company this time around. We’re going to leverage tried and trusted models from https://web.devopstopologies.com/ to take you on this journey because they’re such a good representation of what we see in the real world.

Stage One — Everything is okay

We start off much the same way we did with the people side of things. The company is operating in Silos like “Dev” and “Ops” and thinking:

“Everything is fine the way it is”

Stage Two — Automate all the things

“Let’s do something and call it DevOps.”

In the next stage the organization recognizes they need to change so they “try to do DevOps”. This is where we typically see a few anti-patterns form as people try to find the sweet spot for their new team structures and ways of working — this is also more common than you might think. We often spend a lot of time with clients at this stage and it’s by far where the hardest work gets done.

So let’s explore the swinging pendulum of DevOps Anti-Patterns and appreciate how easy it is to get things wrong — even with the best of intentions.

Anti-pattern 1 — The DevOps Silo: This happens when you create a “DevOps” or “Platform” team that’s separate from your development teams or cross-functional teams. You’ll write some great automation scripts but transformation doesn’t happen here.

Anti-pattern 2 — NoOps: While this doesn’t seem like a terrible idea at face value, downplaying the importance of operations within software development can lead to poor choices that wake you up at 3am.

Anti-pattern 3 — AllOps: Isolating developers is not going to lead to better software delivery, and most definitely won’t lead to transformation in your organization (unless you really need to spend 80% of your time building that wondering k8s cluster).

So at this point you’ve tried a few different changes and had varying success. Hopefully you’ve seen some improvements along the way and you’re ready to take your organization to the next stage.

Stage Three — Transformation

Why don’t we try the “You build it, you run it” model. Now we’re talking!

Operations: Responsible for providing and operating the platform

Developers: Responsible for operating their applications, which are built on top of the enterprise guard rails.

This is where we usually start to see“shifting left” in terms of QA and Security — and that’s a win for everybody.

Here are some key indicators that your organization is maturing in it’s DevOps transformation journey:

Improved collaboration across departments
Value stream mapping exercises are performed to drive process improvement
A learning organization takes shape
Blameless post mortems
Gamedays
Lean principles are embraced — Eliminate waste, Amplify learning, Decide as late as possible, Deliver as fast as possible, Empower the team, Build integrity in, Optimize the whole

Stage Four — The pursuit of Continuous Improvement

Hopefully by now you’ve realized there’s no perfect Venn diagram to determine the success of your organization. There are however some key indicators you can look at for.

Building on the success from true DevOps transformation in Stage Three you should now be capable of deploying multiple times a day with increased certainty and minimal risk. This means more automated tests and better quality code. Removing bottlenecks should second nature and everything should start shifting left.

What we should see now are these new ways of thinking spreading throughout the entire organization, no longer a “DevOps” thing for the I.T department — continuous improvement is happening in all parts of the business. Instead of teams building up towers of influence to protect their corner of the organization people are working together toward a common goal.

The recipe for successful transformation

So far we’ve focused on “how we learn” and touched on “DevOps” but we haven’t really gone into practical advice for how to transform a company. Offering up a personalized plan in a blog post would be foolish but I can definitly set you off on the right path. If you’re a leader trying to kick-start your own company transformation there are some lessons others have learned that will save you years of banging your head against the wall.

Top-Down is half the recipe — Your company needs to hear the right things from the leadership group, and people/teams/departments also need to be held accountable if they’re not supporting the target-state laid out in the vision by leadership group (actions always speak louder than words).
You guessed it Bottom-Up is the second half — Teams on the ground need to be bought-in on the vision and willing to execute. We’re talking about completely changing people’s working lives and the way they think. Start with teams most willing to change, prove what works in small but significant teams and products, and use them as a lighthouse for the rest of the teams.
Find change-agents/champions — These are people within your organization that are not only willing to change, they’re ready to shout it from the rooftops and inspire those around them.
Don’t just hire contractors — find genuine transformation partners and learn absolutely everything you can from them.

Homework for transformation leaders: Take a deep-dive into how Microsoft, GE and Amazon went on their own transformation journeys (and explore what went well, and what they could have improved).

Transformation Partners needs it’s own section…

I can come in and build you a shiny new k8s cluster, maybe I could take 4 of the brightest developers I know and churn out our own applications faster than most of your existing teams, or embed in your teams and give you a report showing how we increased development by 320%. But if everything goes back to how it was after I leave — you’ve made 0% progress towards transformation (also please stop building k8s clusters because it’s cool — focus on if each of your teams can succinctly explain who their customer is, how they obtain feedback in the shortest loop possible and what business value they’re delivering in each two-week period).

So what does a transformation partner look like (and am I about to sell you my services?). The only two things I need to tell you about the company I work for is “we charge reassuringly high rates” and we “fall in love with our customer”.

If you can hire the right people to inspire, lead and transform your organization please do that. I assure you I’ll still find a way to put food on the table.
If you want a power-boost, a consultancy that’s focused on transformation filled with people who’ve done it before should definitely be on your list — Pair them up with your change agents and the willing development teams in your organization, and watch your growth sky rocket (but be prepared to do to work).

Hiring a consultancy for transformation and asking them to do all the work is like hiring a personal trainer, making them do all of your situps, then wondering why they look great but you still haven’t ‘transformed” — Andrew Khoury

Not keen on a consultancy and feel like you’re company only needs a small nudge in the right direction? Sounds like you want to head-hunt a few key people (when you do that make sure they’re empowered to make the changes you want them to make) and have some key mentors by your side to bounce ideas off and keep you honest.

Summary

Many organizations try to replicate the success of others “doing DevOps” by hiring DevOps consultants or building DevOps teams but they often miss the mark in doing so — the journey is more important than the destination. Let’s recap what we learnt on this journey together:

How people learn: We explored how people learn and why it’s important to take the time to bring them along the journey. Trust and empathy are key here.

Learning from your trusted partners: We touched on how valuable it is to have a trusted partner in your corner, with a focus on drawing from their experience and maximizing how much you learn from them. This allows for new ways of thinking to take hold in your organization which ignites the transformation process.

Organizational transformation: We broke down transformation into four key stages:

Everything is fine the way it is
Automation — but not much direction (this is the hardest stage and where we learn what doesn’t work)
Actual transformation (this is where the hard work pays off — We see Improved collaboration, Value stream mapping, a learning organization, Blameless post mortems, Gamedays, Lean principles)
The pursuit of Continuous Improvement (this is where we refine what’s already working)

Approach transformation with a plan — That means Top-Down & Bottom-Up, identifying and empower your change-agents, holding people accountable for transformation and positive behaviors. Don’t forget to find those willing development teams who are ready to transform into cross-functional product teams with with a healthy obsession for their customers & continuous learning and improvement.

Info

Also posted on medium as One DevOps Please — Part 2.

New to DevOps? Start here.

Mon, 18 Nov 2019 13:51:01 +0000

Some time after I got my first job “in DevOps” (I was one of those shiny new DevOps Engineers and really excited about the role) I distinctly remember a frank conversation with the Head of Delivery…

I turned to him and said “You know DevOps isn’t real, right?” to which he replied “I know” (we were talking about job titles) and in that moment I knew we’d get along just fine.

We went on to have an interesting discussion about what DevOps meant to each of us. He explained that our clients, and the industry were looking for “DevOps Engineers” and the term was more a marketing term (it was much easier than the more accurate title combo I was thinking of for my role “CICD Build Automation Engineer / Cloud Engineer / Agile Champion / Infrastructure Developer / SRE / Application Integration Engineer / DevOps Evangelist / Automation Architect / Security Engineer / Ninja”).

Any developer who works in a team that practices DevOps is a “DevOps” engineer. DevOps isn’t a position, but instead a philosophy that must be adopted by the whole organisation in order to solve problems together.

Today we have DevSecOps, and all sort of buzz words to describe different ways of thinking of DevOps. In reality DevOps is rooted in the continuous improvement of the development lifecycle and that’s why it’s so important to understand the dev component.

DevOps (a clipped compound of “development” and “operations”) is a software development methodology that combines software development ( Dev) with information technology operations ( Ops). The goal of DevOps is to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives. [1]

DevOps Resources

There are so many DevOps resources for those getting started, and I like to keep an up to date starter guide here: https://github.com/drewkhoury/devops-101

While everyone has their own definition and way of thinking of DevOps I’ve includes a few of my favourites:

Definitions

Frameworks

DevOps Bookmarks — There are new awesome tools and frameworks being released everyday. This is an open and transparent attempt at aggregating all those.

Lists

AcalephStorage :: Awesome DevOps — A curated list of resources for Devops
joubertredrat :: Awesome DevOps — This is the awesome list with all open source and free applications that you can use in your management.
Awesome SysAdmin — A curated list of amazingly awesome open source sysadmin resources inspired by Awesome PHP.

Misc

Google’s SRE books: https://landing.google.com/sre/books/
How to annoy a web developer — A list of tips on How to annoy web developers, hopefully for knowing and avoiding them.
DevOps Types and Anti-Types
Atlassian Toolchain
GitHub DevOps Tools Showcase
gitflow
DevOps Checklist
15 Point Checklist
The Must Know Checklist For DevOps & Site Reliability Engineers
xebialabs periodic-table-of-devops-tools
xebialabs — release and deployment pipeline
Why the Best Companies and Developers Give Away Almost Everything They Do
The twelve-factor App

Originally posted on: https://www.linkedin.com/pulse/new-devops-start-here-andrew-khoury/

Info

Also posted on medium as New to DevOps? Start here. and LinkedIn as New to DevOps? Start here..

One DevOps Please — Part 1

Wed, 16 Oct 2019 05:57:48 +0000

One DevOps Please — An Enterprise Journey to a DevOpsy-Cloud

“I’ll have one DevOps please.”

Over the last few years large organizations have been coming to consultants like myself to ask for help “Installing DevOps” into their organization. Some of them just want kubernetes clusters, while others are talking about true transformation.

In this two-part blog I want to take you on a DevOps transformation journey. I’m going to share with you my recipe for success and give you all the secrets to unlocking true business value through people and organizational change with a culture of learning at their core.

Because everyone’s definition of “DevOps” can be slightly different, I’m going to set the scene with a simple definition:

“DevOps is really about eliminating (most) Technical, Process and Cultural barriers to between Idea and Execution — using Software.” — Kishore Jalleda

My DevOps Journey

Before we get stuck into how people and companies transform the way they work, let me start by sharing the journey I took to get here and how it’s shaped my ways of thinking.

I started my career by learning. I strived for technical excellence and perfection through delivery. I poured my heart and soul into my craft which started out as web development but quickly grew into cloud and automation.

One thing I realized was that I loved teaching people new things. What took longer to learn was how impactful learning and transformation could be to both teams and companies.

Nowadays my value comes from enabling my teams and coaching my clients. Our team’s success depends on my ability to foster the right skills, remove the noise/blockers and leveraging my previous experience to set us up for success.

How People Learn

I mentioned earlier that some organizations want to “Install the DevOps”. Astute readers will know that DevOps isn’t a piece of software and so hiring consultants to “Do the DevOps for you” won’t give you long term results. The aim for your organization should be to empower your people by giving them opportunities to learn and grow (hint: that’s where the transformation happens).

So how does learning happen? Can you just send your employees on DevOps training and have them come back 1 week later ready to tackle the world’s problems? Unfortunately it’s not that easy, and here’s why. Learning typically follows the following 4 stages:

Step 1: I don’t need to learn . I’m fine just the way I am.
Step 2: I need to learn — can you help me?
Step 3: I know a little about now. I can start to contribute with some help and guidance, thanks!
Step 4: I know now. I can contribute on my own, and I can teach others too now

This model is known as The Four Stages of Competence and it’s a critical part of any “DevOps/Transformation” journey. If you don’t understand how to take people from step one to step two all your efforts will fall on deaf ears. There’s also something called the Dunning Kruger effect — a cognitive bias in which people mistakenly assess their cognitive ability as greater than it is. Psychology can really work against us!

Now we know that unlocking the very best from your employees involves taking them on a learning journey, and unlocking the mind (steps 1–2).

The exciting part of the journey (Steps 2–4) involves knowledge transfer, which I’ll touch on briefly. Something I’m really proud of about Contino is how we help people on their personal journey through the dual-delivery model. This might sound fancy, but it’s just leveraging practical experience to pair-program with people. We also tend to spend plenty of time at the whiteboard teaching, but most importantly we establish trust and show empathy by working along side our clients and having shared goals.

Getting from Step 1 to Step 2

One of the hardest parts of transformation is getting past the very first step — so it deserves it’s own section.

Getting people to recognize that they need to learn something is the first hurdle. Getting them to be vulnerable in the workplace and admit that they need help is often the second one.

Getting individuals to open up and trust you can take time. My job as a consultant is to deliver, but also to fall in love with my customer. I’m not talking about butterflies in my stomach, I’m talking about caring about my team’s success and how happy they are each day. I’m lucky enough to love what I do and have the opportunity to work with teams I genuinely care about.

But sometimes there are other factors at play and empathy alone isn’t enough. It helps to have a trusted partner along with your for the ride, and the key is to get the most value out of them by leveraging their skills and experience to maximize your learning. Let’s have a look at common pitfalls that may hamper learning with your consulting partner:

Hiring a consultancy and having them do all the work (no learning)
Hiring a consultancy and telling them what to do (staff augmentation)
Your team is “too busy” or you aren’t able to grow/hire within your team (you can’t learn if you’re not present)

Person vs Organizational Transformation

So can we transform a company one person or team at a time?

Well, sort of. It’s often easy to find inspired people in an organization who can be your champions. Investing your own time in someone can be very rewarding and the results can be seen relatively quickly within your teams or sphere of influence. The other benefit is that investing in people (or your team) doesn’t usually require CEO sign-off.

While helping people learn new skills and ways of thinking is a great starting point you won’t be able to achieve large scale transformation without a well thought-out (and properly executed) strategy for your organization. You’ll also need buy-in from your leadership.

So how do we achieve organizational transformation, what does it look like and what are the pitfalls? Find out in One DevOps Please — Part 2 where we’ll deep dive into what it looks like to transformation a company (ie Installing DevOps company-wide).

Info

Also posted on medium as One DevOps Please — Part 1.

Automating Google App Engine

Mon, 14 Oct 2019 13:46:01 +0000

Google App Engine is a pretty amazing service. You write code, and Google will make sure it runs, and scales well. This service was available way back in 2008, well before everyone was getting excited over lambdas and serverless.

In my opinion, AppEngine is one of the most underrated services I’ve seen. It’s beautifully simple, and it has planet-level scaling (Google words). You don’t have to configure load balancers, tweak an OS or install/configure a Runtime. The focus is on optimizing for the developer experience, and having your spend more time writing code.

Even though Google makes it easy to get started with App Engine I wanted to reduce the steps even further. I’ve created a GAE demo that offers a local development and deployment environment via docker containers and simple to use commands.

Google App Engine Demo

Local Development

Creating a local development environment can be done with one command:

1~/code/gae-demo $ docker-compose up local

This is what the console output should look like:

This is what the app looks running on your machine:

The demo app comes with instructions and sample code that adds entries to a local datastore.

One of the amazing features of Google App Engine is the local admin environment that offers similar functionality to the online admin console from Google. The demo comes with everything pre-configured and ready to use.

Local Datastore Viewer:

Interactive Console:

Not bad for one line of code!

Deploying

Google has a wonderful feature called “ Projects” that allow you to group related resources to make them easier to manage. When you run the `new-deploy` command the demo will create a new project and related application, then deploy it for you.

1~/code/gae-demo $ docker-compose up new-deploy

Deploying updates to your application is quick and easy too. First specify your project id from the previous step, then run re-deploy.

1~/code/gae-demo $ export CLOUDSDK\_CORE\_PROJECT=devops-demo-xxx ~/code/gae-demo $ docker-compose up re-deploy

That’s it, why are you still here? You could be spending this time deploying your new application.

You can find the full demo and instructions here: https://github.com/drewkhoury/gae-demo

Info

Also posted on medium as Automating Google App Engine.

Learning Ansible — The quick way

Thu, 10 Oct 2019 13:46:02 +0000

Are you strapped for time but wanting to learn Ansible? I’ve got your back! I’ve been using Ansible for several years now, and was first introduced to it as a replacement to Puppet. It didn’t take me long to learn, but it what really amazed me what how much more it could do!

I’ve used Ansible to generate Cloudformation templates, perform Websphere deployments, collect information on multiple hosts and do simple configuration management for hosts as well as testing software.

Some of my favourite things about Ansible are how easy it is to get started, the ability to set and override variables in multiple locations, templating, looping and the simple use of tags to target subsets of tasks.

Looping, templates and variables in Ansible:

Ansible does a great job of system level tasks that you’d normally perform in the shell or by scripts (think file manipulation, installing software, restarting services) but it can also communicate with multiple hosts, and has modules for Cloud providers too. It’s simplicity is where it really shines.

Head over to https://docs.google.com/presentation/d/1jIsDHf-5M1w_KWAyU_R5Uv-iHwvRntUb4cC3yBW3_ME/ to learn more about some of my favourite Ansible constructs and how you can use and how you can use Ansible to automate some of your daily tasks.

Ansible Links

http://docs.ansible.com/ansible/latest/index.html — Ansible Documentation
http://docs.ansible.com/ansible/latest/modules_by_category.html — Module categories
http://docs.ansible.com/ansible/latest/list_of_all_modules.html — All Ansible Modules
http://docs.ansible.com/ansible/latest/YAMLSyntax.html — YAML Syntax … read this a few times, it’ll come in handy
http://docs.ansible.com/ansible/latest/playbooks.html — Ansible Playbooks
https://zaiste.net/posts/ansible_101/ — A nice 5 minute hands-on intro to Ansible
https://gist.github.com/andreicristianpetcu/b892338de279af9dac067891579cad7d — Ansible cheatsheet, a great reference point for just about any bit of Ansible code you’d need to write
https://www.ansible.com/blog/ansible-best-practices-essentials — Best practices
https://serversforhackers.com/c/an-ansible2-tutorial — Detailed Ansible tutorial with explaination and code.
https://gist.github.com/phred/2897937 — pedantically commented playbook

Originally posted on https://www.linkedin.com/pulse/learning-ansible-quick-way-andrew-khoury/

Bonus Points:

If you liked learning about Ansible I have a git repo full of DevOpsy learnings https://github.com/drewkhoury/devops-101

Info

Also posted on medium as Learning Ansible — The quick way.

DevOps in 10 minutes with Google App Engine

Wed, 09 Oct 2019 13:16:01 +0000

Okay, now that I have your attention let’s talk about Google App Engine for a minute and how you can achieve your DevOps dreams with a fully managed serverless application platform.

App Engine history

App Engine launched 10 years ago as a tool to run web applications on Google infrastructure. App Engine only supported Python at the time, but the service offered dynamic webserving, persistent storage (Datastore), and APIs for authenticating users and sending email.

Fast forward to 2019 and the App Engine offering has matured. The standard environment is optimised for rapid scaling (sudden and extreme spikes of traffic) or apps intended to run for free or at very low cost.

Python 2.7, Python 3.7
Java 8
Node.js 8, and Node.js 10 (beta)
PHP 5.5, and PHP 7.2 (beta)
Go 1.9, and Go 1.11 (beta)

If you need something else, the flexible environment offers Python, Java, Node.js, Go, Ruby, PHP, or .NET, or any other software that you can run in a container.

But it’s more than just a code runtime engine. Google markets App Engine as having zero-config deployments and zero server management, allowing you to focus on writing code (that sounds like my kind of DevOps). App Engine offers Monitoring, Logging & Diagnostics, Traffic Splitting and Application Versioning (those features you normally hear about when talking about “DevOps” or CI/CD).

Getting started with App Engine

Assuming you already have a Google Cloud account, creating a deploying your first App Engine App consists three easy steps:

Create a Project to store all your related resources
Create an App
Deploy your App

gcloud projects create project-demo-xxx \
--name="project demo xxx" --labels=type=test

gcloud app create --region=australia-southeast1 \
--project project-demo-xxx

gcloud app deploy --project project-demo-xxx

Normally, navigating a new Cloud vendor, a new product, and new concepts and troubleshooting demo code can be time consuming and error prone. I know, I’ve gone through this process myself.

If you’re interested in giving App Engine a go (and I encourage you to do so, because it’s a fantastic example of what an integrated serverless platform should look like) you can head over to https://github.com/drewkhoury/gae-demo and try it yourself. I’ve streamlined the whole process to try and get you up and running as quickly as possible:

How to authenticate with Google via the command-line
Taking advantage of the local environment Google provides (dev_appserver.py)
Sample Python App that’s optimised for App Engine
Docker containers / Docker compose for consistency (local development and deployment)

Let me know what you think of Google App Engine after you give it a go!

Info

Also posted on:

medium as DevOps in 10 minutes with Google App Engine
and LinkedIn as DevOps in 10 minutes with Google App Engine

Things I learnt working for an I.T Consultancy

Sun, 06 Oct 2019 21:05:20 +0000

This was an old blog post draft that I planned on cleaning up. In the end I decided to post it as the raw stream of thoughts that it is. Each one of these thoughts could be its own blog post.

Things I learnt working for an I.T consultancy, doing the DevOps, building all the platforms, leading teams, and annoying a bunch of people along the way.

aka

Ramblings from a “DevOps Engineer”.

Resource Augmentation vs Outcome based work (the ongoing struggle of chasing high-value outcomes for clients).

The art of getting people to realize stuff (you might know what the “right” answer is for a given technology choice or next step for a project, but how do you bring people along for the journey and give them a chance to input their own ideas).

Hiring, promotions and bonus/incentives.

360 feedback — getting honest feedback, and getting it much more often that every 12 months
how your bonus is allocated — and why the worst time to talk about your bonus is when it’s being handed out
Why/Why/How you should negotiate $ — know your value, understand the ‘buckets’ of money in companies and the incentives from the companies perspective. Most important understand what your drivers are and what will make you happy. Is is just money, or do you have other wants?

Early in my career I used to think that process was the key, and I overlooked people and my teams.

Agile gone bad. How some so amazing has been weaponized over and over. There are literally hundreds of examples of this e.g: “Let’s start reporting on story points per person so we can compare individual performance”.

What’s wrong with LinkedIn, and (some) recruiters. And why my inbox looks like a dating app (with message after message of poorly thought out copy/paste solicitations).

Incentive to send messages (to meet quotas)
Incentive to get you to leave your job

How you should fill a role:

Ask them what they need done, use that to drive requirements for a role
e.g fix our pipelines, scale our app, reduce bugs in our app (by half, e.g from 50 to 25)
write an advanced thank-you letter to give to applicants/new starters so they have a clear goal of what to strive for
Let people pick their team members

How do you mange your projects? (hint: It doesn’t matter what software you do/don’t use — it’s more about how you manage blockers and how focused the team can be). Physical wall vs Virtual (JIRA board), this choice will be specific to the team and it’s needs. You may need to compromise on level of detail — that’s okay.

AWS. Their approach to development teams. Get on-board for building APIs so other teams in Amazon can consume them, and how it launched them into the biggest Cloud provider today.

Trust the people you hire. If you’re hiring people you don’t trust you’re doing it wrong.

One thread on LinkedIn from a hiring manger had the attitude “I’ll pay you 50% market rate until I trust you”. I really hope they were trolling.

As Steve Jobs said: Hire smart people to tell you what to do.

Train/Grow people. Don’t be worried about “What happens if we train them and they leave?” … be worried about “What happens if we don’t train them and they stay”.

Remote work. Pros/Cons.

Pair programming optional, Reviewing code mandatory.

Platforms. “Built it and they will come” is some of the biggest wastage in I.T departments that I’ve seen.

Documentation, too often it’s seen as not important but just like testing it should be part of the work you do not an “I’ll do it later” type thing.

Start thinking of your internal teams as customers (see AWS note about APIs).

Track/share Metrics. Make decisions through data.

Vision is important, have one, or follow someone who does.

Being talented isn’t always a good thing…

e.g being right and proving someone wrong in a group setting
intimidating someone by showing off how smart you are to them
challenging tech leads, delivery leads, sec, ops in a way that gets them off-side

DevOps teams & how they’re AntiPaterns. https://web.devopstopologies.com/

The power struggle between Ops Sec and the rest of the company.

Some people don’t want change.

There’s no one magic solution, or one magical person that can drive transformational change. Support is needed from the top, but also needed at all levels (It’s great to have support from the c-level, but if ops/sec blocks things you’re not going to have a good time).

Technology is actually pretty easy to work with, people are more complicated. If you manage teams meet with them regularly, care about them, and get out of that office when you have your catch-up. I had most amazing Delivery Manager who would catchup with every single team in our company every two weeks. He’d ask about what’s good/bad/blockers and what and what he could do help me. He also asked how I was feeling and if I was challenged. Mostly he actually cared about me and my life because each conversation felt like one with a good friend — and years later we still keep in touch.

Info

Also posted on medium as Things I learnt working for an I.T Consultancy.