Twelve years ago, when I started working as an engineer in IT, there was one sacred word that left no one indifferent: deployment. “A deployment is scheduled,” “I’m preparing a deployment,” “I have a deployment tomorrow” would make everyone in the tech and product teams shiver. It was the event that mobilized all IT teams. If your boss entrusted you with the responsibility of deployment, it meant he had enough confidence in you and that you had proven yourself.
Ah, deployments! For some, they were something stressful and dreaded, for others, they were an opportunity to show everyone (often other departments) who runs the show. In that world, deployments were planned well in advance and all departments were informed.
I was long among those who hated all this protocol around sometimes minor changes. I hated having to wake up early in the morning to follow a procedure that sometimes didn’t go as planned: errors in the procedure, unidentified bugs in testing, performance issues, database crashes, DNS switches that don’t propagate… In short, a catastrophe.
This ceremonial allowed some to remind everyone of their importance in the infrastructure, taking great pleasure in playing the savior when things went wrong. They didn’t shy away from mystifying the process, often neglecting to document some important points to know.
I had such bad memories of the deployment ceremonial that when I started my company and built most of the infrastructure, I was obsessed with these questions. We should have a system where everyone is capable of handling a deployment without it being a big deal.
“How to ensure that deployments are not events? How to ensure that everyone in the team can deploy new application versions (even interns)? How to make dozens of deployments a week without stress?”
So, I’ll share what I implemented at HK-TECH, which allows us today to perform dozens of application updates daily, transparently for the teams.
We have implemented highly automated processes (without procedure) so that the whole team can handle their application deployments without the mental burden or sacralization of something as mundane as updating an application version. We have killed the deployment myth.
1. Make it simple
This is the basis of everything. The applications we build are those we have control over. We avoid third-party products we don’t have advanced knowledge of.
2. Docker, Kubernetes, and micro-services
If we can’t have an application that fits into a Docker image, either we break it down, or it’s trashed. We have banned VMs and chosen to be fully containerized for applications (exception for CI/CD runners).
3. Manual foundation, automatic applications
Manual in this case means it’s not delegated to the CI/CD runner, and automatic is the opposite (CI/CD runners handle it). The foundation of all our infrastructures is executed manually, for example, creating/updating a new cluster, while the entire application update process is automated (Docker build, versioning, deployment…).
4. NoOps
Everyone can deploy a new version. Each dev, according to their rights, can deploy and rollback versions. Gone are the days of the Ops Guru putting on their superhero cape to do deployments.
5. Share Knowledge
No secrets or gurus holding all the trade secrets. Everyone is aware of the architecture, the infrastructure, and how things work.
6. Regular updates of foundational elements
One of the major problems that cause deployments to go wrong is often the obsolescence of surrounding bricks. We ensure every month that our technical foundations are up-to-date and don’t hesitate to upgrade them (sometimes at the risk of breaking some applications), but this choice brings us peace and tranquility daily.
7. Monitor only what’s important
We developed our own tool that exclusively monitors the critical points of our applications according to our standards and alerts us through the right channels. I remember the horrors of a production with 40 alerts for the same product, which sometimes weren’t helpful when the service was down. An alert is a real alert. No false positives that become the norm.
8. Go to Production quickly
We made the choice, sometimes against the grain, to say that with every small change, we go to Production. This helps us identify problems quickly and, most importantly, fix them immediately. You can have 5, 6, 10 test or staging environments before production, but they will never be Production. We chose to have a single non-production environment and sometimes even go directly to production depending on the application’s criticality. This has only advantages: we don’t have to maintain an unnecessary fleet of servers that cost us time and money.
That’s how, with a limited team, without full-time Ops, we have performed nearly 300 productions on about twenty web and mobile applications since the beginning of the year.
For your web and mobile software needs or building your cloud infrastructures, feel free to contact us at contact@thehktech.com. We will be happy to assist you!
I’m Hervé-Gaël KOUAMO, Founder and CTO of HK-TECH, a French tech company specializing in designing, building, and optimizing applications. We also assist businesses throughout their cloud migration journey, ensuring seamless transitions and maximizing their digital potential. You can follow me on LinkedIn (I post in French mostly there): https://www.linkedin.com/in/herv%C3%A9-ga%C3%ABl-kouamo-157633197/
