In Ansible State of Mind – Part 1, we discussed how automation is the first step of the development process. In this post, we want to go into what to consider before we implement automation or Continuous Delivery (CD).
If achieving Continuous Delivery means reaching the Emerald City, where your product delivery and engineering fantasies come true, automation is the Yellow Brick Road that leads you there. As you follow the Yellow Brick Road, you move beyond writing scripts. When you arrive at the Emerald City, you’ll be able to go from “manual” to “push-button” deployment, yet still be able to shift product delivery gears if the situation demands it. Before we start any implementation, here are a few things to evaluate when assessing your organization’s automation maturity:
- Are you automating first? (Covered in the previous post)
- Have you integrated automation as part of development lifecycle? Do engineers actually get sprint cycle to work on automation? Is the automation artifact, which could be a process, a template or a script, part of the acceptance criteria?
- Are you hiring the right people with the right mind-set? Especially since senior DevOps is such a hot/rare commodity, are you training your team with the right approach to Continuous Delivery?
- Do you already have a mature Continuous Integration process? Are you measuring unit test coverage? Are you running static analysis (for security, style, code standards)? Are you versioning/caching artifacts in a way that can support your ideal deployment process?
- Is the delivery part of the pipeline connected to upstream? In other words, is the downstream/delivery part of the pipeline automatically retrieving the latest artifacts and enjoying the inherited confidence through built-in tests.
- Can you catch configuration drift? Are you able to tell the difference from one server to another?
- Are you choosing technologies that can effectively support automated delivery? This circles back to “Automating First” mentality again.
- Do you already have a reliable way to rebuild a system? How are you versioning artifacts or tracking upstream packages/libraries?
- Can your systems survive a reboot? Putting systems to sleep is the easy part:
- Can your application be shut down and brought back up gracefully? (No, “kill -9” is not the right way, nor does it send the same signal as Ctrl-C.)
- Do you have start-up scripts arranged in the sequence that will satisfy all your inter-application dependencies? Or better yet, service discovery set up in such a way that start-up order doesn’t matter, and that service outages are handle gracefully by dependents without intervention.
- Can your source control management (SCM) handle the following three scenarios?
- Deploy a new server/instance/container
- Update an existing server/instance/container
- Self-bootstrap an auto-scaled instance/container
- What is the extent of existing configuration management? What configurations are you tracking and versioning? Are your abstracting/layering configurations for simplicity or security?
- Is your config repo organized and branched in a way that will support multiple environments and/or multiple silo’s in an environment?
- Is your access control built for automation? Can you access your servers securely without manual intervention or exposing credentials? Can your servers access artifact and config repositories securely without manual intervention or exposing credentials? Do you have your asset delivery orchestrator positioned at a place where it can access all of your environments with different credentials? Can you check out only the necessary configs from your SCM without downloading the whole project?
- What can you validate at each step of the deployment?
- Can you validate network connectivity from an upstream component to downstream?
- Do you perform OS/container level health check?
- Do you have wait conditions that depends on whether an application container is ready?
- Do you have different levels of health check built into your application so that monitoring host, network devices (i.e. load balancer, WAF, etc) and deployment orchestrator can do a final smoke test?
- Are you operating on X-as-code? Are you leveraging an infrastructure, platform, or container as code service that has ample API or CLI support?
After taking all things above into consideration, we now can set out to build a CD process. After completing building multiple evolutions of CI/CD practices, we find that introducing and building a strong CM practice is a very good entry point for building automation for Continuous Delivery. Adopting a CM practice early forces you to think automation and what you want to accomplish to support a CD practice eventually.
We named this blog series Ansible State of Mind because using Ansible requires a paradigm shift that we particularly appreciate and hence have adopted when we rolled out Ansible (the config management tool) for a client. “Ansible state of mind” is about focusing on the target state; It is about a configuration driven deployment philosophy that works with any kind of architecture and across all environments. It is about having a CM repository that not only provisions but also audits and maintains your deployments.
It’s worth pointing out that having a strong config-driven deployment practice does NOT prevent you from having an immutable infrastructure. Always re-deploying (instead of updating) a virtual instance/container is just a way of achieving immutability. The configuration repository is, and should will always be, the source of truth for your infrastructure.
Building a CD process is really about the journey along the way. Architecting the Yellow Brick Road is challenging and fun, and it is what Ten Mile Square excels at. Looking for automation wizardry to ship your product faster and more reliably? Give us a call!
Jason Mao and Rob Giseburt authored this post.