Understand the Role of Culture and Resilience
Learning Objectives
After completing this unit, you’ll be able to:
- Identify the three types of organizational culture.
- Use failures to your advantage.
- Carry out blameless postmortems in response to failures.
- Explain the importance of designing systems for resiliency.
Impact of Culture on DevOps
It's often said that DevOps is primarily about culture. But what does that mean? What is culture?
Culture is psychology at scale. When we communicate, we share ideas and feelings, and those feelings can rouse emotion and intentions in others. When we spend time working together, we attune to one another and influence each other. And as people enter and leave the organization, they influence it and shape its culture.
Organization leaders have a particularly strong influence. Humans, like other animals who live in groups, look to their leaders to guide their actions. Because they have a massive impact on their followers, for better or worse, leaders must constantly refine their leadership skills.
Westrum Organizational Culture
Ron Westrum conducted an influential study of culture in organizations such as airlines and healthcare. He discovered that he could predict safety outcomes by looking at how well information flowed in organizations.
Westrum identified three types of organizational culture: pathological, bureaucratic, and generative. Pathological organizations focus on power, and use fear and threats to maintain control. Bureaucratic cultures are characterized by rules, and those who set the rules focus on protecting their departments and turf. Generative cultures, by contrast, focus on mission, and other concerns are subordinated to the mission.
Information flows differently in these three types of cultures. Pathological organizations are likely to suppress unflattering information, and people are reluctant to share insights because they fear punishment. Bureaucratic organizations may hoard information or address issues in a narrow way. Generative, performance-oriented organizations thrive on information, and they actively solicit and share input.
These cultural characteristics are summarized in the table below.
| Pathological |
Bureaucratic |
Generative
|
| Power-oriented |
Rule-oriented |
Performance-oriented |
| Low cooperation |
Modest cooperation |
High cooperation |
| Messengers punished |
Messengers neglected |
Messengers trained |
| Responsibilities shirked |
Responsibilities limited |
Responsibilities shared |
| Collaboration discouraged |
Collaboration tolerated |
Collaboration encouraged |
| Failure leads to scapegoating |
Failure leads to justice |
Failure leads to inquiry |
| Novelty crushed |
Novelty leads to problems |
Novelty implemented |
The State of DevOps reports repeatedly point to generative culture as central to the effective practice of DevOps. Many technical and cultural practices of DevOps, such as continuous delivery and Lean product development, promote such a culture. And a generative culture in turn supports organizational performance and the four key metrics described earlier.
Failing and Pivoting
When you set goals, you take the chance that you won’t achieve them. Subconsciously, we may avoid risk of failure by not setting goals, but only in the pursuit of goals can we uncover the inner and outer obstacles that stand in our way.
But what happens if we fail?
Remember that people with a fixed mindset take failure as a sign that they don't have what it takes to succeed, while people with a growth mindset conclude that they need to apply more effort or change their approach.
Pathological culture punishes failure, while generative culture treats failure as an opportunity for inquiry and improvement.
And scientists respond to an unsuccessful experiment by refining their mental models and generating new hypotheses.
Notice a trend?
The Lean Startup Mindset
That fail-then-pivot trend plays out in the Lean Startup community. Starting up a new business is risky: The vast majority of ideas don’t turn into successful products, and the vast majority of startups fail to thrive in the market.
Eric Ries, author of The Lean Startup, frames this as a learning process: You learn whether your product or service fills a need, and you learn whether people will pay enough for it to sustain your business. This is the most valuable knowledge you can gain, so you want to validate your product idea in the fastest, safest, most economical way possible.
Established businesses that develop new products face the same challenge. You can spend months or years developing products without first validating the product idea with actual customers. When we first start doing something, we know the least about that activity that we’ll ever know. Nevertheless, it’s easy to assume we know what people want, and how to build it. When we’re wrong, failure can be expensive.
We can apply the Lean Startup mindset to developing features for our users. Creating and validating quick prototypes and getting fast feedback helps us learn. Many ideas won’t succeed, but each failure gives us insight into our users.
Blameless Postmortems
When generative cultures face failure, they meet it with inquiry. The blameless postmortem meeting is a powerful example.
What’s a blameless postmortem? When an incident occurs, the team gathers to dig into what led to the incident. They resist pinning blame on any one individual. If the group identifies human error as a cause, they ask more questions. What situation led to a human making this kind of error? Can we prevent that situation by showing information more clearly or building automated safeguards?
The goal is to better understand the circumstances that led to the problem so the team can find the best remedies. While there can be a single failure, there’s rarely a single cause. When teams see all the circumstances that led to a failure, they can apply multiple safeguards to prevent future incidents.
An effective DevOps practice depends on building a just culture where people are treated fairly and mistakes lead to insight instead of punishment. It’s only in an environment free from fear that information can flow, and we can collectively identify systemic problems that underlie failures.
This approach has been key to improvements in fields like manufacturing, healthcare, and transportation, where safety is paramount. Digital technologies also have tremendous potential to impact human health and wellbeing. As software developers, we can learn a lot about how to respond to failures from these other fields.
Building for Resiliency
When teams build their first application or server, they just want to get it working. That alone may seem like a miracle! Then, as teams mature, they recognize that they need backups or some form of disaster recovery. But many teams don’t test their recovery plan, hoping instead that the system will never crash.
If you use the package development workflow, Salesforce scratch orgs encourage you to build for resiliency. Because they expire, you have an incentive to capture changes in version control and to make sure you can easily recreate the environment based on configuration such as Scratch Org Definition Files. Copado lets teams automate every aspect of scratch org setup, including installing packages, loading data, and running scripts.
Similarly, in the org development workflow you should track sandbox contents in version control and automate the setup process, so you can refresh with minimal disruption.
Joan decides to help her team become more resilient by practicing rolling back production changes using Copado. They begin by setting up Git snapshots of their production and testing orgs. The team holds training exercises where multiple people practice rolling back deployments to their testing orgs. They also create a checklist for the team to follow in case a rollback is needed. That checklist ensures the team can move quickly without forgetting important steps in the process.
