I have sought to change organisations, with some success and not a few acknowledged failures. Personal reflection on this experience has been useful and has allowed me to form something that amounts to 'a theory of change'. This is a grand claim, and I understand that this theory is, of course, weakly supported by the sort of evidence that I would demand of others making similar claims.
Before an organisation can change there must be a broadly shared perception that 'things could be better'. Organisations that are hubristic, that display excessive self-confidence, or conversely are hopeless, will be unable to change. It does not matter that there is no consensus on what exactly could be better or the route to better, it simply matters that there is a shared existential dissatisfaction with the status-quo.
The individuals within an organisation must believe they possess 'agency' that is the capacity to act and they must feel that they 'own' the problem that is to be addressed by change. In other words they must feel that it is for them individually and collectively to do something.
Agency and ownership must be accompanied by a sense that the resources to accomplish the change are within reach. Not necessarily within easy reach. A common pattern for comfortable avoidance of change is to form unreasonable expectations about the resources required for change and how they are to be secured.
Critical to the question of resources is the capacity to create the organisational bandwidth for change. Very few organisations believe this bandwidth exists at the start of a change journey, and indeed it may not. It will have to be built, a process that in itself can be quite disruptive.
There must be a compelling vision of the future beyond the change. It is best, naturally, that the vision is created by the organisation as a whole, and broadly subscribed to. This may not always be possible and a key function of change leadership is to shape and convey that vision.
I recall a senior colleague setting out his approach: "Organisations are made of memory-metal ... you can, with the application of a small amount of force, bend them into a new shape ... but once a small amount of heat is applied they will snap back to their previous conformation. Only if you superheat them will they assume a new shape permanently." This is a powerful analogy, it is important to move an organisation to a new equilibrium otherwise a change will not stick. Durable change entails applying significant energy to the system.
It is to be accepted that not everybody will subscribe to the vision, dissent or discomfort are acceptable, but there will need to be an early and visible limit placed on negative or obstructive behaviours. Similarly 'consent and evade' will need to be addressed openly, it can sap organisational will more certainly than outright opposition.
Change often entails loss and universally entails uncertainty. Early on, the losses must be confronted, and the period of uncertainty, possibly extended needs to be acknowledged. Losses of organisational narratives and losses of individual self-esteem and position are the hardest to bear and will require the most attention.
It is important that what is lost or set aside is recognised and memorialised. It will often have been constructed and preserved with pain and effort. The moment needs to be suitably marked perhaps even celebrated. A positive orientation to the future need not be accompanied by a wholesale rejection of what has gone before.
Finally, organisational change is not, contrary to what my prior observations might have suggested, an abstract 'thing', it is highly contingent and contextual, the place and the moment matter and have to be accounted for.
In summary we need “… a bespoke combination of Lenin and Burke, with the 18th Brumaire as the bridging mechanism”. Thanks ‘Anonymous’!
I like your thinking on change. I've been through, instigated and tried to enact change in various civil service organisations. (And would probably say these are representative of many other organisations) . I think it is all, ultimately, about people and the exercise of power in the workplace. Here are some thoughts:
Change is very often something done by senior people to (rather than with) more junior people. The senior people may have good reasons (perceived inefficiency, responding to external pressures, a shift in strategy) or these are a cover for less good reasons (wanting to look good to the boss, feel important, get promoted, accrue more power, pure ideology, needing things to be tidier, or wanting something to focus on and being uncomfortable with feeling bored, or in the worst case creating a crisis in order to be the hero fixing it). They've got where they are by taking risks: they like risk. I've seen changes being driven by all these reasons.
So more junior people often resist change. They may not trust that their seniors know enough about their jobs to get the change right. It seems risky, it might make things worse. They may have seen changes come and go and be cynical about this time. They may be exhausted from the last round of change and just want to focus on their jobs uninterrupted. They feel they haven't had any choice, or been consulted, and become disengaged. The uncertainty may make them feel anxious about their jobs and livelihood/ identity.
The change is often then badly enacted, without good two-way communication, dedicated change resources, or managing the actual process. Whoever is put in charge thought it was a great opportunity for glory and attacks the challenge they've been handed enthusiastically, but soon lose momentum. Nobody is incentivised to enact the change. People put their heads down and try to wait it out.
The senior people get frustrated at what they see as naysayers, blockers, risk aversion and evasion, leaving them with the choice of ignoring, or addressing head on, what they see as poor behaviours. And so change quickly becomes conflict; and all too often become personal. If it goes sour enough, with recriminations and people leaving, someone senior might step in to resolve it, cancel the planned change, move the instigator of the change along, and try to reinstate peace. Seldom will a very senior person back the change in the face of difficulty. Or some kind of watery compromise is reached, change in name not action, allowing everyone to save face.
And so creating and landing real change becomes very hard, if not impossible. I think change is sometimes necessary, but all too often not and people are right to challenge. When it is necessary, it needs commitment both to the outcome and the process, in spite of and expecting conflict, and while doing everything possible to bring people along and empower them, but being prepared to have grown up conversations about whether where the organisation needs to be is right for some of the people in it: and if not to support them in finding a place they will be happier.
Ultimately change makes organisations ask themselves whether they are prepared to make some people unhappy in getting there, and if the cost is worth the benefits.