Vicious and virtuous circles in enterprise architecture practice

In my conversations with EA practitioners, predominantly in Australia, I often hear the stories of disbanded architecture teams and reorganised EA functions. Many companies experience periodical ebbs and flows in the number of employed architects and the overall enthusiasm towards enterprise architecture. Anecdotal evidence from the United States¹ and Europe² suggests that such failures and restarts of EA practices are far from being uncommon there as well. Some time ago, Gartner even estimated that ‘as many as 25% of all organisations may be in this restart situation’^{3 (page 2)}. At the same time, in many companies EA practices seemingly operate rather smoothly and evolve organically without being a subject of constant reorganisations.

Why do EA practices in some organisations run more or less steadily and productively, while in other companies they represent a long series of intermittent fiascos and relaunches, hopes and disappointments? My analysis of various aspects of EA practices reveals a number of factors and mechanisms in their work that form a positive (i.e. self-reinforcing) feedback loop, which essentially turns an EA practice in an organisation into a dynamic system with two possible stable, self-sustaining but opposite states: self-improvement and self-destruction (importantly, the discussion in this article relates primarily to strategic planning as the most prominent process of an EA practice, but is less relevant to other EA-related activities, e.g. solution delivery, repository maintenance and technology standardisation).

In the state of self-improvement, an EA practice stays on the beneficial track of a virtuous circle, when its achievements and successes facilitate further achievements and successes. By contrast, in the self-destruction mode, an EA practice lapses into the dangerous rut of a vicious circle, when its failures and reorganisations contribute to successive failures and reorganisations. The existence of positive feedback mechanisms and the ensuing virtuous and vicious circles partially explains the curious situation with EA practices widely observed in the industry.

Specifics of enterprise architecture practice

EA practices represent complex organisational undertakings involving multifarious actors, activities and instruments. Although they have many unique features distinguishing them from other organisational practices, for the purposes of our discussion it is critical to understand specifically the following three of their features: the primacy of partnership relations, lack of universal approaches and difficulties in evaluating performance.

1. The primacy of partnership relations

First, the core element of an EA practice, undoubtedly, is communication. As EA practices represent joint business and IT planning efforts, they imply active engagement between enterprise architects and senior business managers. Effective engagement, in turn, necessitates finding a common language, reaching mutual understanding and building trustful relationships between these parties, which is easy neither for architects nor for their business counterparts. On the one hand, for enterprise architects, participating in strategic conversations requires not only a deep immersion in the business intricacies, but also establishing a strong personal reputation of trusted partners capable of ‘talking sense’ in the eyes of business leaders. Gaining sufficient business knowledge and earning positive reputation takes considerable time and effort (understanding the corporate IT landscape usually happens much quicker and easier). For instance, some architects and architecture managers participating in my research opined that in a large company, it takes up to two years for any enterprise architect to fully fit into the business and stakeholder context, become truly influential and productive.

On the other hand, for business leaders, accepting enterprise architects as full-fledged members in their inner circles also requires overcoming some deep-seated mental barriers and cultural prejudices, as well as effecting certain changes in their attitude. First, business managers should realise the strategic role of IT and be prepared to participate in IT-related discussions. Second, they should start perceiving architects as real business partners, rather than merely some subservient IT technicians. Finally, business managers should learn to deal with enterprise architects and appreciate their input into planning activities. Therefore, EA practices in organisations, to a large extent, build upon a delicate ‘social capital’ and accumulating this capital demands initiative and deliberate effort from both the business and IT sides.

2. Lack of universal approaches

Second, besides the central element of communication discussed above, an EA practice also includes many other peripheral elements intended to support effective collaboration and decision-making, e.g. processes, documents and approval procedures. As with most management-related disciplines, there are no one-size-fits-all recipes, definitive methodologies or universal approaches for organising an EA practice with all its nuances suitable for every company and situation. EA practices are always idiosyncratic in all their aspects, including roles, artifacts, modelling notations, communication patterns, governance arrangements and even software tools. Establishing a successful EA practice, thus, requires finding organisation-specific ways of getting things done. EA practices generally cannot be imitated from some ideal specimens, copied from other companies or brought into an organisation by outsiders unaware of its internal specifics. Instead, they need to be grown within the organisation by its architecture team based on a genuine understanding of its unique climate and needs, common sense, experience, experimentation and learning.

At the same time, the absence of any universal recipes for establishing an EA practice is readily compensated (or exacerbated, to be more precise) by the presence of countless techniques of questionable applicability actively promoted by their commercially driven salesmen as ‘best practices’, if not ‘industry standards’. Most egregiously, The Open Group widely touts TOGAF as a proven EA methodology and ArchiMate as a standard modelling language for EA-related purposes, persuading a naive audience to believe that 'best practices have been summarized in frameworks like TOGAF and modeling languages like ArchiMate'^{4 (page 25)}, whereas in reality TOGAF with its numerous predecessors proved impractical decades ago⁵ and ArchiMate is actually inappropriate for communication with senior decision-makers⁶. Rampant faddism, irresponsible marketing, the abundance of fictional EA best practices, flawed approaches and misguiding recommendations, as well as ample and aggressive misinformation on enterprise architecture in general, all contribute to the creation of an inebriating atmosphere of sweet deception surrounding enterprise architects in organisations. Instead of staying sober, thinking on their own and following the difficult but the only right way of developing customised planning approaches suitable for a particular company based on its specific needs, architects are constantly offered an ‘easier’ way in the form of ready-to-use but predatory techniques that will most likely ruin their EA practices. Believe in the promises of The Open Group, listen to siren songs of omniscient EA gurus, get fascinated with the thoughts of thought leaders, succumb to the temptation to implement the methodology of a world-famous consultancy, follow the latest fashion - and your EA practice will almost certainly fail.

3. Difficulties in evaluating performance

Third, contextually, EA practices exist to improve the overall quality of decision-making relevant to business and IT alignment in organisations. As they do not produce any material outputs and have no real KPIs, evaluating their actual performance is highly problematic. Even though the effectiveness or futility of an EA practice can be rather evident in the long run, its short-term productivity is nearly impossible to assess objectively, let alone measure quantitatively, on both individual and organisational levels (except for the cases of sheer inadequacy). On an individual level, it is often hard for an enterprise architect to tell whether, or to what extent, his or her activities and efforts are effectual or not, and if not, how exactly they should be improved. For example, an architect can develop various architecture principles, target states or investment roadmaps relatively quickly, but only in a more distant future can it be understood whether these plans are good or bad as well as whether they actually work out as intended, e.g. are respected, followed, resisted or ignored. For this reason, enterprise architects typically cannot adjust their behaviour promptly based on some instant performance feedback, but need substantial time to discern the causal relationships between the exerted efforts and the resulting outcomes. Since the effects of actions are delayed and spread in time, genuine learning for an architect is always a long-term process.

Organisationally, the value of an EA practice, or the lack of thereof, can also be not particularly visible in the short horizon. On the one hand, problems with messy IT landscapes misaligned with business needs simply cannot be resolved swiftly, but require persistent work over a long period. For this reason, even a well-functioning architecture team often cannot demonstrate any tangible outcomes immediately. On the other hand, even if an entire EA function is eliminated at once, the organisation may not notice any instantaneous impact on its operations or degradation of its performance. Hence, both the presence and absence of architecture teams are manifested and become perceptible predominantly in the long run, while their current productivity levels can only be felt subjectively by business and IT leaders based on some weak and unreliable signals. Because of the significant inertia of architectural efforts and the subjectivity of their results, there is arguably no easy way for organisations to understand how efficacious their EA practices today really are, whether something in their work should be amended and how.

Vicious circle in enterprise architecture practice

Together, the three characteristic features of EA practices described above are conducive to the formation of the vicious circle of never-ending reorganisations, failures and restarts. Imagine a situation when an EA team considered non-value-adding was purged leaving a bitter aftertaste of disappointment and scepticism towards enterprise architecture throughout the organisation and a new group of architects was recruited instead to relaunch the failed EA initiative. What is likely to happen in this situation?

The newly hired enterprise architects, even if they are competent, for objective reasons, are not in a position to start productive work from day one. They can neither determine the right way of organising an EA practice without adequately understanding the company with its specifics and needs, nor enter into a constructive dialogue with its business leaders straightaway without acquiring sufficient business knowledge and accumulating the positive reputation of trusted partners, on both collective and personal levels. The former problem is exacerbated, on the one hand, by the general tendency to follow fashions and imitate others in the face of uncertainty and, on the other hand, by the ongoing flood of new ‘panaceas’ peddled by consultancies and gurus which, at best, distract architects from searching for suitable approaches for the needs of their organisations and, at worst, entirely substitute these searches with the selection of latest industry fads. The latter problem is aggravated by the burden of notorious fame of enterprise architecture lingering from the failures of previous EA efforts, which discourages business managers from communicating with architects ⁷,⁸. For these reasons, even proficient enterprise architects may need considerable time to ‘warm up’, study the organisation and its business, understand its decision-making context and build relationships with its key stakeholders before being able to make a valuable contribution (not to speak of unskilful or inexperienced ones).

At the same time, at the early stages of the EA initiative neither enterprise architects nor organisational leaders are able to tell with confidence whether the undertaken actions are correct and whether they are actually on the right path to establishing a successful EA practice. For example, the initial promising agreements reached between architects and business managers, for whatever reason, can be subsequently broken, while the developed good-looking EA artifacts may turn out unusable, irrelevant and eventually end up in a wastebasket. And, even if the nascent EA practice is in fact moving in the right direction, due to its long-term orientation, it often cannot demonstrate any convincing outcomes to assure its status and secure further executive support (not to mention the situations when the practice is going astray).

For the reasons explicated above, the first steps of an EA practice are usually associated with significant uncertainty, surrounded by the aura of confusion and can be metaphorically compared with wandering in the darkness or navigating without a compass. Both business leaders and architects are having doubts about the success of the endeavour, but nobody knows for sure what to do. Numerous techniques are advertised, but very few, if any, of them will work as claimed. Whether the adopted approaches will prove effective is unclear, but there is no easy way to find that out. Experimenting by trial and error may be necessary, but reliable performance feedback to inform learning is unavailable. Continuing top-level sponsorship of EA efforts is required, but whether the organisation is heading towards a successful EA practice is obscure.

In the situation described above, given the infamous reputation and sceptical attitude towards enterprise architecture ensuing from earlier failures, the executive support of the emerging EA practice has a high probability of being discontinued, even before giving it a chance to wade through the initial turmoil and evolve into something valuable (and because of the objective difficulties described above, it may actually never happen). In this case, the EA team will be disbanded, all lessons learned by its members will be forgotten and any social capital accumulated between enterprise architects and business leaders will be wiped out, while the reputation of enterprise architecture in the organisation will be further spoiled. Consequently, the EA function will sooner or later undergo another round of reorganisation only to start again from scratch but, this time, in worse initial conditions than before, thereby increasing the chance of successive failures and reorganisations. The EA practice in the company, thus, descends to the next turn of its vicious circle and converges to a detrimental equilibrium.

Virtuous circle in enterprise architecture practice

A virtuous circle in an EA practice represents a situation quite opposite to the one described above, when exactly the same underlying mechanisms work in the opposite direction and reinforce earlier successes. Business leaders trust enterprise architects, consider them as real partners, appreciate their efforts and are willing to deepen cooperation. Architects, in turn, understand the strategic context, know how to communicate properly with business stakeholders to contribute their expertise and, thereby, further strengthen their reputation.

For you

Be part of something bigger, join BCS, The Chartered Institute for IT.

Being working in the organisation for a while, architects learned a great deal about its unique internal specifics from their own first-hand practical experience. For this reason, they are capable of making sober judgments about the potential applicability of various planning techniques in this particular company, which allows them to propose reasonable enhancements in the adopted approaches and ignore useless industry fads.

Organisationally, senior business and IT executives value the contribution of their EA practice, realise its importance for doing business and are ready to invest additional resources in its further development. As a result, the practice gradually evolves towards greater maturity via well-thought-out incremental improvements.

Swerving from a vicious circle to a virtuous circle

The natural question that arises next is what should an organisation stuck in a vicious circle do to swerve out to the healthy trajectory of a virtuous circle? Unfortunately, as with most aspects of an EA practice, there are no simple, straightforward or universal answers to this question; no blanket prescriptions can be offered. However, certain ideas and considerations that may aid companies to increase their odds of surmounting the difficulties and getting out of the quagmire still can be provided. Some of these suggestions are pretty concrete and intended specifically to break the separate links of a vicious circle and, thereby, help tear the circle itself, whereas others are more general and address the overall philosophy of an EA practice.

More specific suggestions

First, one of the major contributors to a vicious circle is the negative reputation of enterprise architecture demotivating business leaders from dealing with architects, thus, undermining further EA efforts. Since after repeated failures the very term ‘enterprise architecture’ may sound repugnant and is likely to have a negative connotation, organisations can try practicing enterprise architecture without calling it so. As Gartner analysts fairly notice, ‘many restarted programs find that the negative ‘baggage’ associated with the term ‘EA’ is too strong to overcome, and it is simply easier and more effective to call it something else’^{9 (page 4)}.

Second, another important contributor to a vicious circle is the lack of organisational awareness among newly hired architects that creates considerable problems on the way to establishing a successful EA practice. Obviously, architects that have just joined a company cannot understand its unique context in all aspects (e.g. strategy, structure, stakeholders, politics and operations) to commence productive work immediately. However, organisations can at least try to minimise the necessary learning and expedite the onboarding processes by recruiting architects with previous experience in companies with similar characteristics, most importantly, from the same industry sector, maybe of a commensurable size and with analogous culture. Knowledge of industry-specific terminology, business specifics and regulatory framework can raise the status of architects in managerial circles and facilitate partnership relations.

A vicious circle also forms due to the inability of enterprise architects to properly adapt planning approaches to the needs of a particular organisation. This problem is partly caused by the poor understanding of the organisation among newcomers and can be somewhat alleviated by attracting architects from similar companies, as discussed above, but the problem itself is much broader than that. As an EA practice is characterised by rather vague causal relationships between actions and effects, long-term orientation and general ambiguity, distinguishing ‘the signal from the noise’ in its work requires real wisdom that can be acquired, perhaps, only through observing the development of EA practices in different organisations for substantial periods of time. For this reason, newbie architects and traveling EA consultants spending no more than several months in each client company without being able to observe the long-term consequences of their efforts arguably cannot have that long-range view required to understand what approaches can actually prove effective and become institutionalised in a specific organisation. Therefore, organisations can increase their chances of breaking a vicious circle by avoiding hiring novice enterprise architects that can boast only their recent TOGAF certifications as well as seasoned EA consultants with tens of ‘successful’ short-term planning engagement (many of which are likely to have brought only symbolic value or even produced nothing but shelfware), and headhunting sage enterprise architects with a proven track record as permanent employees of EA functions in different companies, who realise the organisation-specific nature of EA practices, have seen many successful and abortive cases with their own eyes and are capable of picking suitable planning approaches for the needs of their organisations.

Finally, a vicious circle closes when an organisation purges its architecture team deemed unproductive, discarding all gathered experience and rudiments of social capital, only to restart its EA practice from scratch in potentially worse initial settings. As the short-term productivity of enterprise architects is difficult to appreciate, even if their efforts will eventually lead to a successful EA practice, and a new EA team may not be in a better position than their predecessors, decisions to cancel and reorganise an EA practice should not be made easily. Namely, organisations arguably should not rush to stop their EA practices prematurely, for example, if they cannot demonstrate value after the first N months of their existence, but only if they are stuck in one place, ceased their development and do not evolve anywhere. However, the timely replacements of individual architects with tarnished personal reputation might be beneficial.

More general suggestions

Besides the specific suggestions for breaking a vicious circle provided above, adopting a number of broader ideas for shaping the right mindset can also be helpful for maturing EA practices. First, establishing a well-functioning EA practice requires a deep understanding of an organisation with its idiosyncratic needs and adapting the applied planning approaches to these needs. Mindlessly mimicking other companies, aligning with ‘industry standards’ and trying to implement random techniques X, Y or Z appearing in news feeds is not going to help (but is likely to harm), only earnest reflections on what approaches can really work in a particular situation and why can lead to success, e.g. how exactly the EA function should be structured to fit the business structure¹⁰, what EA artifacts might be necessary to underpin decision-making processes and what level of architectural agility is desired¹¹. For this reason, organisations should invest in self-awareness to better comprehend their own features, peculiarities and demands, as well as in organisational learning to analyse their own experience with enterprise architecture and introduce thought-out improvements in their EA practices.

Second, EA practices represent continuous activities that get embedded into organisations and are performed there on a permanent basis, rather than temporarily or intermittently, and many of their characteristics, properties and effects tend to unfold gradually over time manifesting clearly predominantly in the long run. For example, social capital enabling partnership between enterprise architects and business leaders needs time to grow. Achieving self-awareness and benefiting from organisational learning also take time. Even realising perceptible value from practicing enterprise architecture requires some time. For this reason, organisations should adopt long-term thinking in relation to their EA practices that implies, but is not limited to, viewing them as strategic assets rather than investments with fixed costs and returns, building durable architecture teams staffed with permanent employees, not temporary contractors or consultants, and seeking lasting value instead of immediate payback.

Lastly, an EA practice represents a subtle and fragile matter that is difficult to create, but easy to destroy. For example, social capital requires time and perseverance to accumulate, but is quickly eroded by the turnover of managers and architects. Expensive lessons learned from tough EA-related experience also leave the organisation together with its members. Institutionalised processes and commonly accepted procedures established over months or years can fade away rapidly as a result of drastic transformations. For this reason, companies should treat their EA practices with care, stabilise the number of architects, avoid their reckless reorganisations and try to introduce deliberated, balanced and evolutionary changes instead. Hysteric, dramatic, hasty or simply arbitrary modifications in an EA practice are likely to cause disruption and thereby inflict harm.

Beyond vicious and virtuous circles

Potential vicious and virtuous circles in an EA practice, as well as the relevant advice for dealing with vicious circles, are shown schematically in Figure 1.

Click the diagram below to view a larger version

Figure 1. Vicious and Virtuous Circles in Enterprise Architecture Practice.

The discussion provided in this article and summarised in Figure 1 may create an impression that an EA practice can only be in one of the two possible binary states: vicious circle or virtuous circle. Of course, this is a significant simplification; the reality can be much more complex than that and the situation in organisations is rarely either ‘black’ or ‘white’. EA practices, as very sophisticated and multifaceted phenomena, can actually be in a virtually infinite number of different states ranging anywhere between the two extremes described above, i.e. combine various features of vicious and virtuous circles. However, articulating vicious and virtuous circles in their purest form helps clearly explain the internal mechanisms of an EA practice enabling the existence of these circles and vividly illustrate them at work.

Moreover, it cannot be claimed that the discussion of mechanisms contributing to the formation of vicious and virtuous circles presented in this article is complete and exhaustive. For example, there are likely to be some other undiscovered factors or forces generating positive feedback in an EA practice and, thus, conducive to self-perpetuating states. Needless to say, the article also does not intend to present a comprehensive overview of all possible problems that can hinder EA practices, but concentrates only on a few general problems that drive EA practices into endless vicious circles.

Likewise, this article does not pretend to provide universal or guaranteed advice as to how organisations can escape vicious circles in their EA practices, but offers only some broad pertinent considerations. Its core message can be formulated simply as follows: please, be aware of dangerous vicious circles in an EA practice and do not repeat the expensive mistakes of companies spiralling in these circles for years.