Note: This is the english version of Jeppe’s qed.dk blog post
Typos and minor text improvements added on the 20th of June 2021.
There is an interesting change in the approach to SOA and it comes from an unexpected area.
I have worked with SOA (a great acronym which really says nothing as it has been watered down) for over 10 years and through my time I have seen companies build more and more hierarchy (or rather layers) and structure into their SOA landscape as a way to achieve “business agility through business and IT alignment and increased recycling”. If you tend to get a gag reflex by hearing that phrase, I feel you.
The idea and the intention is good, but the execution is in many places is to the grade F-. Instead of having achieved agility through loosely coupled services, these companies instead end up with hard-coupled services, an expensive and cumbersome organizational Enterprise Service Bus (ESB), which is placed centrally to coordinate the hundreds to thousands of services we have built. It is a bit like herding sheep with a chest freezer and a rope tied to each sheep.
Herding sheep using a chest freezer and rope
It’s messy and unstable. Not unstable as in the ESB is unstable, but unstable in that we’re desperately trying to create order out of chaos without understanding the mechanisms of action.
Let me make it clear, I don’t believe SOA, nor the ESB, is to be blamed for these failures. In my experience it has been the way SOA was implemented and deployed that has caused the majority of problems.
Everyday life in any major company offers a torrent of new service contracts or amendments to existing service contracts. New systems are purchased and systems are being built to replace the old systems, with the result that there is now one extra system in addition to the old system that often ends up remaining in operation. The growth is organic, but the way we handle it is by introducing more hierarchy, bureaucracy and coordination. It is the forces of nature that rule and govern, and as you know, if you choose to pee against the wind you get wet.
So what do you typically do to control this disorder? You hire Enterprise Architects who can draw stylized diagrams of how the organization and the systems should be. You hire data architects who can draw canonical models across the business in the hope that we have time and money to ensure that everyone has the same understanding of concepts such as a customer. Most of it is has proven to be a waste of money and time.
What else do we do to solve the problem? We introduce a central organizational unit that is responsible for managing the ESB and who heroically, meaning that work hard to make this work, try to ensure that services can talk to each other and new integrations gets built. We have inserted an intermediary who needs to understand the business , service the customer and service providers needs and requirements. That is a lot to expect of an intermediary.
All the control and influence can very easily end up with the ESB group, which thus forms an organizational- and project related bottleneck. This is where responsibility, or in many cases the lack thereof, lives and it is where we quietly end up moving more and more of our business logic in the form of orchestrations, also known as centralization of logic.
Slowly our underlying services gets transformed to CRUD (Create / Read / Update / Delete) Data Services, which gets orchestrated by the ESB. Orchestrations definitely has its place, but it’s my experience thatmany of these orchestrations become unmanageable and entangled.
But surely it cannot be bad?
We achieve high levels of (data) service reuse and isn’t that ‘s why we wanted SOA in the first place?
How many can recognize the following SOA (architecture) picture?
Layered SOA (“best practice”)
What if we drew the same responsibilities in this way?
Layering with a Database
In my eyes both solutions achieve the same result with approximately the same high degree of data and logic coupling. The SOA solution has a slightly weaker technical coupling, at the expense of lower performance and more complex transaction handling. This increases the cost and makes the SOA solution much more technically complicated. We have virtually replaced the database transaction handling and table joining functionality with SOAP, XML and HTTP (or REST, GraphQL for that matter). And no, the solution is not to make the database a key player again. The whole problem has been turned upside down and there are in my opinion much better solutions.
It’s surprising that layered SOA has been touted as best practice when it in reality contributes to higher coupling, both in data coupling and temporal coupling, high latency and heavy technical baggage in order to handle updates of multiple autonomous services?
In the next blog post I will delve more into the problems of layered SOA, the challenges of the centralized ESB as well as how a different approach to service design perhaps can help.
But what about the interesting development, what it is and where it comes from? This will be covered at the end of the next blog post 🙂
Jeppe Cramon's Software development blog 
Nice post Jeppe!
LikeLike