Integration Happy vs. Unhappy Paths
One of the biggest mistakes that I see companies make when setting up an integration for the first time is that they only consider the integration "happy path". They fail to consider all of the possible "unhappy paths", which result in frustrated employees and ultimately unhappy customers.
What is the integration "happy path"? The "happy path" is the pathway that data flows through when when everything works as expected. Take the example of integrating an order from a webstore like Magento to an ERP like Acumatica. The happy path creates a new order in Acumatica whenever a new order is created in Magento. The integration selects the correct customer and the correct items so that the order created in Acumatica is identical to the order created in Magento. The majority of the out-of-the-box, pre-configured, pre-built, "plug-n-play", and dare I say "native" integrations that are available on the market today typically handle the "happy path" very well.
But what about all of the "unhappy paths"? In other words, what about all of the possible scenarios that you are not thinking of that are needed for the integration to work properly for your company? For example, what if you offer a promotional discount? What if certain customers or items are not taxable? What if you offer free shipping for orders over a certain amount? What if a customer specifies a different billing or shipping address? These are all things that you need to think about when building an integration. By turning these "unhappy paths" into "happy paths" and testing through each of these different scenarios, you can minimize the risk of something going wrong with your integration.
Unfortunately there are more "unhappy paths" to consider that go beyond different use cases or scenarios. What happens if the integration attempts to create an order in the ERP and the customer or one of the items purchased does not exist in the ERP? What happens if there is a temporary loss in connectivity that prevents certain data records from syncing in the correct order, or not syncing at all? What happens if one order fails to sync - does that prevent other orders from syncing? What if someone goes into the webstore and adds a new coupon code that fails to sync correctly to the ERP? What if the order changes after it syncs? These are all different "unhappy paths" that must be accounted for, too.
Now I know what you might be thinking - with so many possible "unhappy paths", is it worth integrating in the first place, or should I just find a system that can "do it all"? While finding systems that can do most of the work without the need for integration, its impossible to find a system that can "do it all". And if a system claims to "do it all", it probably does not "do it all" very well. So the need to integrate is inevitable. And a working integration is achievable, you just need to choose the right integration solution that helps you manage the "unhappy paths", including the unforeseen ones because those will prevent you from sleeping at night.
When choosing an integration solution or platform, in addition to figuring out all of the possible "happy paths" and if the integration solution can handle them, you need to understand how the integration solution handles the "unhappy paths" (especially the unforeseen ones). Here is a list of features that you should look for in an integration platform that helps with "unhappy paths":
Configurable: In order to handle all of the possible use cases that may be unique to your company, the integration must be configurable. You should be able to map any field between systems and adapt the integration workflow to your business requirements.
Logging: Having the ability to review everything that has happened is helpful when troubleshooting. Logging is a necessity, but it also can be overwhelming sifting through logs to figure out what happened. So while logging is important, by itself its not enough.
Quarantine: When issues arise in the integration, there needs to be a way for that issue to be isolated off to the side and allow other data to continue to flow. If the integration completely stops when an issue occurs, every small problem becomes a very big problem very quickly.
Self-Healing: Having the ability to resume where you left off and automatically retry when issues occur is extremely helpful. Often times, integration "unhappy paths" are caused by intermittent issues like temporary loss of connectivity or even scheduled system down time. When things stop working due to these types of issues, the best integration platforms can resume where they left off and heal themselves.
Visibility and Health Monitoring: The ability to see exactly what issues currently exist with an integration is paramount to resolving them quickly, and ideally prevent it from happening again. The system should only tell you what is currently wrong, which is different than a system telling you everything that did go wrong (and could have possibly fixed itself). If you have to sift through the logs to figure out what happened, you could spend hours chasing the wrong rabbit down the wrong hole. The system should be telling you exactly where you need to look.
Our staff has been building integrations for over 15 years with various types of systems, including ERP, eCommerce, CRM, as well as many custom and homegrown or specialized systems. One of the reasons why we built AcuSync (an Integration Platform designed specifically for Acumatica) was to help our customers build highly configurable integrations that provide the tools necessary to handle any type of "unhappy path". Every one of our customers that has an ERP has integrated that ERP with some other system, and they all use AcuSync to do that.