Cloud Integration 101

Bob Ivkovic, Principal of IT Architects

January 22, 2018

Before we jump into cloud integration, I have a foundational question: What is cloud? We better understand what it is that we’re integrating before we dive in. I think this is a fair question considering that cloud is a misunderstood term as it applies to information technology. Most people seem to think that cloud is a type of technology, which is partly true, but I tell people that it’s an economic business model. What do I mean by that? Basically, it’s a business model that allows organizations to manage or outsource their IT while they subscribe to the IT services that allow them to carry on their business. Thus, outsourcing applications also means outsourcing the interfaces between these applications, which can co-exist in the cloud as well as on-premise. It’s not always justified to move everything to the cloud. The reason that organizations outsource their IT is because it has become too difficult and expensive to run their own IT shops, effectively and efficiently. They’d rather focus on their business than their IT resources and capabilities. Thus, they’re no longer interested in buying software licenses and owning their IT assets outright, but rather paying a subscription fee to use those IT assets and pay for how much they use, while someone else manages these assets on their own premises, also known as the cloud.

Some technology experts insist that cloud is a modernization strategy allowing organizations to deploy their applications and interfaces on a third-party SaaS (Software-as-a-Service) platform rather than on their own on-premise application landscape. And the term “third-party SaaS” means that it is far away from an organization’s physical location – most likely, somewhere in a cloud managed by someone who’ll be paid handsomely for it. I’d say that one of the greatest benefits of cloud is the increased level of comfort that businesses are going to gain by having their applications and data outside the security of their on-premise firewalls. Like I already said, businesses can hand-off all their IT resources and focus on their business – this includes the hand-off of their system integration landscape.

Simplifying Cloud Integration

Although cloud seems to have a viable value proposition to most organizations, there is a snag: it’s not always that black and white, while integration is very grey. Specifically, we can’t always get everything into the cloud and tell a cloud vendor, “Here it is, all of it, so go ahead and manage all the applications that used to be in our application portfolio……and don’t forget the interfaces”. So, what’s the problem? There’s a lot, which probably deserves another article, but let me give you a couple that seem to be at the forefront. One is that a lot of organizations don’t’ want to give up control of their applications, or at least some of the mission-critical ones. These applications may house some very confidential data, and God forbid that any of it get out to the public, like the Ashley Madison hack where infidelity and perpetrators were exposed publically. Unlike Ashley Madison, there are also companies that store regulatory data, which is prohibited from existing in a public cloud. This is why companies have private clouds, which are internally managed. However, there is another big problem with cloud, and that’s having a hybrid cloud solution. This means that some of your apps are in the cloud being managed by cloud vendors, while others are on-premise and the responsibility of the company. Let’s face it, we have some applications, also called systems of differentiation, that are unique to an organization and cannot be managed well by cloud vendors. Besides, it’s a company’s super users who can do a better job of and want to do manage these differentiated applications for business optimization reasons. The problems is not only being able to integrate all the applications in the cloud, but also integrating them with applications that are on-premise. IT Archtiects, our company, is an expert on cloud integration and the reason that we are busy working with these companies to optimize their integration landscape regardless of where there applications live – on-premise, private cloud, public cloud, hybrid cloud, or even Mars.

So what does this all mean for companies that want to take their IT into the cloud? It’s important for business and IT management to understand that yesterday’s integration approach of manually recreating point-to-point integrations from scratch is no longer fast enough to keep pace with an organization’s rapid growth and integration requirements, especially if we want to be in the cloud and have someone else take responsibility. A more robust approach to application integration is required in order to keep up with a company’s changing business model and its need to adapt to a changing competitive market. There are several considerations to simplify cloud integration as companies transition from a complex application integration architecture (i.e. whatever we have now) to a simple and agile integration platform (which we should have soon if we’re going to be competitive with other companies).

5 Considerations to Simplify Cloud Integration

Like the title says, there are five considerations to keep in mind when simplifying cloud integration. Each of these five approaches is a topic in itself, but I’ll give you the short version so pay close attention. And please remember that these considerations are part of a collective integration philosophy based on an organization’s integration maturity and knowledge base in migrating interfaces and integration solutions to the cloud. I’m pretty sure you’ll take into account all these cloud integration considerations if you want to make your cloud implementation a successful one.

1) Take Advantage of Prebuilt Integrations
Here’s a good philosophy for those who want to streamline their interfaces and integration solutions: since commercially-available integration technology has now hit the market and we no longer have to build interfaces and integration solutions from the ground up, let’s leverage these state-of-the-art integration platforms. This will save us a lot of time and money required to develop interfaces and integration solutions. Furthermore, we are now in a position to leverage this technology for common solutions and avoid what many integration specialists refer to as point-to-point interfaces. The operative word here is “reuse” whicmeans that if we are sending orders from one system to another, we can use one interface regardless of whether it is a sales order, purchase order, transfer order, or whatever other order it is. And, when a department within a company needs an interface to pull orders in from its central ERP, it can leverage an interface from another department that pulls order data from the ERP, regardless if it is pointed at another departmental or target application.

Prebuilt integration frameworks provided by vendor integration platforms offer not only run-ready integration solutions based on different integration patterns that move all kinds of data – such as customers, vendors, assets, orders, inventory adjustments, journal entries, and other master data and transactions types – but also provide the Application Programming Interfaces (APIs) that allow data objects to be created from a source application and loaded directly into a target application. We won’t even talk about all the extra features these pre-built integration solutions provide such as guaranteed data delivery, error-recovery, archiving, push/pull event and time schedulilng triggers, and so on.

2) Get on the Crowdsourcing Bandwagon
This crowdsourcing deal is all about trust, specifically being able to trust where the data is coming from before a company loads source data into its own target applications. I like to use the eBay example where we purchase products on eBay from unknown sellers, especially those with no reviewer ratings and live in foreign countries. It’s no problem buying from a seller with a satisfaction rating of 99% or higher. Trusted data is a matter of having confidence in the seller. Most retail sites provide this confidence if they want to get our business. However, applying this confidence concept into our interfaces/integration solutions is at the heart of a vendor’s integration platforms. For instance, Oracle Integration Cloud Service (ICS) provides crowdsourced input of previous users of Oracle ICS – including users, customers, and partners. Thus, the more users who have matched and activated a data pair between two applications, the higher the recommendation to trust the data. This is a best practice that many organizations are looking for when trusting data from external sources, and becomes very important in procurement and sales systems.

3) Bypass Configuration through SaaS
Companies are tired of time-consuming and error-prone procedures of configuring their integration platforms prior to integrating their applications. The auto-association of Software-as-a-Service (SaaS) applications removes all this heart-ache and gives companies the ability to bypass the complexity of setting up the integration platform for these applications. It also removes the need for specialists and an army to do all this. As more cloud-based applications and services continue to be added to the integration platform, companies are constantly looking to streamline and automate their business processes in order to simplify cloud integration complexities. Remember, cloud-integration solutions available as Platform-as-a-Service (PaaS) have a time-to-market advantage over a company’s on-premise integration platform by eliminating the need to install and configure the platform for specific company needs. However, we’re still not there yet; setting up the platform to be ready to connect to applications is still a big problem. Specifically, it is difficult to determine what security protocols an application requires, and the application-specific security credentials for the integration platform to create, read, update, and delete data. Furthermore, it gets tricky identifying whether there is just one instance of the application deployed or several, and which one is the correct one to be used for integration. I’d say companies that do a lot of integration figure things out more easily by creating an application catalogue, which by the way, is another feature of cloud integration platforms.

4) Empower Users to Integrate Applications
The biggest problem with today’s integration platforms is the fact that you have to be an expert integration developer to use many of its highly advanced integration features. This is all changing with cloud integration platforms, which are catering to end users. This means that non-IT staff – finance, sales & marketing, HR, inventory management, etc – can collaborate with the IT guys to deliver integration solutions to market much faster. Business experts using the applications (or power users who know the ins and outs of the applications) can spearhead application integration without relying on the IT guys, and leaving all the heavy-lifting (i.e. advanced integration solutions) to the integration developers. These new cloud integration platforms have the technological advances that allow users to build the integration pipes between applications, while the IT experts provide guidance and ensure standardization of integration solutions across the enterprise.

5) Prepare Cloud to On-Premises Portability
As more and more application infrastructure goes into the cloud, the higher the benefits: lower costs, faster deployment time, and increase in scalability (both up and down) to meet an organization’s growing demands. Let’s face it, some companies are bound by corporate policies and industry regulations regarding the sensitivity of its data, thus prohibiting its data to be in a public cloud. In such cases, this data could be transitioned to an on-premises private cloud infrastructure. Some data just isn’t meant to be in a public cloud. This also applies to multinational companies that do business in several continents and require a federated integration strategy that takes into account the needs of geographical regions. Just imagine the complexity involved if the global integration solution needs to merge several countries into one, which requires all the different platforms (based on different architectures, standards, and software) to be consolidated.

There are three main factors to consider when assessing complexity and merging existing integration environments into a single cloud-based integration platform:

1. 1. 1. Software Standards between Public & Private Cloud-based Models:
         In general, the integration software components are the same: application connectors or programming interfaces (also known as APIs), a transformation mapper, data enrichment, data filtering, data distribution, integration monitoring, distributed data access/query, and some others. This similarity in integration software components should make migration easier. However, depending on what integration components a company uses today, migration can be complex, especially if legacy and archaic integration code exists. In this case, integration experts will be required to refactor the integration solution into the cloud.
      2. Architecture Standards between On-Premise & Cloud-based Integration:
         As in the prior point, if the data and application architectures are the same between on-premise and cloud, portability will be straight-forward, regardless if it is a private, public, or hybrid cloud. If not, let the vendor experts help you develop a standard cloud architecture and port everything from your on-premises environment.
      3. Business Process Standards between On-Premise & Cloud-based Integration:
         Again, if the industry standard business process modelling notation is the same (such as BPMN), and the same business process execution language (BPEL) can be generated for on-premise and cloud platforms, this simplifies reuse of components when migrating from an on-premise to a cloud integration platform. If you’re not using BPMN and BPEL, then a lot more time and effort, as well as help from the vendor, will be required to move to a modern-day business process standard.

So there you have it, a bunch of things need to be considered when taking integration into the cloud. Of course, the devil is in the details, and hopefully cloud-based integration solutions have the wherewithal to help an organization transition from an on-premise to cloud-based integration platform. And don’t forget, the vendor providing a cloud integration platform is always there to help with it’s own group of cloud integration experts. We’ll talk more about the devil and the details in upcoming advanced articles. I promise.