When companies need to integrate their in-house applications and SaaS applications, old tools and ad hoc methods won't stand up to challenges in orchestration, security, data transformation and more. Old ways die hard, however, so many organizations are behind the curve on enterprise-SaaS app integration, say Gartner Research and 451 Research analysts.
What do organizations have to do to make on-premises and cloud applications play well together? Software integration experts reveal why traditional software integration approaches fail and which new tools and strategies work.
SaaS providers should have bitten the bullet and given their customers integration capabilities, rather than forcing customers to buy them from a third party.
vice president, Gartner Inc.
Application integration got lost in the "cloud rush," said Benoit Lheureux, vice president of research firm Gartner Inc. in Stamford, Conn. Hustling to catch up with virtualization and cloud advances, many organizations failed to perform due diligence on how they'd integrate and synchronize data between on-premises and cloud apps. Ironically, Lheureux added, virtualization and cloud are driving the requirements for integration.
The fact that SaaS vendors don't provide a complete integration solution came as a nasty surprise to many early adopters of cloud apps. "There was a misperception that because every cloud provider had an API, app integration was auto-magical," said Lheureux. Cloud vendors' APIs aren't the answer in most cases, because they're frequently proprietary, long and complicated, he said, referring to Force.com APIs with more than 500 pages of specs.
IT organizations also assumed cloud application integration could be done with traditional integration technologies, like extract, transform, load (ETL), external EDI and FTP services for integration between applications and types of businesses.. The problem is that these tools are expensive, require a lot of skills and aren't necessarily designed to work the way a cloud system would work, said Carl Lehman, who covers enterprise architecture, integration and BPM for 451 Research in New York.
For the last five years, Lheureux has seen companies dealing with integration on a one-off, ad hoc basis every time they got a new SaaS app. "Until you get a few cloud projects under your belt, you don't appreciate just how complex integration is," he said. "Now they have a mess and must start applying a rational discipline across all types of applications."
Hurdles in cloud, enterprise app integration
IT organizations that want to integrate multiple SaaS solutions from multiple vendors with multiple on-premises applications face complex problems, said Lehman, including the following:
- ensuring transformation as data goes from one system to the other, from on-premises to the cloud;
- maintaining the connections and the data exchange with the same level of security that organizations had in the enterprise;
- having the right types of connections to accommodate the volumes of data and interaction between the two systems; and
- orchestrating data flow to get the right data to the right system at the right time, and, in doing so, creating a comprehensive, cohesive information scheme and not silos of information.
Orchestration is the most-needed integration functionality, in Lehman's opinion. Users need a graphic visual interface that allows them to look at the status of quality, security, liability and accuracy of all the data to be able to manage and orchestrate them accordingly.
SaaS providers' app integration capabilities
Today, many SaaS providers specialize in their applications' unique capabilities and don't address integration, Lheureux and Lehman said. They advise users to carefully assess the integration functionality SaaS vendors can provide, plus their own capabilities and which third-party cloud-enterprise application integration solutions could fill the gap between the two.
Focus SaaS vendor technology evaluations on the quality and capabilities of transformation, security, the volume and the orchestration solutions, said Lehman. Make certain that the SaaS vendor is capable of exposing data in a standardized way so it can be used by an integration service provider or an on-premises enterprise integration technology, like an enterprise service bus.
The emergence of third-party integration service providers has kept SaaS application integration projects going. Third-party tools -- from Dell (Boomi), MuleSoft, PathLogic, Jitterbit, Kapow, Talend, Vitria, TIBCO and Red Hat, IBM (Cast Iron) -- assist with SaaS to on-premises application integration. "Their prices are reasonable for small to mid-size companies and [they] also translate departmental solutions in larger enterprises," Lehman said. "I would rely on those types of vendors to assist in integration."
Gotchas when working with integration services providers include pricing models that change with scale, volume or number of users. Be aware of how their pricing models scale so you can budget for that, said Lehman.
Most SaaS vendors will provide greater integration capabilities in the future, he said. Lheureux thinks that time can't come too soon and believes that cloud providers have been remiss in offering integration as an add-on and not an included-in solution.
"Redirecting your customers to a third-party for integration solutions when integration is a requirement in every IT project of substance is falling short of meeting your customer requirements," said Lheureux. "SaaS providers should have bitten the bullet and given their customers integration capabilities, rather than forcing customers to buy them from a third party."
Lheureux pointed to Workday as an example of a company that has ponied up with integration systems, providing the application, middleware and integration middleware in a bundle (or SaaS and iPaaS). He predicts that Microsoft will offer integration as a feature of Azure.
Cloud integration discipline and strategy
To break down barriers to cloud application integration, DevOps should also change from ad hoc to company-wide standard practices. "You can't be successful in managing an evolving, constantly changing application portfolio if you are not really disciplined at doing integration," said Lheureux.
For most organization, there's still time to craft a cloud-to-enterprise application integration strategy. "Right now, everyone's going slowly," said Lehman. "They're not shifting their entire enterprise to a SaaS model. They're doing it departmentally or by function."
Start strategy planning with standardizing ways to master the four main challenges mentioned earlier: transformation, security, volume and orchestration, said Lehman. From there, said Lheureux, create a coherent, disciplined application integration strategy that works across all sources of application data -- in-house or cloud, behind or outside the enterprise firewall. Before buying, make an informed decision about how a piece of the cloud will be integrated.
A coherent strategy doesn't necessarily mean using only one solution, one technology approach or one implementation approach, said Lheureux. Certainly, organizations benefit from using one infrastructure for integration, no matter where application data and process logic resides. Within that framework, however, have a portfolio of technologies and models you can use on a case-by-case basis based on predefined criteria.
A final word of advice: Treat integration artifacts as valuable assets. Maintain them for reuse. Whether you're connecting to a dozen clouds apps or connecting to a thousand external trading partners, all of the interface definitions, maps for translations and so on are valuable for continued use when cloud and/or enterprise applications are updated or replaced and trading partners come and go.
This was first published in November 2012