Search: A Key Building Block for Business Applications

As I wrote previously, the ECM market is going through significant transformation and continued consolidation.  The convergence of 4 different markets are contributing to this evolution.  One of the interesting aspects of this transformation is the emergence of search as a key enabler for business applications.

Basic Premise

Future winners in the evolving ECM market will need to embrace new composition frameworks that can rapidly deliver solutions at the intersection of content, social, search and analytics, and business process management.  Those composition platforms will also need to support hybrid cloud delivery models to deliver optimal business values based on the customer needs.

Search and the Composite Application Platform

The concept of search based applications is defined by Sue Feldman and her team at IDC as "applications that combine search and or text analytics with collaborative technologies, workflow, domain knowledge, business intelligence or relevant web services".

The definition above introduces key distinctions from the traditional definition of search:

1. Search based applications combine a broad range of capabilities:  I find this aspect of the definition particularly interesting.  From my perspective, this makes search technology one of the building blocks of an application composition platform.  It offers both information management capabilities such as information categorization, text mining, indexing, fuzzy matching and information navigation capabilities such as faceted / relational navigation and data visualization in the context of business applications.
 
2. Information is integrated with business processes: this is a critical aspect of search based applications and this is where I believe there is strong overlap with decision management or case management.  In a business context, search supports a decision process.  Those decisions might be more project centric which will require more collaboration and ad-hoc discovery.  They might also be more process or case centric in which a network of processes will manage the orchestration of the decision process.

This requires that search capabilities be weaved into the fabric of business applications.

What about some examples?

First let's look at some examples for best of breed search vendors and how they leverage the concept of search based applications to address their customer problems.

New account opening and single view of the customer

Attivio provides the case study of a large global bank challenged to enhance customer service across 40 countries, 50,000 employees and 100 separate sources of information.  The bank decided to leverage a search based application to provide a unified view of customers resulting in the enhanced productivity of the customer facing agent, improved customer experience, and a higher percentage of first call resolution rates, while decreasing overall support and maintenance cost.

Warranty and quality monitoring

High warranty costs have received significant exposure lately. Think about Johnson and Johnson having to recall 22 products in 19 months adding up to over $900M in revenue loss.  Those are nightmares manufacturing executives cannot ignore.  Getting the information to identify warranty issues before those organizations are at risk requires visibility throughout the entire supply chain.  By integrating data from multiple sources, search based applications provide the right tools to support the discovery and early identification of product quality issues.  Endeca offers Endeca Latitude for Warranty to address this problem.

A New Kind of Business Application Composition Platform

The examples above illustrate the power of search based application.  Those applications become even more powerful when they are combined with an action engine that enables the tracking and processing of warranty issues, the collaboration on criminal investigations or complex financial products.  With xCP, our view is that great search and data visualization capabilities need to be combined with the orchestration of content and information that will drive optimal business decisions: for instance a product recall based on the early identification of warranty issues.

Samir Batla, product manager for search and analytics at EMC IIG has posted a series of blogs posts on how xCP exposes a wide variety of technologies that enables optimal business decisions and how from early warnings, customers can take action to make pro-active decision for their businesses.  As shown below, the anatomy of an optimal business decision requires much more than search.  The combination of the broad set of capabilities of xCP is what enables these optimal business decisions.

The Need for Cloud Integration

The cloud is everywhere those days and the software industry is going through some drastic transformations.  According to IDC, “Software as a Service 2010-2014” report, software will never be the same.  The on-premise software market will go down 19% during this period from a market share of 66%.  In the same period, shares of the public cloud are expected to go up 12% to 22% of the software market revenue.  The Platform as a Service market is expected to grow between 39% and 55% a year for the next 5 years (IDC forecast vs. Forrester forecast).  This is quite some shift...

Forrester Research defines Platform as a Service (PaaS) as:

A complete application platform for multitenant cloud environments that includes development tools, runtime, and administration and management tools and services. PaaS combines an application platform with managed cloud infrastructure services.

A key value of PaaS is the ability to focus on the business application and not on platform technologies.  As PaaS becomes more widely adopted and business applications increasingly built on top of such platforms, the need for cloud integration will become pervasive.

Modern applications require the ability to expose enterprise information in the context of the task at hand.  Adaptive contextual user experiences will become increasingly critical for business users to rapidly make optimal business decisions.  In a PaaS or SaaS world, this will require robust integration with the needed information sources such as:

• Cloud based applications: CRM systems like Salesforce.com, HR systems like Success Factor or Workday, public cloud email systems and productivity applications
• Enterprise applications:  ERP systems like SAP or Oracle Business Applications
• Content management systems internally hosted or in the public cloud
• Legacy systems and applications

However, cloud integration as some serious challenges it needs to overcome.  A July 2010 survey by Saugatuck Technology confirms that data security and privacy as well as data integration concerns are top concerns in deploying cloud based business solutions.

Let’s have a look at some of those challenges:

• Data integrity and information security.  Art Coviello from RSA articulates the problem nicely:  "While cloud computing offers tremendous benefits in cost and agility, it breaks down some of the traditional means of ensuring visibility and control of infrastructure and information. Forcing enterprises to develop trusted relationships individually with each cloud service provider they wish to use is cumbersome and will not scale. New thinking in security and compliance is required to provide a future in which organizations can consume services from a wide variety of cloud service providers on-demand and for all their application needs." 
• Application security integration.  Anyone who has tried to integrate applications together knows that one major integration issue is around access control and particularly role based security.  Protocols  like OAuth have emerged to provide a simple way to publish and interact with protected data.  Other standards like XACML provide flexible solutions for enforcing granular authorization.  Unfortunately, those technologies are not broadly adopted by enterprise application making authorization integration a challenge.
• Data integration.  Many enterprise systems still require proprietary integration mechanisms.  But even cloud based applications require integration with their proprietary APIs.  Cloud based integration should provide a flexible mechanism for data mapping and integration.  As Mark Brennan, Director of Business Applications at Pandora puts it: "traditional approaches to integration are no longer adequate when you have literally dozens of SaaS applications, all needing to talk to each other. If every change, customization or refinement becomes a project, we can’t keep up. To changes all that, we need a solution that elevates us out of code level and putting the power directly into our hands".   Some of those data integration challenges may involve:
• Bi-directional data synchronization
• Transactional integrity and service orchestration
• Solutions to deal with information caching and system latency

Clearly, addressing those challenges will be key to enterprises moving and trusting their enterprise systems to the public cloud.

A New Platform to Deliver Superior Customer Experiences

A few weeks ago, I talked about the new platform and the new technology era of business applications. The new platform must enable business applications to drive optimal business decisions and improve customer experiences. It must provide a breadth of application services that enable businesses to rapidly build intelligent business applications that can adjust rapidly to changes in their business environment.

Several weeks ago, at EMC World 2011, I gave a presentation on How to leverage Documentum xCP and the 5 Cs (correspondences, capture, case, content and compliance) to deliver superior customer experiences while improving business efficiencies.

In my presentation I started with a personal experience. A few months ago, I got an automated phone call from my bank telling me that my debit card had been deactivated and to press 1 if I wanted to reactivate it. Puzzled, I decided to call my bank instead. First, regular customer services was closed; so I decided to get routed to the service where you report card losses. After a 15 minute process, I finally got in touch with a customer representative who told me that this was a scam they were aware of and that they had issued a press release to warn their customers about the scam. Yes, you read right: a press release... As educated and responsible consumers and customers, we are expected to monitor the press releases of the companies we do business with, no less.

Well, in the day and age of personalized services and hyper social interactions, needless to say we expect more. This is where the new platform comes into play.  Imagine what my bank could have done had they been using that new platform:

1. They could have been monitoring social channels and their own customer data to look for patterns that their brand was being used for mischievous purposes.  That's where Big Data comes in.   Leveraging data computing appliances and predictive models, corporations can be on the look for significant patterns or events that are likely to impact their businesses.
2. Once, my bank has found evidence that a scam is underway, they could have leveraged the intelligent correspondence management and process capabilities of xCP to send me multi-channel personalized communications and to make sure that I was aware of the scam.  In those communications, they would have provided me with instructions on how to deal with the situation.
3. They could have been proactive in identifying customers that were victims of the scam and opened cases to help those customers deal with the situation, cancel their debit cards and issue new ones
4. They would be delivering better customer experiences, ensuring the loyalty of their customers while protected their business against potential charges of negligent behavior.

This is the power of the new platform.  Gain insight from social and business data, and take actions on that insight to deliver superior customer experiences.  As the success of companies like Zappos shows, companies that can successfully deliver those superior experiences create long term sustainable competitive advantage.

The Evolution of the ECM Market

When Documentum was founded, the founders' vision was to manage the world's information to enable innovation and scientific breakthrough (like finding a cure for cancer). Since then, what started as a Document Management opportunity has gone a long way. Today, Enterprise Content Management is a mature market providing breadth of technologies that enable organizations to reshape the way project or transactional decisions are made. The ECM market is also a market that has gone through significant transformation lately and continued consolidation.

I see the convergence of 4 markets contributing to the evolution of ECM targeting different interaction types (from adhoc to structured) and application types (from horizontal to vertical).

The Commoditization of Content Services

The unprecedented success of Microsoft SharePoint has commoditized content services and significantly changed the content management market. SharePoint is now a force to be reckoned with. The sprawl of SharePoint sites has also created tremendous opportunities for information governance and given rise to a new generation of anti-SharePoint offerings such as Box.net leveraging the cloud to simplify information sharing.

The Emergence of Social Business Platforms

With the launch of their new Alfresco Enterprise release, Alfresco embraced the concept of Social Content Management. Jive and IBM refer to the same opportunity as Social Business. Whatever the term, in its latest magic quadrant, Gartner recognizes the impact of social interactions as social media create masses of unmanaged content.

Jive and IBM are the clear leaders in Social Business Platforms and the recent partnership between Jive and Alfresco recognizes the need for Social Business Platforms to provide robust content management capabilities. Social Business Platform vendors like Jive have been successful against Microsoft by focusing exclusively on social features and how they can help certain types of employees (salespeople, developers, etc.). Microsoft has also recognized the impact of social on content management and collaboration with SharePoint 2010.

Vendors for Social Business Platforms include: Jive, IBM, Microsoft, Cisco Quad, Acquia / Drupal

When ECM Meets Search Based Applications

The concept of search based applications is defined by Sue Feldman and her team at IDC as "applications that combine search and or text analytics with collaborative technologies, workflow, domain knowledge, business intelligence or relevant web services". This definition is being embraced by search vendors such as Endeca, Sinequa and Exalead (now part of Dassault Systemes). Gregory Greffenstette, Exalead chief science officer, offers a glance into concrete examples of search based applications from logistic track and trace type applications, to CRM, 360 view of the customer and decision intelligence.

At EMC, we have recognized the importance of search based applications and are incorporating search and text mining functionality into our products from CenterStage to Documentum xCP leveraging our Documentum xPlore, Content Intelligence Services and Federated Search Services, now core features of the EMC Documentum Platform.

Other ECM vendors from OpenText to IBM have recognized this trend as well with significant investments and acquisitions in this area. Key to delivering the promises of search based applications will be tight integration with composition tools for a new breed of business savvy developers that will enable the rapid creation of search enabled applications.

The following search vendors have embraced this trend: Autonomy, Microsoft / Fast, Attivio, Endeca, Sinequa, Exalead (Dassault Systemes)

Towards Composite Content Application Frameworks and Dynamic Case Management

As Gartner points out in its latest magic quadrant, there is increasingly more pressure on ECM vendors for ready to use solutions which solve real business problems rather than generic content platforms composed of dozens of loosely coupled modules. The need for more business solutions coupled with the convergence of the ECM and BPM markets has seen the rise of new opportunities for what Gartner calls Composite Content Application frameworks and Forrester calls Dynamic Case Management. While composite content application frameworks address the needs of content solutions, dynamic case management focuses on the convergence of ECM, BPM, business analytics and event processing. The reality is that there is a spectrum of solutions from content to case that require a new generation of composition tools to rapidly deliver business value with the agility to evolve quickly as business needs change. Organizations need more than a repository, they need an application platform and the solutions that come with it.

The following vendors provide composite content application platforms: EMC Documentum xCP, IBM, OpenText, Adobe, SpringCM, Hyland, Nuxeo

Towards the New Information Fabric

As the delivery of composite content application platforms transition to the cloud, this creates an opportunity for a new information fabric that will deliver tremendous business value and flexibility while reducing barriers to adoption. The industry is clearly at an inflection points in the adoption of cloud services. While Public cloud services offer tremendous promises, hybrid cloud deployments are most likely to provide the right balance between information control and flexibility.

Future winners in the evolving ECM market will need to embrace new composition frameworks that can rapidly deliver solutions at the intersection of content, social, search and analytics, and BPM (from content to case). Those composition platforms will also need to support hybrid cloud delivery models to deliver optimal business values based on the customer needs.

Do you share this perspective on the evolution of ECM market? Share your thoughts on this blog or on twitter.

The New Technology Era of Business Applications

After a long hiatus, I have decided that it was time to start blogging again. Last time I posted a blog post was more than 2 years ago. Time flies...

We live in an exciting technology era. Technology transformation is accelerating and what is unique about this transformation is that all layers of the technology stack are experiencing significant changes at the same time. Jeetu Patel wrote a good blog post introducing the Post PC-era. For business applications this has significant implications.

1. The new platform: Business applications require a new platform. That new platform must be scalable, elastic, and rapidly respond to the needs of the business. It also needs a rich set of services to enable businesses to rapidly deliver applications that meet their most critical needs. At EMC Information Intelligence Group, we are investing on both those fronts:
   1. At EMC World, we announced EMC OnDemand to take EMC Documentum into the New Platform era.
   2. Documentum xCP was mentioned in every one of our executives keynotes (by Joe Tucci, Pat Gelsinger, Paul Maritz). xCP provides the breadth of services that enables to rapidly deliver business applications that meet the critical needs of the business. At EMC World, I had a presentation on leveraging the 5Cs (correspondence, capture, case, content and compliance). Those are cornerstone services for the new platform.
2. The new developer: The new developer needs new tools to create applications at a new level of abstraction and to enable her to focus on solving business problems rapidly and not as much on coding or the technology. The new developer needs powerful composition tools. Check out how xCP can empower the new developer to rapidly create sophisticated business applications.
3. The new user: Choice computing is critical to the new user. Check out Jeetu Patel's Blog post on the new user. The new user wants pervasive access to information, user interfaces that are catered to her needs. She needs to make decisions rapidly and needs the right insight to make the right decision with confidence. To that effect, EMC IIG had some significant partnership announcements at EMC World. But beyond the right user interface, the new user needs instant access to the right data, with the insight that enables her to take the right action. She needs to be able to turn information into business advantage. I presented at EMC World on how xCP can enable the new user to do just that

Those are exciting times at EMC IIG, the innovation machines is humming and we are taking part of the transformation of the business application platform on those 3 fronts. This represents an incredible opportunity for the future. I am exciting to be part of this transformation and to lead the development of next generation technology that will enable it.