Posts Tagged ‘ TDWI ’

Data Requirement for Advanced Analytics

TDWI author, Philip Russom has presented a fantastic checklist on the data requirements for advanced analytics.

First, it is a major BI/DW organization which pinpoints the need of different data architectures for reporting and analytics (particularly advanced analytics).

Second, it serves as an important document for data warehousing and modeling experts who usually dont consider the advanced analytics usage when designing the data storage.

Third, it promotes the provisioning of separate analytical data stores that advanced analytics demand.

Fourth, it serves a business case for in-Memory databases.

Standard reporting and analytics (OLAP) suffice well with multidimensional models (high level, summarized data) while advanced analytics require raw transactional data (low level, detail data) along with aggregated data and derived data usually in denormalized forms. The exact nature of the design is determined on the type of analysis to be carried out.

The data integration is also different for data warehousing serving reporting and analytics and for the analytics databases serving advanced analytics. The former mostly rely on ETL while the later is better served up both in practicality and the nature of analysis by ELT.

Secondly, the data integration for data warehousing deals mostly with aggregating, consolidating and changing the schema type from relational to multidimensional. Whereas in analytics database, the data integration is of an advanced mathematical nature where activities like discretization of continuous data, binning, reverse pivoting, data sampling and PCA are heavily employed.

A similar discussion had been carried out sometime ago here.

This white paper makes a strong case.


BI Competency Centers – A Program Management approach towards delivering excelling BI

I recently came across a client whose need’s where genuinely overwhelmed by a bit of frustration and a lack of solution partners who are not just technology implementers. The company had consolidated a lot of software solutions from various vendors, with overlapping features and functionality and had multiple departments participating in a tug of war for taking their departmental (and individual) BI ventures, enterprise wide. Interestingly, there was the case of an orphaned BI initiative there as well, the sponsors of that project had gone elsewhere!

The company came up with the idea of an excellence center (or a competency center) to try to standardize people, processes and technology. Easier said than done, the whole concept IS highly Utopian and is usually touted as a single solution to this fairly universal problem. But to achieve this”excellence”, a lot of background work is required which besides being costly is also time consuming.

To start with, a vision of an excellence center has to be developed. First of all, what DO THESE TERMS REALLY MEAN? Excellence Centers, Competency Centers, Strategy and Delivery Groups etc.It is one of the curse of hypes but a fairly reasonable mapping exists……”Program Management”

A Business Intelligence Program Management which sees BI implementations not just as a technology with limited business benefits but a business initiated venture with a targeted growth plan providing further services, features, ROI and sanity has a very strong case to sell.

According to Gartner Research, “A BICC is a cross-functional team with specific tasks, roles, responsibilities, and processes for supporting and promoting the effective use of Business Intelligence across the organization.”

BI Projects should be looked as ongoing, cyclical and iterative BI processes providing an improved delivery at each iteration. A Competency Center can provide the framework for measuring BI projects and their implementation, it also lets the company experience the cultural and operational transformations taking place as a result of a systematic and pervasive BI establishment. However, considering the different organizational behavior at different sized companies, operating in various verticals in diverse cultural backgrounds cannot be a single, enlightening offering.

It has to be Tailored for each concern whether a corporate or a department. But in general, a few set of services are considered core to the BI concerns in a company, namely,

  1. The Periodic Assessment of ROI and Cost vs Benefits.
  2. The standardization of processes and technology, whcih includes an enterprise level integration infrastructure again for both business and technology.
  3. A well defined and controlled Risk Management perspective on the BI space.
  4. A carefully crafted Knowledge Management initiative including organizational change.
  5. A focused and prioritized agenda on Business User “Buy-In” into the BI environment.

Several companies provide their BICC setup and operations competencies and consultancies these days.  However, there aren’t many best practices or guidelines in choosing the right partner for establishing one. Minimum requirements could be the ability of execute BI projects and programs, strong Human Resources, Business Processes and Systems Integration skills etc.

Although BICCs are ongoing programs, they should be highly target oriented. These milestones and performance targets are based on various assessment calculators which usually come as part of a BICC setup.

A very creative way to visualize the progress and understand the whole philosophy behind the BICCs is wonderful BI Maturity Model for demonstrating the characteristics of a BI program or project, developed by TDWI.

There is also a fairly detailed book on the topic of establishing and developing a BICC, published from SAS and Wiley and Co, Titled:

“Business Intelligence Competency Centers: A Team Approach to Maximizing Competitive Advantage (Wiley and SAS Business Series) by Gloria J. Miller, Dagmar Brautigam, Stefanie V. Gerlach

Although the book is written by one of the BICC consultancy firms, the ideas presented are applicable universally. Their interpretation of the core services offered (or should be) by a BICC have been widely adopted by both the industry and the academia.

Source: Business Intelligence Competency Centers, a Team Approach to Maximize Competitive Advantage" SAS and Wiley Co.

Source: Business Intelligence Competency Centers, a Team Approach to Maximize Competitive Advantage" SAS and Wiley Co.

All of these services are interrelated and each serves as an input to others. Each service also serves more than one goal of the BICC.

For example, the Advanced Analytics service besides providing a greater usability of BI and its infrastructure also increases the ROI. It also presents a strong case for evangelizing BI. It gives the business users an insight on what CAN happen from your BI environment. For organizations not having a sound infrastructure in place, an aggressively advertised advanced analytics service can form the motive to invest in a holistic enterprise information architecture, for example.

Establishing a BICC is a highly subjective matter and varies substantially from case to case. However a template based road map can be followed as one provided in the referred book. Primarily it depends on the existence of a similar setup already in the company, the maturity of the company in terms of its processes and policies for change management and technology, the type of people in terms of domain expertise and skill levels, the budget and time constraints etc.

As part of a general best practice, it is ideal to grow the BICC organically, meaning from bottom up with sponsorship from the top. A departmental wide BICC prototype which is planned for the enterprise but services one smaller concern at a time, like a department and then growing gradually into covering more departments and offering richer services.

Having a centric approach towards managing the concerns of BI is a daunting task but has its dividends promised if done well. The success of BI projects heavily rely on their continuity, reliability, flexibility, visibility and scalabiilty. BICCs offer just that.

The TDWI BI Maturity Model

As most industry experts tout the ongoing nature of Business Intelligence projects, there comes a natural desire to rate the status quo BI in an organization. However, as wide and diverse a Business Intelligence implementation is in terms of the tools, processes, people and culture invovled, it does need an overall benchmarking to assess future directions by setting goals and understanding the shortcomings of current offerings. To measure the ‘Maturity’ of an implementation, several independent organizations and certain vendors have developed their own assessment models. Among such publicly available models, the TDWI’s BI Maturity Model is a top down, vision oriented model which organizations can use to develop a road map. The model is a generalization of multiple BI projects and implementations indicating certain patterns of behavior based on five different aspects:

1. The BI Adoption

2. Organization Control and Processes

3. Usage

4. Insight

5. ROI

Such a maturity assessment is important in terms of gauging the value of the business intelligence inititave. For systems integrators and consultants its serves as a guide to set project milestones, deliverables and management and for C-Level execs to understand a step by step guide to the ROI from BI investments.

The TDWI BI Maturity Model offers a framework to adjudge the current standings of a BI implementation in terms of its adoption, control, usage, insights and finally the ROI.

An associated poster is presented here taken from Timo Elliott’s blog:

Each graph has its own target audience within the stakeholders and serves as a guide to a particular agenda. The management might be more interested in the last graph, the business value and ROI, whereas the business analysts might be more interested in the Insight whil the program implementers will focus on the adoption and usage as their primary concerns.

The Gaps:

While advocating the value of Business Intelligence, it is imperative for organizations to understand the gaps which obstruct their progress, either caused by the management or by the prevailing culture of the organization. It is the crossing of these gaps that defines the change management agenda for the program.

In the BI adoption cycle, the gaps define the points of stagnation which requires consultancy and a self motivated drive by the organization to cross it. But in other cycles the gaps indicate paradigm shift, state transition and time to reap the rewards of the investment. For the Local control vs enterprise standards, this indicates a well placed Business Process Management occasionally coupled by Data Governance initiatives which take over adhoc factors within organizations. In the first gap called the gulf, it is the individuals who feel empowered mostly by self-service BI capabilities but it kind of stops at that until and unless the organization throttles to move towards the second gap, called the chasm where it finds a mutual agreement between the individual entities of control and the corporate governance and management practices which leads to organization enlightenment, becoming a ‘sage’.

For the BI Usage, before the gulf, the organization is equipped with its first batch of power users which identifies freedom from their IT department to provide insight. But here is where the gap occurs. A group of established power users indirectly in control of the BI program lets the organization stagnate and miss out the ability to truly empower all their business users and not just power users. Although an ideal utopian state to accomplish, but it has been the approach of wise men, there is no end to gaining wisdom and becoming a sage. To overcome this chasm, organizations further enhance their incentive systems for business users while providing the next layer of BI services: the customization capabilities to supplement self-service. Here tools become pervasive to the organization staff and their throughput increases.

For BI Insight, the phases between chasms indicates the shortening of the decision making process. Once the second and final chasm is crossed, organizations gain the capability of automated decsison making and a system which supplements business users with true decision support.

Programs such as BI require a dedicated sponsorship from the management to prosper and when they do take interest, expectations of ROI might supersede their actual values. This is partly caused by phenomena well explained in Gartner’s Hype Cycle.

This poster aids the BI teams to better explain the ROI expectations to the management in charge of the programs. What you sow, so shall you reap system applies to BI programs as well.

The model is well explained by the author Wayne Eckerson and certain blogs supplement the material as well.

Some notable links:

1. TDWI BI Maturity Assessment Tool

2. You can’t get there from here!

3. Hows your BI Maturity