Archive for the ‘ Data Warehousing ’ Category

Impact of Business Modeling on Data Warehouse Success

Business Modeling to Process Management professionals may mean something different, but in data management, business models are the logical data structures which capture the meaningful events in context. Here the context refers to dimensions, measures and their specific usage. Data Warehousing activities should be agile, thats the evolving zeitgeist which concepts like DW2.0 (Inmon), Agile BI and Self-Service demand. The fundamental problem with Data warehouse development is catering to change, agility and still serving a diverse user base.

BI processes and tools like Qlikview, Spotfire, Tableau with in-memory, associative models, alternative to OLAP provide the agility but the backbone of the organizational data warehouses are still locked into fixed, rigid and less scalable subject areas.

The impact of conducting Business Modeling sessions with the Business users prior to BI is effective on many fronts:

1. Business Exceptions are captured early.

One of the trick questions in requirements gathering for data warehouses is to extract the exceptions in business rules, processes, events from business users. This phase is an ‘art’ and only few can get a good knack of exposing as many business exceptions early in the design process. The problem with realizing business exceptions late result in architectures with patchy workarounds and fixes which reduce the overall muscle of the data warehouse building.

2. The Essence of Business Processes necessary for Data Warehousing is preserved.

At times, business modeling invoke the discussions among business users who rather than portraying the as-is states, often pertain to discussions and debates on what ought ‘to-be’ states and thus the BI consultant captures the overall strategy and goals of the business process under scrutiny. This info leads him/her to design solutions which adhere to the general gist and thus can manage both as-is and to-be with little modifications or further effort. Decent data warehouse scalability can be achieved.

3. A common language for communication is established between business and IT/BI.

Often times BI consultants & DW Architects force Business Users into understanding the business data model using the underlying database data models with naming conventions, physical layouts and dimensional modeling technicalities which overwhelm the business user. Business Users love when IT talks their language and IT appreciates removing ambiguity and vagueness by using a common lingua franca.

4. Greater sandboxing capabilities

Faster iteration cycles for prototype warehouse models can be reached which will allow for faster convergence to the desired data model(s). Easily an end to end prototype including subject area development, followed by BI can be experienced often revealing obstacles before going for the time consuming and complete implementation.

5. Setting expectations becomes more accurate.

Business Users through sandboxing and through modeling see the highlights of things to come and therefore right sizing is achieved.

6. An initial scope for BI is understood.

Business modeling will reveal to the BI Consultant(s) the major pain points of the business users and having seen the business models with its exceptions and all facets, a rough time estimate can be set and/or a rough activity and project (or phase) scope can be identified.

7. The A-Team is on-board.

By bringing a wide array of people from end users, power users, quality assurance officers, BI consultants and data architects, a company can easily identify the A-Team from the business modeling activity. This A-Team will carry the momentum to execute the project and in the longer run participate in the BICC.

8. Nip the Evil in the Bud, Data Quality and Governance issues start surfacing

Although they seem to be a distraction and are separate programs themselves, Data Governance (and Quality) has to be addressed as early in the Data warehouse design phase as possible since the design elements can be either considerate of cleansing or assume that to be cleansed when it comes to the warehouse. Secondly, Data Quality fixes later in the value chain, (worst case, front end reporting) will cause greater deviations from standards and design and a solution based more on patches and work around.

9. Greater Transparency in design and architectural choices is observed.

Due to Business Modeling, the organization can better track down on individual decisions which underwent while designing micro elements in the design process. Usually data warehouse implementations which don’t start this way, eventually end up with a long audit trail of documentations and ‘memos’ which are hard to capture and even harder to manage.

10. Overall project implementation times in principle, reduce.

Since the initial challenges are dealt upfront or are atleast discussed upfront, the overall project implementation times reduce. Lesser time is required to do exception management, lesser time is required in debating design choices and communication, and lesser number of surprises (in principle) have to be dealt with.

All in all, it is vital to involve the business users in the design process using business modeling approaches. There are several business modeling tools available in the market which assist in the process but at the end of the day, its the process of conducting such sessions that bear fruit.

Advertisement

Data Requirement for Advanced Analytics

TDWI author, Philip Russom has presented a fantastic checklist on the data requirements for advanced analytics.

First, it is a major BI/DW organization which pinpoints the need of different data architectures for reporting and analytics (particularly advanced analytics).

Second, it serves as an important document for data warehousing and modeling experts who usually dont consider the advanced analytics usage when designing the data storage.

Third, it promotes the provisioning of separate analytical data stores that advanced analytics demand.

Fourth, it serves a business case for in-Memory databases.

Standard reporting and analytics (OLAP) suffice well with multidimensional models (high level, summarized data) while advanced analytics require raw transactional data (low level, detail data) along with aggregated data and derived data usually in denormalized forms. The exact nature of the design is determined on the type of analysis to be carried out.

The data integration is also different for data warehousing serving reporting and analytics and for the analytics databases serving advanced analytics. The former mostly rely on ETL while the later is better served up both in practicality and the nature of analysis by ELT.

Secondly, the data integration for data warehousing deals mostly with aggregating, consolidating and changing the schema type from relational to multidimensional. Whereas in analytics database, the data integration is of an advanced mathematical nature where activities like discretization of continuous data, binning, reverse pivoting, data sampling and PCA are heavily employed.

A similar discussion had been carried out sometime ago here.

This white paper makes a strong case.