Spreadsheets a BOON or Curse to Bi?

      I know I am opening a topic where I may get skewed views on the topic. But i guess this is one of the basis for Bi. The Higher Management and most of the end users tend to have a good hand over Spreadsheets. And have to make the decisions at higher levels. Just as the famous saying 'The Devil lies in the DETAIL' when it comes to analysing the data at finer level and making the most of Bi from the data available data appropriate Bi tools are the best bets.

    

      People who have been working in the Business Intelligence delivery area would strongly agree that Spreadsheets are good to an extent of MIS, and Reporting purpose but not really tend to deliver the most for analytical reporting which is the main objective of Bi.

    

      It’s very common scenario; Inspite of the best analytical and Bi tools present in the organisations the end users make an effort to derive analytical information through spreadsheets which are really more suited for standalone reporting and aggregate reporting.

  

      In short; If you want to derive the most from Business intelligence initiative. Bi tools have to be used considerably, the more you use the more value reaped from the Data available within the organisation. 

Historical Big Data in Organisations.

Historical Data, commonly also referred to Big Data is all about finding a needle of value in a haystack of unstructured information. Organisations are now investing in solutions that interpret consumer behaviour, detect fraud, and even predict the future! McKinsey recently released a report stating that leading companies are using big data analytics to gain competitive advantage. They predict a 60% margin increase for retail companies who are able to harvest the power of big data.

To support these new analytics, IT strategies are mushrooming, the newest techniques and building data mining solutions. New data discovery techniques include spectacular visualization tools and interactive semantic query experiences.

Knowledge workers, analyst and data scientists sift through filtered data asking one unrelated explorative question after another.

The real industry challenge is to integrate Big Data it into mainstream IT.

The following Initiatives are outcome of mitigating Operational Risks in Finance and Insurance industry which are a derivative of historical data:

•   BASEL – II/III – A widespread initiative in EU Finance/banking Industry.
•  Solvency II - An Insurance Industry initiative.

  

Agile Development and Bi Solution

      
     Recently I have been experiencing enterprises opting for agile delivery, for a Bi Solution.  Does Agile delivery suit a Bi Solution ?
    
      Business Intelligence helps the organisation/Enterprise to be on its toes. The KPI's cease to deliver the invaluable information over a course of time. Hence the KPI's are refined/revisited from time to time. This is overcome by Agile delivery of Bi Solution.
     
      Agile Delivery is not only applicable to Business Process in Bi; it in-turn also results in Agile Development Bi Cycle made up of Releases. The Agile Delopment Delivery method is another important topic which I would cover  in my latter post
     
     In the current dynamic and economic times of competitive edge. Agile delivery method offers significant value for a Business Intelligence Programme for a Enterprise.

Following are some good refernce books for Agile Development Methodology:

Succeeding with Agile: Software Development Using Scrum
Agile Product Management with Scrum: Creating Products That Customers Love
Agile Software Development, Principles, Patterns, and Practices

ODS, Datamart OR Datawarehouse for Bi ?

        

      Recently, had a Query Regarding ODS, Datamart or a Datawarehouse?

And it was backed having as datawarehouse is the best. But my  Call was IT DEPENDS,yes Indeed it just Depends. Further analysis led us to a Campaign managment System built  over an ODS - operational Data Store.

       These different systems are built for specific purpose to meet the objectives. They absolutely address different purpose in their own way.

Following are the examples where these systems should be Build.

 ODS - possibly for Call Centres or CRM systems having a recent snapshot of the Data

 Datamart - Historical Data for a Department in Enterprise viz. Sales, marketing, Risk

 Datawarehouse - These systems are Built with a purpose to provide a picture of the Enterprise with respect to time based on factual data.

      A CRM and a Campaign management programme could be a derivative of any of the above; as stated earlier, depending what are the KPI's and Objectives of the Project.

      Hence objectives of the Project and KPI's after distilling the requirements are responsible for choosing the right system to serve the organisation.

Business Intelligence & Day to Day life - 02

What is the foundation of Business Intelligence ?

       Firstly, Historical Data wihin the organisation forms the main basis of Business Intelligence Initiative in an organisation. However the sponsorer and Analysts are the key drivers for making the Business better hecne the organisation should consider the following:

- Visionary Analysts and Strong Sponserer.
- Vigilant team Business team to maintain the competitive edge.
- Tech Savvy Technology Enterprise Arch. Group.
- Experienced/Vigilant Team forms the core of a Successful Bi Delivery.

Are you making the Most from BI !

      There is a big difference between having BI capabilities and having a competitive edge because of BI. One organization may have the best BI tools and a great data warehouse, but see little value from it.

Another organization may spend relatively little on BI, and achieve a real competitive edge due to how they utilize their information. BI needs a certain level of tools to be possible, but real BI value comes from experienced BI people.

       You can give someone the best saws and hammers and they can build you a shack that falls down in a few months, or a beautiful house that lasts a hundred years, depending on their skills and experience.

  

Here are some examples of how sucessful BI Programmes:

  

1.  A retail company targeted their marketing efforts to existing customers based on what they've purchased in the past, and then predicting what other types of items they may want to purchase. They increased sales to existing customers by £4 million in a year.
2. A financial services company identifies which customers lost them money and which were the most profitable, and then set up fees so the losing customers dropped off. They saved £400,000 in the first year by having less customers that cost money. 
3. A financial services company analyzes what tasks were not completed on time, and made adjustments to their processes that saved them £250,000 in what would have been lost commissions / fees in 6 months.  
4. A finanicial services company also used BI capablities in areas of Operational Risk. For maintaing the Capital Requirements to be Met.
5. A manufacturer predicts inventory levels based on past sales by customer / product / day so they never run out of products. This is especially important to this company because there is no back-fill situation for their products. When one of their customers needs a product, if there are none on the shelf their customer buys from someone else that day. By being able to predict inventory levels, they increased sales £1.5 million in one year and earned a lot of credibility in the market.

  

 

Business Intelligence & Day to Day life - 01

        I am very sure if I am writing a post on the above topic one post would not be enough.
Also would like to make you aware what this post would be about:
- Is Business Intelligence for Organisation and Business Houses only ?
- Does a common consumer person practice BI ?
- Do consumers come across Business Intelligence scenarios quite ofen ?
- What is the foundation of Business Intelligence ?

Let me stop there and at least answer one question in this first post.
      Business Intelligence is based on following two quotes I came across:
- "Old age is Encycelopedia" - More applicable to the Business aspect. & the other
- "There are only two type of classes the Educated and the Poor; the poor are poor because they are ignorant" - More applicable to the consumers.  

     Bottom line is Both 'The Producers and the Consumers' have to co-exist and each of them would like to perform BEST; by practicing BI - Modern Economics spontaneously.

Datawarehouse or Datamart

       What should an Enterprise go for if starting BI as new Initiative?

       If an organisation is starting a BI initiative and want to get a feel of BI and its capablities. Its prudent to start with a Datamart for following reasons.
- It is confined to smaller subject area in the Enterprise eg: sales, products, Finance.
- You have a small consice group of experts which can deliver an Bi platform to use with ease.
- This will in turn enable to analyse the Benfits of BI to the organisation.

•       Calculating - Return on Investment  - ROI.
•       Comparing with the NPV.
•       Ease of use with the End-User

       The stakeholders interest and strong sponserer for a Bigger BI Iniitiative in the Organisation.

Recipe for Succesful BI Solution

For a Sucessfull BI Program following are most important ingredients:

•  Firstly, A very Strong Program Sponsorer who is backed by the Key Stakeholders
•  Vision about the organisations Goals.
•  Good Knowledgeable Business Analysts in the particular Domain.
•  Information Architect
•  Data Architect/Data Modeller
•  Enterprise Architecture Team.
•  Key Performance Indicators
•  Delivery Methodology.

BI a tool to keep ahead of the rivals

         Business Intelligence is one of key unleashed potential energy for matured Organisations. Compare it with stagnant water- Stagnant water just helps in decaying of things within. Similarly BI acts as free flowing kinetic energy very much like flowing water brining life into pastures new for the Enterprise. Implementing BI in the organistion not only helps you in keeping ahead of the rivals but also to avoid or have checks in place for any operational risks which may not be in the Risk Radar of the Enterprise.
Hence BI is been one of most essential tools(dept) in any successful organisation.

Impementation of BI for example can be done in following ways:
- Understanding the customer.
- Needs of the Market/customer.
- Target Audience.

Following are some of best books helping in understanding the value of BI:
- Business Intelligence: Data Mining and Optimization for Decision Making
- Successful Business Intelligence: Secrets to Making BI a Killer App

Datawarehouse Testing

What are the BEST ways of Datawarehouse Testing?
Following are few Best Ways for Datawarehouse testing.

Data Reconciliation: Comparing the data at attribute level writing queries (SQL) codes to compare the counts between the source and traget to start with. Later to check the data to bechmark the desired Data Quality for the transformed /migrated Data.

Sampling: Sampling is another way of testing the DWH. Taking samples of the Data.

Regression Testing: Datawarehouse tend to grow ove a course of time its a good practice to have Regression testing as a part of each release.

Dimensional Modelling

A methodology of arrangement of Database objects, structures in Facts and Dimensions is Dimensional Modelling.
Dimensions - Are tables mostly having contextual attributes, describing about the charcteristics and features of the Dimension, Object.
           Dimension tables are higly denormalised tables. Have redundant data in columns within the Dimension Table.
Fact - The attributes in this table are facts or numeric value attributes a deivative of the dimensions it contains the measures of the dimension and used to quantify the dimension Object attributes.

The fact table defines the Grain - level of detail stored in the Data Model. Its quite important to finalise the Grain level in the Data Model before the physical design of the Data Model.

A good book to Refer Dimensional Modelling would be:
The Data Warehouse Toolkit:The Complete Guide to Dimensional Modeling

SUN Modelling

SUN Modelling

     Sun Modelling is an user case driven Modelling driven by the Business, to gather and confirm the BI Requirements form the Business Stakeholders. For those familar with UML Modelling. SUN Modelling is a similar tool used in BI area.
     It depicts the logical relationship amongest the entities and attributes which are used for satisfy the KPI's of BI initaitive.

Data Migration

Data Migration as the name suggests is an activity realated to movement of data from one systems to another system. This activity is mainly carried by using following methods:
ETL Tools like:
* Informatica
* SAP Business Object Services
* Ab Initio
* IBM Datastage.
* SQL and PL/SQL              
              Supporting documents required for Data Migration involve:
- Interface/Data mapping Document
- Data Dictionary
- Data Sourcing Strategy

Business Intelligence

Business Intelligence - is the tool to keep an organisation ahead of the rivals, make decisions based on information available within the organisation.

     This information would be used in following Systems.

• Datawarehousing & Data Mining

      Using Historical data withing the organisation to a large extend to make anlytical decisions, A datawarehouse System is, Subject-Oriented, Integrated, Time-Variant, Non-volatile

Subject-Oriented: A data warehouse can be used to analyze a particular subject area. For example, "sales" can be a particular subject.

Integrated: A data warehouse integrates data from multiple data sources. For example, source Company A and source Company B may have different ways of identifying a product, but in a data warehouse, there will be only a single way of identifying a product as bouth the companies belong to same Parent Company.

Time-Variant: Historical data is kept in a data warehouse. For example, one can retrieve data from 3 months, 6 months, 12 months, 5 years, decades or even older data from a data warehouse.

Non-volatile: Once data is in the data warehouse, it will not change. No historical Data is updated. So, historical data in a data warehouse should never be altered.

• CRM - Customer Relationship Management.

          Customer Relationship Management System is also a derivative of Data captured within the organisation.

It is used to study the customer Interaction with the organisation hence comes under the BI Umbrella.

• Campaign Management

          Campaign Management too is a derivative of Data held within the Organisation. Used for running a campaign of product within the organisation. Mostly by the Sales and Marketing Teams.