Analysis of Big Data for Financial Services Institutions

In this blog post, we will look at analysis of stock prices and dividends by industry. This task is important to all participants of Stock Market including individual retail investors, institutional investors such as mutual funds, banks, insurance companies and hedge funds, and publicly traded corporations trading in their own shares.

In this demo, team of Stock Trading Company analyses semi-structured stock data from the New York Stock Exchange (NYSE).

  1. Data Architect collects data and makes information accessible to business. He will use Hadoop-based distribution on Windows Azure and Hive queries to aggregate stock and dividend data by years.
  2. Financial Analyst will analyze stock data and prepare ad-hoc reports to support trading and management processes. She will use Power Query add-in for Excel to join aggregated data from Hadoop with additional information on top 500 S&P companies from Azure Marketplace Datamarket. Additionally she will create ad-hoc reports with Power View for Excel.
  3. Trading Executive is responsible for understanding key decision makers and suggesting best product mix of securities. He will make some modifications to Power View reports provided by Financial Analyst.

Details on how Data Architect aggregates data in Hadoop are available in a separate blog post.

Below you can see some screenshots from the demo.










Aggregating Big Data with HDInsight (Hadoop) on Azure

When we a talking about Big Data we may mean huge amounts of data (high Volume), data in any format (high Variety), and streaming data (appearing with high Velocity). Microsoft provides solutions for all of these “3V” tasks under unified monitoring, management and security, as well as unified data movement technologies. These
workloads are supported correspondingly by SQL Server Database and Parallel
Data Warehouse, HDInsight (Hadoop for Windows or Azure), and Microsoft SQL
Server StreamInsight.


Let us talk about Microsoft Big Data technology for Non-Relational data.

Microsoft’s adaptation of Hadoop technology can be deployed in a cloud-based environment or on-premises. The Hadoop-based service on the Windows Azure platform is a cloud-based service that offers elastic (in a term of data volumes) analytics on Microsoft’s cloud platform. For customers who want to keep the data within their data centers, Microsoft provides Hadoop-based distribution on Windows Server.

In this blog post, we will start diving into Hadoop in Azure technology and Hive queries to analyze semi-structured data in Hadoop.

In addition to traditional data warehousing, when operational data stored in special structures in Enterprise Data Warehouse, we can store all other raw data in “Store it All” cluster. At any moment, we are able to create query to these data to answer some business question. (In addition, we may store the answer in the Data Warehouse if necessary)


Let me introduce the first part of Bid Data Demonstration where Data Architect will store log files with stock prices and dividends in Azure Blob Storage and will use Hive queries to aggregate data by years and stock tickers into separate file.


Here is the video:

Additional materials: Windows Azure Storage Architecture Overview

Oil and Gas Analytics with Power View for SharePoint

Oil and Gas industry requires highest quality of decision-making based on correct and intuitive data. The industry becomes more innovative and requires faster usage of new production methods paying attention to eliminate risks at the same time.

Microsoft Business Intelligence provides tools to analyze historical data in intuitive and interactive way.

In this demo, team of Oil and Gas Company solves business issue through collaboration of following persons:

  1. Chief Executive Officer develops strategy, monitors operating efficiency, and achieves growth of profit and company value.
  2. Business Unit Manager works to ensure business continuity, improve technology, and drive excellence.
  3. HR Executive is responsible for improvement of personnel performance through training and appraisals.

All of them are using the same company portal built on SharePoint portal with a set of pages containing interactive reports (dashboards). These reports based on Business Intelligence Semantic Model (BISM) and visualized with Power View for SharePoint.












Microsoft Customer Analytics Solution for Banking

Understanding customers is the foundation to a sustainable competitive advantage in banking. Such problems as customer churn are very difficult to mitigate and much better just to prevent these events using analysis of customer behavior and preferences. To make it possible it is necessary to extract more value from internal and external data sources, and to be able interactively analyze intuitive data.

Microsoft Business Intelligence allows to solve problem of customer churn, provide better customer service, improve cross-selling, up-selling and enhance share of wallet.

Understanding importance of these tasks Microsoft created Customer Analytics Solution for Banking. In this blog post, I am presenting brief demonstration of this solution. After seeing video recording of demonstration (see below), you will find that marketers and other business users can easily use this solution to answer complex inquiries.

During the demonstration, I will act from points of view of different persons.

  1. The goal of Chief Retail Executive is to develop more profitable customer relationships with end-to-end customer insight with following tasks.
    1. Engage and retain the most profitable customers
    2. Assess segmentation performance and opportunities
    3. Optimize product  mix and pricing strategy
  2. The goal of Chief Marketing Officer is to understand and improve brand perception by integrating a broader set of data with following tasks.
    1. Understand brand perception & consumer sentiment.
    2. Gain insight into customer preferences.
    3. Identify key influencers & social media channels.
  3. The goal of Operator of Call Center is to provide support having complete view of the customer with following tasks.
    1. Provide best customer service quality.
    2. Combine customer sentiment analysis and profitability data.
    3. Track personal performance.
  4. The goal of Contact Center Executive is to maximize customer retention and satisfaction through improved service with following tasks.
    1. Improve  customer service quality.
    2. Track call center performance for each Agent.
    3. Combine customer sentiment analysis and profitability data.


Banking Future Vision

Please take a look at following videos which reflect current vision of Microsoft on future banking.

Analysis of population, GDP, HIV deaths, and number of mobile phones in Africa

In this video Marc Reguera walks you through some stunning visualizations that depict important information about progress in Africa by examining a number of factors from each nation, such as population, GDP, HIV deaths, and number of mobile phones. He interacts with these visualization by simply touching the screen of the stunning piece of hardware that is PPI by Microsoft.  Watch on as Marc shows you how to get from data to powerful business insights and impact in just a few clicks using Power View.

Source: Microsoft BI Channel on

Retail Analytics with Power View

In the following demonstration you can see example of retail analytics. Reports in the demo created with Power View, which is a new tool in Microsoft BI.

In the demo there is analysis of sales promotions, customer demographics, sales channels, retail stores, products and sales performance.

These reports can be created in Excel 2013 based on a tabular BI semantic model (BISM) using PowerPivot for Excel. Analyst can create such data model and reports by himself in days (if data is prepared).

US Economy Analysis with iPad

In the following demonstration Tyler Chessman shows example of US Economy Dashboards and how to use the Microsoft BI Platform with Tablet devices like the iPad.