The Challenges of OLAP in Very Large Data Warehouses
John Hancock of Microsoft will share lessons learned in building a system that supports a 1.2 terabyte data warehouse within an OLAP cube, using market data from a syndicated data publisher.
Microsoft, Unisys, Knosys, and EMC teamed up to demonstrate an application using an extremely large data set in a cube. Using market data from a leading syndicated data publisher, the team has built a system that supports a 1.2 terabyte SQL Server 2000 data warehouse within an Analysis Services MOLAP cube. Through this single cube, users have access to more than 700,000 products and 7.7 billion fact rows. Even complex queries complete in less than a second.
The presentation will give details of the dimension and fact structure, along with an overview of the source data volumes and characteristics. A brief overview of the hardware used will be followed by practical lessons learned in data partitioning, indexing, and aggregation design for large OLAP structures. Data loading techniques including parallel processing will be covered. Client access scenarios (rich and thin clients) and cube processing and query response times will be reviewed.
John is a consultant for Microsoft Consulting Services (MCS), focusing on Business Intelligence solutions. He has previously worked in the UK, US and South Africa as an independent consultant, with clients in manufacturing, retail and the petrochemical industry.
John Hancock, The Challenges of OLAP in Very Large Data Warehouses
Questions? Contact SIG Co-Chairs: