December events to attend in Wheeling, WV

Hello everyone,  I want to share some cool events going down in Wheeling, WV in December. 

If you like wine or technology I highly recommend catching the Greater Wheeling Chapter of the Association for Information Technology Professionals meeting on December 9th.  This meeting is open to the public. We will meet on the forth floor of Wheeling Central High School at 6pm.  All attendees will have the change to share their 2009 accomplishments while we chow down a pizza dinner.  Dinner will be followed by a wine tasting at the Good Mansion Wines store. Check out the event page to signup for the 2009 December meeting.

Are you a young superstar go getter? If so, OVConnect is the group for you.  The mission is to bring young professionals in the valley together.  This months installment includes a holiday toy drive with a happy hour twist.  On December 17th at 5:00pm we will socialize and get our drink on at River City.  In order to attend all you have to do is signup and bring a toy or make a ten dollar donation.  All donations benefit Catholic Charities.

Bonus Pittsburgh Technology Event

Are you a .NET programmer and interested in what’s going on with MVC?  If so, this months installment of the PGH.NET User Group is a must attend free event.  The PGH.NET meeting is on December 8th. You can find more info here or register here.

Pittsburgh SQL User Group: Data Warehousing Presentation

Today, I am giving my first presentation at my local SQL Server User Group.  Below are links to the slides.  I will also add some more resource here after todays presentation.

Introduction to Data Warehousing slides

DEMO: Build Your First Cube

Below is the description of the discussion.

November’s presentation will include an Introduction to Data Warehousing with SQL Server presented by John Sterrett, a Database Administrator and Web Engineering Specialist for Orrick, Herrington & Sutcliffe LLP.  If you are interested in learning how Analysis Services, Integration Services and Reporting Services can provide Business Intelligence this meeting is for you.  We will start with the basic concepts of data warehousing including key terms and definitions.  Next, we will cover some sample business cases while we walk through the process of designing a data warehouse.  Once this session is complete you will be able to develop your first cube.

Date: Monday, November 16, 2009

Time: 11:45 a.m. Registration; 12:00 PM – 2:00 p.m.

Venue: Council Training Room, 2000 Technology Drive, Pittsburgh, PA 15219

Cost: No charge

RSVP: Email Heather Jones at Heather_Jones@Dell.com

If you have any questions, need more information, or would be willing to participate in any upcoming presentations or discussions, please Heather Jones at Heather_Jones@Dell.com.

Data Warehouse: Facts and Measures

A few days ago I wrote a post that gave an introduction to dimensions.  Today, we are going to continue covering the basic concepts included in dimensional modeling by covering an introduction to fact tables and measures.  These posts are all part of the introduction to building a data warehouse with sql server series.

What is a Fact Table?

A fact table is a table that joins dimension tables with measures.  For example, Lets say you wanted to know the time worked by employees, by location, by project and by task.  If you had a dimension for employees, location, project and task you would create a composite primary key using these foreign keys and add an additional column for the time worked measure. (more on measures in a little bit)

Keep in mind that fact tables are huge unlike dimension tables.  Fact tables are usually built to contain a hundred thousand records on the low side up to billions of records on the high side.  Therefore, these tables must be normalized to be efficient.

A fact table is normalized when it consists of the following two things:

    1. A composite primary key using the foreign keys to the dimension tables.
    2. Measure(s) used for numerical analysis.

What is a measure?

A measure is normally an additive numerical value that represents a business metric.  You are also not limited to a single measure.  You can have multiple measures within a fact table.  For example, if your fact table is used to track purchases internationally you might have measures for each type of currency.  If you are building a fact table for the retail industry you might also have the following measures cost, list price, average sale price.

Next, you can take a look at my demo on building your first cube.

Introduction to Dimensions

Today, we are going to quickly cover one of the basic concepts included in dimensional modeling.  We are going over the basics of dimensions, attributes and hierarchies.  We will review how they are related and how they work within Data Warehouses.  If you are looking for an introduction to building a data warehouse click here.

What is a Dimension Table?

A dimension table provides the description behind the analytic numbers.  It describes the who, what, when, where and why behind the facts. Dimensions are normally broken down into groups (tables) and they contain several attributes (columns).   Unlike a fact table the dimension table is not normalized.  Generally, dimension tables have many columns but a limited amount of rows. Dimension tables normally provide two purposes in a data warehouse, it can be used to filter queries and to select data.

Several data warehouses include the following dimension tables products, employees, customers, time, and location.  Lets say your business requirement is to provide an  time tracking data warehouse.  You would to want to implement a employee dimension table that included at least the following attributes (columns) first name, last name, status, start date, hire date, end date, department name title, salary etc….   we could actually go on and on and this is okay as long as the attributes support your business requirements.

image image

What are dimension hierarchies?

Dimension hierarchies provide a way to define a relationship between multiple attributes within a dimension. They are commonly defined as a structure to provide drill up and drill down capabilities.  You can have multiple hierarchies within a dimension. For example, in a time dimension you might want to have two separate hierarchies, one for fiscal year and another for calendar year.  These two dimension could contain the following attributes year, quarter, month, day.

image

Best Practices for designing a Dimension table

  • Use a unique identifier integer column that is auto incremental as your primary key.  This is commonly known as a surrogate key.
  • Use the source’s primary key as an alternating key.
  • Any additional attributes (columns) that describe the business entity

Slowly Changing Dimensions

Now that we know how to build a dimension we need to consider how the data is stored.  Yes, occasionally the attribute data changes.  For example, lets pretend you are a die hard Pirates fan and built a Data Warehouse to do analysis on your favorite players.  How would you handle promotions like Andrew McCutchen getting called up from AAA Indianapolis to the Pirates?

The following are options to handle slowly changing dimensions:

  1. Do nothing (type 0).  This approach is highly not recommended.  This cannot guarantee history preservation and includes the least control over managing changed attributes.
  2. Overwrite old data with new data (type 1)
    This completely overwrites the previous attribute value.  This could be ideal if you had a spelling error or the historical value of this column is not crucial. This is the easiest method to implement but it can be hard to manage.
  3. Create multiple records with different key values (type 2)
    This provides you with the ability to have an unlimited amount of history.  It is usually implemented by adding a start and end date column.  If you have to make changes to past records this could require you to also update fact records.
  4. Create a new column for previous values (type 3)
    This allows only a fixed amount of history to be retained.  If you create two extra columns for the team attribute you can only store three team values (the current value and last two previous values)

I hope you enjoyed this post.  Next we will cover an introduction to facts and measures.

AITP Region 18 Conference

This past weekend I attended the AITP Region 18 Conference in Columbus, OH. This was a great opportunity for me to meet and network with several Information Technology Professionals.  If you have never attended a AITP meeting I highly recommend that you do.

The following was our agenda and a brief afterthought of the topics:

  • Ice Breaker – Got to trade name tags and learn a few facts from a new friend.
  • Leadership – Caught up on the leadership changes within AITP on the national level.
  • VMWare – Learned how VMWare is moving into the cloud.  I was very impressed with the ability to move in and out of the VMWare cloud.
  • Cloud Computing – This really makes it easy to do a SaaS startup.  I am looking forward to trying azure out.

Following the technical sessions we had a great dinner and networking session at the Buckeye Hall of Fame.  Unfortunately, it looked like my presence cursed Ohio State.

Reporting Service Statistics

Question:
How do I obtain statistics for the utilization of SSRS reports?

Solution:

With a little research I found a great sample application on codeplex that provides statistics for SSRS.  The Server Management Sample Reports are available for both SQL Server 2005 & 2008.

The following solution provides answers to the following questions.  What are the top 10 most executed reports? What reports have the longest average execution time?  Who is accessing the reports?

SSRS Analytics

To implement this solution all you have to is download the sample solution and follow the installation guide.  If you have any questions post them here and I will do my best to help you out.

When was that object modified?

Yes, every once in a while when I am deploying an application I ask myself the following question.  What database objects (tables, stored procedures, functions etc..) did I modify with this release?  

Ideally this is documented in the release plan but I will admit I have been known to slip every once in a while.  Therefore, I am showcasing a query that can provide help.  This query was written by Gordon Bell and it can be found here.   It uses the sys.objects DMV that are included in SQL 2005 & 2008. 

I will defiantly throw this script into my bag of tricks. 

select name, modify_date,
case when type_desc = 'USER_TABLE' then 'Table'
when type_desc = 'SQL_STORED_PROCEDURE' then 'Stored Procedure'
when type_desc in ('SQL_INLINE_TABLE_VALUED_FUNCTION', 'SQL_SCALAR_FUNCTION', 
'SQL_TABLE_VALUED_FUNCTION') then 'Function'
end as type_desc
from sys.objects
where type in ('U', 'P', 'FN', 'IF', 'TF')
and is_ms_shipped = 0
order by 2 desc

Get index fragmentation statistics

I recently attended a Pittsburgh SQL Server user group meeting where Brent Ozar gave a presentation on the silent performance killer.  This motivated me to create a stored procedure that could leverage the DMVs in SQL 2005/2008 to gather index fragmentation statistics for all databases on a given server.

Goal

The goal is very simple.  Build a query that could be scheduled to grab statistics that are helpful towards determining if an index needs to be defragged or reorganized.  I would like to throw these results into a table so I could analyze them at a later date.  I would also like to monitor the fill factor and padding to determine if I need to make changes and to analyze if the changes are really helpful.

Download Scripts

The following script uses the following DMV’s sys.dm_db_index_physical_stats, sys.objects and sys.indexes and this script is provided as is.

To download the script click here

To download the create table script for the table used click here


Table Definition

The following is an explanation of the columns.  The following descriptions come from MSDN.

Column Name

Description

databaseName

Name of database, unique within an instance of SQL Server.

objectName Object name.
indexName Name of the index. name is unique only within the object.

NULL = Heap

partitionNumber 1-based partition number within the owning object; a table, view, or index.

1 = Nonpartitioned index or heap.

fragmentation Logical fragmentation for indexes, or extent fragmentation for heaps in the IN_ROW_DATA allocation unit.

The value is measured as a percentage and takes into account multiple files. For definitions of logical and extent fragmentation, see Remarks.

0 for LOB_DATA and ROW_OVERFLOW_DATA allocation units.

NULL for heaps when mode = SAMPLED.

fill_factor > 0 = FILLFACTOR percentage used when the index was created or rebuilt.

0 = Default value

is_padded 1 = PADINDEX is ON.

0 = PADINDEX is OFF.

type_desc Description of index type:

HEAP
CLUSTERED
NONCLUSTERED
XML
SPATIAL

page_count Total number of index or data pages.

For an index, the total number of index pages in the current level of the b-tree in the IN_ROW_DATA allocation unit.

For a heap, the total number of data pages in the IN_ROW_DATA allocation unit.

For LOB_DATA or ROW_OVERFLOW_DATA allocation units, total number of pages in the allocation unit.

date this is the current date GETDATE()

Script

   1:  ALTER PROCEDURE [dbo].[GetStatsForIndexes]
   2:      @PageCount INT = 100
   3:  
   4:  AS
   5:  BEGIN
   6:      -- SET NOCOUNT ON added to prevent extra result sets from
   7:      -- interfering with SELECT statements.
   8:      SET NOCOUNT ON;
   9:  
  10:      -- Declare varables
  11:      DECLARE @dbID INT, @dbName VARCHAR(128), @SQL NVARCHAR(MAX)
  12:  
  13:      -- Create a temp table to store all active databases
  14:      CREATE TABLE #databaseList
  15:      (
  16:            databaseID        INT
  17:          , databaseName      VARCHAR(128)
  18:      );
  19:  
  20:      -- we only want non-system databases who are currenlty online
  21:      INSERT INTO #databaseList (databaseID, databaseName)
  22:      SELECT d.database_id, d.name FROM sys.databases d where d.[state] = 0 and d.database_id > 4
  23:  
  24:  
  25:      -- Loop through all databases 
  26:         WHILE (SELECT COUNT(*) FROM #databaseList) > 0  BEGIN
  27:  
  28:             -- get a database id
  29:          SELECT TOP 1 @dbID = databaseID, @dbName = databaseName
  30:          FROM #databaseList;
  31:  
  32:              SET @SQL = 'INSERT INTO DBA_Tools.dbo.IDX_FRAG (databaseName, ObjectName, indexName, partitionNumber, fragmentation, fill_factor, is_padded, type_desc, page_count, [date])
  33:                  SELECT
  34:                    db.name AS databaseName
  35:                  , obj.name AS ObjectName
  36:                  , idx.name AS indexName
  37:                  , ps.partition_number AS partitionNumber
  38:                  , ps.avg_fragmentation_in_percent AS fragmentation
  39:                  ,idx.fill_factor
  40:                  ,idx.is_padded
  41:                  ,idx.type_desc
  42:                  , ps.page_count
  43:                  , GETDATE() as [date]
  44:              FROM sys.databases db
  45:                INNER JOIN sys.dm_db_index_physical_stats ('+CAST(@dbID AS VARCHAR(10))+', NULL, NULL , NULL, N''Limited'') ps
  46:                    ON db.database_id = ps.database_id
  47:                INNER JOIN '+ @dbName+'.sys.objects obj ON obj.object_id = ps.object_id
  48:                INNER JOIN '+ @dbName+'.sys.indexes idx ON idx.index_id = ps.index_id AND idx.object_id = ps.object_id
  49:              WHERE ps.index_id > 0
  50:                 AND ps.page_count > 100
  51:              ORDER BY page_count desc
  52:              OPTION (MaxDop 1);'
  53:  
  54:          EXECUTE sp_executesql @SQL
  55:          -- remove the database from the databases table
  56:          DELETE FROM #databaseList WHERE databaseID = @dbID
  57:  
  58:          -- get the next database in the databases table
  59:          SELECT TOP 1 @dbID = databaseID, @dbName = databaseName
  60:          FROM #databaseList;
  61:  
  62:      END
  63:      -- temp table is no longer needed, so we will kill it.
  64:      DROP TABLE #databaseList;
  65:  END

Building a Data Warehouse with SQL Server

Today, I am going to start a series of blogs on one of my favorite topics Data Warehousing with SQL Server.  I will start by defining a data warehouse.  I will then give some business cases that explain the benefits of a data warehouse.  Finally, I will start writing about the built in tools SQL Server provides to implement a data warehouse with SQL Server.

What is a Data Warehouse?

In its simplest form a Data Warehouse is a way to store data information and facts in an format that is informational.  Hopefully, you were able to pull this information from the photos above.   Personally, I like to think of a Data Warehouse as a tool used by decision makers to improve decision‐making.

Business cases for a Data Warehouse

The following are several reasons business cases that explain how “insert company name here” can benefit from a data warehouse.

  • A Data Warehouse is fast to query. After the initial setup is complete, queries can run up to 1000% faster in an OLAP database than in an OLTP database.
  • A Data Warehouses improves ROI by allowing end users to make more efficient use of enterprise information so many companies have all the information they need.
  • A Data Warehouse is a good solution for application’s that are great for data entry but lacks in depth reporting and drilldown capabilities.
  • A Data Warehouse could be used to bring several applications and/or data sources together.
  • If you are a service company a data warehouse could be used to analyze work completed to estimate future flat fee engagements. (I can go on and on on these examples…)
  • If you already own SQL Server you can implement a data warehouse solution with the built in tools.  This means no additional cost for software is needed.

What tools are needed to Build a Data Warehouse?

Both SQL Server 2005 and SQL Server 2008 include three tools that are very helpful towards implementing a Data Warehouse. These three tools allow you to create an killer data warehouse.

  • Analysis Services (SSAS)
  • Integration Services (SSIS)
  • Reporting Services (SSRS).

We will go over these tools in great detail in future posts.  For now, let’s just think of SSAS as the tool that provides storage for the data used in cubes for your data warehouse. SSIS is the tool we will use to extract, transform and load (ETL) data into our data warehouse.  SSRS is a tool used to view the data in our data warehouse.

Next post : Introduction to Data Warehouse Dimensions

SQLSaturday Update & Future Presentations

I wanted to give a quick update and thank everyone who has responded to my initial posting about bringing SQL Saturday to Wheeling, WV.    I have spoken to a couple technology groups and several individuals and everyone is very supportive of this event.

The following are upcoming work items:

  1. Establish a budget
  2. Build a support team that will help with organizing the event.
  3. Find a location and pinpoint a date (tentatively First Quarter 2010.)

I look forward to blogging the status of these work items.

On another note, It looks like I will be presenting the following topic “Building a Data Warehouse with SQL Server” twice in November.  I will present first at the Greater Wheeling Chapter of AITP meeting on November 11th.  I will also give the same presentation at the Pittsburgh SQL Server User Group Meeting on November 16th.

You will soon see a series of blog post that go over building a data warehouse.  You can find the first one here.