Thursday, July 16, 2009

Louisville SQL Server Users Group – SSIS (Part 2)

This week, I continued the series of presentations on SSIS that I started for the Louisville SQL Server Users Group.  Here’s a copy of the second presentation in the series for anyone who attended (or missed it).  I extended the demonstration SSIS package that I started, but haven’t quite completed it yet, so I’ll post it in the near future.

 

Unfortunately, I ran out of time to cover the last two topics in the presentation – Event Handlers and DTS Migration.  So, I’ll cover those first in the next session and continue on with as many of the other planned topics as we have time for.

One last thought.  During the meeting, I mentioned BIDS Helper, a must have tool for anyone developing SSIS packages (or SSAS solutions or SSRS reports).  If you don’t have it, get it!

Louisville SQL Server Users Group – SSIS (Part 1)

Back in April, I started a series of presentations on SSIS for the Louisville SQL Server Users Group.  Here’s a copy of the presentation for anyone who attended (or not).  I started building a demonstration SSIS package as part of the presentation and will post it along with one of the next few sessions in the series.

Friday, November 28, 2008

Optimizing Dimension Designs (Series Prelude)

As I indicated in my last post, I presented at the PASS Summit 2008 conference on optimizing dimension designs.  This is a prelude to a series of posts I'll be doing over the next several weeks related to that presentation.

First things first.  If you missed my session, you missed a special appearance by Beaker from The Muppets.  As I mentioned during my talk, I used to work at Intellinet, a great consulting firm located in Atlanta.  While there (from 1997 to 2005), I grew and managed the original BI practice there, trying to manage the likes of Douglas McDowell and Erik Veerman (yes, I'm partially responsible for both of them!)  At any rate, at a Christmas party one year, they presented me with a special award -- my very own Beaker statue.  I certainly don't see the resemblance, but everyone got a kick out of it (and I still have that statue!)  So, to lighten up my session, I started things off right by including this Beaker video -- and explained to everyone that I was a lot like Beaker in that video during the conference as I had to completely rebuild my VPC and demos after an external USB drive crashed on me the day before the conference started -- yikes!

At any rate, enjoy the Beaker video if you didn't catch my session.  And if you managed to catch my session, let me know if you thought Beaker stole the show!!

PS.  Congratulations to Douglas McDowell for becoming a new PASS Board member!

Friday, November 21, 2008

PASS Summit 2008 Session on Optimizing Dimension Designs

Well, its been quite some time since I last blogged.  The summer and fall were a pretty busy time, but I guess everyone is busy, so that's no real excuse.  Although I didn't post about it prior to the PASS Summit 2008 conference, I finished up my presentation at the conference today.  As in prior years, I spoke about dimension designs in SSAS.  This year I included information regarding good dimension designs and a few interesting design scenarios that call for "outside the box" thinking.

Over the next several weeks, I'm planning a series of posts containing information from my presentation.  And I'll hopefully include some details that I had to gloss over based on the limited amount of time available during my session (75 minutes goes by pretty quickly!)  If you happened to catch my session, let me know what you thought -- hopefully it was entertaining if nothing else!  And stay tuned over the coming weeks for additional posts related to optimizing dimension designs.

PS.  If I get permission from PASS, I'll post my slide deck here (waiting to hear if that breaks any speaker rules or not...)

Wednesday, June 25, 2008

Handling Multiple Calendars with a M2M Scenario (Part 2)

In my last post, I started describing a way to handle multiple calendars using a many-to-many scenario.  As I explained, a recent client engagement called for a solution that would allow them to add calendars (customer calendars in particular) without requiring changes to the relational or SSAS models.  In most classic date dimensions, multiple calendars are handled using additional columns for the additional calendar parts that each date has.  These additional columns are then added to SSAS as additional attributes in the date dimension; these attributes are then typically used to create specific user hierarchies for that particular calendar (a Gregorian calendar hierarchy, a fiscal calendar hierarchy, etc.)

The previous post laid the groundwork for creating a multi-calendar solution using a many-to-many scenario.  In that post, I described a new DimCalendar dimension table that would contain a row for each date within each of the calendars needed.  The table contained date-part columns (for things like year, quarter, month, etc.), but these were not specific to a given calendar.  Instead, they contained the appropriate date-parts for a given date within a given calendar.  One thing to note about this approach is that the DimCalendar dimension table will hold N times the number of dates that you might need for all the years you need to represent in your solution.  Thus, if you need 10 years worth of dates, a standard date dimension might contain approximately 3,650 rows; with the DimCalendar table, if you need 4 calendars represented for those same 10 years, the table will contain approximately 14,600 rows.  Given this is a relatively low row count for a dimension table, this shouldn't impose any performance problems with the solution.

With the DimCalendar table available, we can create a new [Calendar] dimension within SSAS using standard dimension techniques.  Using the DimCalendar table design from the last post, the [Calendar] dimension within SSAS might look like this:

 Calendar Dimension Design

A few notes about the design:

  • The [Calendar] attribute contains the calendar names based on the values found in the CalendarName column from the DimCalendar table; the DefaultMember property of the [Calendar] attribute is set to [Calendar].[Calendar].&[Gregorian] so that the default calendar is the Gregorian calendar; this attribute also has its IsAggregatable property set to False.
  • The other attributes are built in a typical fashion, using standard KeyColumns and NameColumn values based on columns available from the DimCalendar table; in addition, standard attribute relationships exist between attributes as would be expected in a date dimension (with the additional step of relating the [Year] attribute to [Calendar]).
  • The [Year-Quarter-Month] and [Year-Week] hierarchies are built in a typical fashion.
  • The [Date Calculations] attribute is based on a named calculation column in the DSV (a standard approach for "hanging" date calculations on an attribute within a date dimension) with a default value of "Current Calendar Date".

With the new [Calendar] dimension in place, we can update the cube to take advantage of it.  Note that in the discussion that follows, I did not remove the existing [Date] dimension from the Adventure Works cube.  Ideally, however, the new [Calendar] dimension would replace it (and could be used in a role-playing fashion just like the [Date] dimension is used already).

The first step to updating the cube is to add a new measure group based on the new FactDateCalendar table.  Remember that this table contains the linkage between the existing DimTime table (which is related to all the other fact tables) and the new DimCalendar table.  The measure group need only contain a single, hidden measure that uses the Count aggregate function to aggregate the row count of the FactDateCalendar table based on its related dimensions.  In the screen shot below, note the diagram showing the relationships between the new tables and the new measure group named "Fact Date Calendar".

New Measure Group

With the new measure group in place, the Dimension Usage tab can be used to relate the [Calendar] and [Date] dimensions to it and create the many-to-many relationship between the new [Calendar] dimension and the other related measure groups.  The following shows the a few of the relationships defined on the Dimension Usage tab.

New Many-to-Many Relationships

Note that the many-to-many relationships use the new "Fact Date Calendar" measure group as the intermediate measure group.  Also note that many-to-many relationships exist between the [Calendar] dimension and the other measure groups (I just chose not to show them all in the screen shot).

With the new measure group in place along with the new relationships, the solution can be deployed, processed, and then queried (using the BIDS browser, the SSMS browser, or a front-end tool like Excel).  The following shows a simple Excel 2007 pivot-table that uses the [Year-Quarter-Month] hierarchy from the new [Calendar] dimension along with measures from the "Internet Sales" measure group.

Simple Pivot-Table with Calendar Dimension

Note the different calendars showing the same grand total, but with different subtotals below that based on the fact that days aggregate to fiscal years differently than they do for calendar years (at least for Adventure Works).  If users want to focus on a particular calendar, they can filter the hierarchy or include the [Calendar] attribute as a page-level filter to get the same affect.  Also note that if a query doesn't include the new [Calendar] hierarchy, everything works as expected given the fact that the Gregorian calendar is set as the default calendar and aggregates and totals work as expected.

Finally, to round out the solution, we can add date calculations to the new [Calendar] dimension and make them work as expected.  Earlier, I mentioned that the [Date Calculations] attribute was added to the new [Calendar] dimension in order to provide a place to "hang" date calculations.  This is a fairly standard approach that has been described (with its various pros and cons discussed) by many different SSAS practitioners over the last several years; thus, I won't go into a lot of discussion about the approach here.  Suffice it to say, the following provides the MDX I added to the calculations script for the cube in order to create three example date calculations (which apply to different sets of calendars -- for example purposes only).

// Date calculations for the [Calendar] dimension

Create Member CurrentCube.[Calendar].[Date Calculations].[Year to Date]
As Null;

Create Member CurrentCube.[Calendar].[Date Calculations].[Year Over Year Growth %]
As Null;

Create Member CurrentCube.[Calendar].[Date Calculations].[Year Over Year Growth]
As Null;

Create Set [Date Calculation Target Measures]
As
{
[Measures].[Internet Sales Amount]
,[Measures].[Internet Order Quantity]
,[Measures].[Internet Order Count]
,[Measures].[Reseller Sales Amount]
,[Measures].[Reseller Order Quantity]
,[Measures].[Reseller Order Count]
};

// [Year to Date]
Scope
(
[Date Calculation Target Measures]
,[Calendar].[Date Calculations].[Year to Date]
);

// [Year-Quarter-Month].[Year] level
(
[Calendar].[Year-Quarter-Month].[Year].Members
) =
Aggregate
(
{[Calendar].[Date Calculations].DefaultMember} *
PeriodsToDate
(
[Calendar].[Year-Quarter-Month].[Year]
,[Calendar].[Year-Quarter-Month].CurrentMember
)
);

// [Year-Quarter-Month].[Quarter] level
(
[Calendar].[Year-Quarter-Month].[Quarter].Members
) =
Aggregate
(
{[Calendar].[Date Calculations].DefaultMember} *
PeriodsToDate
(
[Calendar].[Year-Quarter-Month].[Year]
,[Calendar].[Year-Quarter-Month].CurrentMember
)
);

// [Year-Quarter-Month].[Month] level
(
[Calendar].[Year-Quarter-Month].[Month].Members
) =
Aggregate
(
{[Calendar].[Date Calculations].DefaultMember} *
PeriodsToDate
(
[Calendar].[Year-Quarter-Month].[Year]
,[Calendar].[Year-Quarter-Month].CurrentMember
)
);

// [Year-Week].[Year] level
(
[Calendar].[Year-Week].[Year].Members
) =
Aggregate
(
{[Calendar].[Date Calculations].DefaultMember} *
PeriodsToDate
(
[Calendar].[Year-Week].[Year]
,[Calendar].[Year-Week].CurrentMember
)
);

// [Year-Week].[Week] level
(
[Calendar].[Year-Week].[Week].Members
) =
Aggregate
(
{[Calendar].[Date Calculations].DefaultMember} *
PeriodsToDate
(
[Calendar].[Year-Week].[Year]
,[Calendar].[Year-Week].CurrentMember
)
);

End Scope;

// [Year Over Year Growth %]
// Applied only to the [Gregorian] calendar
Scope
(
[Date Calculation Target Measures]
,[Calendar].[Date Calculations].[Year Over Year Growth %]
,[Calendar].[Calendar].[Gregorian]
);

// [Year] attribute
(
[Calendar].[Year].[Year].Members ( 1 ) : Null
) =
IIf
(
([Calendar].[Date Calculations].DefaultMember, [Calendar].[Year].CurrentMember.Lag(1)) = 0
,Null
,
(
([Calendar].[Date Calculations].DefaultMember) -
([Calendar].[Date Calculations].DefaultMember, [Calendar].[Year].CurrentMember.Lag(1))
)
/
([Calendar].[Date Calculations].DefaultMember, [Calendar].[Year].CurrentMember.Lag(1))
);

Format_String
(
([Calendar].[Date Calculations].[Year Over Year Growth %])
) = "Percent";

End Scope;

// [Year Over Year Growth]
// Applied only to the [Fiscal] calendar
Scope
(
[Date Calculation Target Measures]
,[Calendar].[Date Calculations].[Year Over Year Growth]
,[Calendar].[Calendar].[Fiscal]
);

// [Year] attribute
(
[Calendar].[Year].[Year].Members ( 1 ) : Null
) =
([Calendar].[Date Calculations].DefaultMember) -
([Calendar].[Date Calculations].DefaultMember, [Calendar].[Year].CurrentMember.Lag(1));

End Scope ;



With this script in place, the Excel 200 pivot table shown above can be updated to include the new date calculations (showing just one of the measure included in the set of measures the date calculations apply to).



Excel Pivot-Table with Calendar Dim and Date Calculations



Note that in the screen shot below, a "Reporting" calendar shows up along with the "Fiscal" and "Gregorian" calendars.  The nice thing about this solution is that I didn't have to change the design at all in order to add this new calendar (that's the whole point of the design).  In fact, I had completed the design work described above, deployed and processed the solution, and was browsing it with just the "Fiscal" and "Gregorian" calendars included.  I then used a T-SQL script to add the new "Reporting" calendar by adding rows to the new DimCalendar and FactDateCalendar tables in the relational database.  I then reprocessed the [Calendar] dimension and the "Fact Date Calendar" measure group (without reprocessing any other dimensions or measure groups in the solution) and viola!  The new "Reporting" calendar was established and showed up in my pivot-table.



Hopefully, others out there who may be faced with supporting multiple calendars (beyond the standard "Fiscal" and "Gregorian" varieties) will find this approach interesting and useful.  Let me know what you think...



PS.  If you want to download a copy of the solution, I've made it available as a zip file here:





The SQL database should be restored as "AdventureWorksDW-M2MCalendar" and the SSAS solution should be deployed as "Adventure Works DW - M2M Calendar".  I typically create copies of the base Adventure Works sample databases for creating and testing new ideas and solutions.  Enjoy!