Archive for T-SQL

Cursor Transformation

2015 ArtOne of my favorite presentations to give is called, “Changing Your Habits to Improve the Performance of Your T-SQL.” After giving this presentation in Austin, TX this past January, I had one of my students contact me. He saw in one of my demonstration how bad cursors can perform. He wanted to see if his stored procedure could be written using set theory.

Scenario

He has an application that can only accept 3 data types (NVARCHAR, NUMERIC, and DATETIME), and as all normal database schemas do, his database has more than 3 data types. He needed to identify which column had data types that could not be used and determine which of the 3 data types could be used for that column. The actual conversion was handled by another process, so the information to do the conversion was stored in a table.

The Cursor Approach

His approach was to use a cursor to cycle through all the columns in the provided table, analyze each column, determine the new data type, and store the information in a table variable. After the cursor was completed, the data in the table variable was written to a permanent table for the next process to use.

This approach isn’t necessarily bad. If you are only running it infrequently and you needed to write this stored procedure quickly, then it’s fine. But if this type of stored procedure needs to be run frequently, then it should be rewritten.

The Reason For the Rewrite

Every SQL statement used in a stored procedure can get it’s own execution plan, depending on whether or not it was parameterized. Parameterized queries that are identically written (including case, spaces, and line breaks), can reuse an execution plan. Those that are not, will receive their own execution. Each unique execution plan is stored in the cache. When there are multiple similar execution plans stored in the Cache, it’s called “Cache Bloat.”

Note: This does not apply to databases with the “optimize for ad hoc workloads” setting turned on, but that would be a different blog post.

What does this have to do with Cursors? Each time the Cursor loops (and this includes While loops), each SELECT statement is executed independently and receives its own execution plan. If the query is parameterized, then execution plans can be reused. You can see this by using Extended Events, or Profiler.

Note: If you try this out, don’t do it in production. You’ll be adding load to your SQL Server.

The Cursor

Below is a similar query to what I was sent. This particular solution looks very complicated. There are table variables, one holding the good data types, and one that will hold the needed information for the conversion. There are also several variables to be used in the cursor. Finally the table variable with the needed information, is written to the external table.


USE DemoProgramming
GO
SET NOCOUNT ON;

/* These are the parameters that would be used in the stored procedure.*/
DECLARE
@schema AS NVARCHAR(200) = 'dbo'
,@tableName AS NVARCHAR(200) = 'SalesHeader'
/* table variable will approved data types.*/
DECLARE @legalDataTypes TABLE
(
Data_Type NVARCHAR(500)
);

INSERT INTO @legalDataTypes
(Data_Type)
VALUES
('nvarchar'),
('numeric'),
('datetime');
/* use information_schema.columns to discover information about column types of input table */
DECLARE @ColumnInformation TABLE
(
ColumnName NVARCHAR(100) NOT NULL
PRIMARY KEY
,ColumnDataType NVARCHAR(20) NOT NULL
,ColumnLength NVARCHAR(10)
,NumericPrecision INT
,NumericScale INT
,OrdinalPosition INT
);

INSERT INTO @ColumnInformation
(ColumnName
,ColumnDataType
,ColumnLength
,NumericPrecision
,NumericScale
,OrdinalPosition
)
SELECT
COLUMN_NAME
,DATA_TYPE
,CHARACTER_MAXIMUM_LENGTH
,NUMERIC_PRECISION
,NUMERIC_SCALE
,ORDINAL_POSITION
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = @tableName
AND TABLE_SCHEMA = @schema;

/* The table will be populated iteratively and eventually inserted into the meta data table */
DECLARE @resultSet TABLE
(
ColumnName NVARCHAR(100)
,ColumnDataType NVARCHAR(20)
,ColumnLength NVARCHAR(10)
,Position INT
);

/* These will be set each loop */
DECLARE @columnLength AS NVARCHAR(10)
DECLARE @newDataType AS NVARCHAR(20);

/*setup cursor*/
DECLARE @columnName NVARCHAR(100);
DECLARE @dataType NVARCHAR(100);
DECLARE @characterMaximumLength NVARCHAR(10);
DECLARE @ordinalPosition INT;
DECLARE @numericPrecision INT;
DECLARE @numericScale INT;

DECLARE allColumnsCursor CURSOR
FOR
SELECT
ColumnName
,ColumnDataType
,ColumnLength
,NumericPrecision
,NumericScale
,OrdinalPosition
FROM
@ColumnInformation;
OPEN allColumnsCursor;
FETCH NEXT FROM allColumnsCursor INTO @columnName, @dataType, @characterMaximumLength, @numericPrecision, @numericScale, @ordinalPosition;
WHILE @@FETCH_STATUS = 0
BEGIN
IF @dataType NOT IN (SELECT
Data_Type
FROM
@legalDataTypes)
/* illegal data types*/
BEGIN
IF @dataType IN ('varchar', 'char', 'nchar')
BEGIN
SET @newDataType = N'nvarchar';
SET @columnLength = CAST(@characterMaximumLength AS NVARCHAR(10));
END;
ELSE
IF @dataType IN ('decimal', 'float', 'real', 'money', 'smallmoney')
BEGIN
SET @newDataType = N'numeric';
SET @columnLength = '(19,8)';
END;
ELSE
IF @dataType IN ('bigint', 'smallint', 'tinyint', 'binary', 'varbinary', 'int')
BEGIN
SET @newDataType = N'numeric';
SET @columnLength = '(19,0)';
END;
ELSE
IF @dataType IN ('bit')
BEGIN
SET @newDataType = N'numeric';
SET @columnLength = '(1,0)';
END;
ELSE
IF @dataType IN ('smalldatetime', 'date', 'time', 'datetimeoffset', 'datetime2', 'timestamp')
BEGIN
SET @newDataType = N'datetime';
SET @columnLength = NULL;
END;
ELSE
BEGIN
DECLARE @ret_string VARCHAR(255);
EXEC sys.xp_sprintf @ret_string OUTPUT, '@@columnName = %s has unrecoginzed @dataType = %s', @columnName, @dataType;
RAISERROR(@ret_string,16,1);
RETURN;
END;
END;
ELSE
BEGIN
/* legal data types, don't change datatype but capture correct columnLength */
-- VALUES ('nvarchar'),('numeric'),('datetime');
SET @newDataType = @dataType;
IF @dataType = 'nvarchar'
BEGIN
SET @columnLength = CAST(@characterMaximumLength AS NVARCHAR(10));
END;
ELSE
IF @dataType = 'numeric'
BEGIN
SET @columnLength = '(' + CAST(@numericPrecision AS NVARCHAR(10)) + ',' + CAST(@numericScale AS NVARCHAR(10)) + ')';
END;
ELSE
BEGIN
SET @columnLength = NULL;
END;
END;
INSERT INTO @resultSet
(ColumnName
,ColumnDataType
,ColumnLength
,Position
)
VALUES
(@columnName
,@newDataType
,@columnLength
,@ordinalPosition
);

FETCH NEXT FROM allColumnsCursor INTO @columnName, @dataType, @characterMaximumLength, @ordinalPosition, @numericPrecision, @numericScale;

END;
CLOSE allColumnsCursor;
DEALLOCATE allColumnsCursor;
/* populate two meta data tables*/
INSERT INTO dbo.DataTypeConversion_Cursor
(TableName
,ColumnName
,ColumnDataType
,ColumnLength
,OrdinalPosition
)
SELECT
@tableName
,ColumnName
,ColumnDataType
,ColumnLength
,Position
FROM
@resultSet;

The New Solution

The new solution consists of a permanent table that contains the information, per data type to do the conversion. That allows a join between the conversion information and the metadata found in the system SQL view, “INFORMATION_SCHEMA.COLUMNS”. After the join, the new information can be directly inserted into the permanent table for the next process to consume.


/*Create this table one time*/
CREATE TABLE dbo.DataTypeConversion
(
OldDataType NVARCHAR(20)
,NewDataType NVARCHAR(20)
,columnLength NVARCHAR(20)
)

/*Values for the Data Types that will be converted.*/
INSERT INTO dbo.DataTypeConversion
(OldDataType, NewDataType, columnLength)
VALUES
('decimal', N'numeric', '(19,8)')
,('Float', N'numeric', '(19,8)')
,('real', N'numeric', '(19,8)')
,('money', N'numeric', '(19,8)')
,('smallmoney', N'numeric', '(19,8)')
,('varchar', N'nvarchar', '-1')
,('char', N'nvarchar', '-1')
,('nchar', N'nvarchar', '-1')
,('bigint', N'numeric', '(19,0)')
,('smallint', N'numeric', '(19,0)')
,('tinyint', N'numeric', '(19,0)')
,('binary', N'numeric', '(19,0)')
,('varbinary', N'numeric', '(19,0)')
,('int', N'numeric', '(19,0)')
,('bit', N'numeric', '(1,0)')
,('smalldatetime', N'datetime', NULL)
,('date', N'datetime', NULL)
,('time', N'datetime', NULL)
,('datetimeoffset', N'datetime', NULL)
,('datetime2', N'datetime', NULL)
,('timestamp', N'datetime', NULL)
,('numeric', N'numeric', NULL)
,('nvarchar', N'nvarchar', NULL)

/*paramters that would be used with the Stored Procedure.*/
DECLARE
@Tablename AS VARCHAR(100) = 'SalesHeader'
,@schema AS VARCHAR(20) = 'dbo'

INSERT DataTypeConversion_SetTheory
(TableName
,ColumnName
,ColumnDataType
,ColumnLength
,OrdinalPosition
)
SELECT
c.TABLE_NAME
,c.COLUMN_NAME
,d.NewDataType

/*Conversion for data types*/
,CASE WHEN d.NewDataType = 'nvarchar' THEN CAST(c.CHARACTER_MAXIMUM_LENGTH AS NVARCHAR(10))
WHEN d.OldDataType = 'numeic' THEN '(' + CAST(c.NUMERIC_PRECISION AS NVARCHAR(10)) + ',' + CAST(c.NUMERIC_SCALE AS NVARCHAR(10)) + ')'
ELSE d.columnLength
END AS NewColumnLength
,c.ORDINAL_POSITION
FROM
INFORMATION_SCHEMA.COLUMNS AS c
JOIN dbo.DataTypeConversion AS d ON c.DATA_TYPE = d.OldDataType
WHERE
c.TABLE_NAME = @Tablename
AND c.TABLE_SCHEMA = @schema;

Giving thanks

I want to give thanks to my student, Mark Lai and the company he works for, allowing me to write about this Cursor Transformation. As the geek I am, I thoroughly enjoyed the challenge of thinking outside of the box to rewrite the stored procedure.

T-SQL Tuesday #74 – Knowing When Data Changes Occur in Standard Edition

Robert L. Davis (b|t) is hosting this month’s T-SQL Tuesday blog party. The party was started by Adam Machanic (b|t) in December of 2009.

This month’s T-SQL Tuesday’s topic is on Data Change. This is an important topic for me, since I’m about to create our first Data Warehouse at work. I’ve chosen to champion the companies who are using the Standard Edition of SQL Server.

Crestfallen

I took a class on Change Data Capture this past year at the SQL Saturday in Portland, Oregon. I absolutely loved it. I couldn’t wait to implement it. Then I found out it was an Enterprise Level feature only. I was crestfallen.

I went back to work asking if we could consider using Enterprise SQL Server, I was told that we have too many cores on our Nutanix hardware to make the features we would use worth it. You see, SQL Server pricing is based on the number of cores the underlying hardware has, not the number of cores you utilize in your VM. So what is a girl to do? (And guys, I haven’t forgotten you either.)

Since the Change Data Capture feature is not available to us Standard Edition users, let me take you through another pattern that uses auditing columns on each table.

The Basis of The Pattern I Use

I use a pattern that includes four fields on all transactional tables. This (absolutely) includes lookup tables too. The two table types that are an exception to this pattern are audit tables and error tables. I’ll cover why later in this article.

Four fields include CreatedOn, CreatedBy, UpdatedOn, and UpdatedBy. The dates should be DateTime2. CreatedOn is the easiest to populate. You can create a default on the field to be populated with GetDate().

The other fields can be a little more tricky depending on your environment. There are two patterns for populating CreatedBy as well as two patterns for populating UpdatedBy and UpdatedOn. Below you will find the pros and cons of each.

Pattern One For CreatedBy

The first pattern has the CreatedBy using the SYSTEM_USER function to populate the field. This will retrieve the windows login that the user used to login into the application.

Pros
  • This pattern allows the use of a default on the CreatedBy field. This allows the field to not be dependant on code to populate it.
  • All users have a validated account.
  • Accounts that are used by SSIS or services don’t need to have a row in the User table.
Cons
  • The field will be a varchar or an nvarchar.
  • The CreatedBy field will need to be parsed to find out which user inserted the record for reporting. (One way around this, is to remove the domain before inserting the (n)varchar).

This is my preferred technique since you don’t need to rely on the User table being immediately populated for a user to start using an application. It also makes it easier when the UpdatedBy field is updated through a pattern below.

 

Pattern Two for CreatedBy

This technique is based on an existing User table and uses their ID for the value of CreatedBy.

Pros
  • This pattern allows the CreatedBy field to join to the User table through an integer. Information about the user can be determined easily by joining to the User table but takes less space than the varchar/nvarchar mentioned above.
  • CreatedBy is not relying on windows authentication. This is helpful when not all users of the application have a windows login account.
Cons
  • This pattern requires a row for various tasks in the User table when SSIS or services insert data automatically.
  • This pattern requires CreatedBy to be populated by the application instead of by a default value. This requires working with the AppDev team and getting the AppDev team to write some additional code for you. (If you need to resort to bribing them, I suggest pizza, donuts, chocolate, or beer.)

 

Now let’s talk about UpdatedOn and UpdatedBy. They are a little more tricky to populate since they are populated on an update.

Pattern One For UpdatedBy and UpdatedOn

This pattern requires code to populate the fields since there are no defaults on updated rows. As stated above, you’ll have to work with the AppDev team.

Note: This is not a bad thing. I whole heartedly believe in having a good relationship with the AppDev team. It makes it easier to work with the team when they have the ability to write SQL. It also has the benefit of being included earlier in design sessions. (I personally have been blessed with a great AppDev lead.)

Pros
  • This pattern can be used to populate the UpdatedBy field and even the UpdatedOn fields using similar code.
  • Either User IDs or Windows account can be used to populate UpdatedBy.
Cons
  • The pattern is dependent on the AppDev team to remember to include UpdatedBy and UpdatedOn for every updated row, especially when there are tight deadlines.
  • You need to be vigilant that the fields are always updated, especially with lookup tables.

This is my preferred technique since it doesn’t involve triggers, which we’ll see in pattern two below.

Pattern Two For UpdatedBy and UpdatedOn

This pattern uses the After Update trigger. The reason I choose the After Update trigger, is that it waits until all the changes are made to the table(s) being updated by other triggers.

Note: I do not encourage multiple tables to be updated in a single trigger. (Hmm. This sounds like a good future post.)

Pros
  • This can be used as a stop gap while the AppDev team refactors their code to populate UpdatedBy and UpdatedOn fields as well as CreatedBy.
  • There is no requirement to rely on the application code to populate these fields.
Cons
  • The UpdatedBy can’t be updated with the User’s Id when populating the field.
  • You might not be allowed to use triggers. (Note: Triggers are not bad when used correctly.)

CREATE TABLE dbo.OrderStatusType
(
OrderStatusTypeId INT IDENTITY PRIMARY KEY
,OrderStatusName VARCHAR(50) NOT NULL
,IsActive BIT NOT NULL DEFAULT 1
,CreatedBy VARCHAR(50) NOT NULL DEFAULT SYSTEM_USER
,CreatedOn DATETIME2 NOT NULL DEFAULT GETDATE()
,UpdatedBy VARCHAR(50)
,UpdatedOn DATETIME2
)
GO

CREATE TRIGGER dbo.Trg_OrderStatusType_u ON dbo.OrderStatusType
AFTER UPDATE
AS
SET NOCOUNT ON

UPDATE dbo.OrderStatusType
SET
UpdatedOn = ISNULL(i.UpdatedOn, GETDATE()) -- This makes sure if a NULL was used, it is overwritten with the current Date.
,UpdatedBy = ISNULL(i.UpdatedBy, SUSER_NAME()) -- This makes sure if a NULL was used, it is overwritten with the current user.
FROM
dbo.OrderStatusType AS ost
INNER Join inserted AS i ON ost.OrderStatusTypeID = i.OrderStatusTypeID
GO
INSERT INTO OrderStatusType
(OrderStatusName
,IsActive
)
VALUES
('Created',1)
,('Back Orddered', 1)
,('Shipped',1)
,('Completed',1)

GO

SELECT * FROM OrderStatusType AS ost

UPDATE
OrderStatusType
SET
IsActive = 0
WHERE
OrderStatusTypeId = 2

SELECT * FROM OrderStatusType AS ost

UPDATE
OrderStatusType
SET
IsActive = 0
,UpdatedBy = 'George'
,UpdatedOn = '2016-01-11 17:38'
WHERE
OrderStatusTypeId = 4

SELECT * FROM OrderStatusType AS ost

If you are adding this pattern to existing triggers such as “AFTER INSERT, UPDATE, DELETE”, then make sure to check that the execution of this trigger was based on an update. This can be done by checking if there are any records in the Inserted and deleted tables that are used in triggers.

CREATE TRIGGER dbo.Trg_OrderStatusType_iud ON dbo.OrderStatusType
AFTER INSERT, UPDATE, DELETE
AS
SET NOCOUNT ON

IF EXISTS (SELECT 1 FROM Inserted) AND EXISTS (SELECT 1 FROM deleted)
BEGIN
UPDATE dbo.OrderStatusType
SET
UpdatedOn = ISNULL(i.UpdatedOn, GETDATE()) -- This makes sure if a NULL was used, it is overwritten with the current Date.
FROM
dbo.OrderStatusType AS ost
INNER Join inserted AS i ON ost.OrderStatusTypeID = i.OrderStatusTypeID
END
GO

Exceptions For Audit and Error Tables

These two types of tables are usually populated by inserts only. Most of the time the CreatedOn field is the only field that needs to be added. I would change the name to AuditedOn and ErroredOn respectively since the CreatedOn will most likely represent the creation of the original record.

Why are Lookup tables not in this list? They can be modified over time. Someone may need a value changed or the value needs to be discontinued. The row can’t be deleted when it is discontinued since that would break referential integrity.

In Summary

By using CreatedOn and UpdatedOn you can compare dates when data needs to be processed without having to compare the entire table.

One last thing on the UpdatedBy and UpdatedOn fields. For a long time I insisted that these two fields are NULL during an insert, but I have since changed my mind. By setting the UpdatedBy and UpdatedOn to the same values as CreatedBy and CreatedOn for an insert, you won’t have to deal with the ISNULL function or using the CreatedOn for (some) comparisons.

Thanks For All The Fish

I want to thank Robert L. Davis for hosting this month and I look forward to participating in future T-SQL Tuesday blog parties.

Adventures Into Azure Databases For The First Time (Part 2 of 2)

AdobeStock_52521593_thumb[2]In my last post, I started a two part series to learn how to create and migrate a database from SQL Server 2014 to a new Azure Database.

Recap of My Goals

In Part 1 of this series, I completed the following goals:

  • Activated my free credits through my MSDN license.
  • Created my first empty database.
  • Determined how to connect to the database.

In this post (part 2), my goals are:

  • Poke around the Azure GUI to learn what is available with my free credits.
  • Use my Red Gate tools to create a new schema and load it with data.
  • Learn if I can take a backup from SQL Server 2014 and restore it in my Azure account.
4. Poking Around the Portal

When I was poking around, I found some great features. I discovered these screens by clicking on the database name on the Portal screen (figure A), where I found a series of links at the top that you can click on.

11-29-2015 5-37-20 PM_thumb[1]

 

There is a monitoring screen, which also has some helpful metrics at the bottom, like dead locks. This is an important screen. You don’t want to max out your resources or you’ll be considered a “noisy neighbor” and Microsoft will do something about it.

11-28-2015 5-00-57 PM_thumb[1]

I also found the Scale Out page. This page will allow you to increase or decrease the size of your database. This can be very helpful if your database slowly changes size throughout the month. By changing the service level or the DTU’s to only what you need, you can save money.

5. Upload A Database

I have an existing Database on my SQL Server 2014 server that I want to migrate to my Azure account. I want to first move the schema, then the data. I have always trusted my Red Gate tools to create reliable, deployable scripts. The two tools I’ll be using are SQL Compare and SQL Data Compare.

SQL Compare will compare my existing SQL Server 2014 database schema with my new Azure database schema, which is currently nonexistent. The reason why I like this tool is that it creates serializable transactions for deploying all the individual scripts as one script. If the script fails at any point, then the whole script is rolled back. The second tool, SQL Data Compare, will be used to move the data. I don’t recommend doing this on a gigabyte of data, but for my small database it will work great. It is also a good tool for moving all the values of lookup tables, or for resetting development data to a previous state.

When you open up SQL Compare, you are presented with a screen to set up the two databases. The left hand screen has the database with the SQL Objects you want to deploy and the right hand side has the database you want to deploy to. After you enter the server and login credentials, SQL Compare connects to the Server to obtain the list of databases.

At this point, I learnt another lesson. When I went to get the list of databases from my Azure Server, I was given an error message. It stated that my IP address was not allowed access to the Azure Server. Back to the Portal, I went and I followed these steps:

  1. On the Portal, I clicked on the server name.
  2. I clicked on Configure in the menu bar under the server name.
  3. I was presented with a very helpful page. It told me which IP address was currently accessing the portal. At this point, I clicked on the link “Add to the allowed IP addresses”
  4. Finally, I clicked save at the bottom.

11-29-2015 8-05-39 PM_thumb[1]

 

I went back to SQL Compare, and I was able to retrieve the list of databases from my Azure server. Win!

image

When I clicked the Compare Now button, SQL Compare will compare the two databases and provide a list of deltas. The deltas are broken up into differing groups. The schemas exist in both databases, but differ. One group consists of the schemas that only exist in the database that will be deployed, and the other consists of the schemas that only exist in the database that will be receiving the deployment. You are able to select only the changes that you want to deploy.

image

In this database, there are only two tables and one function that are relevant for my Azure database, so I selected them and then I clicked on the Deployment Wizard button at the top. The next screen asked me if I want to have SQL Compare perform the deployment or to create a deployment script. I chose to create a deployment script. I’m then directed to the screen below.

image

It showed me the script that I can use to do the deployment. When I clicked the “Open Script in Editor” button, a marvelous thing happened. SSMS opened, a tab opened with the script I will be deploying, AND I had been connected to my Azure database with the same credentials I used to create the script.  It also opened the correct database so that I was not left in the Master database.

image

After I executed the script, I had a database schema in the new Azure Utility database.

image

 

Now to get some data.

The SQL Data Compare works exactly like SQL Compare, but it is comparing all the data between two identical (or mostly identical) tables. Note: If the tables don’t have primary keys set, then you can select the field that should be the primary key.

image

Here you can see that I have two tables that I can compare. The ExpandedDate table has +44K records and my Tally table has 1 Billion records to move. The following steps are just like SQL Compare and I can choose whether SQL Data Compare deploys the script or creates a script to be manually deployed.

Since this was my first time migrating data from my local database to my Azure database, I learnt a few things. One, a deployment script with over 1 billion inserts, does not work. (I’m not surprised.)

Two, it took the SQL Data Compare tool a long time to migrate the data. I’m sure my home internet connection (with teenagers using the bandwidth) didn’t help.

6. Restore a local database backup to my Azure Database

While I waited…and waited for my Utility data to be inserted into my Azure database, I did some poking around to see if it was even possible to restore a local SQL Server 2014 backup to an Azure database. Guess what, I found something (And there was much rejoicing).

On CodePlex, I found a SQL Database Migration Wizard that can be used to restore the database. They even had Migration Wizards for 2008R2, 2012, and 2014. SQL Database Migration Wizard v3.15.6, v4.15.6 and v5.15.6

In the post, they also listed under their Reference Materials a “Migration Cookbook” that you can download and use. Migration cookbook now available for the latest Azure SQL Database Update (V12)

Recap

When I first started this two part post, I had only taken classes on Azure topics, but I had never gone through the steps of using the Azure portal. Now I’ve gone on a journey of completing a list of goals, which started with the creation of an Azure account and ended with loading a schema and data to a new database.

I hope you enjoyed this journey and will take a journey of your own exploring what Azure has to offer.

My Eleven Day PASS Summit 15 Experience

“STUEWE!” This is how I knew PASS Summit was about to begin. I was walking to the Friday night SQL Saturday Portland speaker dinner, prior to PASS Summit, when I heard my name being called, well, yelled. Since it was already dark, it took a minute to spot the black SUV with Mike Fal hanging out the window waving at me.

But wait, that is not when my Eleven Day Summit Experience started.

Day One – Thursday

It actually started on Thursday before Summit at LAX. I was there bright and early to pick up two of my Australian friends, Martin Cairney and Ben McNamara, who would be traveling with me to Portland and then on to Seattle. Despite the 14 hour flight, they wanted to go see the Endeavor space shuttle at the California Science Center in LA.

IMG_4924

Day Two – Friday

Friday was spent flying to Portland where we would be attending SQL Saturday Portland. We stayed in an Air B & B house. This is a great way to save some money for lodging and to feel like you’re at home. There is always a coffee machine with decent coffee and the best…no rambunctious kids on the floor above you. WIN!

Friday night, we attended a wonderful speaker dinner (This is when Mike Fal yelled my name out a window of an SUV). Speaker dinners are always fun to attend. You get to catch up with friends and make new ones. Portland is always a bit special, because it’s full of speakers from around the world who made a little extra time to speak at a community event before PASS Summit.

After dinner, we found an amazing restaurant. It had the most scrumptious gluten free bread. I think it was made of clouds.

IMG_4938

Day Three – Saturday

Saturday was spent at one of my favorite SQL Saturdays. Ok, I’ll be honest. ALL SQL Saturdays are my favorite. It’s a great time to do a little extra networking, get to see SQL Family, and meet new people in the community. I also love speaking and that is what I did in Portland. I spoke on SSRS.

2015 PASS Summit12

I was able to attend several wonderful sessions. My favorite was given by Julie Koesmarno and Cindy Gross: Moving beyond Unconscious Bias. I really liked their approach to this topic. Throughout the presentation, they strongly emphasized that we are all good people.

They spoke about how everyone categorizes everything in our lives, but sometimes we have a bias in our categorization, an unconscious bias that we are unaware of. They told us about a study by Harvard called the Implicit Association test, which tests how we categorize various topics. Julie and Cindy then showed us a video of Allen Alda taking the Harvard test. I strongly recommend learning about unconscious bias.

No Portland trip is complete without going to my two favorite places in the evening. We put our names in for our two hour wait at the Multnomah Whiskey Library, and then we headed over to Cassidy’s. They have great food and AMAZING bacon. I ordered enough bacon for all sixteen of us. (And there was much rejoicing). Afterwards we went to the Whiskey Library, where I was finally able to try some Scapa Scotch. I really liked it (Thanks for the recommendation Grant!).2015 PASS Summit13

Day Four – Sunday

One of the cheapest ways to get from Portland to Seattle is the train. When we bought our tickets, they were around $25. Little did we know it would be an unexpectedly amazing day. My friend, Ted Stathakis, was really looking forward to this day too. He loves trains; neither of us realized how amazing it would be.

Last year, there were only five of us on the train. Martin Cairney and I spent the majority of the trip troubleshooting a problem with my VM. Not this time. This time it was completely non-technical. There were sixteen of us in our car, and two who (foolishly) purchased tickets in a nicer car. Why do I say foolishly? Well, we were having so much fun, that they spent the majority of the trip in our car…eating our Voodoo Donuts. Next time, I think they will buy the general tickets. Next time, I’m also going to try getting the WHOLE car too. So if you want to ride with us, keep an eye out for a post from me in the summer.

2015 PASS Summit4

For more details on the fun we had, go look up the twitter handle #SQLTrain.

Day Five – Monday

Monday was a red day. Red Gate day that is. Red Gate puts on a free event called SQL in the City. They have put it on each year on the Monday before the PASS Summit for several years now. They bring quite a few people from the UK to talk about their tools, and they have amazing presentation. This year, they went with a common theme: Continuous Integration. I participated in the Lightening Talks. I spoke on how to use their DLM Dashboard. This is a great tool that keeps an eye on the databases you connect to the DLM dashboard. You can monitor who modified the database and which scripts they ran on the database. Did I mention this is a free tool?

2015 PASS Summit5

Monday night is the annual Networking dinner put on by Steve Jones and Andy Warren. This is a “must attend” event. Why? It’s all about networking and integrating 1st timers into the community. This year, a bunch of us crammed into a booth with four 1st timers. At least two were from Europe. I enjoyed getting to know them and they enjoyed being introduced to all the people that came by our table to say hi.

Note: Remember, networking isn’t just about talking tech. It’s about building relationships so that you can ask technical questions in the future. If you need ideas on how to meet people, take a look at my blog post, Top Ten Ways To Create Your Meet and Greet List For Summit.

Day Six – Tuesday

Tuesday was a relaxing day before the busyness of Summit. I started the day off with breakfast with Chris Yates. This is a rare pleasure. Chris and I met through the SQL Community on Twitter. We now blog together occasionally and I speak at the SQL Saturday in his hometown.

Before I took time to prep for my Wednesday morning presentation, my apartment mates and I went to the Starbucks where they roast all of their coffee beans. I had my first french press, and, I must say, I’m hooked. 2015 PASS Summit2-001

After I was done with my presentation preparation, it was time for the opening event for PASS Summit. This is a whirlwind event. I feel like a butterfly, fluttering from conversation to conversation. This year, I felt like a match maker too. I had three people from my company attending with me. I haven’t had that happen since the mid 90s! I enjoyed introducing them to a number of people.

I also did something different. I went looking for the lone 1st timers. The wallflowers who don’t know who to talk to. I found one. He was eating by himself near a wall, watching everyone by himself. I went up to him, introduced myself, and got to know him. I then asked if there was someone he wanted to meet. He wanted to meet Pinal Dave from India. Pinal and I know of each other, but we’ve never had the pleasure of shaking each other’s hands. So that’s what we did. Luckily, Pinal was standing nearby. Pinal and I finally shook hands, and I introduced him to the 1st timer. That made me very happy. Win!

2015 PASS Summit14

Day Seven – Wednesday

This was my day <deep breaths inserted here>. I was first up to bat <deep breaths inserted here>. I’m nervous as I walk to my room. I see my room and I get calmer. I climb up on the stage and I get calmer. I get my laptop set up and I’m ready to go. Boom.

As you’ve probably noticed, I really believe in networking. My class filled up with plenty of time to spare before the presentation, so I did the same thing I did last year. I had everyone stand up and introduce themselves to their neighbors. I stood on stage with a huge grin on my face, then I stepped off stage and introduced myself to people in the front row.

200 hundred people came to hear my presentation on Sophisticated Techniques in SSRS. I enjoyed every minute of the session. We did have a couple of exciting moments in class though. There was a laptop on the table (not mine) that started beeping. I thoroughly enjoyed slamming nicely shutting the laptop lid. There was also a crash of dishes behind me in the hallway. I said “Opa!” in my head and kept on presenting.

2015 PASS Summit15

With my presentation done, I was able to enjoy classes the rest of the conference.

One class I took was not found in a classroom, but on the edge of a planter on the ground floor. I started talking with this guy (sorry I can’t recall his name, but he was awesome). He was telling me how he had submitted his session, but wasn’t selected. After he told me his topic, I told him I would have taken his class. His presentation was right up my alley. So he asked if I would like to see it. I said yes and we had a wonderful one-on-one discussion, complete with demos on his laptop. Right there. On the edge of a planter. It was my favorite class. THAT is one the many special things about PASS Summit.

One of the things I was really happy about this year was the change in how special diets were handled. In the past, they have always messed up my meals. I have a lot of food sensitivities, so I understand it’s difficult to accommodate my diet, but I still need to eat. This year, they had our names on our meals, and they had the more common special requests handled in the regular food line. (Thank you)

Day Eight – Thursday

I was invited again to live-blog the keynote on Thursday. You can see my comments here. I was very sad to hear that this would be the last keynote for PASS Summit given by David DeWitt and Rimma Nehme. I’m happy that I’ve been able to see two of their keynotes.

2015 PASS Summit Notes1

Before lunch, I thoroughly enjoyed directing people to lunch (I’ll tell you a secret. I think I enjoyed twirling my sign a little too much. Open-mouthed smile).

Thursday is the Women In Technology Day (WIT). WIT puts on a great lunch, and I was in the perfect place to direct people to the regular lunch and the WIT lunch. One of the traditions of WIT day is for men and women to wear kilts. This tradition was started by Grant Fritchey many years ago. This was my fourth year participating.

2015 PASS Summit16

Day Nine – Friday

Friday is full of sessions and bitter sweet goodbyes. Now, I don’t want you to think I only had fun this week. I did attend sessions. I have proof. I also took notes.

2015 PASS Summit17

 

… And I had fun.

 

2015 PASS Summit9

Summit is not complete without at least one night hanging out with Jason Strate at Bush Garden participating in SQL Karaoke. I was super happy to make it there multiple times this year. Last year, I wasn’t able to go at all due to all the commitments I had at Summit.

2015 PASS Summit8

Day Ten – Saturday

Saturday was a vacation day. I spent the day with my Australian friends. These are friends that I’ve made through Summit. I even worked with a some of them for a few months. In the evening, Julie Koesmarno and I threw a dinner party for a few of our friends who were still in town. We were even able to invite some friends we met this year.

2015 PASS Summit19-001

Day Eleven – Sunday

Sunday I had to come back to reality on my flight home. While I had an amazing time in Seattle speaking, learning, networking, and reconnecting, I was happy to see my family and they were happy to see me.

LucybyVictoria

T-SQL Tuesday #72 Summary – Data Modeling Gone Wrong

This month marks the 72nd T-SQL Tuesday.  Adam Machanic’s (b|t) started the T-SQL Tuesday blog party in December of 2009. Each month an invitation is sent out on the first Tuesday of the month, inviting bloggers to participate in a common topic. On the second Tuesday of the month all the bloggers post their contribution to the event for everyone to read. The host sums up all the participant’s entries at the end of the week. This month I’m the host and the topic is …

Data Modeling Gone Wrong

The purpose of SQL Server, is to make sure that the databases are kept safe and run as optimally as possible. The problem is, if the data model is flawed, or not maintained, then no matter how optimally the SQL Server is configured, the database won’t be able to function efficiently.

Below you will find the summary of the 19 posts from this blog party.

Summary

Note: This is a heap. The summary is based on the order received… Well, except Ed’s I inserted him at the top. You’ll see why.

Ed Leighton-Dick
SQL New Blogger Challenge: Week 2 ideas
I found several posts in my comments that simply reiterated my invitation. I didn’t list them below. I am, however listing this one. Ed is challenging new bloggers. I love this idea. He not only broke down my invitation for new bloggers, but gave some ideas on how to find a topic that will fit into the prompt. Thanks go out to Ed for encouraging the “next generation” of SQL Bloggers.

Thomas Rushton – The Lone DBA
T-SQL Tuesday #72 – Data Modeling Gone Wrong
He wrote about generically named Fields and overloaded Fields. This is a very important topic and I’m glad it’s at the top.

Robert Pearl
T-SQL Tuesday No.72 – Is Your Data Model, Normal?
Robert started off his post with, “Well, I’m not a data modeler, nor do I play one in the corporate world.” He then goes on to tell us how important data modeling is. Finally, he goes over the basics of the 3 forms of normalization, which he feels everyone should know. (And I wholeheartedly agree with him.)

Chris Yates

T-SQL Tuesday #72 Invitation – Data Modeling Gone
Wrong

Chris, one of my dearest friends, wrote a fabulous topic on what you should think about and do prior to creating that first table. He then went on to talk about some good general practices to keep in mind while creating the data model.

Tamera Clark
#TSQL2SDAY – Data Modeling Gone Wrong

I really liked Tamera’s approach to the topic. She goes into the realities of teams not admitting there are problems due to

  • “Reasons”
  • Applications
  • Egos

I’ve been there, seen that and have the t-shirt.

Andy Galbraith
T-SQL Tuesday #72 – Implicit Conversion Problems

This is one of those topics that people, who don’t write SQL, often forget about. Implicit conversations and how they impact queries. This is a great read.

Tim Peters

That time a beer broke my database – T-SQL Tuesday
#72

Tim has a great example of one of his data modeling experiences about finding out (after the data model has been deployed) that another table to hold multiple breweries was needed.

Side note: He has a great website, Beer Intelligence Project, where he has documented and visualized new beers. I think he probably enjoyed the research tremendously too.

Rob Volk
T-SQL Tuesday #72: Data Modeling
As usual, Rob tells a great story. This one is a fictitious story about how a small ice cream store grew over time. It’s told from the point of view of the database. The database started as a small database and grew until paralysis hit, I mean performance issues hit.

Steve Jones
Poor Data Modeling – T-SQL Tuesday #72
Steve also has a great topic about data modeling morphing into a difficult model. I’ve worked with the same data models that he describes here. That of a hierarchal data model that can get out of hand if it is not planned properly. The moral of the story is, when a data model is being designed, be forward thinking.

Mike Fal
#TSQL2SDAY: You’re Gonna Cluster that GUID
Mike attacked a very controversial data type: The GUID <sinister music inserted here>. This is a great topic to add to this collection of posts. Mike makes a great argument on why GUIDs should be indexed. He also specifies a specific use case for it.

Aaron Bertrand

T-SQL Tuesday #72 : Models Gone Wild!

I think Aaron and I were separated at birth. I preach the same list of “database sins”. In Aaron’s post he goes over two categories of “sins”, bad names and incorrectly used data types. I think the only thing I would add to his list would be to not use the default SQL Server constraint names. Give them meaningful names!

Kenneth Fisher
Why are you still using datetime?
Kenneth brings a great question to the table. Why are you still using datetime? He goes into the different date data types that are available and why we should be taking advantage of them.

Anders Pedersen
T-SQL Tuesday #72: Data modeling gone extremely wrong
Anders gives a good example of how an over-normalized database can slow down your application.

Terry McCann
Tally/Number tables in SQL Server #TSQL2sday
Terry took a different approach. He looked at the prompt from the point of implementing some best practices instead of identifying bad ones. He wrote about how Tally/Number tables can help with queries to find missing data.

Rob Farley
What’s driving your data model?
Like Chris Yates, Rob wrote about considering the data models purpose BEFORE creating it. The difference between the two posts, is Rob took it from the data warehouse data model point of view.

Malathi Mahadevan
TSQL Tuesday #72 – Data Modeling gone wrong
My good friend Mala, had a great topic for her post. She talked about the “one table to rule them all” pattern that she encountered at one of her jobs. I really liked one of her sentences in her summary, “The big lesson that came out of it was that the size of the schema or fewer tables do not really mean a simpler/easier to manage design, in fact it can be the exact opposite.”

Sander Stad
Data Modeling Gone Wrong
Sander painted a story of XML woe. I want you to be prepared when you read his excellent post. He has an image of an execution plan that will make your hair stand on end. Make sure you are sitting down.

Jens Vestergaard
#TSQL2SDAY – Data Modeling Gone Wrong
Jens has a horrifying tale of bad field name combined with the lack of foreign keys. I may have bad dreams about this one tonight. He does have a great reaction GIF that summarizes how any of us would feel about being faced with the database he describes.

(Update…I forgot to add my own post. Oopse.)

Mickey Stuewe
T-SQL Tuesday #72 – Bad Decisions Made With Surrogate Keys
In my post, I go into when you should and should not use Surrogate Keys as primary keys. I then give a case on how surrogates can cause duplicate data in a many to many table.

Thanks for all the fish!

First, I would like to point out how cool this blog party is. Even though Adam lives in the US, this is not just a US blog party. It’s international. In my list I have posts from The Netherlands, the UK, Denmark, and Australia. I think that is really cool. The one thing that I would like to see is, more women joining the blog party. Including myself, there were only three women who participated. While I’m very happy that Tamera and Mala joined the party, I would like to see more in the future.

T-SQL Tuesday #72 – Bad Decisions Made With Surrogate Keys

This is my second time hosting the t-SQL Tuesday blog party. The party was started by Adam Machanic (b|t) in December of 2009.

This month’s invitation topic is on Data Modeling Gone Wrong. Being a Database Developer, I deal with bad database design decisions daily. One of my app-dev teammates loves to tell me that the bad decisions were made because I didn’t work there yet. (That makes me laugh.)

Surrogate Keys vs Natural Keys

The point of surrogate keys is to represent complicated natural keys as the primary key of the table. Both the surrogate key and natural key will yield a unique key for the row. Sometimes that unique natural key is the entire row. When possible, it is better to use the natural key since it is the true representation of the row. Unfortunately, this is not always practical. Let’s look at some examples.

TSQLTuesday72 Image1

In the employee table it would take four fields to make a primary key from the natural key (first name, last name, social security number, and birthdate). Note: This is assuming this table is only used in the US and the employees have social security numbers. The reason the birthdate is also needed is due to the fact that social security numbers can be reused after someone has passed away. For the employee table it makes sense to have a surrogate key since it would be cumbersome to use all four fields as foreign keys in other tables.

The StateList (representing each state in the United States) is a good example of using the natural key as the primary key. Each state only uses two characters to represent the State so CHAR(2) can be used for the natural key and the primary key. This would provide the added benefit of not needing to join back to the StateList to get the two character representation of the State abbreviation…unless additional information about the state is needed. So what is the point of this table? Well, by having it, you are guaranteed referential integrity on the StateCode field by having a foreign key back to the StateList table. You don’t have to worry that someone puts ZZ as a StateCode.

Danger, Will Robinson!

One of the problems I’ve seen with careless use of surrogate keys are the duplication of natural keys. Quite often it’s overlooked that the natural key still needs to have a unique constraint. Without it, the reporting team ends up having to use MAX or DISTINCT to get the latest instance of the natural key, or SSIS packages are needed to clean up the duplicates. This can be compounded with many-to-many tables.

Many-to-many tables allow two tables to be joined multiple times. An example can be seen in the car insurance industry.  If you have multiple people on the same insurance and they are registered to drive multiple cars, then a many-to-many table would be created to capture the data.

If a surrogate key is used on the many-to-many table in order to provide uniqueness and if the natural key does not have a unique constraint, then duplicate natural key combinations can occur. This can be obfuscated if there is additional information in the table. Maybe the amount the car is insured, is also maintained in this table. Let’s take Victoria’s insurance as an example. If Victoria is in the table with her 1971 Corvette listed twice with two different insurance amounts listed, which one is the current one? The better pattern in this case would be to use the natural key.

TSQLTuesday72 Image2

Conclusion

Surrogate keys are very useful, but it should not be assumed that they should be used for all tables. The natural key should always be considered first. If the natural key is too complicated to be used as foreign keys in other tables, then the surrogate key is a good choice. Just remember to ALSO put a unique constraint on the natural key.

Thanks for all the fish

I had several people tell me on Twitter that they were going to write their first blog post for this t-SQL Tuesday blog party. I want to thank them ahead of time for taking the leap into the blogging world to share their experiences and expertise in their fields.

T-SQL Tuesday #72 Invitation – Data Modeling Gone Wrong

This month marks the 72nd T-SQL Tuesday.  Adam Machanic’s (b|t) started the T-SQL Tuesday blog party in December of 2009. Each month an invitation is sent out on the first Tuesday of the month, inviting bloggers to participate in a common topic. On the second Tuesday of the month all the bloggers post their contribution to the event for everyone to read. The host sums up all the participant’s entries at the end of the week. This month I’m the host and the topic is …

Data Modeling Gone Wrong

The purpose of SQL Server, is to make sure that the databases are kept safe and run as optimally as possible. The problem is, if the data model is flawed, or not maintained, then no matter how optimally the SQL Server is configured, the database won’t be able to function efficiently.

I would like to invite you to share some data modeling practices that should be avoided, and how to fix them when they do occur.

Rules for T-SQL Tuesday Blog Party

Rule 1: Make sure that you include the T-SQL Tuesday image at the top of the post which will help identify your post as a T-SQL Tuesday blog post.  Please include a link back to this invitation too.

Rule 2: Publish your post sometime next Tuesday using GMT. Here’s a link to a GMT time convertor.  For example, in California, that would cover 5 pm Monday to 5 pm (PDT) Tuesday.

Rule 3: Come back here and post a link in the comments so that I can find all the posts for the round up.

Rule 4: Don’t get yourself fired. Make sure that you either generalize your post or get permission to blog about anything from work.

Rule 5: If you roam the Twitterverse, then don’t forget to Tweet about your blog post with the hashtag #tsql2sday.

Rule 6: Go read someone else’s blog on the subject!

Final Rule: Have fun!

Top 10 Ways To Create Your Meet and Greet List For Summit?

There was a time not to long ago when I didn’t know how to meet others in my profession. I didn’t know about SQL Saturdays or PASS Summit. The conferences that I was starting to attend were full of people who really didn’t want to network. But I’m persistent. I was determined to find a community of professionals who wanted to network. Then I found my first SQL Saturday and I fell in love with the community that PASS helps create the world over. I now have friends on almost every major continent? (Are there any SQL professionals in Antarctica? If so, I want to meet you.)

Who you gonna meet?

As each PASS Summit approaches, I make a list. I check it twice. And I decide who I’m going to meet. My question to you is, who are you going to meet at PASS Summit (or at your next SQL Event)? How do you decide who you want to meet? If you are looking for ideas on how to make your list, and you should have a list, keep reading.

My Top Ten List of How I Pick People to Meet

 

1. Set up a Twitter account to get to know the #SQLFamily community

The first thing I did before my very first Summit was creating a Twitter account. The SQL community has a huge presence there and it is a great place to get to know people from all over the world. You’ll not only connect with other individuals, but you’ll also see tweets of links to great articles that people share. You’ll also have a place to ask others how they solved the problems you are now facing through the hashtag, #SQLHelp.

I had several people on my first “Meet and Greet” list whom I had met this way. One of them was Ed Watson, whom I’m still friends with.

IMG_3221We chatted on twitter often. It was great meeting Anil in person.

Note: I recommend reading this before acquiring your first Twitter account. http://www.brentozar.com/twitter/book/

2. Consider the bloggers you follow

I checked out the list of blogs that I read and compared the authors to the list of attendees to see if any of the bloggers I knew were attending.

1651Ola Hallengran is known for his maintenance scripts.
We connected at a karaoke bar.

 

3. Ask the people in your local community if they are attending

They will be able to introduce you to other people during the event. I met several people at the SQL Saturday in San Diego who were also going to Summit. They were happy to introduce me to people at the various events we attended.

723I know Phil from the local user groups.
He helped introduce me to other people.

4. Consider the speakers of the sessions you are attending

As you determine which of the sessions you want to attend, read up on the instructors. They all have small bios on the PASS Summit site. You can also check out their blogs. If they have something in common with you, or if they really helped shape your career, then add them to your list. Just don’t make your entire list out of the speakers. You need variety.

IMG_3245

Jes Borland is an amazing speaker.
I’m so happy I’ve gotten to know her.

Note: I would recommend introducing yourself to speakers you want to meet at various after parties, during lunch, or as you see them in the halls. They are usually super busy right before their sessions setting up and right after their sessions answering questions.
But wait! There’s more!

Those are the easy ways to create a list before the event. But don’t stop building your list after the event starts. The list you bring with you is just the beginning. Keep reading to find out how to add to your list during the event.

5. Go to the networking parties in the evenings

At PASS Summit, there is a Networking Party put on by Andy Warren and Steve Jones. GO TO IT. Sit with people you don’t know. I know I will be. This event is not a sponsored event. In other words, you need to pay for your food and drink, BUT the networking is free and encouraged. Register for it here.

When I went to my first one, I met the lovely Viki Harp. She introduced me to Wendy Pastrick who whisked me away to meet Pam Shaw. It was actually amazing that I ate any food at all. It was so much fun meeting new people.

MickeyAndJasonJason and I are connecting at the
Summit Networking event.

6. Sit with new people during breakfast and lunch

At my first Summit, I only knew the people I had met at my first SQL Saturday, and I was very determined not to eat a single meal by myself. So I didn’t. Every morning, I got on Twitter and asked if I could join anyone for breakfast at the Daily Grill. I used the hashtag #Summit2012 (this year it will be #Summit2015, obviously). And guess what. I never ate alone. This wonderful woman, Monica Rathburn asked me to join her almost every morning.

1143This was my last breakfast at my first Summit.
We started with four people.

7. Consider people in your sessions

Introduce yourself to people sitting around you before the session starts. Or strike up a conversation about the session with someone after the session has ended.

722Ritu and I connected when we realized
we kept attending the same sessions.

8. Hang out at the Community Zone at PASS Summit

This is a great place to meet people. Why? That is the purpose of the Community Zone. Usually there is a schedule for various groups of people to be in the community. So, if you really want to meet the Australians, then show up during the hour to hang out there. If you want to meet people from your own region, then come when they are scheduled to meet up in the Community Zone. (The best part is there are awesome bean bag chairs to sit in.)

1648Tjay and I ran into each other in
the Community Zone.

9. Attend the after parties

Attend as many after parties and other non SQL events as you can. Yes, quite a few of them have drinking, and that might be an issue for you, but not all of them do.  Here are some other events that have little to no drinking that are usually found at PASS Summit:

  • Running. That’s right, there is a large group of runners who get up when I’m still dreaming and go for a run. They usually have cool SQL shirts and Jes Borland is usually found leading the pack of SQL runners.
  • Board game night. Last year there were a couple of nights where people gathered around board games to talk and have fun.
  • PASS Prayers. This is a Christian group who meets for prayers and fellowship in one of the hotel lobbies in the morning, again when I’m still dreaming.
  • Photo walk: This is a great way to learn about Seattle, get a good walk outside, and get to know other SQL Photography lovers.

2014-11-07 21.39.38We were hanging out at an event in the evening.

Note: All the events that PASS knows about will be put on this page a few weeks before PASS Summit starts.

10. Attend other events that occur around PASS Summit

Last but not least, attend Redgate’s SQL in the City event on the Monday before PASS Summit. This is an amazing free event put on by Redgate. They have several speakers speaking on various topics. There’s also a free lunch and networking at the end of the event. When you are done, you can head over to the Networking dinner I mentioned in No. 5 above.

downloadSebastian Mein and I had a photo op with
the lovely Carly from Redgate visiting from the UK.

Hi. My name is…

One of the things you can do when you are talking to people is give them your card. Wait…You don’t have one? That is easily fixed. Vista Print is where I make mine and they always seem to have discounts. Since the card is about YOU and not your company, just put the contact info you are interested in sharing. I put my name, title, email, and a picture of myself.  My first year, I came home with 50 cards from other people. I wrote on the back where I met them. The following year I went through them and I was amazed at how many I still remembered and even interacted with through other SQL events and through social media (mostly Twitter).

Back to talking to people

So, you’ve got your list and you are standing in front of someone you wanted to meet. Now what?

If they’re not considered “famous”, then ask them if you could buy them a drink (coffee, soda, or bottled water works, too) or ask if they have time to meet in the Community Zone to chat. The Community Zone is usually full of awesome bean bag chairs to sit and talk in.  Tell them you wanted to meet them to talk about xyz, and xyz doesn’t have to be about SQL. Maybe you both enjoy art, or learning about Whiskey distilleries. (Oh wait, that’s me.)

2014KY4ChrisYatesChris and I will be reconnecting over breakfast this year.

If you consider them “famous”, thank them for writing/speaking/inspiring. If they have time to talk, tell them about yourself and maybe ask them a question about SQL.

What if you are shy or an introverted?

You can still make connections. You only really need to make one strong connection. It’s ok if it takes more than one Summit to develop. I have SQL Family friends that are shy/introverted. I make sure they go out to some of the events and are having a good time. I help with making introductions for them to make connections with people on their “Meet and Greet” list.

Anecdote: My first Summit I met someone who was shy. We saw each other again our second Summit, but it wasn’t until our third Summit when we developed a stronger connection. I know it was hard for them, but they called me and asked if they could go with me to an event. They weren’t comfortable going by themselves. I was happy to go to the event together. I was also happy to introduce them to other people in the community. I’m really looking forward to spending more time with them this Summit.
Follow the White Rabbit

In the end, it’s all about making connections. If Neo hadn’t followed the white rabbit, he wouldn’t have met Trinity who took him to Morpheus. These connections are not just for the yearly PASS Summit. They are there for as long as you nurture them. Some of the people I’ve met, I only see at Summit, some I see four or five times a year, and some I talk to every day through social media and Skype. These connections help remind me that I’m not crazy when the App Devs tell me that Foreign Keys are bad, and they help me when the poo hits the fan and I need to restore data in a way I’ve never done before. Finally, they are there for me because we are all part of this huge community, affectionately dubbed “SQL Family”.

Questions Answered From Presentation: SSRS 101 Creating Reports for Diagnostic Data

On Tuesday, July 7th, 2015, I had the privilege of speaking for the DBA Fundamentals Virtual Chapter. Thanks to the 375 who attended and to all who will be watching the recording. While I can’t read the comments during my presentation (way too distracting), I did enjoy reading the questions and comments that were sent to me. Below you will find answers to the questions I was sent.

Questions Answered

1. Are you talking about SSRS 2016 or 2014?

I was presenting using SSRS 2012, but the demos and discussions applied to versions 2008 through 2014.

2. When should you use SSRS RDL vs. RDLC (Remote Definition Language Client-side)?  Should you be able to invoke RDL from a web app (say web forms, asp.net) just as easily as RDLC?

The difference is RDLC reports will run on the client side. They don’t need to access the report server. RDL reports can be invoked on the client side, but they are rendered on the report server then delivered to the client. The RDLC reports would be part of an application and will take up client resources to generate.

3a. What is the difference between SSRS and Reports Application in Visual Studio 2010 (full version)?

3b.Why use SSDT over Visual Studio?

I liked both of these questions, and they have the same answer, so I wanted to group them together.

SSRS (SQL Server Reporting Services) files are generated in two applications, Report Builder and Visual Studio. Report Builder exists on the Reporting Server called Report Manager. Visual Studio is the Integrated development environment (IDE) that you write code in. Microsoft created (at least) two distributables for Visual Studio. The one that Application Developers, is only known as Visual Studio and the one that comes with SQL Server called SQL Server Data Tools (SSDT). The IDE is the same for both of them. When they co-exist on the same computer, they will be integrated. If you only have the SSDT version, then you can only create SSRS, SSIS, and SSAS projects. You can also download over project types for Visual Studio, such as PowerShell projects.

Note: For SQL Server 2008 it was called Business Intelligence Development Studio (BIDS). It was renamed to SSDT in version 2012.

3a. Is it better to use SSDT or Report Builder? I have been using Report Builder for my reports so far…

The answer to this depends on your comfort level for creating reports. Report Builder is meant for the power users in the business. SSDT is meant for developers. I’ve personally never used report builder, but that’s because I’ve been writing reports since before Report Builder was introduced. If you continue to create reports in Report Builder, then take a look into generating Report Parts. Those were introduced in SQL 2012 and are supposed to be “building blocks” to help create more complicated reports within Report Builder.

4. If you don’t have the source of the report can you get it back from the web

I am happy to tell you, yes! (and there was much rejoicing!)

In the Report Manager, pull down the menu to the right of the report and select download. You can also go into the properties of the report and see the download option in the toolbar for that report.

Answers201507_01

5. How did you link the two datasets?

The two datasets weren’t linked like you would see writing a JOIN statement. They were filtered using the same parameter. Datasets can’t be joined together within the report. There are a couple of functions introduced in SQL 2008 R2 that allow you to reference a single value from another data set, but that is as close as they have gotten.

6. How is the security configured within the data sources ? How can the double-hop authentication issue be avoided when accessing data from multiple servers ?

You have 4 different security options that you can use. Books online can tell you the details on each of them.

Answers201507_02

I personally use a SQL Server login, only used for reports to access the data. I then use windows authentication to access the reports themselves. Each data source can connect to a different server and even use a different SQL Server account.

I’m not sure I’m answering your question adequately since it can be read a couple of different ways. If you are looking for a solution to a double-hop authentication problem, then take a look at this article. I’ve faced this once, but it was about 5 years ago. The Double-Hop Authentication Problem

Send me an email if you still have questions on this topic.

7. I saw that you saved the password for the data source (during development). How can we ensure that it is encrypted when deployed to the server?

I would have to do some research to see how the password is sent during the publishing event, but I do know it is stored encrypted.  In fact, the Connection String, UserName, and Password are all encrypted. Go into the ReportServer database and run this query: SELECT * FROM ReportServer.dbo.DataSource.

If you are still concerned about the Data Source during the publishing event, then you can create the data source manually in Report Manager. As long as the SSRS project has the property “OverwriteDataSources” set to false, the data source will remain the one you manually created.

8. What if you want to show multiple databases (in your report)?

You can create as many data sources as you need and they can each point to a different database on a different SQL Server.

If you are gathering diagnostic data, like we were doing in the demo, then you might want to consider a two step approach. The first step would be to have an SSIS package or a PowerShell script retrieve the diagnostic data from each server on a schedule and save the data to a central database. Then you can have the report pull the diagnostic data from that central database.

9. Can hidden parameters have values passed to them, for example: can a hidden parameter list containing state names have “CA” passed to it when the report is opened?

Yes. That would be done in the parameter properties on the report in the Report Manager, from code, or from another report. You can also create Linked Reports in the Report Manager and change the values of the hidden parameters.

10. Is there any difference between previewing the report than just running it by r-clicking on the report?

Not really.

11. Can you set border style in your template? So it’s not there at all?

Yes. If you were to save a new tablix in the template with the borders set to None for each row. Then you would copy that tablix for each tablix you need. Unfortunately you can’t modify the actual tools in the toolbox.

12. Any strategy to fine tune multiple drop down option/filter in SSRS report.

Yes.

1. Make sure that the Queries that are populating the drop down lists are FAST.

2. If the drop down lists are filtering each other, known as cascading parameters, then have them filter within the report, instead of making a round trip back to SQL Server for the filtering. This would be done by applying the filter to the 2nd parameter in the filter property instead of attaching it to the parameter property.

3. Make sure that the predicates of each of the queries and the final data set have proper indexes.

4. If the queries that the drop down lists are using are really slow due to too many JOINs, then consider using an SSIS package to create a static table that is updated every X minutes/hours with the latest data. Then use that table for the parameter lists.

13. This is not the question you are looking for. Move along. Move along.

14. How to display a message in the report if the dataset doesn’t return any records ?

This is an excellent question and the answer is not used enough.

Set the NoRowsMessage property on the tablix (Table/Matrix/List) control. You can get to it through the property window. The value of this property is displayed when there is no data to show. The value can even be an expression.

15. How to get a big report to limit to 4000 records but well distributed in terms of days in a month?

I would love to have more information on why you have this requirement. It is an interesting one. Here is what you can do. Note: It will slow the query down though, so make sure you have good indexes on the predicates.

1. I have 1 million rows of random dates and numbers. This happens to be a very narrow table.

Answers201507_03

2. I use the ROW_NUMBER() function. It will give a sequential number to each row based on the partition. I partitioned the data based on Year, Month, Day. If your date field does not have time, then you can partition based on the date field. Within each partition, the data is sorted by Date then TestDataID (the PK). This will help guarantee the same order each time.

3. I determined how many days were in the range of dates I’m selecting from, divided 4000 by that number, and select only those row number values per date. This will provide an even distribution across days.

4. Since you can’t guarantee that the number of days will divide evenly into 4000, you need to either have less than 4000 rows returned, or more than 4000 returned by adding 1 to the number created in step 3.

Answers201507_04


–1 million rows
DECLARE @StartDate AS DATE = ‘1/1/1972’;
DECLARE @EndDate AS DATE = ‘1/1/1973’;

SELECT DATEDIFF(DAY,@StartDate, @EndDate) AS NumberOfDays;
WITH cte_ranking
AS
(
SELECT TOP 100 PERCENT
TestDataID
,SomeDate
,SomeTextNumber AS SomeNumber
,ROW_NUMBER() OVER (PARTITION BY dateyear, datemonth, dateday ORDER BY datevalue, TestDataID) AS RN
FROM
DemoProgramming.dbo.TestData AS td
JOIN dbo.DimDate AS dd ON td.SomeDate = dd.DateValue
WHERE
td.SomeDate >= @StartDate
AND td.SomeDate <= @EndDate
ORDER BY datevalue
)
SELECT
cte.TestDataID
,cte.SomeDate
,cte.SomeNumber
FROM
cte_ranking AS cte
WHERE
cte.RN <= (4000/DATEDIFF(DAY,@StartDate, @EndDate)) + 1;

–Returned 4038 rows

[\SQL]

16. With subscriptions is it possible to make one subscription to use the current fiscal month to generate report vs. creating 12 subscriptions for each fiscal month?

Unfortunately no. You can do it based on calendar months, but not based on fiscal months where the first day of the fiscal month may not be the 1st day of the calendar month. I would love to see this feature.  If this is a must have requirement, then email me and we can talk about some “creative” solutions.

17. What are the different security roles on report server side?

The SSRS development team were kind enough to add the definitions on the security screen.

Answers201507_05

 

18. Can you append the date/time to the file name that gets created with a subscription that saves to a fileshare?

Yes. Add the @ExecutionTime parameter to the filename.

19. How do you set-up the email option for the subscription?

First your Report Manager needs to be setup to send email. I usually have my Sys Admin help me with that.

After that, it’s a matter of populating fields (and there was much rejoicing.)

If you use the standard subscription, then the setup screen will look like figure A below. If you are creating a data driven subscription, then the setup screen will look like figure B below. Either way, you need to set all the properties. Then select a schedule for the emails to go out on.

I would highly suggest using Active Directory groups for the To list. This make is easier to manage when people change jobs. You may also want the email to go to yourself for a few days or weeks to make sure it is going out as predicted.

figure A

Answers201507_06

figure B

Answers201507_07

 

Thanks for all the fish

I wanted to give a shout out to Glenn Berry for letting me use his diagnostic queries for my demos. You can find the full diagnostic script, per SQL Version here on Glenn’s website.

The downloads for this presentation are available on the DBA Fundamentals Meeting Archives page and will be available on my website under Resources shortly.

T-SQL Tuesday #66: Monitor ALL Reports

Catherine Wilhelmsen (b|t) is hosting this month’s T-SQL Tuesday blog party. The party was started by Adam Machanic (b|t) in December of 2009.

This month’s invitation is about monitoring.

This topic is a tough one. I didn’t want to pick just one topic. Especially since I just installed SQL Sentry’s Performance Advisor in production and the Redgate’s DLM Dashboard in Dev. Both products are helping me see my environments clearer.

But what about the need to monitor other items that don’t have a cool tool you can buy off the shelf? Or products like SSRS that have built in metrics collection?

That is when you need to roll your own code to capture metrics to help monitor your own, unique environment.  Take Excel reports as an example, or really any application that is used for reporting. It would be wonderful if you could capture metrics and monitor usage of ALL reports across your environment in a consistent manner. Let’s take a look at a way that can be accomplished.

In the beginning

The first thing you need to do is some planning. You need to know what kind of data you want to capture, and how you are going to update the existing reports to capture that data.

I happen to really like the way that SSRS collects usage metrics. So, for my Excel reports, I wanted to collect similar data. This will allow me to have one report to show how all my reports are performing. Maybe my top 10 slowest reports are from 3 different reporting platforms.

I created a couple of tables to hold the non-SSRS report metrics and meta data. I then created a stored procedure to insert the data into the tables. Finally, I added the stored procedure to the bottom of the stored procedures used by the Excel reports I wanted to keep track of.

Step 1

I created a table that held the information I wanted to collect.


CREATE TABLE dbo.ReportType
(ReportTypeID INT IDENTITY PRIMARY KEY
,ReportTypeName AS varchar(20) NOT NULL
)

INSERT INTO dbo.ReportType(ReportTypeName) VALUES ('Excel');
INSERT INTO dbo.ReportType(ReportTypeName) VALUES ('PowerPoint');

CREATE TABLE dbo.ReportLog
(ReportLogID INT IDENTITY PRIMARY KEY
,ReportTypeID INT NOT NULL
,ReportPath VARCHAR(255) NOT NULL
,ReportName VARCHAR(255) NOT NULL
,UserName NVARCHAR(255) NOT NULL
,SprocName VARCHAR(255) NOT NULL
,ReportParameters VARCHAR(500) NULL
,TimeStart DATETIME NOT NULL
,TimeEnd DATETIME NOT NULL
,Row_Count INT NOT NULL
)

Step 2

I created a stored procedure that I could have at the bottom of the stored procedures the reports are using.


CREATE PROCEDURE dbo.InsertReportUsage
(
@ReportTypeID INT
,@ReportPath VARCHAR(255)
,@ReportName VARCHAR(255)
,@UserName NVARCHAR(255)
,@SprocName VARCHAR(255)
,@ReportParameters VARCHAR(500)
,@TimeStart DATETIME
,@TimeEnd DATETIME
,@Row_Count int
)
AS
BEGIN TRY
SET NOCOUNT ON

INSERT INTO ReportLog
(
ReportTypeID
,ReportPath
,ReportName
,UserName
,SprocName
,ReportParameters
,TimeStart
,TimeEnd
,Row_Count
)
VALUES
(
@ReportTypeID
,@ReportPath
,@ReportName
,@UserName
,@SprocName
,@ReportParameters
,@TimeStart
,@TimeEnd
,@Row_Count
)
END TRY
BEGIN CATCH

DECLARE
@ErrorMessage AS nvarchar(3000)
,@ErrorSeverity AS int

SET @ErrorMessage = ISNULL(DB_NAME(DB_ID()) + N'.' + SCHEMA_NAME(SCHEMA_ID()) + N'.' + OBJECT_NAME(@@PROCID, DB_ID()),
N'SQL Object Name Not Available') + N': Error: ' + CONVERT(nvarchar(10), ERROR_NUMBER()) + N' Line: '
+ CONVERT(nvarchar(5), ERROR_LINE()) + N' - ' + ERROR_MESSAGE()

SET @ErrorSeverity = ERROR_SEVERITY()
RAISERROR(@ErrorMessage, @ErrorSeverity, 1)
END CATCH

SET NOCOUNT OFF
GO

Step 3

I added some common fields at the top of the stored procedure for my report. Then, at the bottom I called the stored procedure that will record the report usage metrics.

Since I didn’t want to hard code the report path or the report name in the stored procedure, I passed those values from Excel.

I was also able to dynamically capture the stored procedure name, and the system user’s credentials.

Here is an example of a report that lists the names of employees.


CREATE PROCEDURE dbo.PersonReport
(
@LastName varchar(100)
,@ReportPath VARCHAR(255)
,@ReportName VARCHAR(255)
)
AS
BEGIN TRY
--SET NOCOUNT ON  (Don't set)

--------------Excel Log Block--------------
DECLARE    @SprocName VARCHAR(255) = DB_NAME(DB_ID()) + '.' + SCHEMA_NAME(SCHEMA_ID()) + '.' + OBJECT_NAME(@@PROCID, DB_ID())
DECLARE    @ReportParameters VARCHAR(8000) = 'LastName=' + @LastName
DECLARE    @TimeStart DATETIME = GETDATE()
--------------Excel Log Block--------------

SELECT
p.BusinessEntityID
,p.Title
,p.FirstName
,p.MiddleName
,p.LastName
FROM
Person.Person AS p
WHERE
p.LastName LIKE @LastName + '%';

--------------Excel Log Block--------------
EXEC ReportManagement.dbo.InsertReportUsage 1, @ReportPath, @ReportName, SUSER_NAME(), @SprocName
,@ReportParameters, @TimeStart, Get_date(), @@RowCount
--------------Excel Log Block--------------
END TRY
BEGIN CATCH

DECLARE
@ErrorMessage AS nvarchar(3000)
,@ErrorSeverity AS int

SET @ErrorMessage = ISNULL(DB_NAME(DB_ID()) + N'.' + SCHEMA_NAME(SCHEMA_ID()) + N'.' + OBJECT_NAME(@@PROCID, DB_ID()),
N'SQL Object Name Not Available') + N': Error: ' + CONVERT(nvarchar(10), ERROR_NUMBER()) + N' Line: '
+ CONVERT(nvarchar(5), ERROR_LINE()) + N' - ' + ERROR_MESSAGE()

SET @ErrorSeverity = ERROR_SEVERITY()
RAISERROR(@ErrorMessage, @ErrorSeverity, 1)
END CATCH

SET NOCOUNT OFF
GO

Last Step

Finally, I created a stored procedure that joined the metrics from the ExecutionLog3 View that is provided in the ReportServer database for SSRS report usage metrics with the metrics I captured in the ReportLog table.

Now I can generate an overarching report for all reports in my environment.

Thanks for all the fish

Thanks go out to Catherine Wilhelmsen for hosting this month’s T-SQL Tuesday blog party. She is one of my favorite people to catch up with during PASS Summit. I hope you spend some time catching up with her on her blog.

%d bloggers like this: