Robert F Bouillon


I was creating an SSIS package, seemingly simple. One of the tasks I needed to do involved copying our production database to a seperate DB instance for staging. I wanted an SSIS package that would stage production so we could push and test changes.

My first problem is that I tried to create a simple Transfer Database task. When I tried to run the task, I got an error stating that our Stored Procedure, DtsRun, could not be scripted. It's an encrypted stored procedure.

A little annoyed, I decided that I'd do a transfer SQL objects instead. Now I don't want to hard-code my objects to xfer based on the current schema and edit this SSIS script every time we make a DB change; I just want it to copy ALL objects except for my encrypted proc. I'll just build a list with with another task.

I decided that I'll write a script that generates a list of proc names, and I'll pass that to the Copy SQL Server Objects task. Well apparantly I can't assign a collection value with a script. I have to create ANOTHER package that modifies THIS package because (1) The entire package isn't exposed to the script task and (2) object variables can't be used in expressions.

Am I missing something So because I use a single encrypted stored procedure, I have to write 2 fairly complex packages to copy my database That's just stupid. I can understand if SQL can't move the encrypted package while the database is online. I could deal with that. But the extremes I have to go to JUST to copy my database are truly rediculous.

I was really excited about SSIS, but I've been really dissapointed. The performance of the designer on my machine is really poor (I have a Core T2700 with 2GB RAM). Just about every error message I've gotten, both design-time and run-time, I've had to google because they're so obscure. And if you can only store blittable types and strings in variables, that severely limits the functionality of a package (without going to programmatic extremes).

Most of all, I'm really dissapointed in the expression system, If the CLR is loaded into the SSQL/SSIS runtime, then why am I coding it using ANOTHER proprietary coding syntax I thought .NET was meant to keep people from having to redesign the wheel

Microsoft is constantly pushing C#, yet I can only script in VB.NET, which means I now have code-bases in different languages.

Overall, I'm really dissapointed with SSIS. The usefulness of the added functionality in SSIS packages is shrowded by the massive list of nuances.




Re: SSIS - Several Bugs in creating a simple task

Jamie Thomson


Robert F Bouillon wrote:

I was creating an SSIS package, seemingly simple. One of the tasks I needed to do involved copying our production database to a seperate DB instance for staging. I wanted an SSIS package that would stage production so we could push and test changes.

My first problem is that I tried to create a simple Transfer Database task. When I tried to run the task, I got an error stating that our Stored Procedure, DtsRun, could not be scripted. It's an encrypted stored procedure.

That kinda makes sense does it not The whole point of encrypting it is so no-one can get at it. That's my understanding anyway.

Robert F Bouillon wrote:

A little annoyed, I decided that I'd do a transfer SQL objects instead. Now I don't want to hard-code my objects to xfer based on the current schema and edit this SSIS script every time we make a DB change; I just want it to copy ALL objects except for my encrypted proc. I'll just build a list with with another task.

I decided that I'll write a script that generates a list of proc names, and I'll pass that to the Copy SQL Server Objects task. Well apparantly I can't assign a collection value with a script. I have to create ANOTHER package that modifies THIS package because (1) The entire package isn't exposed to the script task and (2) object variables can't be used in expressions.

I've never used Copy SQL Server Objects Task but can you not loop over your objects using ForEach Loop's SMO Enumerator (or other) and move them one by one.

Robert F Bouillon wrote:

Am I missing something So because I use a single encrypted stored procedure, I have to write 2 fairly complex packages to copy my database That's just stupid. I can understand if SQL can't move the encrypted package while the database is online. I could deal with that. But the extremes I have to go to JUST to copy my database are truly rediculous.

Have you tried the Import/Export wizard to generate a package to do this

Robert F Bouillon wrote:

I was really excited about SSIS, but I've been really dissapointed. The performance of the designer on my machine is really poor (I have a Core T2700 with 2GB RAM). Just about every error message I've gotten, both design-time and run-time, I've had to google because they're so obscure. And if you can only store blittable types and strings in variables, that severely limits the functionality of a package (without going to programmatic extremes).

What else would you like to store I haven't yet come across anything that cannot be stored in vars.

Robert F Bouillon wrote:

Most of all, I'm really dissapointed in the expression system, If the CLR is loaded into the SSQL/SSIS runtime, then why am I coding it using ANOTHER proprietary coding syntax I thought .NET was meant to keep people from having to redesign the wheel

Yeah alot of people don't like this but there are good justifications for it. Quite simply, the expression syntax is optimised for what it is provided for. Greasy fast speed. And faster than .Net. Hopefully someone from MSFT will reply will teh proper justification.

Robert F Bouillon wrote:

Microsoft is constantly pushing C#, yet I can only script in VB.NET, which means I now have code-bases in different languages.

It is a problem yeah. The reason is that the scripting environment is VSA - something completely seperate to SSIS. VSA only supports VB.Net. The inclusion of scripting was late in the dev cycle for SSIS v1 and hence they didn't have time to build something that used C#. Given the choice between having VB.Net or nothing at all I know what I'd got for. In the next version VSA is expected to get replaced with VSTO and this will give full C# support.

-Jamie







Re: SSIS - Several Bugs in creating a simple task

Robert F Bouillon

Jamie Thomson wrote:

-clipped-

That kinda makes sense does it not The whole point of encrypting it is so no-one can get at it. That's my understanding anyway.

In a sense, I totally agree, however in I'm not trying to access the contents, and I have the permissions I need to backup/restore. The task is meant to transfer a database. Whether an object is encrypted or not should be irrelevant to the task. The transfer works in offline mode, but in online mode I get the above stated error. I'm not trying to access/modify it, I need to move it, and I have the SQL permissions to do so.

After some thought, I could increase the Minimum Error Count to 2, but my package will break if any more Ecrypted Stored Procs are added. I'd have to add another script that counts the number of encrypted procs and sets the Min Err count manually. Again, it seems like a LOT of work to copy a database, and if it's a known issue, should there have been functionality built into the xfer database task to compensate

I should just be able to drop-and-go with the xfer database task. Any intricacies related to the mechanics of performing the move should be handled by the task itself, and that should include encrypted procs.

Jamie Thomson wrote:

-clipped-

I've never used Copy SQL Server Objects Task but can you not loop over your objects using ForEach Loop's SMO Enumerator (or other) and move them one by one.

Well that's really the main issue. The TablesList is a StringCollection property. Properties that are objects can't be set with expressions. This means that variables that are objects have little value other than to pass objects between script tasks. The ForEach loop sets a property on every iteration. There's no way for me to, within the same package, programmatically create a list and pass the list to the Copy SQL objects task. The only way for me to do this would be to create another package that manipulates this package programatically. I'm assuming there was a good reason why the Script Task doesn't have visibility outside of itself other than variables

It doesn't make sense to me why the script task can't programatically change properties of other tasks at run-time. Combined with the property/object/expression limitation, any properties within SSIS that are not strings or numbers can ONLY bet set at design time, unless you create another package that alters your package programatically. That's not what I wouldhave expected from SSIS, or MS for that matter. It's a LOT of work to perform mostly simple tasks. I want to build a list of objects, and pass that list to the Copy SQL objects task.

I would assume I could just create a variable, drop my list in that variable, and then assign that variable to a property with an expression. That seems logical and simple. I guess the point I'm trying to make is that if it was a design requirement that the tasks should only communicate with each other through variables, then limiting expressions to strings and numbers greatly imhibits the ability of SSIS, negating a lot of the value SSIS and variables provide. Also, if it was a known limitation that expressions couldn't assign obejcts to properties, the next logical step would be to allow the scripts to programatically change properties. Disallowing changing properties at run-time completely because of the limitations of the expression language (which arguably shouldn't even exist) or because someone chose to limit the visibility of the script object seems like a decision made in err.

Jamie Thomson wrote:

What else would you like to store I haven't yet come across anything that cannot be stored in vars.

In theory, you should be able to store and assign any object type that can also be a Task Property. As of right now, you can't programatically alter any properties unless they are a number or string, because of the limitation.

Jamie Thomson wrote:

Yeah alot of people don't like this but there are good justifications for it. Quite simply, the expression syntax is optimised for what it is provided for. Greasy fast speed. And faster than .Net. Hopefully someone from MSFT will reply will teh proper justification.

Not that I don't believe you, but it's hard for me to believe speed was the driving factor. .NET is compiled. An expression syntax would be interpreted, unless SSIS compiles it (which again, would seem like they're re-inventing the wheel). I would assume that once a package is run for the first time, all of the .NET would be compiled and cached. At that point I can't see it being slower at all than an interpreted expression syntax. If they are compiling the syntax, it's hard for me to believe that the minute amount of speed in using a custom-compilation would offset the functionality provided by enabling the .NET framework.

Jamie Thomson wrote:

It is a problem yeah. The reason is that the scripting environment is VSA - something completely seperate to SSIS. VSA only supports VB.Net. The inclusion of scripting was late in the dev cycle for SSIS v1 and hence they didn't have time to build something that used C#. Given the choice between having VB.Net or nothing at all I know what I'd got for. In the next version VSA is expected to get replaced with VSTO and this will give full C# support.

In theory, wouldn't adding C# support to VSA also implicitly add C# support in 2005 Is there a reason they're moving to another framework rather than just adding C# support to VSA

I also found a couple of bugs so far. Where do I report them. If I try and set the ListTables property from the VS property window, when I hit ADD, I get a "Could not find a ctor for the String type" error. When adding an item to the project, if I click around and then press cancel, I get some "could not find proxy" error or something like that.

Thanks for your help so far Jamie.






Re: SSIS - Several Bugs in creating a simple task

Jamie Thomson

Robert F Bouillon wrote:

Jamie Thomson wrote:

-clipped-

That kinda makes sense does it not The whole point of encrypting it is so no-one can get at it. That's my understanding anyway.

In a sense, I totally agree, however in I'm not trying to access the contents, and I have the permissions I need to backup/restore. The task is meant to transfer a database. Whether an object is encrypted or not should be irrelevant to the task. The transfer works in offline mode, but in online mode I get the above stated error. I'm not trying to access/modify it, I need to move it, and I have the SQL permissions to do so.

After some thought, I could increase the Minimum Error Count to 2, but my package will break if any more Ecrypted Stored Procs are added. I'd have to add another script that counts the number of encrypted procs and sets the Min Err count manually. Again, it seems like a LOT of work to copy a database, and if it's a known issue, should there have been functionality built into the xfer database task to compensate

I should just be able to drop-and-go with the xfer database task. Any intricacies related to the mechanics of performing the move should be handled by the task itself, and that should include encrypted procs.

Interesting. I can't speak from a position of knowledge here so had better not say anything except to say that if you think the task is lacking in some way,log it at http://connect.microsot.com/sqlserver/feedback

Robert F Bouillon wrote:

Jamie Thomson wrote:

-clipped-

I've never used Copy SQL Server Objects Task but can you not loop over your objects using ForEach Loop's SMO Enumerator (or other) and move them one by one.

Well that's really the main issue. The TablesList is a StringCollection property. Properties that are objects can't be set with expressions.

OK. That makes sense. Clearly there's something lacking here although in 2 and a half years of using this thing you're the first person I've heard that's come across this problem. Again, I'm going to take the coward's way out and ask could you submit this at Connect.

Robert F Bouillon wrote:

This means that variables that are objects have little value other than to pass objects between script tasks. The ForEach loop sets a property on every iteration. There's no way for me to, within the same package, programmatically create a list and pass the list to the Copy SQL objects task. The only way for me to do this would be to create another package that manipulates this package programatically.

Yeah, i get this now. Like I say, I don't think there;s a way of doing it!

Robert F Bouillon wrote:

I'm assuming there was a good reason why the Script Task doesn't have visibility outside of itself other than variables

It doesn't make sense to me why the script task can't programatically change properties of other tasks at run-time. Combined with the property/object/expression limitation, any properties within SSIS that are not strings or numbers can ONLY bet set at design time, unless you create another package that alters your package programatically. That's not what I wouldhave expected from SSIS, or MS for that matter. It's a LOT of work to perform mostly simple tasks. I want to build a list of objects, and pass that list to the Copy SQL objects task.

I have to be honest. This is the first situation I have come across where the expression syntax has not been "up to the job". Its not my place to explain the rationale for not giving us the ability to manipulate the pckage at execution time from a script task - I do think there are good reasons though.

Robert F Bouillon wrote:

I would assume I could just create a variable, drop my list in that variable, and then assign that variable to a property with an expression. That seems logical and simple. I guess the point I'm trying to make is that if it was a design requirement that the tasks should only communicate with each other through variables, then limiting expressions to strings and numbers greatly imhibits the ability of SSIS, negating a lot of the value SSIS and variables provide. Also, if it was a known limitation that expressions couldn't assign obejcts to properties, the next logical step would be to allow the scripts to programatically change properties. Disallowing changing properties at run-time completely because of the limitations of the expression language (which arguably shouldn't even exist) or because someone chose to limit the visibility of the script object seems like a decision made in err.

Jamie Thomson wrote:

What else would you like to store I haven't yet come across anything that cannot be stored in vars.

In theory, you should be able to store and assign any object type that can also be a Task Property. As of right now, you can't programatically alter any properties unless they are a number or string, because of the limitation.

Jamie Thomson wrote:

Yeah alot of people don't like this but there are good justifications for it. Quite simply, the expression syntax is optimised for what it is provided for. Greasy fast speed. And faster than .Net. Hopefully someone from MSFT will reply will teh proper justification.

Not that I don't believe you, but it's hard for me to believe speed was the driving factor. .NET is compiled. An expression syntax would be interpreted, unless SSIS compiles it (which again, would seem like they're re-inventing the wheel). I would assume that once a package is run for the first time, all of the .NET would be compiled and cached. At that point I can't see it being slower at all than an interpreted expression syntax. If they are compiling the syntax, it's hard for me to believe that the minute amount of speed in using a custom-compilation would offset the functionality provided by enabling the .NET framework.

Hopefully someone from MSFT will answer this - because they can answer it better than I can.

Robert F Bouillon wrote:

Jamie Thomson wrote:

It is a problem yeah. The reason is that the scripting environment is VSA - something completely seperate to SSIS. VSA only supports VB.Net. The inclusion of scripting was late in the dev cycle for SSIS v1 and hence they didn't have time to build something that used C#. Given the choice between having VB.Net or nothing at all I know what I'd got for. In the next version VSA is expected to get replaced with VSTO and this will give full C# support.

In theory, wouldn't adding C# support to VSA also implicitly add C# support in 2005

Absolutely

Robert F Bouillon wrote:

Is there a reason they're moving to another framework rather than just adding C# support to VSA

I think VSA is now considered 'legacy' within MSFT. SSIS don't own it yet they are the only people actually using it (that's what I've heard anyway). This, and the fact that there are other limitations with VSA, is the reason for moving VSTO. I think I even heard one person from the SSIS team say that they're having trouble finding anyone in Microsoft that actually assisted in building VSA. That was a long time agio tho so i might be paraphrasing.

Robert F Bouillon wrote:

I also found a couple of bugs so far. Where do I report them.

http://conect.microsoft.com/sqlserver/feedback

Robert F Bouillon wrote:

If I try and set the ListTables property from the VS property window, when I hit ADD, I get a "Could not find a ctor for the String type" error. When adding an item to the project, if I click around and then press cancel, I get some "could not find proxy" error or something like that.

Thanks for your help so far Jamie.

No problem. Not that I've helped much other than confirm what you already knew.

-Jamie






Re: SSIS - Several Bugs in creating a simple task

Robert F Bouillon

Thanks for your help.

I know i'm being a bit harsh on SSIS.

A logical assumption would be that it is difficult/impossible to limit the scope of the expression syntax if .NET is used. I don't know enough about .NET security policies to know if this holds true. It's just bothersome to fumble around yet another proprietary syntax, especially since I know .NET could have been used, and the .NET FCL has a lot more functionality than the expression syntax provides. I'd prefer not to assume, though :)

Thanks for all of your help. Perhaps I just expected too much of SSIS.

This is where I found the 'dynamic package' solution:
http://sqljunkies.com/WebLog/knight_reign/archive/2005/12/31/17731.aspx





Re: SSIS - Several Bugs in creating a simple task

Jamie Thomson

My pleasure.

I would be grateful if you could submit something on Connect around the issue of not being able to set some task properties (specifically connections) using expressions. I think they need to come up with a fix for that.

-Jamie






Re: SSIS - Several Bugs in creating a simple task

Robert F Bouillon

Sure thing. I have a small llist of bugs/suggestions up there now. I'll gladly add that one.

--ROBERT





Re: SSIS - Several Bugs in creating a simple task

Jamie Thomson

Robert F Bouillon wrote:

Sure thing. I have a small llist of bugs/suggestions up there now. I'll gladly add that one.

--ROBERT

Fancy emailing me a list of them I like to add things to my watch list that I'm interested in.

Thanks.

-Jamie