Jon Galloway

Orcas VPC Download info

Greg pointed me to the Visual Studio Orcas CTP Virtual PC image download. Downloading now.

First of all, I'm really happy to see a VPC image download. We've been asking for this for a little while, and it's great to see it happen.

Make sure you follow the download instructions carefully. There are two downloads - a base image and a difference disk image. That's smart for bandwidth, since future releases will only require download of the new difference disk, but it might throw you off if you only download the difference disk.

I see from Greg's screenshot he's just using IE to download them:

When I'm downloading a GB or more, I like to go with a download manager like the Download Them All Firefox extension. Back when I was using IE, I used Download Express. There are a lot of good reasons for using a download manager for big downloads:

• They download in multiple concurrent chunks, which is usually noticibly faster on large downloads.
• They let you pause the downloads. That was handy when I wanted to watch the OnTen Zune video.
• If something goes wrong part way through - which always seems to happen on huge downloads - download managers let you resume the download.
• They have really cool download progress graphs.
• They don't display negative progress - see Greg's -318% completion status above. ;-)

However you decide to get them, go get them!

[Tool] Data Dictionary Creator - Rapid database documentation

[UPDATE] A newer version of DDC has been released to Codeplex

Summary

Data Dictionary Creator (DDC) is a simple application which helps you document SQL Server databases. It stores all the information in Extended Properties, so it's easier to keep the documentation in sync with the database as it changes. You could do a lot of this through SSMS (SQL Server Management Studio), but

1. It's a lot more work, which means it's a lot less likely to be kept updated.
2. The SSMS approach only gives you one column (Description); DDC gives you as many as you want. That's probably one of the best features DDC adds - it lets you tailor your documentation to your database and business.
   
3. SSMS requires you to write your export via tricksy SQL; DDC exports to WordML, Excel, HTML, and XML.

Download DataDictionaryCreator from CodePlex.

How to use it

Connect to database

Enter a connection string (you know the drill). You can alternatively double click the connection string textbox for the standard data connection (DataLink) dialog. The connection string box turns light blue (as shown below) when you're connected.

Set up any Additional Properties you want to track

The Additional Properties box takes a comma delimited list of properties. In the example above, I've included a Source and Audit column. You can change it at any time, so don't worry about it too much if you're not sure right now.

Auto Fill the key column descriptions

Click the "Set descriptions for all keys" button to automatically pre-fill descriptions for all primary and foreign key columns. This is optional, but it saves some work and helps you to see some progress quickly.

Fill in the descriptions

Go to the Edit tab and do the actual work. Nope, there's no GhostDoc action here - you have to fill in the information. It's pretty easy to do, though - select a table, fill in the table description, and fill in some documentation fields. All changes are saved as you work (when you leave each grid cell). Then pick another table and continue. You can do as much or as little as you like; you can make additional changes whenever you want.

Notice that the Audit and Source columns have been added based on my previous settings. 

Export

Go back to the Setup tab and click the Export button. You can select HTML, Excel, WordML, or XML. All exports go first to a native XML format, then through an XSL transform, so you can customize the export by modifying the XSLT files or use them as the template for a transform to any other format you'd like.

Repeat

The best part of this is that it's not a one time thing. Anyone who works on the database can continue to update the documentation as they go.

How would you do this in SSMS?

Oh, you're still stuck on that, huh? Okay, if you want to do this through SQL Server Management Studio (SSMS), the best way is through the Database Diagram interface. Editing through the Modify Table screen is really inefficient since you can only update the description for one column at a time. The Database Diagram screen has a custom view which allows you to add the description column so you can update all column descriptions for a table at a time.

Here's how to set that up

1. Create a table database diagram
2. Switch the table view to "Custom View" - right-click the table name and select custom view
3. Customize the view - right-click the table, select "Modify Custom", and add the Description as shown in the screenshot below

That helps a bit, but you need to go to each table to make updates. If you want to export it, take a look at Raymond Lewallen's SQL script to generate a data dictionary table.

Tech Stuff

I made heavy use of the SQL Server Management Objects (SMO) provided with SQL Server 2005. They make dealing with database metadata really easy, so I could concentrate on the harder stuff, like setting up control containers and persisting user settings. ;-) The source code is included under BSD license, so feel free to take a look and reuse anything you find useful. Please submit any changes you'd like me to include in the next release. The easiest thing to modify is the export XSLT files, which are pretty plain right now.

Minor Update 9/29/06

Version 1.1.0 adds:

• T-SQL export to allow copying documentation between database instances (thanks, Tyler!)
• Loads previously used additional properties from database on connection (thanks for the suggestion, Phil, although you made me do all the work...)
  
• User interface inhancements (thanks, Phil)
  
• Bug fixes (I wrote 'em, I fixed 'em)
  

VS2003 and Vista - Is the sky really falling?

As part of a recent Visual Studio 2005 SP1 announcement, the Corp VP of Microsoft's Developer Division stated that Visual Studio.NET 2003 won't be supported under Windows Vista. Frans Bouma, Paul Wilson, and others have done a good job of raising the level of awareness on the issue. I agree that it's not, you know, a good thing, but I wanted to hear how big a problem it really is.

Does "not supported" mean it will blue screen your computer and set you on fire, or does it mean it'll mostly work with the occasional annoyance? It seems like it may be the latter.

Scott Hanselman, who has actually been using Vista, told me that "Some obscure things like autoregistration of COM Interop Assemblies doesn’t work. But it’s working fine for me."

Scott Guthrie left a comment on Paul Wilson's post saying that the problem is really with "advanced debugging", and it will "mostly work" under Vista:

The big technical challenge is with enabling scenarios like advanced debugging. Debuggers are incredibly invasive in a process, and so changes in how an OS handles memory layout can have big impacts on it. Vista did a lot of work in this release to tighten security and lock down process/memory usage - which is what is affecting both the VS debugger, as well as every other debugger out there. Since the VS debugger is particularly rich (multi-language, managed/native interop, COM + Jscript integration, etc) - it will need additional work to fully support all scenarios on Vista. That is also the reason we are releasing a special servicing release after VS 2005 SP1 specific to Vista - to make sure everything (and especially debugging and profiling) work in all scenarios. It is actually several man-months of work (we've had a team working on this for quite awhile). Note that the .NET 1.1 (and ASP.NET 1.1) is fully supported at runtime on Vista. VS 2003 will mostly work on Vista. What we are saying, though, is that there will be some scenarios where VS 2003 doesn't work (or work well) on Vista - hence the reason it isn't a supported scenario. Instead, we recommend using a VPC/VM image for VS 2003 development to ensure 100% compat.

I sure would have preferred it if that kind of detail - what won't work, and why not - had been more officially announced, and sooner. However, for now it looks like rather than dealing with complaints about it not completely working, Microsoft's positioning this as another "doesn't work by design - not a supported scenario" thing.

Kind of reminds me of the IE7 Standalone thing. Even though it is technically possible to do and works well enough for development purposes, the official answer is that we need to be developing inside a virtual machine.

Command Line Confusion

I kept getting wierd errors in a simple console application which takes a regular expression as an argument. The regular expression kept failing with an "Illegal \ at end of pattern" error. The odd thing was that I was properly escaping the \ as \\.

After some testing, I think I've convinced myself that this had nothing to do with regular expressions; there's something wacky about the way the .NET Framework handles commandline arguments which end in \\". Here's a simplified argument test application: 

using System;

public class Test
{
    public static void Main(string[] args)
    {
        Console.WriteLine("Environment.CommandLine: " + Environment.CommandLine);
        for(int i=0;i<args.Length;i++)
        {
            OutputArg(i,args[i]);
            Console.WriteLine();
        }
    }
    
    static void OutputArg(int i,string arg)
    {
        Console.WriteLine("Environment.GetCommandLineArgs()[{0}]: {1}", i+1, Environment.GetCommandLineArgs()[i+1]);
        Console.WriteLine("args[{0}]: {1}",i,arg);
        //foreach(char c in arg) Console.WriteLine(c);
    }
}

First, let's test it to see how it handles quoted and unqoted strings:

C:\Temp>CommandLineArgumentsTest.exe "monkey" potato
Environment.CommandLine: CommandLineArgumentsTest.exe "monkey" potato
Environment.GetCommandLineArgs()[1]: monkey
args[0]: monkey

Environment.GetCommandLineArgs()[2]: potato
args[1]: potato

Everything sees pretty good there - we've passed a quoted string and an unquoted string, and both work just fine.

Now, let's try it with some strings that end with two back slashes (\\):

C:\Temp>CommandLineArgumentsTest.exe "\\test\\" "\\test\\\"
Environment.CommandLine: CommandLineArgumentsTest.exe "\\test\\" "\\test\\\"
Environment.GetCommandLineArgs()[1]: \\test\
args[0]: \\test\

Environment.GetCommandLineArgs()[2]: \\test\"
args[1]: \\test\"

See what I'm talking about? The last \ vanishes like X-Files was all up in this place. Normally when things get strange, I turn to my good friend Reflector for some insight, but mscorlib.System.Environment.GetCommandLineArgsNative() is rather un-surprisingly native code and is thus well sheltered from Reflector's prying eyes.

After a bit of thought it looks like the second backslash is being "used up" by escaping the final quote. Let's try it with an unquoted argument:

C:\Documents and Settings\Jon\My Documents\My Code Snippets>CommandLineArgumentsTest.exe \\test\\
Environment.CommandLine: CommandLineArgumentsTest.exe \\test\\
Environment.GetCommandLineArgs()[1]: \\test\\
args[0]: \\test\\

Sure enough, everything's great there. That doesn't solve the problem, though - if there's a space in the string, it needs to be quoted so it's handled as one argument (e.g. "\\t e s t\\"). Well, we have enough info at this point to hack together a solution, but I don't like it at all. To pass \\test\\ as a quoted command-line argument, we need to pass in "\\test\\\\":

C:\Temp>CommandLineArgumentsTest.exe "\\test\\\\"
Environment.CommandLine: CommandLineArgumentsTest.exe "\\test\\\\"
Environment.GetCommandLineArgs()[1]: \\test\\
args[0]: \\test\\

It's still kind of odd to me, though - why doesn't it display that quote in \\test\\\", since it's been escaped? If the GetComandLineArg() is seeing the last two characters as \", wouldn't it make more sense for args[0] to be \\test\" rather than \\test\ ? As I understand it, escaping in both C# and RegEx is supposed to be handled in a left to right fashion, but this seems to be working right to left.

[tool] TimeSnapper - Never lose work again

TimeSnapper is one of my favorite applications. It does one thing and it does it well - it takes a screenshot every few seconds, then lets you browse through your history by date and time. Most people focus on this as a way to simplify filling in time sheets, but I've come to rely on this program as a safety net. This thing has saved my butt countless times:

• I'd spent many long hours working with Roiy on the Monoppix project, but since we're separated by about 7500 miles I doubted we'd ever get to meet in person. We finally had a chance - he had a brief layover on a flight through Los Angeles, so we planned to meet for sushi. Roiy sent me his contact information via MSN Messenger. I normally use Skype chat, which I have set up to log chat transcripts, so I assumed my chat was logged. Two weeks later, when it was time to leave for the two hour drive to LA, I realized that I'd never set up Messenger to log my chat transcripts. I thought I was screwed, but then I remembered that I always have TimeSnapper running. It took a couple of minutes to find the chat window with all the information I needed.
  
  
• I'd been working for a while on a difficult SQL query which needed to clean up and import some really ugly legacy data. Since I work from home, I decided to bang on it late one night (both the benefit and peril of working from home) and I got it working. It was late, and I forgot to save my work. The next morning, Windows Update had rebooted my computer and the work was gone.¹ Rather than try to recreate the work, I just viewed the previous night's work on TimeSnapper. Done and done.
  
  
• I'm frequently involved in deploying my own work to both staging and production servers. I've always viewed staging deployments as a practice run for for the production deployment, but lately I've begun using TimeSnapper movies from the staging deployment as a guide to the production deployment. Of course, it's not the only tool in my deployment toolbox - I automate as much as possible, pre-stage whatever I can, and write up deployment plans. However, it's hard to think of more functional deployment documentation than a video showing each step of a successful deployment.
  
  
• We recently helped a client to migrate several older sites to a new server. I handled everything via Remote Desktop. The move went well, the client signed off, and the old server was decomissioned. A while later, the client had a question about the configuration of the old server (which they'd since decomissioned and re-formatted). I was able to answer their question by scanning through screenshots of my computer while I'd been working on the remote server.
  
  
• Often, I'll get so caught up in leaving a comment on someone else's blog that I end up putting more effort into it than a post on my own  blog. Then I'll submit it and it just goes away. Is it awaiting moderation? What's going on? I'll check back a few days later and it won't have been posted, but other comment spam's there so I doubt it's been moderated. Who knows... at least with TimeSnapper, I can OCR the comment and either resubmit it as a comment, or turn it into my own blog posting. This is a kind of trivial example, but it's a good example of how you never have to lose work of any kind when everything's automatically journaled.
  
  
• I'd helped my 5 year old daughter draw a picture on my computer. She was really proud of it, so we printed a copy. Some random household disaster struck and we closed the program and commenced bailing water or cicling the wagons or something. Later I realized that the printer had been out of ink, and the picture hadn't printed. Fortunately, I was able to bring up the masterpiece in TimeSnapper and re-print it before anyone found out. Crisis averted.

The point of all these stories is that TimeSnapper is useful for so much more than filling in time sheets (although that's what got me started with it). Like backup and anti-virus software, TimeSnapper is the kind of application that hums along quitely in the background until you really need it; at that time you're either happy you've had it running or really sorry you haven't.

I've been planning this post for a while, and happened to finally get to it on the weekend that TimeSnapper Professional was released. TimeSnapper Classic has been and apparently will always be free, and was what I was using for all of the above stories. I highly recommend you give TimeSnapper a try.

Notes:

1. The main objection I hear when I show off this program is disk usage. It is possible to fill up a drive if you're taking high resolution screenshots, but this isn't really a problem if you make proper use of the image format and archival options. I run my main computer at 1920x1200 resolution (it's a 24" widescreen), and I've found that 100% quality EMF images are around 300K. TimeSnapper only takes a new screenshot if it's changed from the previous one, so that works out to 400 - 600MB for most workdays. I keep 24 days² of history, which is probably excessive, but the stories I've listed above (and countless others I'm not mentioning) have convinced me I'd rather be safe than sorry.  That means that my current ScreenShots directory is just under 14GB. Again, I'll remind you that I'm running at a high resolution and keeping over three weeks of history, but even at that, so what? 14GB costs less than $20, and it's already saved me way more than that in terms of lost work. Most computers come with 160GB minimum these days, and I'd argue that this is a very good use of that enormous amount of space.
2. TimeSnapper stores the screenshots as images rather than video. That's a little less space efficient, but it makes sense for the history browser interface, which lets you quickly look up an arbitrary time (e.g. last Thursday at 10:46 AM). It's also worthwhile when you really need to see exactly what was on the screen to have a full quality image rather than compressed video.
3. Disclosure: I haven't been asked to write this review, and I'm not getting anything for it.
4. .NET developers may be interested in knowing that TimeSnapper is a .NET application. It was the Grand Prize Winner in the great Larkware Contest of 2005.
5. The title of this post is a perhaps a little bit extreme. It's pretty hard for me to lose work with TimeSnapper recording everything that's happened on my screen for the past 24 days, but I'm not going to tempt fate by saying it's impossible. It covers the thing that regular backups and good source control use alone won't catch.

Sometimes it's easier to just hex edit the assembly

Summary

PalPal's Single Item Purchase links are not technically valid - the query portion of the URL doesn't start with a question mark. This confuses ASP.NET 2.0's Response.Redirect(url) call, which causes the return URL to be malformed. Rather than spend dozens of hours upgrading to the newest version of the store software, I spent 15 minutes with Reflector, WinMerge, and a hex editor to modify the .NET assembly.

The Obligatory Human Interest Back Story

I'd been working on setting up an e-commerce site for my wife for a long time. The first version ran on ZenCart, a fork of the popular osCommerce PHP / MySql system. The PayPal integration never really worked, and I decided to spend a few bucks and switch to what I know - ASP.NET.

The second version of the site is built on DotNetNuke with the CATALooK store module. We went with DNN for the admin and editing features, so once it's up and running she can run it by herself. We looked at several store modules, including the official DotNetNuke Store Module, and found that CATALook is way ahead of all the other stores and very reasonably priced (The CATALooK support alone has been worth the price).

We put a huge amount of effort into a new skin that uses clean (table free) HTML, although the DNN modules spew so many tables it feels pointless at times.  The hardest part was removing the formatting from the  CATALooK modules, since many of them used the XML skins approach. Finally it was done, and we placed a test order...

It went through, but the return link from PayPal to our site was broken. The querystring parameters were incorrect, and the URL began with http:/ rather than http://. Problems with PayPal integration - deja vu, huh? This time, though, I had a support contact. They immediately got back to me and told me it had to do with ASP.NET 2.0's handling of the PayPal link; the new version of CATALook (just released) would fix it. The new version did indeed fix the problem with PayPal.

Unfortunately, this version was the time that the CATALooK folks decided to convert the skinning model from XML based skins to ASCX based skins. While I vastly prefer ASCX based skins, the this wasn't a convenient time for me to spend a lot of time converting my skins. I gave it a shot, but the conversion process didn't go smoothly. My wife and I were lamenting the fact that we had to put off launching her 99% complete site to complete a difficult upgrade when all we really needed was a tiny part of the new version - the fix to the PayPal redirect. "We're going to have to do all this work just to change a few characters," I said.

Hey, that gives me an idea...

Technical Background on the Problem

The problem is both PayPal and .NET's fault. PayPal's Single Item Purchase / Buy Now [pdf] links are not (as I read RFC 3986) valid URL's. Here's an example; note that the query portion of the URL (starting with business=store@test.com...) isn't prefixed with a question mark (?), it's prefixed with a forward slash (/):

It looks like there's a bug in the System.Uri.CombineUri() function which is called by Response.Redirect(). It looks for a substring of ? to figure out where the query portion of the url starts, doesn't find it, and processes the entire URL as if it had no query. I understand that it's being broken by a malformed URL, but I'd still like it to fail a little more gracefully.

Anyhow, the solution is to use an alternate Buy Now link format which replaces that last slash with a quesion mark:

https://www.paypal.com/xclick?business=store@test.com&item_name=Baseball+Hat&item_number=123&amount=5.95&return=http%3A//www.test.com/thankyou.htm

Armed with that knowledge, I was able to find that exact bug fix in the store system's via Reflector very quickly.

Could it really work?

I assumed that I'd need to disassemble and recompile the assembly, but luckily the entire URL was set as a string constant. On the off-chance it might work, I opened the assembly in a hex editor. I used the hex editor which is built into Notepad++, which totally rocks. First I searched for "xclick", but I didn't find it. I was pretty sure it was there, since Reflector had shown it, so I converted it to UTF-16:

 
1. Bring up the find dialog
2. Enter "xclick" (of course, any search string could be sustituted here)
3. Changed the datatype from "String" to "Hex Pattern"
4. Inserted 00 values after every hex pair, so "78636c69636b" became "780063006c00690063006b00"
5. Tried the search again

Sure enough, that got it. I changed the following character from a forward slash to a question mark, saved, and tested. That got it - I was able to stay on my current release of the software, but I had my one character bug fix.

Disclaimers

Of course, if the DLL had been signed I'd have been out of luck, but fortunately it wasn't and all was good. Yes, I should upgrade to the new version of the store software as soon as possible, but I was happy to be able to decouple PayPal bugfix from the skin system change.

SQL Reporting Services - CSS fix for Firefox

SSRS 2005 is pretty slick, but the HTML is just terrible. Reports are displayed in an IFRAME that's deep in nested table land, and the IFRAME's height setting only works in IE. The end result is that reports don't display correctly in Firefox - the IFRAME's height defaults to a few hundred pixels, so you only see the top 2 inches of the report. However, they did the right thing by designating CSS classes for most of the important elements, so we can fix it by adding a min-height setting. I'm sure there are other issues with getting SSRS to display correctly in Firefox, and possibly other answers (let me hear them in the comments below), but this CSS fix at least lets the reports show.

Add the following to the ReportingServices.css file (by default, it's found in "C:\Program Files\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportManager\Styles\"):

 /* Fix report IFRAME height for Firefox */

.DocMapAndReportFrame
{
    min-height: 860px;
}

::Add to C:\Program Files\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportManager\Styles\ReportingServices.css
SET CSSFILE=%ProgramFiles%\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportManager\Styles\ReportingServices.css
IF NOT EXIST "%CSSFILE%.bak" COPY "%CSSFILE%" "%CSSFILE%.bak"
echo. >> "%CSSFILE%"
echo. >> "%CSSFILE%"
echo /* Fix report IFRAME height for Firefox */ >> "%CSSFILE%"
echo .DocMapAndReportFrame >> "%CSSFILE%"
echo { >> "%CSSFILE%"
echo     min-height: 860px; >> "%CSSFILE%"
echo } >> "%CSSFILE%"

Notes / Disclaimers / Retractions

This just adds a min-height attribute to the class used for the IFRAME. Of course, you can set the min-height to another value if you'd like; if you make it larger than your end user's screen height they'll see a scroll bar and may go into convulsions.

This change isn't needed for IE7. One of the big changes to IE7's CSS handling is that it will stop treating height and width as min-height and min-width, but IE7 and Firefox still treat height=100% differently (at leat for IFRAMES).

Please let me know if there's a better way to fix this, more to be fixed, etc. Please also let Phil know what you'd do if you won the lottery.

Update

This only fixes height for charts. Another common problem is width - for instance, tablular reports end up smashed horizontally. I haven't found a CSS fix for this; the best option I've found so far is to add an empty textbox that stretches the width of the report.