Recipe: Deploying a SQL Database to a Remote Hosting Environment (Part 1)
Scenario:
You finish building a great ASP.NET application, have everything tested and working right on your local system, are taking full advantage of the new ASP.NET 2.0 Membership, Role and Profile features, and are ready to publish it to a remote hosting environment and share it with the world.
Copying the .aspx files and compiled assemblies to the remote system is pretty easy (just FTP or copy them up). The challenge that confronts a lot of developers, though, is how to setup and recreate the database contents - both schema and data - on the remote hosted site. Unfortunately there hasn't historically been a super-easy way to accomplish this.
The good news is that this week the SQL Server team published the release candidate of a new SQL Server Hosting Toolkit that will make it much, much easier to deploy your SQL solutions remotely to a hosted environment. The toolkit allows you to work with SQL Express, SQL Server 2000, and SQL Server 2005 databases locally, and then easily transfer your schema and data and install them into a shared hosting remote SQL Server account.
SQL Server Hosting Toolkit
The SQL Server Hosting toolkit is available for free, and ships with a Database Publishing Wizard that supports two database hosting deployment scenarios:
1) The Database Publishing Wizard enables you to point at a database you are working with on your local system, and then automatically create a .SQL script file that contains the setup logic needed to re-create an exact replica of the database on any remote system. This .SQL script includes everything needed to create the database schema (tables, views, sprocs, triggers, full-text catalogs, roles, rules, etc - full details here), as well as populate the new database with the same table row contents as your local database (this is analogous to the MySQL dump utility). The benefit of having this setup logic encapsulated in a single .SQL file is that most hosters already support the ability for you to upload .SQL files to their hosted environments and run these scripts via their hosting admin control panels. Assuming you have a web hoster that supports this today, you can immediately start using the Database Publishing Wizard to easily deploy your sites without requiring anything to be installed or configured by the hoster.
2) The Database Publishing Wizard also enables you to point at a database you are working with on your local system, and then use web-services to transfer and recreate the database in your remote hoster environment (without you having to create the .SQL file or use the hoster admin control panel to run it). This publishing option does require that a SQL Publishing web-service be exposed in the hosting environment, and the SQL Server Hosting Toolkit includes a free implementation of this SQL Publishing web-service that we'll be working with hosters to aggressively deploy.
The Database Publishing Wizard enables you to use either SQL Express or SQL Server 2000/2005 locally, and then use either SQL 2000 or SQL 2005 in the remote hoster environment. It does not require that the versions of SQL match - so you can use SQL Express 2005 locally and then upload to a SQL 2000 server in the hosting environment without having to change any of your code.
The Database Publishing Wizard also supports handling the built-in ASP.NET 2.0 Membership, Role Management, Profile and Health Monitoring schemas. A lot of people have run into issues because the built-in .SQL scripts that ship by default with ASP.NET for setting up these schemas require DBO permissions at install-time for the SQL scripts -- which a lot of hosters don't support (note: the scripts do not require DBO permissions at runtime - only for install time, but this can sometimes still be a blocker in itself unless the hoster is willing to install them for you). The Database Publishing Wizard on the other-hand does notrequire DBO permissions when installing the ASP.NET Membership, Roles and Profile schemas/data, and should enable you to deploy the ASPNETDB tables + sprocs just as easily as any other database using the Database Publishing Wizard.
First Tutorial: Deploying a SQL Express Database to a SQL Server Hosting Account (using .SQL files)
I'll be doing a series of posts over the next few weeks showing how to use the various features within the SQL Server Hosting Toolkit. This first tutorial in the series covers how to use it to easily generate a .SQL installation file of a local SQL Express database that you can then copy to a remote hosting account and use to re-create a SQL Server database for you to use with your hosted site.
Step 0: Download and Install the Database Publishing Wizard
The first step we'll need to-do is to make sure we have the Database Publishing Wizard from the SQL Hosting Toolkit installed. Click here to download it and install it.
The Database Publishing Wizard comes with support for both a GUI based wizard, as well as a command-line utility. The GUI based wizard can be run either standalone or via context-menu support that it adds to the Server Explorer in both Visual Studio 2005 and Visual Web Developer Express. For the purposes of this tutorial we'll be using this later Server Explorer integration - which makes publishing really easy.
Step 1: Create an ASP.NET web-site that uses a local SQL Express or SQL Server database
To help with this demo, we will use the built-in Personal Starter Kit template that ships with both VS 2005 and Visual Web Developer Express. To create a new web project based on it, select File->New Web Site within VWD or VS and choose the "Personal Starter Kit" template in the New Web-Site dialog. By default the personal starter kit application is configured to use SQL Express (which is free and can be downloaded here). When run the sample looks like below:
After creating the application, you can then run the web admin tool (choose the WebSite->ASP.NET Configuration menu item in VWD/VS) and create a new user and add them to the "admin" role for the site. You can then login as this new admin user and try uploading new pictures and/or customizing the existing ones on the site (note that both the picture meta-data, as well as the raw image binaries are stored in a database when you do this):
Once you are all done with the above steps we'll have two SQL Express databases installed within the \app_data directory for our project. One of the SQL Express databases is named personal.mdf and contains the tables and stored procedures specific to our web-site (photo and album tables, as well as basic content management support). The other SQL Express database is named aspnetdb.mdf and contains the database storage for the default ASP.NET 2.0 Membership, Role and Profile providers (which the application above is using for login and admin purposes).
Step 2: Creating .SQL Installation Scripts for our Database
Now that we've created a new application + local database, and added custom data to it (new users + their role membership, as well as new photos and albums), we want to deploy the application to a remote hosting server.
The first step we'll take is to create .SQL script files that will enable us to automate re-creating the exact same database schema + database content on our remote hosting account. To-do this we'll use the Database Publishing Wizard we installed as part of the SQL Hosting Toolkit.
To begin, click on the "Server Explorer" tab within Visual Studio or Visual Web Developer to see the databases that the application is using:
As you can see in the above picture, we have two SQL Express databases that we are using: ASPNETDB.MDF and Personal.MDF. To generate .SQL installation files for each one, simply select the database in the solution explorer, then right-click and select the new "Publish to Provider" context menu item (added by the Database Publishing Wizard) setup on it:
This will launch the Database Publishing Wizard and allow us to walkthrough scripting the installation of our database. As I mentioned in the intro of this blog post, the Database Publishing Wizard supports two deployment options: 1) To generate .SQL install script files that you can copy to your remote hoster and run using their existing web admin control panel tools, or 2) To upload the database directly using Database Publishing Web-Services on the hoster web-site.
For this first tutorial, we'll be using the .SQL script file approach - so keep the default radio button selected and provide a name for the .SQL install script file you want to generate:
When you click "next" you'll be given the option to customize some of preferences when creating the .SQL setup file. Note that you can control whether to drop existing objects within the target database, whether you want to target SQL 2000 or SQL 2005 with the script, and whether you want to setup both the schema and data, or just the schema, or just the data:
For this tutorial just keep the defaults selected, and hit next and generate the .SQL script:
You now have a Personal .SQL file that contains a script that you can run on any SQL server to re-create all the tables, sprocs, views, triggers, full-text catalogs, etc. for a database, as well as import and add all of the table row data that was in the database at the time the .SQL file was created.
The .SQL file itself is just plain text - so you can open it up with any text editor to see it and/or customize it with your own statements:
Notice above how the .SQL file includes both the SQL DDL needed to create the Photos table (including all of its constraints and primary-key/foreign-key relationships), as well as the SQL to insert data within the table once it is created (in the case above it is even inserting binary data for the photos - since they are stored in the database).
Once you repeat these steps for the ASPNETDB SQL Express database as well you'll have two .SQL installation scripts that you can use to automatically re-create your SQL database on any SQL Server:
Note that the .SQL files we built can be used to create two separate databases on a server, or they can both be run against the same database to create a single database that has a unified set of tables, sprocs, and data for the application. To accomplish this, simply run both scripts against the same database, and assuming no table or sproc names conflict, you'll have a single database containing everything. This later option is very useful when you have a hosting account that only provides 1 database instance for you to use!
Step 3: Using our .SQL files to create our remote databases
Now that we have our .SQL files, we can go about using them to install our database at our hoster. Exactly how we use the .SQL files to install the database will vary depending on how the hoster gives us access to our SQL account. Some hosters provide an HTML based file-upload tool that allows you to provide a .SQL file - which they will then execute against the SQL database you own.
Other hosters provide an online query tool (like below) that allows you to copy/paste SQL statements to run against your database. If you have a hoster which provides an online query tool like this, then you can open the .SQL file with a text-editor and copy/paste the contents into the query textbox and run it.
The quality of the SQL tools that different hosters provide varies quite a bit. In testing the Database Publishing Wizard we found that some custom-made SQL admin tools provided by hosters had issues where they incorrectly parsed valid SQL statements (in particular GOTO statements). This page describes one issue you might see some hosters have with GOTO statements, along with a workaround you can use. To help improve the overall quality of SQL hosting admin tools, the SQL Server team early next year is going to be shipping the source to a free SQL HTML admin tool that hosters will be able to integrate into their experiences. Hopefully this will help improve the standard experience with all Windows hosters.
If your hoster has no usable HTML web admin tool for allowing you to easily manage your SQL database, then you can also just write a simple ASP.NET page that you FTP (along with your .SQL file) to your web-site and then hit to read the .SQL file on the server in as text, and then pass it as a string to ADO.NET to execute. This will give you the same result as the query analyzer above - and fully create your database for you.
Step 4: Updating our connection-string within web.config
Once we've got our data uploaded within a database at our hoster, we'll want to upload our .aspx files, assemblies and content to the remote site (typically this is done over FTP).
The last step we'll need to take is to open up our web.config file and update the <connectionStrings> section to point at our new database location at the remote hoster. Note that you'll need to get the exact SQL server, database name, and username/password account to use from the hoster.
Using our personal starter kit example above, we'd change the <connectionStrings> section within its web.config file from the default connection-string (which uses two SQL Express database in the local \app_data directory):
<connectionStrings>
<add name="Personal" connectionString="Data Source=.\SQLExpress;Integrated Security=True;User Instance=True;AttachDBFilename=|DataDirectory|Personal.mdf" />
<remove name="LocalSqlServer"/>
<add name="LocalSqlServer" connectionString="Data Source=.\SQLExpress;Integrated Security=True;User Instance=True;AttachDBFilename=|DataDirectory|aspnetdb.mdf" />
</connectionStrings>
<add name="Personal" connectionString="Data Source=.\SQLExpress;Integrated Security=True;User Instance=True;AttachDBFilename=|DataDirectory|Personal.mdf" />
<remove name="LocalSqlServer"/>
<add name="LocalSqlServer" connectionString="Data Source=.\SQLExpress;Integrated Security=True;User Instance=True;AttachDBFilename=|DataDirectory|aspnetdb.mdf" />
</connectionStrings>
To instead use a single SQL Server 2000 database (the "scottguDB" database on the "Server123" box).
<connectionStrings>
<add name="Personal" connectionString="Data Source=Server123;Initial Catalog=scottguDB;Integrated Security=True" providerName="System.Data.SqlClient" />
<remove name="LocalSqlServer"/>
<add name="LocalSqlServer" connectionString="Data Source=Server123;Initial Catalog=scottguDB;Integrated Security=True" providerName="System.Data.SqlClient" />
</connectionStrings>
<add name="Personal" connectionString="Data Source=Server123;Initial Catalog=scottguDB;Integrated Security=True" providerName="System.Data.SqlClient" />
<remove name="LocalSqlServer"/>
<add name="LocalSqlServer" connectionString="Data Source=Server123;Initial Catalog=scottguDB;Integrated Security=True" providerName="System.Data.SqlClient" />
</connectionStrings>
We were able to use a single database (instead of the two above) because we we ran both .SQL files against the single database - which merged all schema and data into a single database instance.
Step 5: We are done
Now we can run the application remotely in a hosted environment, and it should just work.
Summary
The Database Publishing Wizard that ships as part of the SQL Hosting Toolkit should make creating .SQL files for any database (SQL Express or full SQL Server) really easy. You can use this to easily dump your local database and then use it to re-create the exact same database on a remote system.
In future tutorials I'll also show how you can actually re-create your database remotely without even having to generate a .SQL file (instead you can publish the database directly from VS to your hoster over a web-service). Stay tuned for details on how to-do this soon.
Hope this helps,
