Quickly and Safely Move a Microsoft SQL Server Database (MDF and LDF Files) to a New Physical Location (Including Setting Read Write Access)

To move a Microsoft SQL Server database to a new physical location you need to detach, copy, reattach and set permissions for the database MDF and log LDF files associated with the database. I needed to do it as the drive on which each type of database required file (MDF/LDF) was located was to be separately backed up, as per the site backup regulations. I was using SQL Server 2008 R2.

Microsoft’s recommended way of doing this is to use SQL Management Studio. Create a query and detach your database “mydb” by entering and running the following:

use master
go
sp_detach_db ‘mydb’
go

Now copy the MDF and LDF files to their new location and reattach (locations and file names are for this example only):

use master
go
sp_attach_db ‘mydb’,’E:\DATA\mydb.mdf’,’F:\DATA\mydb_log.ldf’
go

You can check the basic properties of the database (and that the file locations have been correctly set) using:

use mydb
go
sp_helpfile
go

The final thing I needed to do was to set the file/folder permissions so that the database could go from read-only to read/write. I first set the folder permissions for my two new folders (“E:\DATA\” and “F:\DATA\” as above). To do this I needed to add the following user to the security settings with full control:

sqlservermssqluser$COMPUTERNAME$mssqlserver

This didn’t actually set the files (server permissions error) so I set the permissions for the “E:\DATA\mydb.mdf” and “F:\DATA\mydb_log.ldf” files individually the same way.

Now open up another query window and type the following to enable read/write access to the database:

use master
go
alter database mydb set read_write with no_wait
go

Now your database is set up exactly as it was previously, only the associated files have moved physical location.

Easily migrate content between SharePoint servers using stsadm.exe and fix the HTTP 401.1 site collection administrator local login problem

I needed to migrate content from one install of SharePoint Web Services 3.0 on Windows 2003 to another physical server and ran into an issue with checking my site locally before changing all the DNS records over. The actual export and import process is quite easy but can take a while if there are a lot of subsites or files within your SharePoint portal.

Migrating between SharePoint installs using stsadm.exe

The easiest way to export an entire SharePoint site is to use stsadm.exe, which is typically located in “C:Program FilesCommon FilesMicrosoft Sharedweb server extensions12BIN” and is available when you have installed SharePoint. I created a reference to this in the PATH variable so I could use it everywhere to make things easier. Thankfully the documentation is good (thanks Microsoft) and you can find more explicit details on exporting and importing from TechNet. Note that this is similar to other versions of SharePoint and more information on various methods of migration can be found on Microsoft Support, including moving databases directly. Chris O’Brian also has a good post about the various approaches.

The simple stsadm command I used to export my portal (including all files and subsites etc) at http://portal.sitename.com was:

stsadm -o export -url http://portal.sitename.com -filename sitename.bak

This produced a lot of ~30mb files and a good log of everything that took place. All the Active Directory user permissions were also included in the export, which was one of my big worries moving to a new server. For local users you have more of a problem as these don’t exist on the new server and need to be recreated.

To import your site back into another SharePoint install, you have to first make sure there is an existing web application and associated site collection (on the root “/”) before copying over all your exported files to the new server. The first time I tried I assumed it would regenerate the site collection based on my export from scratch, but apparently not. The command I used to import was:

stsadm -o import -url http://portal.sitename.com -filename sitename.bak

Now even though you will probably have a lot of files from the export process, you do not need to specify them all, just the main one, in this case “sitename.bak”. After a while your new site will be populated with all the content from your export and is ready for testing before you go live and change your DNS records to point to the new server.

Testing your newly migrated site locally, avoiding HTTP 401.1

As I was using remote desktop to access my new server to run the stsadm.exe import command I wanted to test the site locally by logging in with my site collection administrator details before changing the DNS over. To do this I set up a reference in my hosts file “C:Windowssystem32driversetchosts” on the new server to point http://portal.sitename.com to localhost (127.0.0.1) then tried to visit http://portal.sitename.com within my remote desktop session. This is where I hit a HTTP 401.1 login error due to a security setting built into Windows 2003, even though I tried logging in with the correct site collection administrator details.

This is apparently a security fix to Windows Server 2003 SP1 that stops reflection attacks and according to Microsoft “authentication fails if the FQDN or the custom host header that you use does not match the local computer name”. The details on how to fix this are located at Microsoft Support and I’ve noted the easiest way to fix this by removing the loopback check entirely.

First you need to disable strict name checking by editing the registry on your server. Open the registry editor (run regedit.exe) and go to “HKEY_LOCAL_MACHINESystemCurrentControlSetServicesLanmanServerParameters”. Now click “Edit -> New -> DWORD Value” and name it “DisableStrictNameChecking” and then right click and set it to decimal “1″.

Now go to “HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlLsa” and click “Edit -> New -> DWORD Value” and name it “DisableLoopbackCheck” and then right click and set it to decimal “1″ as well.

You need to restart your server for the changes to have any effect and once you have you should be able to log in to your local site at http://portal.sitename.com without hitting a HTTP 401.1 error with your site collection administrator details. Now you can test out the site before changing the DNS records to point to the new server and removing your host file record.

Migrating data from MySQL to Microsoft SQL Server (and away from MySQL slowness)

I hit a stumbling block with MySQL that has made me seriously rethink using it in future. A simple JOIN query between two tables (one with 1k rows, 1 with 660k rows) was taking a very long time in MySQL. I had a look at indexing etc but couldn’t get the execution time down below 220 seconds on the test machine, no matter how the query was structured. I even considered adding redundant columns to the dataset to allow for indexing using integers, rather than varchars. While trying a few things I decided to test the exact same query on the same data set in Microsoft SQL Server Express 2005, where it took 4 seconds (1 second after the query had been cached).

The difference is obviously massive, far more than I was expecting. I then ran a few tests using DISTINCT, GROUP BY and COUNT(*) and noticed that in every instance, MySQL was slower than SQL Server, even on identical test machines. Considering the database is expected to grow massively over the next year we decided that it would be sensible to consider alternatives to MySQL. We already have a license for SQL Server so the decision was pretty easy (also, all my testing is currently done using SQL Server Express 2005).

Migrating the data across is not easy using .sql exports from MySQL Workbench as the data set is large (60mb of .sql) and the output file needs editing for SQL Server compatibility. Annoyingly SQL Server Express 2005 doesn’t support multiple INSERT VALUES queries (please upgrade to SQL Server 2008!) making it even more difficult.

I found a post on CodeProject by Niklas Henricson that explains how easy it is to migrate tables from MySQL to SQL Server using an ODBC connector from MySQL. You can download the ODBC connector from MySQL and create a connection which can then be easily used in SQL Server Management Studio to transfer data between MySQL and SQL Server. A summary of Niklas’ steps are:

  • Open your ODBC Data Source Administrator from the Control Panel -> Administrative Tools. Under the tab labelled as “System DSN”, press the “Add” button.
  • On the “Create New Data Source” dialog that appeared, choose MySQL ODBC 5.1 Driver and then press the “Finish” button.
  • After that, a MySQL connection configuration dialog will appear. Add your MySQL database account information in it, preferably the “root” account which has full access to your databases in MySQL. Do not change the port to anything other than 3306, unless during your MySQL server installation, you have defined something else.
  • Press the “Test” button to ensure your connection settings are set properly and then the “OK” button when you’re done.
  • In this state, you are ready to establish a link towards MySQL database from your Microsoft SQL Server Management Studio. Open a query window and run the following SQL statement:

EXEC master.dbo.sp_addlinkedserver
@server = N’MYSQL’,
@srvproduct=N’MySQL’,
@provider=N’MSDASQL’,
@provstr=N’DRIVER={MySQL ODBC 5.1 Driver}; SERVER=my.server.ip.or.name; _
DATABASE=databasename; USER=username; PASSWORD=password; OPTION=3′

  • In the query window, run the following SQL statement to import table tablename from the MySQL database databasename, into the database in Microsoft SQL called SQLServerdatabasename.

SELECT * INTO SQLServerdatabasename.dbo.tablename
FROM openquery(MYSQL, ‘SELECT * FROM databasename.tablename’)

The potential for ODBC based migration of data is brilliant and openquery can be scripted to run across all the database tables. I am currently looking at a way of bulk importing an entire database from MySQL to SQL Server and will update this post in the future.

There are a few things to be wary of, such as keys, data types, creation of identity specifications etc.. but this can all be sorted once the data is migrated across. The speed is pretty good and shouldn’t be too much of a concern as this is a one time operation.

TOAD for Oracle Freeware

There is a free version of TOAD, an old, restricted version. It can’t do some of the stuff the commercial version can (I’m currently looking for migration tools) but it looks to be a much nicer way of talking to your Oracle DB than the Oracle tools.

You can get hold of it here.

I’m still having problems with Migration between Access and Oracle using ANY of the Oracle tools, especially Oracle SQL Developer, which throw all kinds of exceptions even when I try to do something as simple as import .csv data.

UPDATE:

Get used to seeing “Feature available in commercial version” as I did on “Menu -> Database -> Import”. Why not hide these features instead of dangling the functionality in front of us! (obviously it’s to make you go out and buy the commercial version, but its still a bit annoying).