Microsoft CRM 4.0 Data Migration Manager versus Scribe

Some Gaps in Troubleshooting and Data Map Development Functionality

   

This is not a comprehensive gap analysis by any means, but I've been looking at data mapping and troubleshooting capabilities in Microsoft's Data Migration Manager in CRM 4.0 and I wanted to take a second to share a few of the findings. I realize this tool isn't designed to specifically replace applications like Scribe, but I thought it would be of interest to those already well versed in Scribe who were wondering about the Data migration manager's functionality in the area.

   

The general idea of troubleshooting errors or rejected rows in the Data Migration Manager is "fix it in the source data". For developing the data mappings for larger data conversions, Microsoft suggests preparing a subset of the source data ( 5-10 records per record type) for working out the mapping details and errors. (NOTE: be sure all child records have their parent records in the subset of test data) In Scribe, you could simply create a filter on the source data to accomplish this. A "Data Map" in the DMM is comparable to a DTS in Scribe, but obviously not nearly as complex as a DTS. There are some predefined data maps (comparable to adapters in Scribe) for data sources such as ACT! 6, Outlook 2003, Outlook 2007 and salesforce.com, but that's it. There was a blurb in the help file that suggested other data maps existed, so I contacted Microsoft Resource Center and they confirmed that there are no other predefined data maps for the DMM at this time. Below is a list of Scribe features that seem to be missing in the Data Migration Manager as well as a few things that seem cumbersome to me in regards to troubleshooting and developing data mappings for conversions.

   

Scribe Features that seem to be Missing in the Data Migration Manager

• No Target Variables
• No Step Control
• No Test Job Functionality (for troubleshooting record by record )
• No Formulas (as they exist in Scribe - see below in "Things that I find cumbersome"
• No ability to stop a data migration DURING the migration (you must wait until the migration processes all records in the source files to review errors)
• No integration capabilities
• No "Update" functionality for conversions - if data needs to be updated you must delete the original dataset and rerun the data migration in "Express Mode" using a modified data map

   

Things that I find cumbersome in the CRM 4.0 Data Migration Manager

Formulas (in Scribe speak) are called "Transformations" in the Data Migration Manager. They cannot be created in the DMM user interface. To modify data (the way you would with a formula in Scribe) you need to do the following:

• Create a new data map for the conversion (complete with all field and column mappings etc.)
• Run the data migration (this creates a data map XML file)
• Delete the data you just migrated (because it needs to be modified)
• Export the XML file for the migration that you just created
• Modify the XML file to include the needed data transformations (requires XML coding)
• Rename and Import the XML file back into the DMM (you can't/shouldn't import a file of the same name back into the DMM)
• Rerun the modified data map in "Express Mode" (Express mode simply skips the field and column mapping steps)

   

Hard coding the value of a field in Scribe is quite simple; however Microsoft suggests modifying the source data to accomplish this in the Data Migration Manager.

   

Troubleshooting and data map development is an iterative process. You continue to rerun the conversion until no errors are received…working through each set of errors as you go. The general process is below.

• Run the full data migration on the subset of test data
• export and review errors in Excel (DMM's version of rejected rows table)
• correct errors in source data documents…or modify XML file manually as mentioned above and import
• delete the data you just migrated (or you will get more errors)
• Rerun the data and repeat process…

     

     

Below is an example of a exported data map from DMM (comparable to a Scribe DTS). This is what you need to modify and import into DMM to include a data transformations (accomplished with formulas in Scribe).

   

   

Screen clipping taken: 3/13/2008, 9:33 AM

   

Below is an example of an Excel error report as exported from the DMM (comparable to Scribe's rejected rows table).

   

#		File Name		Line No.	Description		Column Heading	Column Value	More Information
1		contacts.csv		2	A duplicate lookup reference was found.		account	Test01
2		contacts.csv		3	A duplicate lookup reference was found.		account	Test02
3		contacts.csv		4	A duplicate lookup reference was found.		account	Test03
4		contacts.csv		5	The lookup reference could not be resolved.		account	Test05

   

Pasted from <file:///F:\administrator_1996_data_conversion_errors_Wednesday_%20March%2012_%202008%208_14_14%20AM.csv>