Know issues when upgrading from OM 2007 R2 to OM 2012
Well first of all I really encourage you to upgrade to OM 2012 because it really brings a lot of features and fixes as well as cool integration with the other System Center 2012 products. Now thinking about a product which is very complex, also supports localization and allows a lot of flexibility and extensibility, then we also have to take in account that for different custom extensions or in certain situations, when upgrading from one major version to another, we can come across some issues ...
First of all before upgrading please remember because this is very important: CHECK EVERYTHING you find on TechNet and in the documentations before upgrading to be sure that you have not missed ANY step!
REMEMBER: If the upgrade of the old RMS and Management Group (databases) will fail, then you will LOSE your RMS and Management Group (databases).
This is why you must allways BACKUP both of your DATABASES, RMS Encryption Key so that you can rebuild your environment in the case of an upgrade failure before trying again.
Now I will write here about a list of known issues with the upgrade which you have to take into account and check before upgrading:
1. Upgrade failure when OM User Roles exist that have non-ENU locale
=========================================================================
There is an issue with the upgrade if there was any user roles created in OM which do NOT have ENU locale.
The errors in the Upgrade Log file will look like this:
Error: :Failed to update user role: : Threw Exception.Type: Microsoft.EnterpriseManagement.Common.NullConstraintException, Exception Error Code: 0x80131500, Exception.Message: Parameter UserRoleDisplayName cannot be null.<br> Error: :StackTrace: at Microsoft.EnterpriseManagement.Common.Internal.SecurityConfigurationServiceProxy.UpsertUserRolesV2(ICollection`1 urUpdateResults, ICollection`1 urScopeUpdateResults, ICollection`1 urViewScopeUpdateResults, ICollection`1 urTaskScopeUpdateResults, ICollection`1 urConsoleTaskScopeUpdateResults, ICollection`1
To verify if you would be impacted by this issue, you can run the following SQL Query on the Operational database:
SELECT<br> DISTINCT UV.UserRoleId AS [UserRoleId],<br> LT.LTValue AS [RoleName],<br> LT.LanguageCode AS [LanguageCode]<br> FROM UserRoleView AS UV WITH(NOLOCK) <br>INNER JOIN LocalizedText AS LT WITH(NOLOCK)<br> ON UV.UserRoleId = LT.LTStringId <br>WHERE LT.LanguageCode <> 'ENU'
If you get any results back then please run this query to fix this and then upgrade to OM 2012:
DELETE LocalizedText <br> WHERE LTStringId IN ( <br> SELECT DISTINCT LT.LTStringId <br> FROM UserRoleView AS UV WITH(NOLOCK) <br> INNER JOIN LocalizedText AS LT WITH(NOLOCK) <br> ON UV.UserRoleId = LT.LTStringId <br> WHERE LT.LanguageCode <> 'ENU' <br> ) AND LanguageCode <> 'ENU'
Additionally please also verify this SQL query:
SELECT *<br> FROM LocalizedText<br> WHERE<br> LTValue LIKE '%Operation%Manager Report%' AND<br> LTStringId NOT IN (<br> SELECT DISTINCT(LTStringId)<br> FROM LocalizedText<br> WHERE LanguageCode = 'ENU'<br> )
Again here if you notice that you have any result then pease run this query to solve the issue:
UPDATE LocalizedText<br> SET LanguageCode = 'ENU'<br> WHERE<br> LTValue LIKE '%Operation%Manager Report%' AND<br> LanguageCode <> 'ENU' AND<br> LTStringId NOT IN (<br> SELECT DISTINCT(LTStringId)<br> FROM LocalizedText<br> WHERE LanguageCode = 'ENU'<br> )
So now all the users should have the ENU locale in the database.
*ANOTHER* thing which is important here and what you should check, is that for every UserRole you *DO* have a DisplayName. If a UserRole does not have a DisplayName, the Upgrade will fail.
To check for this, open the OM PowerShell, and type: Get-UserRole | ft Name, DisplayName
If you get any results back where the DisplayName is missing, then open the OM Console and add the DisplayName there and then check again to see if you still have any UserRole with no DisplayName.
2. Upgrade failure if any discovered object exists that has a really long display name
==========================================================================
The next issue is caused by any discovered object which has a really long display name. In the upgrade process the DisplayName from the BaseManagedEntity table is copied over to the DisplayName field of all the MT_* tables. The problem here is that the DisplayName from BaseManagedEntity supports a much longer charater string then the DisplayName filed of the MT_* tables do < NVARCHAR(MAX) vs NVARCHAR(256) >.
If you are impacted by this issue you will see the following in the upgrade log file:
:Threw Exception.Type: System.Data.SqlClient.SqlException, Exception Error Code: 0x80131904, Exception.Message: String or binary data would be truncated.<br> String or binary data would be truncated.<br> The statement has been terminated.<br> The statement has been terminated.<br> Error: :StackTrace: at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)<br> at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()<br> at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)<br> at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async)<br> at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)<br> at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()<br> at Microsoft.EnterpriseManagement.OperationsManager.Setup.DBConfigurationHelper.DBConfiguration.RunSqlCommandsList(IEnumerable`1 sqlCommands)<br> Error: :RunStandardScripts failed with the following exception: : Threw Exception.Type: System.Reflection.TargetInvocationException, Exception Error Code: 0x80131604, Exception.Message: Exception has been thrown by the target of an invocation.<br> Error: :StackTrace: at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeType typeOwner)<br> at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeType typeOwner)<br> at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)<br> at System.Delegate.DynamicInvokeImpl(Object[] args)<br> at Microsoft.EnterpriseManagement.SetupFramework.ActionEngine.Action.Run(String displayStringNamespace, ProgressData progressData, Func`2 progressDelegate)<br> at Microsoft.EnterpriseManagement.SetupFramework.ActionEngine.InstallStep.Run(String displayStringNamespace, ProgressData progressData, Func`2 progressDelegate)<br> Error: :Inner Exception.Type: System.Data.SqlClient.SqlException, Exception Error Code: 0x80131604, Exception.Message: String or binary data would be truncated.<br> String or binary data would be truncated.<br> The statement has been terminated.<br> The statement has been terminated.<br> Error: :InnerException.StackTrace: at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)<br> at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()<br> at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)<br> at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async)<br> at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)<br> at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()<br> at Microsoft.EnterpriseManagement.OperationsManager.Setup.DBConfigurationHelper.DBConfiguration.RunSqlCommandsList(IEnumerable`1 sqlCommands)<br> at Microsoft.EnterpriseManagement.OperationsManager.Setup.DBConfigurationHelper.DBConfiguration.RunSqlStringsFromSqlFile(String sqlFileName)<br> at Microsoft.EnterpriseManagement.OperationsManager.Setup.DBConfigurationHelper.DBConfiguration.RunSqlScripts(String configStep)<br> at Microsoft.SystemCenter.Essentials.SetupFramework.InstallItemsDelegates.OMDatabaseProcessor.RunStandardScripts()<br> Error: :FATAL ACTION: RunStandardScripts<br> Error: :FATAL ACTION: DatabaseActions<br> Error: :ProcessInstalls: Running the PostProcessDelegate returned false.<br> Always: :SetErrorType: Setting VitalFailure. currentInstallItem: Database Configuration<br> Error: :ProcessInstalls: Running the PostProcessDelegate for OMDATABASE failed.... This is a fatal item. Setting rollback.
Now to check if you would be impacted by this issue, you can run this following SQL Query on the Operational database:
SELECT<br> DisplayName,<br> LEN(DisplayName)<br> FROM BaseManagedEntity<br> WHERE LEN(DisplayName) > 255
So if you get ANY results back, make sure that you backup your databases and execute this SQL query which will fix this and let you upgrade even with objects with display names longer than 256:
DECLARE<br> @TableName SYSNAME,<br> @ColumnName SYSNAME,<br> @Statement NVARCHAR(2000)<br> SELECT @ColumnName = ColumnName<br> FROM ManagedType AS MT WITH(NOLOCK)<br> JOIN ManagedTypeProperty AS MTP WITH(NOLOCK)<br> ON MT.ManagedTypeId = MTP.ManagedTypeId<br> WHERE<br> MT.TypeName = N'System.Entity' AND<br> MTP.ManagedTypePropertyName = N'DisplayName'<br> SELECT TOP 1 @TableName = MT.ManagedTypeTableName<br> FROM ManagedType AS MT WITH(NOLOCK)<br> JOIN DerivedManagedTypes DMT<br> ON MT.ManagedTypeId = DMT.BaseTypeId<br> WHERE<br> MT.IsAbstract = 0 AND<br> MT.IsDeleted = 0 AND<br> DMT.BaseTypeId = DMT.NonAbstractBaseTypeId AND<br> DMT.Level = 0 AND<br> MT.ManagedTypeTableName IS NOT NULL<br> ORDER BY MT.ManagedTypeTableName<br> WHILE (@@ROWCOUNT <> 0) BEGIN<br> SET @Statement = N'<br> IF NOT EXISTS(<br> SELECT *<br> FROM sys.columns AS C WITH(NOLOCK)<br> JOIN sys.types AS T WITH(NOLOCK)<br> ON C.user_type_id = t.user_type_id<br> WHERE<br> C.name = ''' + @ColumnName + ''' AND<br> C.object_id = OBJECT_ID(''' + @TableName + ''') AND<br> T.name = ''nvarchar'' AND<br> C.max_length = 8000<br> ) BEGIN<br> ALTER TABLE ' + @TableName + '<br> ALTER COLUMN ' + @ColumnName + ' NVARCHAR(4000)<br> END'<br> EXEC(@Statement)<br> SELECT TOP 1 @TableName = MT.ManagedTypeTableName<br> FROM ManagedType AS MT WITH(NOLOCK)<br> JOIN DerivedManagedTypes AS DMT WITH(NOLOCK)<br> ON MT.ManagedTypeId = DMT.BaseTypeId<br> WHERE<br> MT.IsAbstract = 0 AND<br> MT.IsDeleted = 0 AND<br> DMT.BaseTypeId = DMT.NonAbstractBaseTypeId AND<br> DMT.Level = 0 AND<br> MT.ManagedTypeTableName IS NOT NULL AND<br> MT.ManagedTypeTableName > @TableName<br> ORDER BY MT.ManagedTypeTableName<br> END
3. Upgrade failure when certain custom\unsealed Management Packs exist in OM
==========================================================================
There are some Management Packs which cause the upgrade process from 2007 R2 to 2012 to fail because of the fact that this/these Management Pack(s) fail to be upgraded to 2012 Schema Version.
Here is what the upgrade log file will look if you ran into this issue:
Info: :Updating Schema from v1 to v2<br> Error: :Error:failed to migrate old MPs. This is a fatal error<br> Error: :ImportManagementPacks failed with the following exception: : Threw Exception.Type: System.Reflection.TargetInvocationException, Exception Error Code: 0x80131604, Exception.Message: Exception has been thrown by the target of an invocation.<br> Error: :StackTrace: at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeType typeOwner)<br> at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)<br> at System.Delegate.DynamicInvokeImpl(Object[] args)<br> at Microsoft.EnterpriseManagement.SetupFramework.ActionEngine.Action.Run(String displayStringNamespace, ProgressData progressData, Func`2 progressDelegate)<br> at Microsoft.EnterpriseManagement.SetupFramework.ActionEngine.InstallStep.Run(String displayStringNamespace, ProgressData progressData, Func`2 progressDelegate)<br> Error: :Inner Exception.Type: System.ApplicationException, Exception Error Code: 0x80131604, Exception.Message: Failed to migrate OM MPs to OM12 schema<br> Error: :InnerException.StackTrace: at Microsoft.SystemCenter.Essentials.SetupFramework.InstallItemsDelegates.OMDatabaseProcessor.ImportManagementPacks(String installMode, ProgressData progressData, Func`2 progressDelegate)<br> Error: :FATAL ACTION: ImportManagementPacks<br> Error: :FATAL ACTION: DatabaseActions<br> Error: :ProcessInstalls: Running the PostProcessDelegate returned false.<br> Always: :SetErrorType: Setting VitalFailure. currentInstallItem: Database Configuration<br> Error: :ProcessInstalls: Running the PostProcessDelegate for OMDATABASE failed.... This is a fatal item. Setting rollback.
Here are the known issues which could cause this failure and the SQL Queries with which to check if you would be impacted or not, including of course the solution for each of them:
Cause 1: OM07 allows display strings targeted to element not existed however OM12 does not so the following import fails.
To check if you would be impacted by this scenario, please execute the following SQL Query on your OM Database:
SELECT M.MPName, M.MPFriendlyName, D.ElementName<br> FROM ManagementPack AS M WITH(NOLOCK)<br> INNER JOIN DisplayStringView AS D WITH(NOLOCK)<br> ON M.ManagementPackId = D.ManagementPackId<br> WHERE<br> D.MPElementId <> M.ManagementPackId<br> AND D.MPElementId NOT IN (<br> SELECT MPElementId<br> FROM MPElementView<br> )<br> AND M.MPName NOT IN (<br> 'System.Health.Internal',<br> 'System.BaseliningTasks.Library',<br> 'Microsoft.SystemCenter.ClientMonitoring.Internal',<br> 'Microsoft.SystemCenter.Notifications.Internal'<br> )<br> ORDER BY M.MPName, D.ElementName
If you get back any results, then for all display strings targeted to D.ElementName in these Management Packs, first check whether element name is correct. If it’s not misspelling issue, remove these display strings and if it is a misspelling issue, then correct it so that it matches the actual element name.
Cause 2: Missing MP reference to System.Library for Management Packs having Console Tasks defined.
To check if you would be impacted by this scenario, please execute the following SQL Query on your OM Database:
DECLARE @SystemLibaryId UNIQUEIDENTIFIER<br> SELECT @SystemLibaryId = ManagementPackId<br> FROM ManagementPack WITH(NOLOCK)<br> WHERE MPName = 'System.Library'<br> SELECT DISTINCT(M.MPName), M.MPFriendlyName<br> FROM ManagementPack AS M WITH(NOLOCK)<br> INNER JOIN ConsoleTask AS C WITH(NOLOCK)<br> ON M.ManagementPackId = C.ManagementPackId<br> WHERE NOT EXISTS (<br> SELECT *<br> FROM ManagementPackReferences AS R WITH(NOLOCK)<br> WHERE<br> R.ManagementPackIdReffedBy = M.ManagementPackId<br> AND R.ManagementPackIdSource = @SystemLibaryId<br> )
If you get any results back, then modify these Management Packs and add the Reference to the System.Library Management Pack.
Cause 3: Duplicate entries for CategoryOverride, ModuleOverride or MonitorOverride.
To check if you would be impacted by this scenario, please execute the following SQL Query on your OM Database:
SELECT MPName, MPFriendlyName<br> FROM ManagementPack WITH(NOLOCK)<br> WHERE MPXML IS NULL
If you get any results back, then uninstall these Management Packs and reinstall them after upgrade.
NOTE: If you are unsure of what do to here, then for *extra* safety and if it is possible, just remove all your 3rd party and custom Management Packs and import them back into OM after it was successfully upgraded to 2012.
4. Upgrade fails due to the fact that the db_owner user of the OM database is a SQL user and NOT a Domain Account
======================================================================================================
If we have SQL installed in Mixed Authentication Mode, if the db_owner of the OM database is a SQL User and NOT a Domain Account, the upgrade will fail with this error message:
"operation database configuration failed to install"
Here is what the upgrade log file will look if you ran into this issue:
: Error: :PopulateUserRoles: failed : Threw Exception.Type: System.Runtime.InteropServices.COMException, Exception Error Code: 0x80070539, Exception.Message: The security ID structure is invalid. (Exception from HRESULT: 0x80070539)<br> : Error: :StackTrace: at Microsoft.Mom.Sdk.UserRoleSetup.SetupProgram.populateUserRoles(String adminRoleGroup, String sdkAccount, InstallTypes installType, String installDirectory, Boolean overwriteExistingUsers)<br> at Microsoft.EnterpriseManagement.OperationsManager.Setup.ServerConfiguration.PopulateUserRoles(String adminRoleGroup, String sdkAccount, String installDirPath)<br> : Error: :FATAL ACTION: PopulateUserRoles<br> : Error: :FATAL ACTION: DatabaseActions<br> : Error: :ProcessInstalls: Running the PostProcessDelegate returned false.<br> : Always: :SetErrorType: Setting VitalFailure. currentInstallItem: Database Configuration<br> : Error: :ProcessInstalls: Running the PostProcessDelegate for OMDATABASE failed.... This is a fatal item. Setting rollback.
To solve this issue make sure that you set SQL Authentication to Windows Authentication and that you change the db_owner of the OM database to a Domain Account - it has to be a domain account, not a local account. Please *also* make sure that there exists no SQL User which also has the db_owner rights on either of the OM databases. If such SQL Users exist, then just make sure that you remove the db_owner role from the OM databases.
5. Unable to see any performance or availability details using Widgets
=============================================================
When using Performance Widgets you may experience one or more of the following symptoms:
- While creating Performance widgets, none of the performance objects/counters are shown in the wizard.
- You are unable to retrieve availability details using OpsMgr state widgets.
- Network dashboard views are not showing any availability details or performance details.
- The Reset Health State task fails with the exception below shown under the task status view:
A module reported an error 0x80070490 from a callback which was running as part of rule "System.Health.ResetState" running for instance "<instance name>" with id:"{GUID}" in management group "<Management Group Name>".Error Code: -2130771918 (Unknown error (0x80ff0032)).
To check if you are impacted by this issue, run this SQL Query on the operational database:
IF EXISTS (<br> SELECT *<br> FROM __MOMManagementGroupInfo__ AS MGI WITH(NOLOCK)<br> JOIN MT_ManagementGroup AS MG WITH(NOLOCK)<br> ON MGI.ManagementGroupId = MG.Id_6B1D1BE8_EBB4_B425_08DC_2385C5930B04<br> ) BEGIN<br> PRINT 'OK - issue NOT found.'<br> END ELSE BEGIN<br> PRINT '!!! THE ISSUE EXISTS !!!'<br> END
Now if you get the message that the issue exists, then execute this SQL Query to fix it:
UPDATE __MOMManagementGroupInfo__<br> SET ManagementGroupId = (<br> SELECT TOP 1 Id_6B1D1BE8_EBB4_B425_08DC_2385C5930B04<br> FROM MT_ManagementGroup WITH(NOLOCK)<br> )
Now you should be able to see perfromance or availabilty data in the Widgets.
CHEERS! :D