BAM Portal Errors – Failed to list permissions for BAM view. System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values

I decided to put the name of the error in the post title … but this post also could be called “Why you shouldn’t delete Active Directory accounts (if you don’t know where it is being used!)”… but before I tell you why, let me try to explain the problem and put some context on it.

Last week I had a funny surprised when I tried to access my BAM Portal in my BizTalk Server 2006 environment

An unspecified error has occured.
Use the navigation bar on the left to access Business Activity Monitoring views.
If the problem persist, contact you System Administrator.

I just love this type of errors! Because I’m also one of the System Administrator, meaning that I was f*$#%& .

My first reaction is that it could have been some connectivity problem, and as the error indicates, I tried to navigate the remaining views to see if the problem remained… and surprisingly this problem occurred only in certain views!

PROBLEM

So at this point I knew it was happening some problem. Of course the first thing we should do is to check the Event Viewer to see if we can find more details about the error and I found three errors related between themselves and associated with BAM:

Starting from below:

• The first error was:

Current User: DOMAIN\sandro
EXCEPTION:
Microsoft.BizTalk.Bam.Management.BamManagerException: Failed to list permissions for BAM view. —> System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values.
at System.Data.SqlClient.SqlBuffer.get_String()
at System.Data.SqlClient.SqlDataReader.GetString(Int32 i)
at Microsoft.BizTalk.Bam.Management.SecurityModule.ListViewPermissions(String viewName, String& dboUsername)
— End of inner exception stack trace —
at Microsoft.BizTalk.Bam.Management.SecurityModule.ListViewPermissions(String viewName, String& dboUsername)
at Microsoft.BizTalk.Bam.WebServices.SecurityHelper.VerifyViewPermissions(String viewName, IPrincipal user, BamManager bamManager, Boolean throwIfNoPermissions)
at Microsoft.BizTalk.Bam.WebServices.SecurityHelper.VerifyViewPermissions(String viewName, IPrincipal user, BamManager bamManager)
at Microsoft.BizTalk.Bam.WebServices.Management.BamManagementService.GetViewDetailsAsXml(String viewName)

Observation: Impossible, I’m BizTalk Administrator but most important my user is the owner of this views!!!

• The second error was:

Current User: DOMAIN\sandro
EXCEPTION: System.Web.Services.Protocols.SoapException: Internal Server Error.

Observation: says absolutely nothing … trash!

• And finally the third error was:

(BAMPortal.PortalApplication) Void LogAllErrors(System.Exception[]): System.Web.HttpException: Error executing child request for /BAM/Pages/Search.aspx. —> System.Web.HttpUnhandledException: Exception of type ‘System.Web.HttpUnhandledException’ was thrown. —> System.Web.Services.Protocols.SoapException: System.Web.Services.Protocols.SoapException: Internal Server Error.
at Microsoft.BizTalk.Bam.WebServices.Management.BamManagementService.GetViewDetailsAsXml(String viewName)
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at Microsoft.BizTalk.Bam.WebServices.ManagementService.BamManagementService.GetViewDetailsAsXml(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.BamDefinitionCache.FetchViewDefinition(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.BamDefinitionCache.GetBamDefinition(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.BuildColumnsCollection()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.EnsureColumnsCollection()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.ColumnsOfType(ColumnTypes type)
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.EnsureInstanceColumns()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.get_InstanceColumns()
at BAMPortal.ColumnsChooser_ascx.GetColumns()
at BAMPortal.ColumnsChooser_ascx.GetAvailableColumns()
at BAMPortal.ColumnsChooser_ascx.ReconcileColumns()
at BAMPortal.ColumnsChooser_ascx.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
— End of inner exception stack trace —
at System.Web.UI.Page.HandleError(Exception e)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest()
at System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context)
at System.Web.UI.Page.ProcessRequest(HttpContext context)
at ASP.pages_search_aspx.ProcessRequest(HttpContext context)
at System.Web.HttpServerUtility.ExecuteInternal(IHttpHandler handler, TextWriter writer, Boolean preserveForm, Boolean setPreviousPage, VirtualPath path, VirtualPath filePath, String physPath, Exception error, String queryStringOverride)
— End of inner exception stack trace —
at System.Web.HttpServerUtility.ExecuteInternal(IHttpHandler handler, TextWriter writer, Boolean preserveForm, Boolean setPreviousPage, VirtualPath path, VirtualPath filePath, String physPath, Exception error, String queryStringOverride)
at System.Web.HttpServerUtility.Execute(String path, TextWriter writer, Boolean preserveForm)
at System.Web.HttpServerUtility.Transfer(String path, Boolean preserveForm)
at System.Web.HttpServerUtility.Transfer(String path)
at BAMPortal.navbar_ascx.TreeViewNav_NodeClicked(Object sender, TreeNodeEventArgs eventArgs)
at Microsoft.BizTalk.Bam.Portal.ClickableTreeView.OnTreeNodeClicked(TreeNode node)
at Microsoft.BizTalk.Bam.Portal.ClickableTreeView.RaisePostBackEvent(String eventArgument)
at System.Web.UI.WebControls.TreeView.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest()
at System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context)
at System.Web.UI.Page.ProcessRequest(HttpContext context)
at ASP.pages_view_aspx.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Observation: much information … yet without giving me many tips on the problem that was happening.

Actually the first error is what brings us a better sense of the error… it’s a permission problem to access the view… but why? and what was really causing this problem?

CAUSE

• BAM Management Utility (BM.exe) doesn’t provide the capability to grant group permissions to BAM views instead you need to use user’s accounts. So these issues may occur if the user account which was granted permission to BAM objects are deleted from Active Directory or from Local Computers.

When you access to a certain view in BAM Portal the services invoked by the Portal will try to check the permission for ALL the users associated to this specific view and not only my user, so if a user was deleted from AD or from the local computer without first removing it from the view, the services will fail while attempting to map the account name with Security ID and you will get with this annoying problem: “Data is Null. This method or property cannot be called on Null values.”

You may experience any one or more of the following symptoms:

• When you access to certain views in the BAM Portal
• When you try to execute any kind of operation using BM.exe tool against certain views, like “bm.exe get-accounts”; “bm.exe remove-account” or “bm.exe remove-view”

SOLUTION

• You have to manually delete those user accounts from SQL Server.

How can I really solved this problem?

So now I know the problem, the cause and the solution… but is it that simple?
… NO of course!

First problem: How can I really know with account(s) is causing the problem?

You can have many users in your organization associated with the view and may have passed months or years since the last time we associate the users account to this view… so it is really a problem to find with user is causing the problem.

• Option 1: you can ask!
  • I know that if I ask who was deleting accounts in the AD or which accounts have been deleted, I will get the typical response… no one or no account has been deleted!!! Don’t go there is an endless road.
• Option 2: You can use BM.exe!
  • Unfortunately we also can’t use BM tool to ask with account have permission to this view (bm.exe get-accounts), we get the following error:
    • ERROR: Failed to list permissions for BAM view.
      Data is Null. This method or property cannot be called on Null values.
• Option 3: See in the documentation
  • Another endless road . This type of documentation should exist, but few companies actually have them!

I finally decided to try to make a query directly in the BAM Primary Import database (BAMPrimaryImport) to attempt to get a list of all the account that were associated with this view in order to validate with my system administrators if they all existed in AD, this was the result:

WITH Query AS (
SELECT
    [UserName] = CASE princ.[type]
                    WHEN 'S' THEN princ.[name]
                    WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI
                 END,
    [UserType] = CASE princ.[type]
                    WHEN 'S' THEN 'SQL User'
                    WHEN 'U' THEN 'Windows User'
                 END,
    [DatabaseUserName] = princ.[name],
    [Role] = null,
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --database user
    sys.database_principals princ
LEFT JOIN
    --Login accounts
    sys.login_token ulogin on princ.[sid] = ulogin.[sid]
LEFT JOIN
    --Permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = princ.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col ON col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
LEFT JOIN
    sys.objects obj ON perm.[major_id] = obj.[object_id]
WHERE
    princ.[type] in ('S','U')
UNION
--List all access provisioned to a sql user or windows user/group through a database or application role
SELECT
    [UserName] = CASE memberprinc.[type]
                    WHEN 'S' THEN memberprinc.[name]
                    WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI
                 END,
    [UserType] = CASE memberprinc.[type]
                    WHEN 'S' THEN 'SQL User'
                    WHEN 'U' THEN 'Windows User'
                 END,
    [DatabaseUserName] = memberprinc.[name],
    [Role] = roleprinc.[name],
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --Role/member associations
    sys.database_role_members members
JOIN
    --Roles
    sys.database_principals roleprinc ON roleprinc.[principal_id] = members.[role_principal_id]
JOIN
    --Role members (database users)
    sys.database_principals memberprinc ON memberprinc.[principal_id] = members.[member_principal_id]
LEFT JOIN
    --Login accounts
    sys.login_token ulogin on memberprinc.[sid] = ulogin.[sid]
LEFT JOIN
    --Permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col on col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
LEFT JOIN
    sys.objects obj ON perm.[major_id] = obj.[object_id]
UNION
--List all access provisioned to the public role, which everyone gets by default
SELECT
    [UserName] = '{All Users}',
    [UserType] = '{All Users}',
    [DatabaseUserName] = '{All Users}',
    [Role] = roleprinc.[name],
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --Roles
    sys.database_principals roleprinc
LEFT JOIN
    --Role permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col on col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
JOIN
    --All objects
    sys.objects obj ON obj.[object_id] = perm.[major_id]
WHERE
    --Only roles
    roleprinc.[type] = 'R' AND
    --Only public role
    roleprinc.[name] = 'public' AND
    --Only objects of ours, not the MS objects
    obj.is_ms_shipped = 0
)
SELECT * From  Query
WHERE Role like '%name_of_the_view%' AND ObjectType like 'View'

It may not be sophisticated or the best way but it served my purposes and was much easier and effective than be looking one by one in the SQL Server management Studio.

Now that I had a list of users, it was easy to validate with the system administrators which account had been deleted.

After so much effort and work trying to find the user that was causing the problem I found a simple and more effective solution to this problem in this post on MSDN Blogs: BizTalk Error: BAM Management Utility Errors, using this query:

USE BAMPrimaryImport
GO
Select Name,SID,SUser_SName(SID) as UserAccount from sysusers
WHERE ISLogin = 1 AND issqluser = 0 AND isntuser = 1

In the result for the DB roles, the third column for Windows users (and groups) should display the User Name (or group name) and can’t be null.

So if you execute the above query it will give the user(s) account(s) which was deleted from Domain Controller or Local Computer.

Second problem: How can I manually delete the account?

Yep … this question may seem a bit ridiculous … but the truth is, manually delete the account was not so simple after all… at least for me it wasn’t.

You can’t just delete that account from SQL Server. To do that you need to follow the steps:

• Go to SQL Management Studio, expand the BAMPrimaryImport Database
• Go to Security –> Roles –> Database Roles –> BAM_ManagementWS
• Right click BAM_ManagementWS and go to properties option which will open a new window (Database Role Properties – BAM_ManagementWS)

• In that window go to Securables, choose the NT User account which needs to be deleted.

• In the bottom portion you will see that NT User Account having “View Definition” Permission. Uncheck that option then click OK.
• Now you should able to delete the user from Security –> User –> <NT User Account>

However if you try to delete the user without following these steps, like I did , you get stuck in another huge problem:

TITLE: Microsoft SQL Server Management Studio
——————————
Drop failed for User ‘domain\user’.  (Microsoft.SqlServer.Smo)
——————————
ADDITIONAL INFORMATION:
An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)
——————————
The database principal has granted or denied permissions to objects in the database and cannot be dropped. (Microsoft SQL Server, Error: 15284)

So we need now to manual revoke the Grant access to BAM_ManagementWS by execute the following query:

REVOKE VIEW DEFINITION ON USER::[domain\user] TO [BAM_ManagementWS] AS [domain\user]
GO

Security Considerations for the BAM Portal

Using the principle of least privilege, user accounts should have restrictive permissions to perform routine tasks in the BAM portal. Keep the following points in mind as you set up your user accounts for BAM to balance security with appropriate access for users.

User accounts

User accounts with minimum permissions are not able to use the BAM portal distributed navigation feature. To be able to use this feature, these accounts must have sufficient permissions to allow access to the Web services on the remote computer as well as on the local computer.

User accounts for the BAM Web services must have permissions to access all referenced databases and must be a member of the BAM_ManagementWS role in the referenced databases.

For the following user types, you should be aware of these considerations:

• Domain Users: These users must have access permissions on remote computers that host BAM Primary Import databases that are being accessed.
• Local User: Users who are assigned this role cannot use distributed navigation.

Administrator accounts

Administrators must be members of the securityadmin or sysadmin groups to grant permissions to domain users.

To run the BAM Management utility, you must be at least a database operator for the BAM databases.

Final Notes

I would like to thank Nino Crudele for the help that he gave me to solve this problem and to my coworker and friend José Barbosa who helped me creating these SQL scripts and as result solving the problem.

And a final note for all system administrator… please don’t delete Active Directory accounts… instead disable them!!