Two technical issues over the weekend (II)

The second issue was about a two-node SQL Server cluster down.

The Symptom:
1) Users cannot connect to any of the databases, no database is available;
2) Cluster is gone as cluster service down locally on each node (cluster administrator);

Check & confirmation:
Error “1075 The dependency service does not exist or has been marked for deletion.” reported while attempting to manually re-start the cluster service. Rebooting the nodes does not fix the issue either.

Cause:
The Windows Time service on which the cluster service depends disappeared from both nodes, as shown in the screen-shot(in which the Windows Time service actually presents).

Workaround:
Windows Time service uses w32time.exe and w32time.dll. And these two needs to be repaired or replaced by WFP (Windows File Protection) to get the Windows Time service back. WFP is from %windir%\system32\dllcache, and a Windows Administrator needs to run from a command prompt on both nodes for the reparation:

sfc /scannow

Detailed information about WFP may be found from:
http://support.microsoft.com/kb/q222193/

Once the foregoing command executed successfully, go back to the services panel to start the Windows Time service first (if not started) and then bring back on the cluster service on both nodes.

The next few tasks are for a DBA to bring online the database resources; verify the health of the databases; check connectivity both internal and external; and check OS event logs. A clean reboot then is demanded for both nodes, provided that all the DBA checks and verifications are clean.

The very last step is a reallocation of the databases to move them to their preferred nodes.

By now a healthy cluster is back in service again.

Advertisements
This entry was posted in Blogroll. Bookmark the permalink.

One Response to Two technical issues over the weekend (II)

  1. 木钉 says:

    Hard to comment on something that I have no clue.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s