Friday, July 18, 2008

July 19-20 Weekend Upgrades

Parts of the hosting system will be unavailable as we're expanding the virtualization environment.   We'll be bringing down the entire server farm to add additional RAM as well as upgrade the virtualization software.  The entire upgrade process will last less than 1 hour but we need to complete full system backups which will take significantly more time. Keep an eye on this blog as we'll post our activities as they happen.  Most of the downtime is expected Sunday morning.
 
-Karl J

Thursday, July 10, 2008

Hosting SQL Server failure 7/10/08

The primary SQL Server HOSTSERV1 suffered a failure last night preventing users from launching TrialWorks.  The issue was corrected approximately 9:05 AM but some databases did not immediately come online - this behavior is by design.  When SQL Server recovers from a failure all databases are placed in a "Recovery" mode and checked for consistency before being brought online. This process may take several minutes to complete for each database.   At the time of this e-mail approximately 5% of the databases were still in process of coming online.

The failure was caused by a Microsoft update which failed to properly install at 3:00 AM EST.  We apologize for any inconvenience this may have caused.
 
 
Administrative Note (cause of issue):
 
Event Type: Error
Event Source: MsiInstaller
Event Category: None
Event ID: 1023
Date:  7/10/2008
Time:  3:03:35 AM
User:  NT AUTHORITY\SYSTEM
Computer: HOSTSERV1
Description:
Product: Microsoft SQL Server 2005 - Update 'GDR 3068 for SQL Server Database Services 2005 ENU (KB948109)' could not be installed. Error code 1603. Additional information is available in the log file C:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Hotfix\SQL9_Hotfix_KB948109_sqlrun_sql.msp.log.
 
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 7b 31 33 30 41 33 42 45   {130A3BE
0008: 31 2d 38 35 43 43 2d 34   1-85CC-4
0010: 31 33 35 2d 38 45 41 37   135-8EA7
0018: 2d 35 41 37 32 34 45 45   -5A724EE
0020: 36 43 45 32 43 7d 20 7b   6CE2C} {
0028: 31 44 37 38 35 45 42 33   1D785EB3
0030: 2d 45 36 44 44 2d 34 46   -E6DD-4F
0038: 41 41 2d 39 34 39 43 2d   AA-949C-
0040: 44 32 45 35 44 33 37 36   D2E5D376
0048: 43 37 39 32 7d 20 31 36   C792} 16
0050: 30 33                     03     
 
/Author: Karl J