« Previous Next »

Thread: dbFullSql timeout issue

Last post 08-26-2009 3:36 PM by Woland. 5 replies.

Average Rating Rate It (5)

RSS

Page 1 of 1 (6 items)

Sort Posts:

  • 08-24-2009, 6:32 PM

    • Woland
    • Not Ranked
    • Joined on 04-07-2003, 5:38 PM
    • Los Angeles
    • Posts 12

    dbFullSql timeout issue

    I have a manifest file that needs to run the certain SQL script that sometimes (depending on the server load) will execute longer than 20 seconds. That I believe causes dbFullSql to generate timeout expired exception shown below:

     "Error: Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

    Error count: 1"

     I tried passing "connect timeout=380" to the connection string setting but that doesn't seem to have any affect. The solutions I saw online envolved splitting complex SQL into multiple SQL files. In my case it is a simple SQL file that generated the dates table for the reporting purpose and should typically finish in 10 seconds, but due to SQL load, might take longer in certain condition.

     Is it possible to encrease the timeout on Sql Provider?

     

  • 08-24-2009, 8:45 PM In reply to

    • yaminij
    • Top 50 Contributor
    • Joined on 09-25-2007, 12:41 AM
    • Posts 187

    Re: dbFullSql timeout issue

    It looks like your SQL Server is not responding. Can you do a -debug and send us the stack trace. Please make sure you are able to connect to the database using this connection string by other means (for example ADO.NET APIs).

    Thanks
    Yamini Jagadeesan, SDET, Microsoft IIS Team
  • 08-25-2009, 6:47 PM In reply to

    • Woland
    • Not Ranked
    • Joined on 04-07-2003, 5:38 PM
    • Los Angeles
    • Posts 12

    Re: dbFullSql timeout issue

    This is one of 20 SqlScripts executed as part of the install, the error happens almost at the end. SQL Scripts before that particular one are executed correctly, so I do not think it is a problem with a connection string. That said, I created a sample, which only has two SQL scripts.

    Since I couldn't find a way to attach a file, I'll simply descrive the package:

    The structure is following:

    /SqlScripts
       sql_DataLoad.sql
       sql_schema.sql
    Manifest.xml
    Parameters.xml

    Now contents:

       sql_schema.sql
    ------------------------------------
    /****** Object:  Table [dbo].[ReportingDates]    Script Date: 07/21/2009 17:25:58 ******/
    IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[ReportingDates]') AND type in (N'U'))
    DROP TABLE [dbo].[ReportingDates]
    GO

    /****** Object:  Table [dbo].[ReportingDates]    Script Date: 07/21/2009 17:25:58 ******/
    SET ANSI_NULLS ON
    SET QUOTED_IDENTIFIER ON
    IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[ReportingDates]') AND type in (N'U'))
    BEGIN
    CREATE TABLE [dbo].[ReportingDates](
     [DateKey] [int] NOT NULL,
     [DateFull] [datetime] NULL,
     [CharacterDate] [varchar](10) NULL,
     [FullYear] [char](4) NULL,
     [QuarterNumber] [tinyint] NULL,
     [WeekNumber] [tinyint] NULL,
     [WeekDayName] [varchar](10) NULL,
     [MonthDay] [tinyint] NULL,
     [MonthName] [varchar](12) NULL,
     [YearDay] [smallint] NULL,
     [DateDefinition] [varchar](50) NULL,
     [WeekDay] [tinyint] NULL,
     [MonthNumber] [tinyint] NULL,
     CONSTRAINT [PK__ReportingDates__7B712C3B] PRIMARY KEY CLUSTERED
    (
     [DateKey] ASC
    )WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
    ) ON [PRIMARY]
    END
    GO
    ------------------------------------------------

        sql_DataLoad.sql
    ----------------------------------------------
    DECLARE @Date DATETIME
     SET @Date = '1/1/2000'    

     WHILE @Date < '1/1/2020'
     BEGIN
         INSERT INTO ReportingDates
         (
             DateKey, DateFull, FullYear,
             QuarterNumber, WeekNumber, WeekDayName,
             MonthDay, MonthName, YearDay,
             DateDefinition,
                    CharacterDate,
                    WeekDay,
                    MonthNumber
         )
         SELECT
             CONVERT(VARCHAR(8), @Date, 112), @Date, YEAR(@Date),
             DATEPART(qq, @Date), DATEPART(ww, @Date), DATENAME(dw, @Date),
             DATEPART(dd, @Date), DATENAME(mm, @Date), DATEPART(dy,@Date),
                   DATENAME(mm, @Date) + ' ' + CAST(DATEPART(dd, @Date) AS CHAR(2)) + ',  
               ' + CAST(DATEPART(yy, @Date) AS CHAR(4)),
               CONVERT(VARCHAR(10), @Date, 101),
               DATEPART(dw, @Date),
               DATEPART(mm, @Date)
       
         SET @Date = DATEADD(dd, 1, @Date)
     END

    GO

    -------------------------

    manifest.xml

    -------------------------------

    <MSDeploy.iisApp>
      <dbfullSql path="SqlScripts\sql_schema.sql" transacted="false" />
      <dbfullSql path="SqlScripts\sql_DataLoad.sql" transacted="false" />
    </MSDeploy.iisApp>

    ----------------------------

    Parameters.xml

    -----------------------------------

    <parameters>


      <parameter name="Database Server" description="Location of your database server (i.e., localhost, server name or IP address)." defaultValue="(local)" tags="SQL, dbServer"></parameter>
      <parameter name="Database Name" description="Name of the database for your application." defaultValue="sdk" tags="SQL, dbName">
        <parameterEntry type="TextFile" scope="msdeploy.config" match="DatabaseName" />
      </parameter>

      <parameter name="Database Username" description="Username to access your application database." defaultValue="myuser" tags="SQL, dbUsername">
        <parameterEntry type="TextFile" scope="msdeploy.config" match="DatabaseUserName" />
      </parameter>
     
      <parameter name="Database Password" description="Password for the database username." tags="New, Password, SQL, dbUserPassword">
        <parameterEntry type="TextFile" scope="msdeploy.config" match="DatabasePassword" />
      </parameter>

      <parameter name="Database Administrator" description="Administrator username for your database." defaultValue="sa" tags="SQL, dbAdminUsername"></parameter>
      <parameter name="Database Administrator Password" description="Password for the database administrator account." tags="Password, SQL, dbAdminUserPassword"></parameter>

      <parameter name="Connection String All 1" description="Automatically sets the connection string for the connection request." defaultValue="Connect Timeout=380;Data Source={Database Server};Database={Database Name};uid={Database Username};Pwd={Database Password};" tags="Hidden">
        <parameterEntry type="ProviderPath" scope="dbfullsql" match="sql_*"/>
     </parameter>

    </parameters>

    -----------------------

    I zip it as a test.zip and run it with a following command:

    "c:\Program Files\IIS\Microsoft Web Deploy\msdeploy.exe" -debug -dest:iisApp="Default Web Site/SDKSample" -verb:sync -source:package="test.zip" -setParam:name="Database Server",value="my-sqlserver" -setParam:name="Database Username",value=sa -setParam:name="Database Password",value=mcsqladmin -setParam:name="Database Administrator",value=sa -setParam:name="Database Administrator Password",value=password

    After that i get the following error:

    C:\*\ReadySamples>test

    C:\*\ReadySamples>"c:\Program Files\IIS\Microsoft Web Depl
    oy\msdeploy.exe" -debug -dest:iisApp="Default Web Site/SDKSample" -verb:sync -so
    urce:package="test.zip" -setParam:name="Database Server",value="my-sqlserver" -se
    tParam:name="Database Username",value=sa -setParam:name="Database Password",valu
    e=mcsqladmin -setParam:name="Database Administrator",value=sa -setParam:name="Da
    tabase Administrator Password",value=mcsqladmin
    Action: Adding MSDeploy.iisApp (MSDeploy.iisApp)
    Action: Adding child dbFullSql (SqlScripts\sql_schema.sql)
    Action: Adding child sqlScript (MSDeploy.iisApp/dbFullSql[@path='data source=my-
    sqlserver;initial catalog=sdk;user id=sa;connect timeout=380']/sqlScript)
    Action: Adding child dbFullSql (SqlScripts\sql_DataLoad.sql)
    Action: Adding child sqlScript (MSDeploy.iisApp/dbFullSql[@path='data source=my-
    sqlserver;initial catalog=sdk;user id=sa;connect timeout=380']/sqlScript)
    Microsoft.Web.Deployment.DeploymentException: An error occurred during execution
     of the database script. The approximate location of the error was between lines
     '1' and '30' of the script. The verbose log may have more information about the
     error. The command started with :
    DECLARE @Date DATETIME
     ---> System.Data.SqlClient.SqlException: Timeout expired.  The timeout period e
    lapsed prior to completion of the operation or the server is not responding.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolea
    n breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObj
    ect stateObj)
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cm
    dHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, Tds
    ParserStateObject stateObj)
       at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName,
    Boolean async)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult res
    ult, String methodName, Boolean sendToPipe)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Web.Deployment.DBStatementInfo.Execute(DbConnection connection,
    DbTransaction transaction, DeploymentBaseContext baseContext)
       --- End of inner exception stack trace ---
       at Microsoft.Web.Deployment.DBStatementInfo.Execute(DbConnection connection,
    DbTransaction transaction, DeploymentBaseContext baseContext)
       at Microsoft.Web.Deployment.DBConnectionWrapper.ExecuteSql(DBStatementInfo sq
    lStatement, DeploymentBaseContext baseContext)
       at Microsoft.Web.Deployment.ScriptToDBProvider.Add(DeploymentObject source, B
    oolean whatIf)
       at Microsoft.Web.Deployment.DeploymentObject.AddChild(DeploymentObject source
    , Int32 position, DeploymentSyncContext syncContext)
       at Microsoft.Web.Deployment.DeploymentSyncContext.HandleAddChild(DeploymentOb
    ject destParent, DeploymentObject sourceObject, Int32 position)
       at Microsoft.Web.Deployment.DeploymentSyncContext.SyncChildrenOrder(Deploymen
    tObject dest, DeploymentObject source)
       at Microsoft.Web.Deployment.DeploymentSyncContext.HandleAddChild(DeploymentOb
    ject destParent, DeploymentObject sourceObject, Int32 position)
       at Microsoft.Web.Deployment.DeploymentSyncContext.SyncChildrenOrder(Deploymen
    tObject dest, DeploymentObject source)
       at Microsoft.Web.Deployment.DeploymentSyncContext.ProcessSync(DeploymentObjec
    t destinationObject, DeploymentObject sourceObject)
       at Microsoft.Web.Deployment.DeploymentObject.SyncToInternal(DeploymentObject
    destObject, DeploymentSyncOptions syncOptions, PayloadTable payloadTable, Conten
    tRootTable contentRootTable)
       at Microsoft.Web.Deployment.DeploymentObject.SyncTo(DeploymentProviderOptions
     providerOptions, DeploymentBaseOptions baseOptions, DeploymentSyncOptions syncO
    ptions)
       at MSDeploy.MSDeploy.Execute()
       at MSDeploy.MSDeploy.Main(String[] unusedArgs)
    Error count: 1

     

    --------------------

    Hope this helps diagnosing the issue. Again, it will only produce error if SQL Server is under load, i believe if you encrease the date in the dataload script, you can get the same exception under IDLE SQL Server as well. One thing to keep in mind, is that the issue is not how long the script takes, but rather with an ability for msdeployer to handle long running SQL Scripts since that is something I can't control as a deployment package developer.

     Thanks.

  • 08-25-2009, 9:09 PM In reply to

    • yaminij
    • Top 50 Contributor
    • Joined on 09-25-2007, 12:41 AM
    • Posts 187

    Re: dbFullSql timeout issue

    These are great details. I have a repro now. Let me get back to you on this. Thanks a lot!

    Thanks
    Yamini Jagadeesan, SDET, Microsoft IIS Team
  • 08-26-2009, 1:52 AM In reply to

    • madjos
    • Not Ranked
    • Joined on 05-23-2008, 8:17 PM
    • Posts 11

    Re: dbFullSql timeout issue

    Hi Woland,

    The connection timeout feature of connection strings only applies to setting up an initial connection with SQL Server, not executing individual commands. In our next release (coming soon, maybe within a month) we will provide a commandTimeout setting that you can add in your manifest.xml entry file to increase the timeout.

    <dbfullsql path="SqlScripts\sql_DataLoad.sql" commandTimeout="60" />

    Unfortunately this setting does not work with our current bits. However, I would still recommend splitting up your loop into batches so that they can execute in a shorter time even under some load (within the 30 second default timeout that SQL Server uses). This will provide a better experience to users running the script as they will be able to see progress on the client side. The UI will be able to show progress as each batch executes. One way to do this would be to create a stored procedure that say inserts a years worth of data and call that 20 times separated by GO statements.

    On a related note: any reason you are not transacting your sql scripts? I cant see anything in them that requires non-transacted execution. By enabling transactions you will get a cleaner install experience. Otherwise you risk leaving the installation in a dirty state if the user cancels, or there is some other error.

    Hope that helps,

    madhur

  • 08-26-2009, 3:36 PM In reply to

    • Woland
    • Not Ranked
    • Joined on 04-07-2003, 5:38 PM
    • Los Angeles
    • Posts 12

    Re: dbFullSql timeout issue

    Madhur,

    thanks for the reply. This is what I figured and just hoped it would make it into the release before the final release.

    It might be a good idea to split that SQL into multiple files, but I would also feel much better if I can encrease default timeout so probability of that happening is very small.

    As for transacting SQL Scripts, I have about 20 of them. I'm pretty sure i can't execute all of them as part of the transaction. Also they are designed to remove all the data before executing.

    Thanks,

    hope next version is coming out soon.

Page 1 of 1 (6 items)
Microsoft Communities