seth.king

I've got an ASP.NET application that uses a state machine workflow to track the lifecycle of an entity. In the development region (XP SP2 and IIS 5.1) the workflow works fine. When I try to run the same code on our test server (Server 2003 and IIS 6.0) the application hangs when trying to execute the workflow.

The trace log is set as follows:

<system.diagnostics>
<switches>
<add name="System.Workflow LogToTraceListeners" value="1" />
<add name="System.Workflow.Runtime" value="All" />
<add name="System.Workflow.Runtime.Hosting" value="All" />
<add name="System.Workflow.Runtime.Tracking" value="All" />
<add name="System.Workflow.Activities" value="All" />
<add name="System.Workflow.Activities.Rules" value="All" />
</switches>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="customListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="WFTrace.log" />
</listeners>
</trace>
</system.diagnostics>

In the trace log on my development box, the debug statements and trace statements show the following (just a small section):

TSYS TVI: 8/16/2007 6:20:41 PM DEBUG: TDFTransactionService.Commit(Transaction, ICollection): Entering Method. || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:46 PM DEBUG: TDFTransactionService.Commit(Transaction, ICollection): Calling TechnicalDesignFormDAL.CreateNewTDF() || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:46 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Entering method. || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:46 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): userID = TSYS\SKING || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:47 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Executing the stored procedure PKG_MIG_TDF.Create_New_TDF || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:48 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): inserted the tdf. technicalDesignFormID 859689, lastChangedTimeStamp 8/16/2007 2:20:47 PM, openDate 8/16/2007 2:20:47 PM || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:48 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Closing the data reader || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:48 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Exiting method. returning TSYS.TVI.Entities.Migration.TechnicalDesignFormBE || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 6:20:48 PM DEBUG: TDFTransactionService.Commit(Transaction, ICollection): Exiting Method. || Priority: 0 || Severity: Verbose

System.Workflow.Runtime.Hosting Information: 0 : SqlWorkflowPersistenceService(00000000-0000-0000-0000-000000000000): inserting instance: 5fe0189b-5dbd-443b-b418-fa23ab27f7ca, unlocking: False database: tvi_wf_test

System.Workflow.Runtime.Hosting Information: 0 : Received OnIdle Event for instance, 5fe0189b-5dbd-443b-b418-fa23ab27f7ca

TSYS TVI: 8/16/2007 6:20:51 PM DEBUG: WorkflowMediator._workflowRuntime_WorkflowIdled: Workflow 5fe0189b-5dbd-443b-b418-fa23ab27f7ca idled. || Priority: 0 || Severity: Verbose

System.Workflow.Runtime.Hosting Information: 0 : WorkflowRuntimeTongue TiedchedulePersisted event raised for instance Id 5fe0189b-5dbd-443b-b418-fa23ab27f7ca

TSYS TVI: 8/16/2007 6:20:51 PM DEBUG: WorkflowMediator._workflowRuntime_WorkflowPersisted: Workflow 5fe0189b-5dbd-443b-b418-fa23ab27f7ca persisted. || Priority: 0 || Severity: Verbose

This shows that my transaction service (TDFTransactionService) is called by the WorkflowEnvironment.WorkBatch and that the transaction service calls to the data layer (TechnicalDesignFormDAL). The GREEN line shows that a new record was inserted successfully into the database and an ID was generated properly. The WorkflowMediator subscribes to all public events of the WorkflowRuntime and the debug statements are written when the event is raised. You can see that after the "False database" line, the workflowIdled event fires and then the workflowPersisted event fires, as it should.

In the trace log on the test server, the last few lines after executing the workflow are:

TSYS TVI: 8/16/2007 4:08:05 PM DEBUG: TDFTransactionService.Commit(Transaction, ICollection): Entering Method. || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TDFTransactionService.Commit(Transaction, ICollection): Calling TechnicalDesignFormDAL.CreateNewTDF() || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Entering method. || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): userID = TSYS\SKING || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Executing the stored procedure PKG_MIG_TDF.Create_New_TDF || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): inserted the tdf. technicalDesignFormID 859687, lastChangedTimeStamp 8/16/2007 12:08:10 PM, openDate 8/16/2007 12:08:10 PM || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Closing the data reader || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TechnicalDesignFormDAL.CreateNewTDF(): Exiting method. returning TSYS.TVI.Entities.Migration.TechnicalDesignFormBE || Priority: 0 || Severity: Verbose

TSYS TVI: 8/16/2007 4:08:10 PM DEBUG: TDFTransactionService.Commit(Transaction, ICollection): Exiting Method. || Priority: 0 || Severity: Verbose

System.Workflow.Runtime.Hosting Information: 0 : SqlWorkflowPersistenceService(00000000-0000-0000-0000-000000000000): inserting instance: e790cc57-fa89-4766-aa20-c50f861d90ba, unlocking: False database: tvi_wf_test

Again, this shows me that the transaction service is being called correctly by the WorkflowEnvironment.WorkBatch and that it's then calling to the data layer. The GREEN line again shows that the record is inserted into the database correcly and an ID is generated so we know that is not the problem. The last line is the one that is hinting to me that persistence is causing my symptoms. It is after this that the following event is logged in the event view. Notice how the time on the error is 9 seconds after the last debug statement is written (db time is four hours ahead of the server time)

Event Type: Error
Event Source: .NET Runtime 2.0 Error Reporting
Event Category: None
Event ID: 1000
Date: 8/16/2007
Time: 12:08:19 PM
User: N/A
Computer: RFVMDEV2
Description:
Faulting application w3wp.exe, version 6.0.3790.1830, stamp 42435be1, faulting module orageneric10.dll, version 10.2.0.1, stamp 4313c2da, debug 0, fault address 0x001dca8e.

The only difference between the test region and development is the server and IIS version. My code on development is hitting the same application db and workflow db.

Is there something that I'm missing in the configuration of the server that would cause this issue Does anyone know how to resolve it

Thank you,

S.K.



Re: Windows Workflow Foundation WF on Windows Server 2003 not persisting

seth.king

In looking at the error description in the event log more closely, it looks like the problem is in the orageneric10.dll. Yes, I am hitting an Oracle 10g database. I'm going to try to reinstall the oracle client on the box and see if that makes any difference. Has any one seen this issue with orageneric10.dll I ran DebugDiag on w3wp.exe and got a bit more insight into the issue.

the assembly instruction at orageneric10!kpummmemst+1c2 in E:\oracle\product\10.2.0\client_1\BIN\orageneric10.dll from Oracle Corporation has caused an access violation exception (0xC0000005) when trying to read from memory location 0x00000000 on thread 42.

From my understanding, 0x00000000 = null, right So does this mean that MSDTC and oracle client aren't cooperating somehow

Thanks,


S.K.





Re: Windows Workflow Foundation WF on Windows Server 2003 not persisting

seth.king

I've found the solution to this issue. With the amount of pain and agony that I had to go through, I'm a bit sad to say that the fix was simply to reinstall my Oracle Client on the app server. Apparently the client became corrupted and reinstalling fixed this issue.

Hope that helps anyone else facing similar issues.

Cheers!

Seth