Allan Rees

Hi All

I am maintaining an application that loads xaml activities into workflows at runtime.

I issue i am trying to solve is that as the application runs, memory does not get released and over time the application throws a System out of memory exception and dies.

I have used the Ants 3 memory profiler to see what is either hogging memory or causing a proliferation of objects.

The two issues i see as problem areas are

1. I need to replace the activity names in the workflow files passed from the store with unique names to avoid name conflicts during runtime, this is becomming an expensive operation

private TextReader GetScrubbedContents(string xml)

{

XmlDocument document = new XmlDocument();

document.LoadXml(xml);

XmlNamespaceManager nsManager = new XmlNamespaceManager(document.NameTable);

nsManager.AddNamespace("x", "http://schemas.microsoft.com/winfx/2006/xaml");

XmlNodeList nodes = document.SelectNodes("//*[@x:Name]", nsManager);

foreach (XmlNode node in nodes)

{

node.Attributes["x:Name"].InnerText =

string.Format("Activity{0}", Guid.NewGuid().ToString().Replace("-", "_"));

}

TextReader textReader = new StringReader(document.OuterXml);

return textReader;

}

Is there a better way to do this

2. WorkflowRuntime instances are not getting unloaded from memory, this problem is raised by Rory Primrose http://neovolve.com/archive/2007/04/17/does-the-workflowruntime-have-a-memory-leak.aspx, and he solved it by wrapping the runtime as a singleton.

I have done this as well

public class WorkflowRuntimeWrapper : IWorkflowRuntime

{

private WorkflowRuntime runtime;

public WorkflowRuntimeWrapper()

{

runtime = new WorkflowRuntime();

runtime.ServicesExceptionNotHandled += new EventHandler<ServicesExceptionNotHandledEventArgs>(runtime_ServicesExceptionNotHandled);

runtime.Started += new EventHandler<WorkflowRuntimeEventArgs>(runtime_Started);

runtime.Stopped += new EventHandler<WorkflowRuntimeEventArgs>(runtime_Stopped);

runtime.WorkflowAborted += new EventHandler<WorkflowEventArgs>(runtime_WorkflowAborted);

runtime.WorkflowCompleted += new EventHandler<WorkflowCompletedEventArgs>(runtime_WorkflowCompleted);

runtime.WorkflowCreated += new EventHandler<WorkflowEventArgs>(runtime_WorkflowCreated);

runtime.WorkflowIdled += new EventHandler<WorkflowEventArgs>(runtime_WorkflowIdled);

runtime.WorkflowLoaded += new EventHandler<WorkflowEventArgs>(runtime_WorkflowLoaded);

runtime.WorkflowPersisted += new EventHandler<WorkflowEventArgs>(runtime_WorkflowPersisted);

runtime.WorkflowResumed += new EventHandler<WorkflowEventArgs>(runtime_WorkflowResumed);

runtime.WorkflowStarted += new EventHandler<WorkflowEventArgs>(runtime_WorkflowStarted);

runtime.WorkflowSuspended += new EventHandler<WorkflowSuspendedEventArgs>(runtime_WorkflowSuspended);

runtime.WorkflowTerminated += new EventHandler<WorkflowTerminatedEventArgs>(runtime_WorkflowTerminated);

runtime.WorkflowUnloaded += new EventHandler<WorkflowEventArgs>(runtime_WorkflowUnloaded);

}

and am using Spring.Net to create a singleton instance of the WorkflowRuntimeWrapper.

But my problems persist

Does any one have any comments, suggestions or ideas to help solve this

Regards

Allan



Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Kavita Kamani - MSFT

Hi Allan

Have you looked at this

http://msdn2.microsoft.com/en-us/library/Aa973808.aspx

Specifically, I was wondering if the following two issues are happening in your case -

start up and creation costs - how many instances of WorkflowRuntime are you using today If more than one, can you share them

are you using UnloadOnIdle to unload the workflow when it is idle rather than keeping it in memory

Kavita






Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Allan Rees

Thanks for responding Kavita,

It is a good article, and i had not read it.

But i do not think that the two issues you mention are to blame because

1. WorkflowRuntime is a singleton in the system so only one is used, which negates the creation cost issue

2. Yes I am using UnloadOnIdle with a SqlInstanceOwnership of 10 mins.

Thank you for your thoughts

Regards

Allan





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Kavita Kamani - MSFT

Ok makes sense. Were you able to resolve anything after the article, if not, I'll ask someone who might be more familiar with this area to help you out.




Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Allan Rees

Thanks, if you do know anyone please put me in contact with them.

Cheers

Allan





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Kavita Kamani - MSFT

Is it possible for you to post more code - one of developers was trying something out for you and said he could see the memory leak only when the runtime wasnt cached




Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Allan Rees

Ok here we go

This is the class that starts it all off.

/// <summary>

/// Starts the global redemption workflow instance

/// </summary>

/// <param name="promoId">The promotion id</param>

/// <param name="accountNumber">The account number claiming</param>

/// <param name="parameters">The parameters for the workflow</param>

/// <param name="isTest">Is the workflow a test</param>

public PromoRedemptionResponse ProcessRedeemFlow(int promoId, string accountNumber, int casinoId, Dictionary<string, object> parameters,

bool isTest)

{

try

{

// Set the custom workflow parameters

Dictionary<string, object> typedParameters = new Dictionary<string, object>();

typedParameters.Add("AccountNR", accountNumber);

typedParameters.Add("CasinoId", casinoId);

typedParameters.Add("PromoId", promoId);

typedParameters.Add("ContextVariables", parameters);

log.DebugFormat("Creating {0} workflow instance", "RedemptionFlow");

IWorkflowInstance instance = runtime.CreateWorkflow(typeof(RedemptionFlow), typedParameters);

log.DebugFormat(

"Making the workflow aware that it is changing and then adding the {0}, {1} and {2} plugins ",

"Eligibility", "Calculation", "Redemption");

WorkflowChanges wfChanges = new WorkflowChanges(instance.GetWorkflowDefinition());

List<WorkflowFile> workflowFiles =

workflowFileHandler.GetPromoWorkflowFiles(promoId, !isTest);

// Get the custom eligibility, calculation and redemption workflows for the promotion

SequenceActivity inclusionActivity =

activityLoader.GetPluginActivity(ActivityType.Eligibility, workflowFiles);

SequenceActivity calculationActivity =

activityLoader.GetPluginActivity(ActivityType.Calculation, workflowFiles);

SequenceActivity redemptionActivity =

activityLoader.GetPluginActivity(ActivityType.Redemption, workflowFiles);

activityReplacer.ReplaceActivity(wfChanges, "InclusionPlaceHolder", inclusionActivity);

activityReplacer.ReplaceActivity(wfChanges, "CalculationPlaceHolder", calculationActivity);

activityReplacer.ReplaceActivity(wfChanges, "RedemptionPlaceHolder", redemptionActivity);

// Apply runtime changes and start the workflow

instance.ApplyWorkflowChanges(wfChanges);

log.DebugFormat("Starting workflow instance {0}", instance.InstanceId);

instance.Start();

//TODO: make the timeout configurable through the constructor

return responseStore.SubscribeAndWaitForMessage<PromoRedemptionResponse>(instance, 60000);

}

catch (Exception ex)

{

log.Error(123076,

string.Format("An error occurred when trying to start the redemption flow. Error: {0}", ex.Message),

ex);

}

return null;

}

this class loads the workflow that comes from the databases into the plugin

using System;

using System.Collections.Generic;

using System.IO;

using System.Workflow.Activities;

using System.Workflow.ComponentModel.Compiler;

using System.Workflow.Runtime;

using System.Xml;

using log4net.Ext.EventID;

using Osiris.PromoManager.Contracts;

namespace Osiris.PromoManager.Workflow

{

//TODO: The activity loader should cache the xaml - must also invalidate it on a change

/// <summary>

/// The plugin activity loader is responsible for loading the required workflow type from

/// the xaml workflows for the specific promotion or casino

/// </summary>

[Serializable]

[CLSCompliant(false)]

public class PluginActivityLoader : IPluginActivityLoader

{

private static readonly IEventIDLog log = EventIDLogManager.GetLogger(typeof (PluginActivityLoader));

private IWorkflowRuntime workflowRuntime;

/// <summary>

/// The default constructor

/// </summary>

/// <param name="workflowRuntime">The workflow runtime</param>

public PluginActivityLoader(IWorkflowRuntime workflowRuntime)

{

this.workflowRuntime = workflowRuntime;

}

/// <summary>

/// Gets the plugin activity from a list of workflow files for a promotion

/// </summary>

/// <param name="activityType">The current activity type</param>

/// <param name="workflowFiles">A list of workflows that contain the current activity type</param>

/// <returns>The current activity type workflow activity</returns>

public SequenceActivity GetPluginActivity(ActivityType activityType, List<WorkflowFile> workflowFiles)

{

try

{

log.DebugFormat("Attempting to load {0} plugin flow", activityType);

// Loop through the workflow files and get the file for the current step, either Eligibility, Calculation or Redemption

WorkflowFile workflowFile = null;

foreach (WorkflowFile file in workflowFiles)

{

if (file != null)

{

if (file.StepName == activityType.ToString())

{

workflowFile = file;

break;

}

}

else

{

throw new NullReferenceException("The workflow file is null");

}

}

if (workflowFile == null)

{

string error = string.Format("The workflowFiles parameter did not contain a workflow file for step {0}",

activityType.ToString());

log.Error(123072, error);

throw new ArgumentException(error, "workflowFiles");

}

// Add referenced assemblies to the TypeProvider

// TypeProvider typeProvider = new TypeProvider(workflowRuntime);

// typeProvider.AddAssembly(typeof(CustomActivity).Assembly);

// workflowRuntime.AddService(typeProvider);

// Creating workflow from XAML file

using (XmlReader workflowReader = XmlReader.Create(GetScrubbedContents(workflowFile.FileContent)))

{

IWorkflowInstance workflowInstance = workflowRuntime.CreateWorkflow(workflowReader);

SequenceActivity plugin = workflowInstance.GetWorkflowDefinition() as SequenceActivity;

workflowInstance.Terminate();

return plugin;

}

}

catch (Exception ex)

{

log.Error(123073, "An error occurred when getting the plugin activity", ex);

throw;

}

}

/// <summary>

/// Takes the xml passed, and replaces the activity names with unique names to avoid

/// name conflicts during runtime

/// </summary>

/// <param name="xml">The xml to examine and change</param>

/// <returns>The clean xml contents</returns>

private TextReader GetScrubbedContents(string xml)

{

XmlDocument document = new XmlDocument();

document.LoadXml(xml);

XmlNamespaceManager nsManager = new XmlNamespaceManager(document.NameTable);

nsManager.AddNamespace("x", "http://schemas.microsoft.com/winfx/2006/xaml");

XmlNodeList nodes = document.SelectNodes("//*[@x:Name]", nsManager);

foreach (XmlNode node in nodes)

{

node.Attributes["x:Name"].InnerText =

string.Format("Activity{0}", Guid.NewGuid().ToString().Replace("-", "_"));

}

TextReader textReader = new StringReader(document.OuterXml);

return textReader;

}

}

}





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Allan Rees

this is the runtime wrapper

using System;

using System.Collections.Generic;

using System.Collections.ObjectModel;

using System.Workflow.Runtime;

using System.Xml;

namespace Osiris.PromoManager.Workflow

{

public class WorkflowRuntimeWrapper : IWorkflowRuntime

{

private WorkflowRuntime runtime;

private WorkflowInstance instance;

public WorkflowRuntimeWrapper()

{

runtime = new WorkflowRuntime();

runtime.ServicesExceptionNotHandled += new EventHandler<ServicesExceptionNotHandledEventArgs>(runtime_ServicesExceptionNotHandled);

runtime.Started += new EventHandler<WorkflowRuntimeEventArgs>(runtime_Started);

runtime.Stopped += new EventHandler<WorkflowRuntimeEventArgs>(runtime_Stopped);

runtime.WorkflowAborted += new EventHandler<WorkflowEventArgs>(runtime_WorkflowAborted);

runtime.WorkflowCompleted += new EventHandler<WorkflowCompletedEventArgs>(runtime_WorkflowCompleted);

runtime.WorkflowCreated += new EventHandler<WorkflowEventArgs>(runtime_WorkflowCreated);

runtime.WorkflowIdled += new EventHandler<WorkflowEventArgs>(runtime_WorkflowIdled);

runtime.WorkflowLoaded += new EventHandler<WorkflowEventArgs>(runtime_WorkflowLoaded);

runtime.WorkflowPersisted += new EventHandler<WorkflowEventArgs>(runtime_WorkflowPersisted);

runtime.WorkflowResumed += new EventHandler<WorkflowEventArgs>(runtime_WorkflowResumed);

runtime.WorkflowStarted += new EventHandler<WorkflowEventArgs>(runtime_WorkflowStarted);

runtime.WorkflowSuspended += new EventHandler<WorkflowSuspendedEventArgs>(runtime_WorkflowSuspended);

runtime.WorkflowTerminated += new EventHandler<WorkflowTerminatedEventArgs>(runtime_WorkflowTerminated);

runtime.WorkflowUnloaded += new EventHandler<WorkflowEventArgs>(runtime_WorkflowUnloaded);

}

public event EventHandler<WorkflowRuntimeEventArgs> Started;

public event EventHandler<WorkflowRuntimeEventArgs> Stopped;

public event EventHandler<WorkflowEventArgs> WorkflowAborted;

public event EventHandler<WorkflowCompletedEventArgs> WorkflowCompleted;

public event EventHandler<WorkflowEventArgs> WorkflowCreated;

public event EventHandler<WorkflowEventArgs> WorkflowIdled;

public event EventHandler<WorkflowEventArgs> WorkflowLoaded;

public event EventHandler<WorkflowEventArgs> WorkflowPersisted;

public event EventHandler<WorkflowEventArgs> WorkflowResumed;

public event EventHandler<WorkflowEventArgs> WorkflowStarted;

public event EventHandler<WorkflowSuspendedEventArgs> WorkflowSuspended;

public event EventHandler<WorkflowTerminatedEventArgs> WorkflowTerminated;

public event EventHandler<WorkflowEventArgs> WorkflowUnloaded;

public event EventHandler<ServicesExceptionNotHandledEventArgs> ServicesExceptionNotHandled;

public IWorkflowInstance CreateWorkflow(Type workflowType, Dictionary<string, object> namedArgumentValues)

{

instance = runtime.CreateWorkflow(workflowType, namedArgumentValues);

return new WorkflowInstanceWrapper(instance);

}

public IWorkflowInstance CreateWorkflow(XmlReader workflow)

{

WorkflowInstance instance = runtime.CreateWorkflow(workflow);

return new WorkflowInstanceWrapper(instance);

}

public void StartRuntime()

{

runtime.StartRuntime();

}

public void AddService(object service)

{

runtime.AddService(service);

}

public ReadOnlyCollection<WorkflowInstance> GetLoadedWorkflows()

{

return runtime.GetLoadedWorkflows();

}

private void runtime_WorkflowUnloaded(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowUnloaded;

if (handler != null)

{

WorkflowUnloaded(sender, e);

}

}

private void runtime_WorkflowTerminated(object sender, WorkflowTerminatedEventArgs e)

{

EventHandler<WorkflowTerminatedEventArgs> handler = WorkflowTerminated;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowSuspended(object sender, WorkflowSuspendedEventArgs e)

{

EventHandler<WorkflowSuspendedEventArgs> handler = WorkflowSuspended;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowStarted(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowStarted;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowResumed(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowResumed;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowPersisted(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowPersisted;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowLoaded(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowLoaded;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowIdled(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowIdled;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowCreated(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowCreated;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowCompleted(object sender, WorkflowCompletedEventArgs e)

{

EventHandler<WorkflowCompletedEventArgs> handler = WorkflowCompleted;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_WorkflowAborted(object sender, WorkflowEventArgs e)

{

EventHandler<WorkflowEventArgs> handler = WorkflowAborted;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_Stopped(object sender, WorkflowRuntimeEventArgs e)

{

EventHandler<WorkflowRuntimeEventArgs> handler = Stopped;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_Started(object sender, WorkflowRuntimeEventArgs e)

{

EventHandler<WorkflowRuntimeEventArgs> handler = Started;

if (handler != null)

{

handler(sender, e);

}

}

private void runtime_ServicesExceptionNotHandled(object sender, ServicesExceptionNotHandledEventArgs e)

{

EventHandler<ServicesExceptionNotHandledEventArgs> handler = ServicesExceptionNotHandled;

if (handler != null)

{

handler(sender, e);

}

}

public void StopRunTime()

{

runtime.StopRuntime();

}

}

}

this is the instance wrapper

using System;

using System.Workflow.ComponentModel;

using System.Workflow.Runtime;

namespace Osiris.PromoManager.Workflow

{

/// <summary>

/// Decorates a WorkflowInstance

/// </summary>

public class WorkflowInstanceWrapper : IWorkflowInstance

{

private WorkflowInstance instance;

/// <summary>

/// Constructs this decorator with the instance it decorates

/// </summary>

/// <param name="instance">The instance it decorates</param>

public WorkflowInstanceWrapper(WorkflowInstance instance)

{

this.instance = instance;

}

/// <summary>

/// Gets the Unique Identifier for the workflow instance

/// </summary>

public Guid InstanceId

{

get { return instance.InstanceId; }

}

/// <summary>

/// Apply changes to the workflow instance

/// </summary>

/// <param name="wfChanges">The changes to apply to the flow</param>

public void ApplyWorkflowChanges(WorkflowChanges wfChanges)

{

instance.ApplyWorkflowChanges(wfChanges);

}

/// <summary>

/// Gets the workflow definition for the instance

/// </summary>

/// <returns>The definition of the instance</returns>

public Activity GetWorkflowDefinition()

{

return instance.GetWorkflowDefinition();

}

/// <summary>

/// Starts the instance

/// </summary>

public void Start()

{

instance.Start();

}

/// <summary>

/// Aborts the workflow instance

/// </summary>

public void Abort()

{

instance.Abort();

}

/// <summary>

/// Terminates the instance

/// </summary>

public void Terminate()

{

instance.Terminate(null);

}

}

}

Please shout if you need any thing else

Thanks again for yuour assistance

Allan





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Shelly Guo - MSFT

Dear Customer,

We have identified the memory leak of WorkflowRuntime as a bug in our V1 product. Using a singleton WorkflowRuntime like you do is your only mitigation right now. We are going to fix the issue in the next release of WF.

For you second question, changing the activity name is going to be an expensive operation that cannot be avoided. The way you did it using XmlDocument is slow not only because processing using XmlDocument is expensive, but also that by changing the XML you passed to WorkflowRuntime to call CreateWorkflow, you cannot take advantage of our caching mechanism to speed up the creation of the workflow. We cache the workflow definition based on a value calculated using the XML text.

The general rule is that you should not change ¡°meta property¡± values for each instance of a workflow. So my suggestion is try not to use the activity name as the workflow instance identifier, instead, add a new property to you workflow as the instance identifier, make this property an ¡°instance property¡± instead of ¡°meta property¡±. You can then assign value to this property by passing it as a parameter when you call CreateWorkflow.

Shelly





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Allan Rees

Thank you all for your attentive response

Regards

Allan





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Anonymous

Hi !

"next release of WF" = Visual studio 2008 release (Fwk 3.5)

Thanks.





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Anonymous

Hi

And is it possible to use wf 3.5 classes in an asp.net 3.0 application/VS2005 solution





Re: Windows Workflow Foundation WorkflowRuntime Memory Leak Issue

Wenlong Dong - MSFT

Yes, it is. WF3.5 is backward compatible with WF3.0 and thus works seemlessly with existing applications including VS2005 and .NET3.0 applications. There is no "asp.net 3.0", . I guess you are talking about .NET 3.0.