TanTS

Hi all,
I am trying to write a program that runs another console program as a sub process using the System.Diagnostic.Process. I need to redirect both the standard output and standard error to the main program. Problem I ran into was trying to display the text in the exact same order as they would when output to a console.

So assuming my console program is a simple loop like below:

for(int i=0; i<10; i++)
Console.WriteLine("Standard output:" + i );

for(int i=0; i<10; i++)
Console.Error.WriteLine("Standard error:"+i);

How do I get the main program to write "Standard output 0"...."Standard output 9" followed by "Standard error 0" ... "Standard error 9"

I am using the asynchronous read and write for the console program (i.e. using BeginOutputReadLine and BeginErrorReadLine) and I have hooked up eventhandlers to the ErrorDataReceived and OutputDataReceived events. I tried debugging using breakpoint and it seems the events are not triggered in the correct order. Occasionally I get one ErrorDataReceived event before all OutputDataReceived events have occured.

Hope someone can advice on this. Thanks in advance.

TanTS


Re: Visual C# General Maintaining message order from standard output and standard error

TaylorMichaelL

The output streams are buffered. There is no way to get the true sequential order of the items inserted into the streams. In fact it makes little sense as both streams can be written too at the same time. They are independent of each other. Therefore the best you can do is get the sequential output from each one as they arrive.

Generally this is not an issue though as almost all console apps use standard output for both output and error messages. The error stream is used by some apps but the messages are generally duplicates of the errors generated in the output stream.

Michael Taylor - 6/20/07

http://p3net.mvps.org





Re: Visual C# General Maintaining message order from standard output and standard error

IanG

Generally speaking you shouldn't need it, but sometimes you need to do this:

Code Snippet

Console.Out.Flush();

when you want to synchronise your output. Normally it's not necessary because the default output stream auto-flushes. But if you've supplied your own, that might not be the case.

However, it seems more likely that the real problem here is simply that you're using asynchronous APIs. If you kick off two asynchronous operations simultaneously there's no guarantee that you will receive notifications in the same order in which the underlying events actually occurred.

Even if the .NET Framework raises the events in the correct order, you're at the mercy of the OS scheduler. Suppose the source process generates some output on STDOUT. .NET will raise the OutputDataReceived event in your monitoring process. But suppose the OS decides to run the scheduler at that point - it might decide that the thread on which the OutputDataReceived event was raised has used up its quantum, so it'll put it to sleep, allowing some other thread to run. This could happen before you get to your breakpoint.

Now suppose your source process generates some output on STDERR. .NET will now raise your the ErrorDataReceived event in your monitoring process, and your event handler runs. Some point later the OS scheduler will run again, and will eventually decide to allow the thread it put to sleep earlier to run again, and your handler for the OutputDataReceived event gets to run.

Note that .NET raised the events in the correct order here - it raised OutputDataReceived before ErrorDataReceived. But your code ended up running in the wrong order simply because the OS scheduler happened to perform a context switch at just the wrong moment.

There's no way around this. It's a fundamental problem with trying to read simultaneously from two streams: no mechanism exists for ensuring that the order in which data entered the streams is preserved. Ordering within a single stream is preserved, but ordering across all the streams is not. If you care about ordering you absolutely have to use a single stream.

The same problem would exist in Win32 by the way. This isn't a .NET issue. It's more fundamental than that.

So, how can you achieve the results you desire One way is to ensure that STDERR and STDOUT are the same thing. If these are both going into the same stream, then you can guarantee ordering. But of course you then need some way to work out which message is which - they're all merged into a single stream. Alternatively, you could have your source program tag each message with a sequence number - that way you can detect when your reads have happened out of order, and correct for it. Or you could arrange for some sort of lock-step protocol - when the source process wants a sequence point maybe it could send some special message to say that, and then maybe wait on an Event object. The monitoring process can signal that event when it has received the sequence point, and this way you know you're in sync because the sending process was waiting for that signal before proceeding.