JRMcFetridge

was hoping to use Streams as way around the problem of not knowing the size of file contents being returned on a file transfer operation but still run into the old maxReceivedMessageSize problem . Of course the simplest answer is to set this to a large number but how large We have no idea how large some of these file might be and setting it to Int32.Max is not real good idea. So it seems that using Streams and sending and receiving the data in chunks is no adavantage especially given the extra coding effort to do the "chunking". I guess the only real application of Streams is in some media app where one can start viewing the view before it is totally sent.

If Streams is not the answer ,does any one know a method to handle unknown and maybe huge file sizes being returned by a service.

My client side app.config looks like and yes I know maxReceivedMessageSize is too small

<bindings>

<netTcpBinding>

<binding name="basicTCP" closeTimeout="00:01:00" openTimeout="00:01:00"

receiveTimeout="00:10:00" sendTimeout="00:01:00" transactionFlow="false"

transferMode="Streamed" transactionProtocol="OleTransactions"

hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxBufferPoolSize="524288"

maxBufferSize="65536" maxConnections="10" maxReceivedMessageSize="65536">

<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"

maxBytesPerRead="4096" maxNameTableCharCount="16384" />

<reliableSession ordered="true" inactivityTimeout="00:10:00"

enabled="false" />

<security mode="Transport">

<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />

<message clientCredentialType="Windows" />

</security>

</binding>

</netTcpBinding>

</bindings>

<client>

<endpoint address="net.tcp://localhost:8004/Utilities" binding="netTcpBinding"

bindingConfiguration="basicTCP" contract="LookForEDIFiles.Utilities.IUtilities"

name="basicTCP">

<identity>

<userPrincipalName value="JRMcFetridge@gainesville.infiniteenergy.com" />

</identity>

</endpoint>

</client>




Re: Windows Communication Foundation (Indigo) streaming and unknown file size

Christian Weyer

maxReceivedMessageSize is one knob to prevent several kinds of attacks, such as DOS.

I think if your client is talking to 'well known' services then you are safe to open up this value to the max.

This is what I have done in the past in several projects without any problem.

Streaming is very powerful, if it fits your scenario.




Re: Windows Communication Foundation (Indigo) streaming and unknown file size

Eugene Osovetsky

Take a look at http://msdn2.microsoft.com/en-us/library/ms733135.aspx

WCF was designed with this scenario in mind. In streaming mode (if you're sure that your code only streams, never buffers), it is safe to set the maximum message size to a very large value. The value itself is of course application-specific (e.g. if you are streaming from an untrusted source to a file on disk think how much disk space you would allow the incoming message to consume). The danger of course is if either your code or WCF inadvertently starts buffering instead of streaming. To prevent this in WCF code, two things need to happen: set the "maximum buffering" quota (separate from the "maximum message size" quota) to a reasonable number, and make sure to use safe values (e.g. the default values) for the XML Reader quotas (most of these quotas are intended specifically to protect against excessive buffering in streaming mode). Read the document I link to above for more details.






Re: Windows Communication Foundation (Indigo) streaming and unknown file size

JRMcFetridge

Thanks for the great info but I guess i am still a little in the fog as it seems to me that a lot of buffered i/o is occuring even in streaming mode. My server code (client is of course reversed) looks like:

//"chunk" the output to the outbound stream

while (( length = pgpReader.OutputStream.Read(buffer, 0,2048)) > 0)

outF1.Write(buffer, 0, 2048);

outF1.Seek(0, SeekOrigin.Begin);

So there really is NO advantage in this technique in regards to memory footprint as it will end up being buffered anyways. I might as well just pass the stream itself back and save myself some keystrokes.






Re: Windows Communication Foundation (Indigo) streaming and unknown file size

David Kreutz - MSFT

If by buffered, you mean stored somewhere, then yes, you have to use a buffer, usually a byte array for reading and writing raw streams. I don't believe passing the stream itself back will work either, i'm not sure if it will let you write to a recieved stream while it is still recieving data from the client but i could be wrong.

I'm not sure how you felt you could get around using memory though, you have to store the data at some point on each end, otherwise why send it As far as your original question goes, if you just want to send a large message, then yes setting a high maxMessageSize is your only option. If you can bound the message you can of course set it lower, but it really depends on your scenario.






Re: Windows Communication Foundation (Indigo) streaming and unknown file size

JRMcFetridge

I guess I have not explained my problem well,so will try again. My service can return a unknown length of file and yes I know that the client must eventually allocate a correpsponding large buffer. However with streaming I hoped to send the file in chunks and use a memory stream to handle the input on the client side. As each block arrives on the client it is written to the memory stream which expands to hold the contents. When the input stream is exhausted then I grab the buffer from the memory stream. My client only knows the file length at this point. I hoped the only constraint would be the buffer size that I was chunking with. e.g . 2048. I guess am wrong and will go back to just passing the entire buffer without streaming and set maxMessageSize to int32.max. Too bad!




Re: Windows Communication Foundation (Indigo) streaming and unknown file size

David Kreutz - MSFT

Ok, I think i understand what you are saying. You want to read from the incoming stream into a second stream, keeping track of how many bytes are read as you go so when you finally write the object you know how big to make your buffer. I don't see why that wouldn't work, you can do whatever you like with the buffer once you've read the incoming data from your channel. You'll still need to set maxMessageSize to a large value, however if you are concerned about that your other option would be to generate soem sort of custom logic and chunk your file at the application level and send a series of smaller messages, however this does get expensive as you haev overhead for each message. Another strategy would be to return the file as a String object. You can simply pass the String as your return parameter, and .NET would allocate the size of the String on the recieve size appropriately, you could then convert that back into whatever type you needed.