Anarchy

I am using the FileSystemWatcher to trap when a 3rd party system creates, then renames a file, and I then process it. It is likely to happen that the 3rd party system (which runs every 1 min) could create numerous files (which it creates as a .tmp, then renames to a .req) very quickly

So, I have created a test app which creates files, and another test app which catches the renamed event, and deletes the file. The problem is that if I create 500 files, I do not get 500 events. What happens is that I'll get events up to about 80-90, skip lots (100-300) then get another sequence, skip a few, etc.... If I put a delay in the app creating the files, It tends to work, but not always.

Why am I not getting ALL of the events for the files being renamed

Here is the code to create the files:

Code Snippet

Imports System.IO

Public Class Form1


Private Sub CopyFiles()

Dim Text As String = String.Empty
Dim tmpFileName As String = String.Empty
Dim tmpExt As String = ".tmp"
Dim reqExt As String = ".req"
Dim ExtEx As String = "C:\WebInterface\Manpack3\EXPORTSExternal\"

For nI As Integer = 1 To CInt(Me.NumericUpDown1.Value) ' 200

Text = File.ReadAllText("C:\WebInterface\Manpack3\EXPORTSExternal\~\00002890_0703141306.req") <--

tmpFileName = nI.ToString("0000")

If File.Exists(ExtEx & tmpFileName & tmpExt) Then
File.Delete(ExtEx & tmpFileName & tmpExt)
End If
File.WriteAllText(ExtEx & tmpFileName & tmpExt, Text)
'MAS.IO.FileIO.WriteFile(ExtEx & tmpFileName & tmpExt, Text)

If File.Exists(ExtEx & tmpFileName & reqExt) Then
File.Delete(ExtEx & tmpFileName & reqExt)
End If
File.Move(ExtEx & tmpFileName & tmpExt, ExtEx & tmpFileName & reqExt)

If CInt(Me.NumericUpDown2.Value) > 0 Then
System.Threading.Thread.Sleep(CInt(Me.NumericUpDown2.Value))
End If

Next


End Sub


Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Me.Enabled = False
CopyFiles()
Me.Enabled = True
End Sub
End Class

You will need to point ExtEx to a folder on your PC (not network, effects are not as bad!) which is where the FSW points to, and change the Text = File.ReadAllText(... line to point to any valid xml file, which is used as a seed

You will need a form with a Button1, a NumericUpDown1 to set the # of files to create, and a NumericUpDown2 to set the sleep between files (in milliseconds)

Here is the code with the FileSystemWatcher: The _fsw.Path needs to match the ExtEx above

Code Snippet

Imports System.IO
Imports System.Threading

Public Class Form1

Private WithEvents _fsw As New System.IO.FileSystemWatcher

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

Debug.WriteLine("~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ " & Now & " ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~")
With _fsw
.Filter = "*.req"
.Path = "C:\WebInterface\Manpack3\EXPORTSExternal"
.NotifyFilter = NotifyFilters.FileName
_fsw.EnableRaisingEvents = True
End With

End Sub

Private Sub _fsw_Renamed(ByVal sender As Object, ByVal e As System.IO.RenamedEventArgs) Handles _fsw.Renamed
Debug.WriteLine("_fsw_Renamed:" & e.FullPath)
TestOpenFile(e.FullPath)
'Dim T As Thread = Nothing

'T = New Thread(New ParameterizedThreadStart(AddressOf TestOpenFile))
'T.IsBackground = True
'T.Start(e.FullPath)
End Sub


Private Sub TestOpenFile(ByVal obj As Object) 'ByVal FileName As String)

Dim FileName As String = TryCast(obj, String)
Dim DelFile As Boolean = True

Dim xDoc As New Xml.XmlDocument

Try
xDoc.Load(FileName)

Catch ex As Exception
Debug.Write("!!!OpenEx:" & ex.Message)
DelFile = False

End Try

If DelFile Then
Try
File.Delete(FileName)
Debug.WriteLine("~deleted OK!")

Catch ex As Exception
Debug.Write("!!!DelEx:" & ex.Message)

End Try
End If

End Sub

End Class

and here is the output from the immediate window...

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 24/09/2007 15:36:57 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0001.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0002.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0003.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0004.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0005.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0006.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0007.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0008.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0009.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0010.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0011.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0012.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0013.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0014.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0015.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0016.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0017.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0018.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0019.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0020.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0021.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0022.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0023.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0024.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0025.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0026.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0027.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0028.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0029.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0030.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0031.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0032.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0033.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0034.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0035.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0036.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0037.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0038.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0039.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0040.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0041.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0042.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0043.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0044.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0045.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0046.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0047.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0048.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0049.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0050.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0051.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0052.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0053.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0054.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0055.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0056.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0057.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0058.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0059.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0060.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0061.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0062.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0063.req <-- jumps from 63-284 here

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0284.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0285.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0286.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0287.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0288.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0289.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0290.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0291.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0292.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0293.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0294.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0295.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0296.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0297.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0298.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0299.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0300.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0301.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0302.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0303.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0304.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0305.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0306.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0307.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0308.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0309.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0310.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0311.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0312.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0313.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0314.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0315.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0316.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0317.req <-- jumps from 317-479 here

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0479.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0480.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0481.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0482.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0483.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0484.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0485.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0486.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0487.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0488.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0489.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0490.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0491.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0492.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0493.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0494.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0495.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0496.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0497.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0498.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0499.req

_fsw_Renamed:C:\WebInterface\Manpack3\EXPORTSExternal\0500.req

What gives



Re: Visual Basic General Not getting all renamed events from FileSystemWatcher

Anarchy

*KoffBump*





Re: Visual Basic General Not getting all renamed events from FileSystemWatcher

Anarchy

btw, when I do get a Renamed event, is this on a separate thread for each file





Re: Visual Basic General Not getting all renamed events from FileSystemWatcher

Anarchy

anyone





Re: Visual Basic General Not getting all renamed events from FileSystemWatcher

Anarchy

so all the big brains in here, and no-one can tell me if the FSW actually raises renamed events for every file being renamed

I've had problems with FSW before with the created event http://forums.microsoft.com/MSDN/ShowPost.aspx PostID=602148&SiteID=1, and spent days on a workaround, I've now had to do the same with the renamed event, and my confidence is through the floor.

FSW is deficient, and not fit for purpose. It doesn't do what it should do, and is too unreliable to be of any use in a production system.

Microsoft should be ashamed (or sued) for releasing software like this, and you now owe me about 6 days of my life I've wasted on this.





Re: Visual Basic General Not getting all renamed events from FileSystemWatcher

Anarchy

2nd bump of many





Re: Visual Basic General Not getting all renamed events from FileSystemWatcher

DMan1

http://msdn2.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx

Note that a FileSystemWatcher does not raise an Error event when an event is missed or when the buffer size is exceeded, due to dependencies with the Windows operating system. To keep from missing events, follow these guidelines:

  • Increasing the buffer size with the InternalBufferSize property can prevent missing file system change events.

  • Avoid watching files with long file names. Consider renaming using shorter names.

  • Keep your event handling code as short as possible.






Re: Visual Basic General Not getting all renamed events from FileSystemWatcher

bill_csharper

Quite agree that FSW has a very flaky implementation. The worst thing to me is that the (very complicated) logic behind it is poorly documented. They (MSFT) should provide a full list of all possible combinations of actions/settings etc and document exactly what happens in each scenario. Yet, as always, the guys who wrote the online docs just dumbed things down to make it look simple & easy, and those MVPs just kept making lame excuses or "guesses" without providing any real solutions.