File verification

Windows Forms General

Scotty2012

I am trying to verify that some files on a hard drive are also listed in a database, and that file names listed in a database are also on the hard drive. I want to know if there's a more efficient way than what I'm doing.

The files are divided randomly into some folders (so not to kill windows with millions( ) of files per folder).

The files are stored like this

Mainfolder
__12345
__23456
__34567

Each folder location is stored in the db with the filename. So, I've opened a connection to the DB and am using a SqlDataReader to get a list of the folder names in the main folder. Then, I am using a FileInfo[] to store the file names of the first subfolder in the SqlDataReader.

To verify both ways, I am looping through the FileInfo[] to get the filename(if i have a file name, I'm assuming the file exists, duh). Each iteration of the loop then opens an additional connection and datareader to check if the current filename exists in the db. Then closes the SqlConnection and the SqlDataReader in the loop before the next iteration.

it just seems like I'm opening and closing a lot of connections, is that a problem
Top

Re: Windows Forms General File verification

VS_Programmer

I would use the File.Exists(filepath) to check if the file exists. (returns true or false)

Hope this helps


Top

Re: Windows Forms General File verification

Derek Smyth

Hi,

What I would do is load all the data from the database into a data table. This will get all the information you need in one local place. A little hit to get the data, depending on number of records, but it could be a smaller hit than constant communication with the database. It's also fairly easy to code using the data adapter.

I'd then loop over each row in the table using VS_Programmers suggestion of File.Exists(), simply build the paths and do the check. This covers file verification from the database to the folder. If you need to you can update the records in the data table with files that are verified.

Going the other way checking files against the database I would use recursion. Recursion is always a good approach to use when parsing over a file/folder tree structure. Also any application that uses recursion is cool by default.

Code Snippet

static void Main(string[] args)
{

Program prg = new Program();
prg.CheckFile(Directory.GetCurrentDirectory());
}

public void CheckFile(string root)
{
DirectoryInfo dirInf = new DirectoryInfo(root);


FileInfo[] sgfFiles = dirInf.GetFiles("*.*");

foreach (FileInfo sgfFile in sgfFiles)
{
//check your datatable for file
}


DirectoryInfo[] subDirs = dirInf.GetDirectories();
foreach (DirectoryInfo subDir in subDirs)
{
CheckFile(subDir.FullName);
}
}

The above example demonstrates a recursive method. Basically the method calls itself passing in a directory who's files are checked before calling the same method again with a new directory. You would check that the file was in the data table where indicated.

Once your finished checking your files I'd then use the data adapter again to update your database. Connection only opens twice and your application processing will be faster because it's working from data in memory. That I feel is a better approach but there are many more other ways you can do it, so my recommendation might not be suitable based on some requirements of your project but that is how I would approach it, get the data from database, parse the data to files, parse the files to data, modify data as appropriate, and then update the database with the modified data.

Hope that helps.





Top

Re: Windows Forms General File verification

Scotty2012

Thanks for your response. I am using recursion to check the physical files, but for each file found, it opened a new dataReader, checked the DB, then closed the dataReader. So, it ran for a week and checked about 700,000 files both ways. I'll try the data table and see how that works out.


Top

Re: Windows Forms General File verification

Derek Smyth

Hi again,

Tell me does your application run as a single one off check or does it run as a service does it use the FileSystemWatcher

If you run as a single one off check then the datatable would be better. If it runs as a service or reacts to a new file being placed into a folder then the datareader would be more suitable. Without knowing all the requirements of the software or how the software runs it's difficult to suggest an idea, maybe one day I'll remember that. I assumed it would be a one off check, a user clicking a button to perform the check.

So tell me a bit more about how the software works and I'll resuggest if my first post isn't a good idea. What do you mean it ran for a week





Top

Re: Windows Forms General File verification

Guang-Ming Bian - MSFT

hi Scotty2012

I think opening and closing a lot of connections is not a good idea. you can set connection as a global variable. Before you looping the FileInfo[], you open the connection, Then after looping all the FileInfo[], then close the connection.





Top