MSGhazi

I can already use the WebClient.downloadfile method to download one file from a URL. What do I do if I want to download ALL the files at a URL, including subfolders Is there a nice clean way to do this I'm thinking I might have to somehow read all the filenames at the url, and fill a string array, then go through and download each file individually.

Any help is appreciated...

Mike


Re: .NET Framework Networking and Communication How to use a WebClient

Michael Nemtsev

Use HttpWebResponse to grab the page and parse its content, extracting the urls from there






Re: .NET Framework Networking and Communication How to use a WebClient

frederikm

Hi

if you're trying to scrape an entire site you do have to know all the files in the site.

Instead of downloading a file, try downloading a string, parsing that for:

- links

- images

- ....

afterwards, write the links and images and so on to a download queue

and write the string out to a file

Hope this helps you out






Re: .NET Framework Networking and Communication How to use a WebClient

timvw

If your intention is to experiment with programming (webrequest and pattern matching) then you already have an answer...

If you simply wanted to get some work done i would recommend that you simply spawn a System.Diagnostics.Process and use a win32 version of wget..





Re: .NET Framework Networking and Communication How to use a WebClient

MSGhazi

Basically I want to have a website where there are some image files stored. I want the program, on command to check the website and download any image files present. So I guess my answer is to maintain a text file on the site containing the urls to all the images present. The program will download the file, read it's contents then download all the files individually.




Re: .NET Framework Networking and Communication How to use a WebClient

Michael Nemtsev

Yep. Exactly




Re: .NET Framework Networking and Communication How to use a WebClient

MSGhazi

This may be a redundant question, but is there a way to get (as a string) a list of the files in a known directory on a website




Re: .NET Framework Networking and Communication How to use a WebClient

frederikm

hi

not unless you have browsing rights on the directory, then you could parse them from the html you get back..

otherwise, like in my post above, just go to one page, get out all the images, links and put them in download queues...

Hope this helps you out