Mark Shen

Hi all,

I have a situation here. I need to read a large log file, in general will be more than 100MB, and with more than 1 million lines of text. I need to load them into a ListView to display. The problem I have is: usually after I load everything, my application will take about 300MB in memory. I'm wondering how to free my memory and make it lighter.

Part of my code is below:

using (StreamReader sr = File.OpenText(ilf.fileName))

{

string oneLine = String.Empty;

string lineNo = String.Empty;

string msg = String.Empty;

while (!sr.EndOfStream)

{

oneLine = sr.ReadLine();

lineNo = oneLine.Substring(0, oneLine.IndexOf(' '));

msg = oneLine.Substring(oneLine.IndexOf(' ') + 1);

ListViewItem lvi = new ListViewItem(lineNo);

lvi.SubItems.Add(msg);

listResult.Items.Add(lvi);

}

}

Any suggetion will be greatly appreciated.

Thanks a lot.



Re: Windows Forms General How to load more than 1 million items in ListView without killing the memory?

nobugz

You should use the VirtualMode property of the ListView control, available in .NET 2.0. Sadly, the MSDN library doesn't have an example. Set the VirtualListSize property to the number of items you have and implement the RetrieveVirtualItem event to supply the listview with the data it needs to paint the item. Yell if you need code.

You probably should also think about how practical it will be for the user to browse through one million items. Some kind of filter might be useful. And there's cursory evidence that ListView gets flaky once you have more than 65535 items; I'm not sure that is a problem too in virtual mode.





Re: Windows Forms General How to load more than 1 million items in ListView without killing the memory?

Mark Shen

Thank you very much, nobugz.

I just noticed that after I minimized the window, and re-maximized it, the memory usage will be around 60MB, which is significantly less than 150MB. It seems like right after I loaded all the items in the listview, there are some unnecessary memory held by the application. Is that true and how can I release those memory explicitly in the app

Does the string operations in the code snippet cause any memory problem If so, how to handle it Thanks a lot again.





Re: Windows Forms General How to load more than 1 million items in ListView without killing the memory?

nobugz

The framework aggressively trims the working set size of your program when you minimize the main window. That doesn't mean your program suddenly uses less memory, the data now resides in the page file. Displaying one million items in a listview simply takes a lot of memory, there are no shortcuts for that other than virtual mode.