Akai

Hi

i need to grab some search results from a *.aspx URL. If this website uses dropdown list for its search function (that is like when i select an option from the dropdown list, the website will auto refresh and display the results). Since this web is dynamic so the url address remains exactly the same as the main page.

is there any method in C# where i can select the options of the dropdown list so that i can select the options that i want programatically and grab the search results I have looked at the html source, whenever the user select an option from the dropdown list, it will call a javascript function, refresh and display the results.

Thank You



Re: .NET Framework Networking and Communication Select options from a dynamic web programatically

Dave Murray (MSFT)

The trick is the javascript function that gets the new results. Chances are that it's calling a web service to download the new data. You simply want to make the same request, and the parse the data you need from the response. If it's not easy enough to figure out from the page's source, a network monitor like NetMon will let you see what's being sent to the webservice, and what it's returning. HttpWebRequest should be able to help you make the request.

Good luck!

-dave





Re: .NET Framework Networking and Communication Select options from a dynamic web programatically

Akai

Hi all

hmm... i tried using HttpWebRequest to POST the data back to the url. HTML_A is the main page source before any selection is made. HTML_B below is the part of the source from the url which i have selected the dropdown list from Internet Explorer....

HTML_A:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
<title>XXX</title>
<meta content="Microsoft Visual Studio .NET 7.1" name="GENERATOR">
<meta content="Visual Basic .NET 7.1" name="CODE_LANGUAGE">
<meta content="JavaScript" name="vs_defaultClientScript">
<meta content="
http://schemas.microsoft.com/intellisense/ie5" name="vs_targetSchema">
<LINK href="../font.css" type="text/css" rel="stylesheet">
<script language="JavaScript" src="../js/mouseover.js"></script>
<!--<script language="javascript">

window.onload = function()

{
history.go(+1);
}

</script>-->
<script language="javascript">
function addiris()
{
var df = prompt(" What do you want to save this query as ");
{
document.Form1.txtiris.value = df;
__doPostBack('btniris','')
alert('Link added to My iris NextBus');
top.location.href =nextbus.aspx'
}

}

</script>
</HEAD>

<body bgColor="#f1f2f3" MS_POSITIONING="GridLayout">
<form name="Form1" method="post" action="bus_serviceopt.aspx" id="Form1">
<input type="hidden" name="__EVENTTARGET" value="" />
<input type="hidden" name="__EVENTARGUMENT" value="" />
<input type="hidden" name="__VIEWSTATE" value="dDwyMTQ2ODEwNzQ7dD............."/>

....

<td vAlign="middle"><select name="drpservice" onchange="__doPostBack('drpservice','')" language="javascript" id="drpservice" class="bodytxt" style="width:96px;">
<option value=""></option>
<option value="001N">001N</option>
<option value="002">002</option>
<option value="003">003</option>
<option value="045">045</option>
<option value="048">048</option>

<option value="051">051</option> -----> no options selected yet.
<option value="052">052</option>
<option value="159">159</option>
<option value="160">160</option>
<option value="CT8">CT8</option>

</select></td>

.....

HTML_B:

.....

<td vAlign="middle"><select name="drpservice" onchange="__doPostBack('drpservice','')" language="javascript" id="drpservice" class="bodytxt" style="width:96px;">
<option value=""></option>
<option value="001N">001N</option>
<option value="002">002</option>
<option value="003">003</option>
<option value="045">045</option>
<option value="048">048</option>
<option selected="selected" value="051">051</option> -----> the option that i selected from the dropdown list in IE.
<option value="052">052</option>
<option value="159">159</option>
<option value="160">160</option>
<option value="CT8">CT8</option>

</select></td>

.....

I tried POST the entire HTML_B back to the url and perform a GET again but the html code that i got doesn't reflect any options being selected. I guess the main problem is we need only to POST the options selected back to the correct position to the url... But how do we go about doing that My C# code that i tried using....

using System;

using System.IO;

using System.Net;

using System.Text;

using System.Globalization;

using System.Web;

class ConsoleApp

{

static void Main(string[] args)

{

Specific_Browser();

}

public static void Specific_Browser()

{

StreamReader sr_input = new StreamReader(@"C:\MK Projects\ConsoleApp\DataInput.txt"); -----> contains HTML_B

string input = sr_input.ReadToEnd();

StreamReader sr = null;

HttpWebRequest WebRequestObject = null;

WebResponse WebResponseObject = null;

try

{

ASCIIEncoding encoding = new ASCIIEncoding();

byte[] data = encoding.GetBytes(input);

HttpWebRequest WebRequestObject1 = (HttpWebRequest)WebRequest.Create(http://www.sbstransit.com.sg/iris3/bus_serviceopt.aspx); --------------------> the URL i need to get information from

WebRequestObject1.Method = "POST";

WebRequestObject1.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)";

WebRequestObject1.ContentType = "application/x-www-form-urlencoded";

WebRequestObject1.ContentLength = data.Length;

WebRequestObject1.KeepAlive = true;

Stream newStream = WebRequestObject1.GetRequestStream();

// Send the data.

newStream.Write(data, 0, data.Length);

newStream.Close();

Console.WriteLine("Posted");

WebResponseObject = WebRequestObject1.GetResponse();

//Retrieve data from web

WebRequestObject1.Method = "GET";

WebResponseObject = (HttpWebResponse)WebRequestObject1.GetResponse();

sr = new StreamReader(WebResponseObject.GetResponseStream());

String Results = sr.ReadToEnd();

FileStream outfile = new FileStream(@"C:\\MK Projects\\ConsoleApp\\DataOutput1.txt", FileMode.Create, FileAccess.Write);

StreamWriter sw = new StreamWriter(outfile);

sw.Write(Results.ToString());

sw.Close();

Console.WriteLine("Retrieved!");

}

finally

{

try

{

sr.Close();

}

catch { }

try

{

WebResponseObject.Close();

WebRequestObject.Abort();

}

catch { }

}

}

Sorry for all the lenghty codes.... i really need some help from here...

Thanks alot!!!





Re: .NET Framework Networking and Communication Select options from a dynamic web programatically

Dave Murray (MSFT)

You don't want to post the entire contents of the page, but only a short string describing all of the elements of the <form> node thats being submitted. The general format is:

[name]=[value]&[name]=[value]&...

For instance:

__EVENTTARGET=drpservice&__VIEWSTATE=...&drpservice=051

-dave





Re: .NET Framework Networking and Communication Select options from a dynamic web programatically

Akai

Hi

if i got a HTML line from the web source when no values were input into this particular textbox

<td vAlign="middle">

<input name="txtsvcno" type="text" maxlength="4" id="txtsvcno" class="bodytxt" style="width:56px;" />

</td>

and when i input a value of "051" into the web, i got the follow HTML line of code....

<td vAlign="middle">

<input name="txtsvcno" type="text" value="051" maxlength="4" id="txtsvcno" class="bodytxt" style="width:56px;" />

</td>

obviously, the difference is in the additional value parameter. So how do i input the text that i want programmatically Does it looks something as follows

string data = "txtsvcno=051&__VIEWSTATE=XXXX..."

Thank You