dmadrian

I've been doing load testing for serveral months on a test rig with 8 agents. My tests have been working well until a couple of days ago when I started seeing tests fail and event log entries on the agents involved with the following messages:

Unable to connect to the controller on 'wal1bajc.cis.concentra.corp:6901'. Computer name could not be obtained.

Unable to connect to the controller on 'wal1bajc.cis.concentra.corp:6901'. The write operation failed, see inner exception.

I noticed that disk space was low on the controller (less than 200 MB) but didn't see any improvement when I made over 1GB available. This problem only seems to happen when I'm running all or most of my agents. For example, if I see this problem on Agent 6 on a test involving 7-8 agents, I can rerun the test for a single agent and see the same data pushed through Agent 6 work successfully.

Can anyone provide any ideas regarding this problem or how to troubleshoot it

Thanks,

David Madrian



Re: Visual Studio Team System - Web and Load Testing Unable to connect to the controller

dmadrian

I turned on logging for my agents and controller and reproduced the problem described above. Below is the part of the log file that appears to coincide with the "unable to connect to the controller" problem. Has anyone seen this before or do you have any ideas for how to address this problem.

Thanks,

David Madrian

[V, 1560, 6, 2007/04/30 16:54:36.307 954959404649] QTAgentService.exe: Connection to controller is up.
[V, 1560, 6, 2007/04/30 16:54:39.291 954970088426] QTAgentService.exe: Connection to controller is up.
[V, 1560, 6, 2007/04/30 16:54:42.291 954980826536] QTAgentService.exe: Connection to controller is up.
[I, 1560, 6, 2007/04/30 16:54:45.291 954991555936] QTAgentService.exe: Connection to controller is down.
[I, 1560, 6, 2007/04/30 16:54:45.291 954991556873] QTAgentService.exe: Attempting to connect to controller using uri: tcp://wal1bajc:6901/ControllerAccessManager.rem
[I, 1560, 6, 2007/04/30 16:54:45.307 954991651088] QTAgentService.exe: Failed to connect to controller.
[V, 1560, 6, 2007/04/30 16:54:45.354 954991766406] QTAgentService.exe: Connecting to controller failed with the following exception: System.IO.IOException: The write operation failed, see inner exception. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Runtime.Remoting.Channels.SocketStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security.NegotiateStream.StartWriting(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.NegotiateStream.ProcessWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
--- End of inner exception stack trace ---

Server stack trace:
at System.Net.Security.NegotiateStream.ProcessWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.NegotiateStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.Runtime.Remoting.Channels.ChunkedMemoryStream.WriteTo(Stream stream)
at System.Runtime.Remoting.Channels.Tcp.TcpClientSocketHandler.GetRequestStream(IMessage msg, Int32 contentLength, ITransportHeaders headers)
at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.SendRequestWithRetry(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream)
at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.VisualStudio.TestTools.Execution.IControllerAccessManager.GetControllerObject(ExecutionAuthenticator authenticator)
at Microsoft.VisualStudio.TestTools.Agent.AgentService.ConnectToController()
[V, 1560, 6, 2007/04/30 16:54:47.026 954997787823] QTAgentService.exe: timeoutthread completed calling callback objects:4/30/2007 4:54:47 PM
[I, 1560, 6, 2007/04/30 16:54:48.307 955002336632] QTAgentService.exe: Connection to controller is down.
[I, 1560, 6, 2007/04/30 16:54:48.307 955002337706] QTAgentService.exe: Attempting to connect to controller using uri: tcp://wal1bajc:6901/ControllerAccessManager.rem
[V, 1560, 6, 2007/04/30 16:54:48.432 955002835285] QTAgentService.exe: Successfully connected to controller.





Re: Visual Studio Team System - Web and Load Testing Unable to connect to the controller

Michael Taute - MSFT

Who do you have the agent configured to run as

The user that the agent service runs as has to be in the team test agent service user group on the controller machine or you will get this error.

--Mike





Re: Visual Studio Team System - Web and Load Testing Unable to connect to the controller

dmadrian

Thanks for your response Mike. I checked and confirmed that the user running the agent service is a member of the TeamTestAgentService user group on the controller. If this were the problem, I'd also be wondering how agents that fail to connect when several agents are running are able to successfully connect when only one agent is running.

I also noticed yesterday that, if I try restarting the same test several times in a row, there will usually be one or more agents unable to connect to the controller. However, sometimes all the agents do connect successfully.

--David





Re: Visual Studio Team System - Web and Load Testing Unable to connect to the controller

slumley MSFT

Are you still having these problems




Re: Visual Studio Team System - Web and Load Testing Unable to connect to the controller

dmadrian

No, this is no longer a problem. Thanks.



Re: Visual Studio Team System - Web and Load Testing Unable to connect to the controller

Ed Glas - MSFT

Were you able to determine a root cause for the problem

Ed.






Re: Visual Studio Team System - Web and Load Testing Unable to connect to the controller

dmadrian

The problem we were experiencing was that the load test client wasn't receiving data from a web service call. In the process of looking for the cause of this problem, I noticed the communications problems I listed previously on this thread. However, I'm a bit embarrased to admit that I discovered these were not the cause of our problem. We had the system under enough stress that the web service call was timing out and the exception about the agent not being able to connect to the controller actually appeared to just be the result of me restarting the test rig from the administration dialog box in Visual Studio.

Sorry about the misleading question, but maybe this will help someone else avoid a similar confusion in the future. Thanks for your help and follow up.

~David