External lync user can not start audio with internal user

Recently I was troubleshooting a case where externally connected internal Lync user could not establish audio call with internal client. From client log we see

INVITE (message shows 5 candidates, we also see that hotspot firewall blocks UDP, so this client will have only limited choise of TCP to establish media flow. Number of IP4 candidates should be 6+ in ideal situation),

SIP Trying

SIP OK (we get SIP responce from another end and list of candidates from the other side), we can now exchange IM chats

SIP ACK and then suddenly SIP BYE. No audio call is esablished.

IN SIP BYE we observe an error:

ms-client-diagnostics: 23; reason="Call failed to establish due to a media connectivity failure when one endpoint is internal and the other is remote";CalleeMediaDebug="audio:ICEWarn=0x4000322,LocalSite=10.79.2.125:49172,LocalMR=xxx:50452,RemoteSite=10.72.3.29:52206,RemoteMR=yyy:50175,PortRange=49152:49191,LocalMRTCPPort=50452,RemoteMRTCPPort=50175,LocalLocation=1,RemoteLocation=2,FederationType=0,NetworkName=test,Interfaces=0x14,BaseInterface=0x4,BaseAddress=10.79.2.125:49158"

if we lookup ICEWarn=0x4000000 in the resource kit: http://www.microsoft.com/en-us/download/details.aspx?id=22644

we could see that problem is coming from egde firewall:

0x4000000

TCP-TCP connectivity checks failed over the TURN Server.

This is indicating that TURN TCP-TCP connectivity check was tried and it failed. The failure indicates that port 443 was not opened on the firewall. If one of the TURN servers was 2007 A/V Edge Server. The administrator needs to open ports from 50,000 through 59,999 TCP to all external Audio/Video Edge services in the environment. This flag isn’t expected and may result in an ICE protocol failure.

Then I have to check if 443 is open, but since I have no access to the Cisco ASA, I use telnet to test the port. The problem with windows telnet is that I can not change IP source address. On the edge if you run telnet www.google.com 443 you will use address of first IP address (Access). That was ok. But to test thirst IP (AV) i had to download ncat http://nmap.org/dist/nmap-6.40-win32.zip
and then run from edge cmd  (http://manpages.ubuntu.com/manpages/natty/man1/ncat.1.html)
ncat -s IP_ADDR_AV_EDGE www.google.com 443
and it was immediately rejected meaning that this direction the port is closed.

When escalated incident was solved by ASA admin, audio calls started flowing.

For more detailed information on connection scenarios refer to http://www.shudnow.net/2010/12/06/lync-server-2010-port-ranges-and-audiomedia-negotiation/

UPDATE: it may happen that firewall is ok, but the VM was migrated to a ESX host that has different VLAN setup and this may prevent traffic also.