Troubleshooting The vMotion migration failed because the ESX host were not able to connect over the vMotion network.
Today I will take you thought the troubleshooting for vMotion using CLI, If you have not done network troubleshooting on esx host you will find some really good esxi commands on this post. And if you are using a converged infrastructure its important to understand end to end connectivity especially if you have different vlans for each service like vMotion FT, Management traffic and VM-Traffic.
I build a brand new Data Center and after configuring the dv switch I decided to test the vMotion first and this is the error I saw on the first attempt the vm was stuck at 14 % and after a while it gave the below error message.
as shown below:
Migrate virtual machine error message
If you read the above error message you will find the IP’s of the host which I am migrating from and to.
for example lets consider the below two host this troubleshooting is done on esx-02. vmnic2 is FT vmnic3 is vMotion so I am comparing the mac address of booth the host on the ucs service profile to compare tht the vnic template has the right mac address and if they are connected to the right vlan. Read the rest of this entry
I came across a strange issue where 2 blades was unable to ping I could get 2 ping and then Request time outs
Problem : 2 Servers( in the same chassis server 3,4 ) unable to ping their gateway. Ping drops are after 1-2 packets. We cannot ping/ssh to them from outside and they cannot be added to vCenter. Other blades in the same chassis are working ( 1,2)
Steps Taken :
- Confirmed that we do not have any IP conflicts
- Made sure that we do not use the same subnet anywhere else in the network
- Also checked and confirmed that the Mgmt(CIMC) subnet is different
- Started a ping from inside a Host ( Server-3 ) to check if you are able to get to the gateway
- Checked vobd logs to find if you can find any entrie for duplicate ip/mac
- Checked the Mgmt Interface : vmk0
- It is connected to Port Group : Management Network on VLAN ID, Check if we are using the right vlan
- Found that the MAC Address of the vmk0 is same as vnic (vmnic0) : This is a known issue VMware KB article here. This is the vnic mac address of the service profile which is attached to the esx blade.
- Deleted and re-created the vmk0 interface
- Confirmed that the MAC Address has changed
- Started a continuous ping which work fine now
- Host can now be added to the vCenter
So let me go through of the process of how we found the duplicate mac address.
The vnic mac address was taken by vmknic as you might know vmknic mad starts with 00:50:56 but in my case it was same as the mac address of vnic from the service profile. Read the rest of this entry