It has been a long time since I have published any post. I have been quite busy these days.
I am going to attend Cisco Live in Berlin, Germany next week. So I just thought this is going to be a good opportunity to share my views of Cisco Live and the latest and greatest technologies Cisco has to offer.
For those of you who are interested to keep an eye on Cisco Live Cisco Live webcasts and broadcasts are free to attend!
To attend a webcast, you will need a Cisco Live account. Register now for your free account or log in if you already have one
I am going to show you how to configure the snmp agent on a Linux based distribution – this example is tested on CentOS and Red Hat Linux
Purpose:
The purpose of installing SNMP (Simple Network Transfer Protocol) is to monitor host resources like CPU, Memory, Network and Disk Utilization etc.
SNMP is a network management protocol used to send and recive
messages between NMS (Network Management Systems). SNMP allows NMS to
manage network devices remotely, by using MIBs .
You might be wondering what are MIB ?
MIB’s are Management Information Base, It can be best described as information in the form of variable that would be useful to manage the specific item. The developer who creates a MIB has to create a OID aka Object Identifiers
Now you might be wondering whats OID ? An object identifier is simply a unique id represented by a series of numbers…
Today I will take you thought the troubleshooting for vMotion using CLI, If you have not done network troubleshooting on esx host you will find some really good esxi commands on this post. And if you are using a converged infrastructure its important to understand end to end connectivity especially if you have different vlans for each service like vMotion FT, Management traffic and VM-Traffic.
I build a brand new Data Center and after configuring the dv switch I decided to test the vMotion first and this is the error I saw on the first attempt the vm was stuck at 14 % and after a while it gave the below error message.
as shown below:
Migrate virtual machine error message
If you read the above error message you will find the IP’s of the host which I am migrating from and to.
for example lets consider the below two host this troubleshooting is done on esx-02. vmnic2 is FT vmnic3 is vMotion so I am comparing the mac address of booth the host on the ucs service profile to compare tht the vnic template has the right mac address and if they are connected to the right vlan. Read the rest of this entry →
I came across a strange issue where 2 blades was unable to ping I could get 2 ping and then Request time outs
Problem : 2 Servers( in the same chassis server 3,4 ) unable to ping their gateway. Ping drops are after 1-2 packets. We cannot ping/ssh to them from outside and they cannot be added to vCenter. Other blades in the same chassis are working ( 1,2)
Steps Taken :
Confirmed that we do not have any IP conflicts
Made sure that we do not use the same subnet anywhere else in the network
Also checked and confirmed that the Mgmt(CIMC) subnet is different
Started a ping from inside a Host ( Server-3 ) to check if you are able to get to the gateway
Checked vobd logs to find if you can find any entrie for duplicate ip/mac
Checked the Mgmt Interface : vmk0
It is connected to Port Group : Management Network on VLAN ID, Check if we are using the right vlan
Found that the MAC Address of the vmk0 is same as vnic (vmnic0) : This is a known issue VMware KB article here. This is the vnic mac address of the service profile which is attached to the esx blade.
Deleted and re-created the vmk0 interface
Confirmed that the MAC Address has changed
Started a continuous ping which work fine now
Host can now be added to the vCenter
So let me go through of the process of how we found the duplicate mac address.
The vnic mac address was taken by vmknic as you might know vmknic mad starts with 00:50:56 but in my case it was same as the mac address of vnic from the service profile. Read the rest of this entry →
This week I have seen an interesting issue on Windows Servers 2008 /2012 . Just to let you know these are VM’s and I was unable to connect to these server. So I login on vCenter to check the windows host and found the server was having network error.
After checking the event viewer i found this error
The system detected an address conflict for IP address 0.0.0.0 with the system having network hardware address xx-xx-xx-CE-44-3F. Network operations on this system may be disrupted as a result. Time stamp 28/10/2014 hh:mm:ss.
Quick Fix
Reboot the server and it got the network back. I have to mention giving ipconfig was showing the correct IP address even when it was having the network error.
Further investigation reviled few interesting and worrying facts, apparently this issue is only affecting Windows Vista and above, we also found that it is a known issue, However it will arise only when a windows server is rebooted.
Root Cause
The root cause is part of the detection flows defined by the RFC 5227 (IPv4 Address Conflict Detection).
The error is caused by the method used by Windows to detect an address conflict (http://tools.ietf.org/html/rfc5227#section-2.1.1) and one of the packets used by the cisco security feature called “ip device tracking”, used for the NAC Layer 2 validation.
Unfortunately the IOS version (15.2) used by the most of the Cisco core switches, enables this feature by default and there is no way to disable it. The only options are: downgrade the IOS or tuning some parameters.
Solution
a. downgrade the IOS
b. modify a parameter of Cisco ‘IP device tracking’ feature in order to potentially solve this issue.
I am writing this post to show the process on how to provision LUN’s from Cisco UCS Invicta to ESX host in 3 steps hope you enjoy the post.
Note:
Create a default LUN and add MAP ID 0 ‘zero’ when ever you are provisioning a new ESX rule of thumb for Invicta is add a default lun lun0_ default with MAP ID 0
Keep all the LUN’s with the same MAP id on each host when provisioning them. ( Caution : If you do not give the same MAP ID for a LUN which is already provisioned and the lun has VM running. The LUN comes up as a new lun to be added to VMware so the Administrator if he is not aware of this issue may end up formating it assuming its a new LUN to provision using VMFS 5.
You need to remember that you can right click and drag and drop luns to map them.
In our environment I have created a default lun with 10 GB and never provision this on VMware, however always add this default lun first with MAP ID 0 to any ESX host you want to provision.
Login to the invicta with admin or superuser account.
Step 1: Creating a LUN
Click on LUN Configuration and LUNs and Click on Create LUN
As I have mentioned in the Notes at the top that we need to first create a default lun which is what I am creating below. just fill in the details I call it lun0_default
The size of this lun can even be 1 GB as you are not going to use this lun . I have put 10 GB a I have 65 TB of thin provisioned disk and I am not going to provision this lun its just what is recommended by Cisco documentation to have a default lun 0 provisioned on each host with MAP ID 0
I have created another LUN of 1 TB calling it lun1 as shown below
Step 2. Adding a Host Initiator Group
Click on LUN Configuration and Initiator Groups and Click on Create New Group
If you are looking at this Failure Reason: advanced boot or secure boot configuration cannot be applied to the specified server There are not enough resources overall, You might have come across this error.
Although the above error looks like a very generic error when you apply a service profile to a Server. Most of the on-line search takes you in the direction of making sure you are using the legacy CD ROM drive.
With the process of elimination I found that while using the Flex Flash ‘SD Card’ in Boot Option this error was trigged. once I removed the SD Card, I was able to apply the profile with out any problems. But I wanted to use the Flex Flash with RAID-1,( SD Cards) to boot. more info on Cisco UCS FlexFlash Configuration of Secure Digital (SD) Card
As you might be aware that boot from SD card is a new feature introduced in version 2.2 , we then started looking at he version of UCS Manager and Firmware running on the environment. I had recently upgraded this UCS to the 2.2(1d) so that was not an issue.
We thought to double check the firmware configuration, when going through this options we create a Host Firmware Policy specifically only for M3 and just selecting the M 200 M3 CIMC and Bios with the latest version of firmware. more info on firmware can be found CISCO UCS Firmware Update Process
Once this option was applied to the Service profile The blade started to boot up.
The only thing different to this was I had created a custom firmware on Host Firmware Policy called 2.2 firmware but ended up selected quite a lot of servers with latest version of software .. However in my UCS Environment I have only got M 200 M3.
So to fix this issue, make sure in ‘Host Firmware Package’ make sure you have customized firmware profile for every different server you have, so that you can individually apply it to the service profile you are assign to that specific server.
Example M2 and M3 Blades we can have two Host Firmware Packages which you can add to specific Service profiles individually.
If you are reading this post you might have come accross the Error “The firmware of the following components cannot be updated because they are using host/management firmware policy:”
This week I was workig on upgrading the firmware of CISCO UCS from version 2.1.(1 f) to 2.2.(1d). When doing the firmware update after uploading the firmware imagers to the FI, I went to the ‘Update Firmware’ and Selected ‘All’ and select ‘Bundle’ to select the specific bundile I wanted to install in my case 2.2(1d)
After making the selection as shown above Click Apply and OK. Which is when I saw the error message below, This bacally means all the chassis CIMC Controler / Adapter, BISO cannot be updated because the service profile attached is using hosted management of firmware.
So, their is no ways way out form this as if you say Yes then these server will miss out form updates.
One option we have tested is de-associate the sersvice profile , as this is Server Firmware so, its related to Service profile association. I was able to Update Firmware successfully.
Howerver I am still invistigating how to avoid this step if I have to do a Firmware upgrade on Production Live enviernment, this is a Show Stoper. Luckely for us we have migrated all our production to a diffrent UCS enviernment, so we did not have any thing running on this UCS.
I belive the way the service profile firmware police should be done in a diffrent method, and then associated to each Physical Blades.
I will keep you posted, once I get another process of doing the Update Firmware with out de-associating the Service Profiles. For more information on how the firmware is done check out my post CISCO UCS Firmware Update Process
Cisco has recently launched a lot of Firmware for CISCO UCS, it is becoming rather hard to keep up with the new release. We had 5 New Release in Version 2.2 starting from 2.2.(1b), 2.2.(1c), 2.2.(1d), 2.2.(1e), 2.2.(2c).
The good thing is, in every release CISCO is adding some good features. In one of the release 2.2.(1d) they have introduced boot from SD, FlexFlash, this option was not available on the older versions of UCS Manager.
If you are intreasted in this, and want to find how you can boot the ESXi or any other OS off the Secure Digital (SD) FlexFlash and how this can be implemented do check my post here: Cisco UCS FlexFlash Configuration of Secure Digital (SD) Card
So, for the newbee to Cisco UCS, its confusing about the firmware. In this post I am trying to clarify few aspects of this, feel free to post comment below if you have any questions and I will get back to you.
The CISCO UCS firmware is bundled into two packages as given below:
1. Infrastructure Firmware
In Infrastructure Software firmware we update the firmware of the following
Cisco UCS Manager software
Kernel and system firmware for the fabric interconnects
I/O module firmware
2. Server Firmware
Cisco UCS B-Series Blade Server Software Bundle
CIMC firmware
BIOS firmware
Adapter firmware
Board controller firmware
Third-party firmware images required by the new server
This also includes the firmware of Adapter, BIOS, FC Adapters, HBA Option ROM, Storage, Controller, Local Disk
Cisco UCS C-Series Rack-Mount UCS-Managed Server Software
CIMC firmware
BIOS firmware
Adapter firmware
Board controller firmware
Third-party firmware images required by the new server
As most of you might be aware the UCS manager is the GUI Java based which allows you to fully manage the CISCO Unified Computing System.
Note I have noticed that the Java version 7 update 40 is only compatible with the UCS Manager.
So, where does the UCS Manager run ?
This was one of my first questions, Well the Software of the UCS Manager runs on the FI ( Fabric Interconnect 62xx) as we have a pair of Fabric Interconnect it has got resilience.
The UCS System is designed to be highly scalable and available, so upgrading the firmware should not bring any outage to the environment. However Cisco still suggests to arrange a Change windows and get this done out of hours or the quite period of your business.
This UCS Manager is bundled in the Infrastructure Firmware Software along with the Kernal update of the FI which is the NXOS, Finally we have the I/O Module firmware which is a part of the Infrastructure firmware software too.
The Other Bundle is the Server Firmware which has the firmware for BIOS, the Blades, Adapter, Board controller etc.
Cisco UCS FlexFlash /SDCard Configuration for Booting Server Operating System.
As most of the Cisco UCS users might be aware that Cisco has now add a lot of improvements and new features on the latest upgrade 2.2(1e).
One of the feature which we have started using is Cisco FlexFlash which is an SD Card. Its fully support if you buy the SD Card from Cisco and we can even do mirroring on the Flash cards for redundancy, we will be installing ESXi Server 5.5 on these blades after mirroring them.
So where and How to install the Secure Digital (SD) Card ?
If the CISCO UCS B200 Blade is in the Chassis you will need to power it off and slide it out just half way through and you should find the SD Card slot on the left
for more information on how about this can be found on the following link http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/hw/blade-servers/B200M3.html
As you might be aware that CISCO UCS blade is just as equal to a brick and all the configuration is done on the logical server called Service profile. you will need to create a new / clone an existing Service profile template and modify with the below changes which I have shown below on this post.
I have to make you aware that if you want to use SD card with Mirroring or RAID 1 you have to create the following new policies and create a Boot policy as mentioned above.
1. Disk policy with Mirror and any configuration
2. We have to create two Scrub policy under policy ( you can call it any thing you want which makes you understand what configuration you have selected)
Flex Flash Scrub
No Scrub
So where will you find the Flex Flash Controller?
Click on the Left pan on Equipment and Select the Chassis then the Server, on the right you will find the below details
click on Inventory and select the last tab which is Storage on the Storage tab you will find FlexFlash Controller
Note: this FlexFlash is only support if you are running 2.2 (d1) firmware.
Just redone my Home LAB as I need more resources especially the Memory. I recently bought a DL380 G5 which is 2U server.with 24 GB Memory and Intel. Xyon CPU. I have configured RAID 5 with 4 SAS 2.5 Disk they are 148 GB each.
I am going to show you how to configure the snmp agent on a Linux based distribution – this example is tested on CentOS and Red Hat Linux
Purpose:
The purpose of installing SNMP (Simple Network Transfer Protocol) is to monitor host resources like CPU, Memory, Network and Disk Utilization etc.
SNMP is a network management protocol used to send and recive
messages between NMS (Network Management Systems). SNMP allows NMS to
manage network devices remotely, by using MIBs .
You might be wondering what are MIB ?
MIB’s are Management Information Base, It can be best described as information in the form of variable that would be useful to manage the specific item. The developer who creates a MIB has to create a OID aka Object Identifiers
Now you might be wondering whats OID ?
An object identifier is simply a unique id represented by a series of numbers and point that has a unique corresponding name. The format of OID is a sequence of numbers with periods /dots in between.
example of Print Device OID is like this:
1.3.6.1.2.1.43.10.2.1.4.1.1
This is HP Printer Total Page Count.
To learn more about MIB’s and OID’s please Google it, as I am still learning about it too its a huge topic on its own.
Now that you have a basic understanding of how SNMP works lets start off with the installation of NET-SNMP package ( oh by the way formally it was known as UCD-SNMP)
Installing NET-SNMP on Linux System:
To install the snmp (net-snmp ) I will recommend you to install the other package net-snmp-utils this will be useful for testing as it has handy tools like snmpwalk etc.
You can also watch the snmp installation video created by me.
The Video has the full installation process.
[root@Client ~]# yum install net-snmp net-snmp-utils
Loaded plugins: downloadonly, fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.coreix.net
* extras: mirrors.coreix.net
* updates: mirror.krystal.co.uk
Setting up Install Process
Package 1:net-snmp-utils-5.5-44.el6_4.1.x86_64 already installed and latest version
Resolving Dependencies
–> Running transaction check
—> Package net-snmp.x86_64 1:5.5-44.el6_4.1 will be installed
–> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================
Package Arch Version Repository Size
=========================================================================================
Installing:
net-snmp x86_64 1:5.5-44.el6_4.1 updates 303 k
Total download size: 303 k
Installed size: 816 k
Is this ok [y/N]: y
Downloading Packages:
net-snmp-5.5-44.el6_4.1.x86_64.rpm | 303 kB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
Installing : 1:net-snmp-5.5-44.el6_4.1.x86_64 1/1
Verifying : 1:net-snmp-5.5-44.el6_4.1.x86_64 1/1
Installed:
net-snmp.x86_64 1:5.5-44.el6_4.1
Complete!
[root@Client ~]#
Once the package is installed we can edit the snmpd.conf, before doing that I would suggest you make a copy of the orignal file
#cp /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.bak
Edit the snmpd.conf file with vi editor
#vi /etc/snmp/snmp.conf
1. Locate the line
com2sec notConfigUser default public
and comment it with #or just change the community name to something secret I have changed it to sizmic
com2sec notConfigUser default sizmic
2. Locate the lines and comment them with #
view systemview included .1.3.6.1.2.1.1
view systemview included .1.3.6.1.2.1.25.1.1
add a new line
view all included .1 80
3. Locate the line
# group context sec.model sec.level prefix read write notif
access notConfigGroup “” any noauth exact systemview one none
change the systemview to all
access notConfigGroup “” any noauth exact all none none
the same changes are highlighted in the image
Save the file
once the file is configured save it by pressing escape key then : wq!
Start the snmp service
once you save the file you can give the following command to start the snmp service
#service snmpd start
Note: each time you make any change to the snmpd.conf file make sure to restart the snmpd service service
For restarting the snmpd service the command is
#service snmpd restart
Configure the snmp service to start automatically:
once the service is started we need to make sure the Linux server knows to keep the service running in an event of reboot.
give the command
#chkconfig snmpd on
this enable snmpd on the following runlevel 2, 3, 4, 5 ( this is required to make sure the snmpd service is running after a reboot to the linux system)
to make sure the service is running on the above runlevel you can use the following command
I have created a Video which can take you through these changes,belive me I had spend a long time to learn about how snmp monitoring works and I will be happy to help if you have any questions
I hope you find this post useful. Don’t forget to leave comments if you find it useful.
I have recently recovered a Administrator Password on Server 2008 R2 last week. so I tested it at home with my Windows 7 Laptop and works like a charm.
Prerequisite
1. Windows media for the OS you are trying to recover the Administrator password.
2. Follow my steps given below 🙂
Reboot the Server or your system with the right media CD/DVD and select the appropriate language.
I have some steps to explain you how to do it!
Click Next and select Repair my Computer
The next screen will will identify all the windows installations and highlight in my case Windows 7 which is installed on C: drive
select Next
Select the Command Prompt
Follow the steps given in the Command window bellow:
switch to Windows\System32 and copy Utilman.exe Utilman.exe.bak ( this is just to create a backup file of Utilman.exe)
then the most important step Copy cmd.exe to Utilman.exe
as shown below copy cmd.exe Utilman.exe then you will get a prompt say yes by ‘y’
You should see a confirmation that one file copied
Restart the Server or in my case Laptop.
Copy cmd.exe over Utilman
Once the Server/ Laptop is rebooted press Windows key + U or click on the icon of Utilman, and you will see the command prompt.
give the following command to change the administrator password
net user administrator * (if your administrator account is renamed you can give that user name as I have given Ibrahim.quraishi below)
you will asked the enter the password ( the characters will not appear but dont worry as it will not show for security)
yo will be asked to reenter the password again as a confirmation.
Then you should be able to use the new password which you have set for the Administrator to login to the server.
It will not sound new if I say that Citrix XenApp server is deployed on a VMware ESXi /vSphere, as VMware is proven tested production.
I was recently tasked to clone a Citrix XenApp server for a new project, then add it back to the same XenApp Farm.
I had to use XenPrep to strip down the SID and GUID and to make it a clean new machine to add it to XenApp Farm.
Prerequisite after Cloning the server:
Isolate the network
login from local Admin account Administrator / pwd
Remove from domain
Change the server name
Change IP address
Might need to reboot the Server
enable the Xenprep service if its disabled change it to manual and start
Copy the Xenprep.msi file
Install Xen prep
If you have to add it back to the Citrix farm we need to strip down all the SID’s
xenprep.exe \XenApp ;
after the command does its job.
Add it back to the domain
reboot the server
What you shoul see:
you will Citrix IMA error that the service was started and stoped or did not start etc.
if you try to enabel the service it will not start.
you will find the server will not be added to the Citrix farm
Fix:
The Independent Management Architecture (IMA) service fails to start.
IMA service is not started.
start ->; Run ->; cmd
run the “dsmaint recreatelhc” command
once the fix is executed you should see the server on the Citrix farm
I am sure many of us being Systems Administrator have struggled at time to keep the free space on Servers. especially the system partition on Windows 2003 Server right ?
Even after deleting the log files, some how the OS grows as it has to get its monthly security updates from Microsoft and other software installs .. etc.
I know, most people try to use DiskPart command line utility, but it does not work on the system partition or I should say OS partition in lay man’s term.
In the new OS like Windows 2008 pr 2008 R2 etc., we don’t need to worry about this as the OS has new feature to shrink or expand the system partition with out rebooting the server. which mean no down time.
So, how can we expand or extend the NTFS formatted C drive on 2003 Server with out rebooting it?
Solution is a utility called ExtPartfrom Dell. its a simple command line utility easy to use tool to expand or extend the size of your C drive provided you have more disk space or another partition that is un-formated.
Dell Basic Disk Expansion
The ExtPart utility provides support for online volume expansion of NTFS formatted basic disks. This is a self extracting file that will install the extpart.exe utility. No reboot is necessary.
Please note some people think that ExtPart does not work on 64 bit OS of Windows 2003, but just to let you know it does work, its just the extractable binary is only 32 bit so it will give you an error that its a 32 bit application if you execute it on a 64 bit windows.
I have used it recently, deployed a virtual machine from a template, the template had the OS installed on only 20 GB. I added more size to the HDD from the data store, from the windows side in the disk management I can see the raw partition, all I had to do is create a new partition and used the dell ExtPart follow the steps and I will extend the C drive, you will not need to reboot the server.
The solution is to extract the binary after downloading on a 32 bit OS and use it on a 64 bit OS and its work.
Their you go! you can now expand the C drive with out rebooting the OS 🙂
the link given below takes you to Dell website what has the ExtPart utility :
