NFS Disconnects in VMware vSphere 5.5 U1

UPDATE [Jul 18 23:30 PDT]:  We continue to receive a lot of calls from customers hitting this issue, so I wanted to share an official position from NetApp CSS (support) to both our customers, as well as our field and partner community to advise customers on information you should know, as of today, about the current status of the 5.5U1 APD issue:

Build 1881737 (ESXi 5.5 Express P4) corrects for this issue, however, it is not fully qualified by NetApp yet. We are anticipating it will be on our IMT in the next week or two.

Our current fully-supported recommendation remains to back the ESXi hosts down to 5.5 flat (build 1331820) and if the SSL Heartblead issue is of concern, have them apply patch ESXi 550-201404401-SG located here: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2076121

Hopefully, within the next week or so, QA/Interop will complete testing and have this latest P4 patch listed on the IMT. When it hits the IMT, I’ll be sure to continue to update you all here.


UPDATE [Jun 12 11:40 PDT]: VMware has released Patch 4 {link} to address the NFS APD issue.  It is recommended to update immediately from current 5.5U1 installs.


UPDATE [Apr 18 17:00 PDT]: VMware has released KB 2076392, noting that this is a known issue affecting ESXi 5.5 Update 1 hosts with connected NFS storage. VMware is working towards providing a resolution to customers. To work around this issue, VMware recommends using ESXi 5.5 GA.   It was also brought to my attention today that 5.5U1 had not made it on to the NetApp IMT yet, as the QA teams had not finished their thorough interop testing.  This is one of those lessons where one must pay attention to the IMT’s for all your gear and software before upgrading whimsically.  #FoodForThought


UPDATE [Apr 17 16:25 PDT]: For now, if this condition is being experienced, the recommendation is to downgrade ESXi to 5.5.

"REMAIN CALM!"

“REMAIN CALM!”

Recently, an issue was uncovered by several NetApp customers running NFS in vSphere 5.5U1 where their datastores would go offline randomly, multiple times throughout the day. If you have not yet upgraded to 5.5 U1, DON’T! There is an ongoing internal thread at NetApp about this issue, so if you’re a NetApp employee, make sure you’re following the server-virt distribution list. When I first heard the news, my first inclination was to post an alert on twitter.  Little did I know how widespread this had become.

My first troubleshooting thought was that this was another iteration of the vSphere 5.5 change of the NFS queue depths from 64 to 4 Billion.  I can confirm that it is NOT related to the issue found in KB 2016122. VMware has confirmed the issue in vSphere and is working closely with NetApp to determine root cause, and we should expect a public KB very soon. This post will be updated with findings as they’re released.  Stay tuned…

[sexy_author_bio]

0 0 vote
Article Rating
Subscribe
Notify of
guest
17 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
trackback

[…] For further information you can see Datacenter Dude’s Nick Howell’s article – NFS Disconnects in VMware vSphere 5.5 U1, and keep an eye out on the VMware knowledge […]

melaniable
melaniable
04/17/2014 20:06

Thank you!

trackback

[…] Nick Howell (who posted this alert first on Twitter)  NFS Disconnects in VMware vSphere 5.5 U1 […]

Björn Lagerwall
Björn Lagerwall
04/18/2014 01:03

Thank you. Any word from other suppliers utilizing NFS?

toffitomek
toffitomek
04/18/2014 12:49

I was told by Nutanix they not recommend 5.5 U1 due to this same issues…

Troy
Troy
04/21/2014 04:41
Reply to  toffitomek

This is correct if you are using Nutanix hardware it also is effected. I believe it is any vendor that users NFS.

trackback

[…] had already reported on this on twitter and the various blog posts but I had to wait until I received the green light from our KB/GSS team. An issue has […]

trackback

[…] Howell (autor del twit inicial) informó de ello con más detalle en su blog DatacenterDude Duncan Epping también informa del problema en su blog Yellow Bricks Michael Webster informó de […]

trackback

[…] NFS Disconnects in VMware vSphere 5.5 U1 – In a sad day for NFS users, vSphere 5.5 update 1 apparently causes your NFS datastores to give up the ghost periodically. I’ve experienced this in my lab and can confirm it’s 100% true (I also saw an issue on the vMotion VMkernel port, too). How did this pass QA? […]

trackback

[…] had already reported on this on twitter and the various blog posts but I had to wait until I received the green light from our KB/GSS team. An issue has […]

trackback

[…] had already reported on this on twitter and the various blog posts but I had to wait until I received the green light from our KB/GSS team. An issue has […]

sam
sam
05/08/2014 12:07

Yes same with Nutanix systems. We opened a case with VMware beginning of April and with Nutanix at the same time. Nutanix did thorough testing and the assumption was made that it is most probably an issue with vSphere 5.5 U1 (Nutanix was in direct contact with VMware at that time). We did the ESX downgrade on April 10th which immediately solved the problem. The official VMware KB was created about a week later or so I believe.

Thomas
Thomas
06/11/2014 04:49

VMware released a patch for this yesterday, haven’t tested it yet though.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2077360

ryanbe
ryanbe
07/18/2014 07:16

Between this and NetApp KB 1014463 (IMT note 7291) I’ve been recommending against ESXi 5.5 for my customers. Am I being overly cautious?

trackback

[…] disconnects in vSphere 5.5 Update 1 This is good info for NetApp and vSphere customers who are using NFS.  It has a good timeline on the issue and was […]

Marino Duregon
Marino Duregon
09/04/2014 11:55

Nick, any chance you can post an update on this topic ? Has Patch 4 been officially validated by Netapp CSS ?

Scott McD
Scott McD
09/23/2014 12:34

Sorry to Bump this Nick, but is there any news on this? 5.5 U2 is released now and was wondering if Netapp has qualified any of these updates or resolved the issues with Vmware. THanks!

17
0
Would love your thoughts, please comment.x
()
x
Login
Loading...
Sign Up

New membership are not allowed.

Loading...