New vscsiStats Excel Macro

I wrote an Excel Macro to process vscsiStats data and turn it into pretty charts & graphs. I shared that macro with my friend Matt Kelliher and after showing him how to use it, he suggested and made a modification to it. The latest version will still process the data and create charts but then it will also export the charts as PNG, create an HTML file and put thumbnails of the charts in. You can then click on any of the charts for a full-screen view. Handy method of presenting the data in a concise format.

Let me know what you think of the file, you may download a copy here. You are welcome to download and use this macro but please leave the comments at the top (feel free to buy us a beer or two too)

This macro has been tested in Excel 2007 and 2010 beta. I must say though,  it runs much more slowly in these than in 2003.

Note: SAVE your spreadsheet first before running this version of the macro, it uses the current save location as the starting point for creating the HTML and saving the images.

VMware / Sun analytics

I was going to do a post about vscsiStats processing in excel just in and of itself. But today an opportunity presented itself that I hope to be able to exploit.
Seemed like our Vmware View Manager was dog slow. Boss was complaining at me about it. I thought, OK, this is an opportunity to gather some vscsistats and process them to see what’s going on with the storage on this thing. As a starting point, our View Manager is a Win2k3 box, 2gig ram running on a Dell Poweredge 2950 ESX4. Backend storage is provided via NFS from a Sun 7410c.
First thing I did was login to ESX with putty, clear vscsiStats and then start gathering statistics. I collected for 60 minutes, exported the data and then processed it in Excel.

hmmm. interesting:

IO Lengths

Looks reasonable enough, let’s look at read and write throughput as reported by vscsiStats:

hmmm. 215K average total throughput to storage. That’s not enough to cause the performance degradation that we are seeing. Another bit of calculation in Excel showed me that the View Manager was averaging 22IOPS. Again, not enough to cause any of the problems we are having with it.

So, I then logged into the Sun 7410 head that runs NFS for this ESX cluster. A few minutes of time looking at the analytics on the sun and I had my answer.

The green shaded area is NFSv3 ops per second to/from the view manager. Yep, take another look 6750 ops per second in that spike. I left the other boxes listed in there although their names have been hidden to protect the innocent. The next highest is 530 IOPS for an SAP server, then 294 and 92 for more SAP servers. One of my file servers is next at 61.

Clicking on the list for View Manager to open the hierarchy, I saw that there were two files listed that made up that 6750 ops, one of them named …delta.vmdk and one of them a vswp file.

The vm has a snapshot and about half of the ops are going to the delta file. Then for some reason the View Manager is exceeding its allocated 2gig of ram and using the virtual swap space.

I’m not an expert at any of these tools, but it sure is nice when you can use the tools you have to track things down and find suspected causes of the problems. I will be making changes to this machine first thing in the morning to remove the snapshot and increase the ram and set a reservation so it doesn’t use a vswp file.

I’ll post again once I gather more stats to see if this has helped with performance.