Share via

Quantification vs Gut feeling

  • Article

I was working on a pretty tricky issue a while ago. It was related to the Visual Studio IDE high memory consumption and slowness and I found that there are a lot of files in the Source Folder. So much so, that my gut feeling was saying that it is the source of all evil. But, troubleshooting doesn't really go well with gut feeling.You need quantification... you need proof!

Have a look at the image below (I created these junk folders just to give you an idea of how it roughly looks like). In the following folder structure, almost every folder contains a folder called _wow. The project was a big one and contained hundreds of folder and thousands of file... yeah, honestly! I just wanted to know what is the size of all the _wow folders combined. Oh, and I forgot to say that I am very lazy and instead of relying on counting them manually I thought of using a couple of tools to make my job easier.

image

It is pretty simple, and believe me this trick is really handy sometimes!

So, the first thing I did was to switch over to the cmd window and navigated to MyFolder. Then I listed out the directory with dir /s /b > MyOutput.txt which gives an output something like this...

C:\temp\MyFolder>dir /s /b
C:\temp\MyFolder\1
C:\temp\MyFolder\2
C:\temp\MyFolder\3
C:\temp\MyFolder\_wow
C:\temp\MyFolder\1\55
C:\temp\MyFolder\1\66
C:\temp\MyFolder\1\77
C:\temp\MyFolder\1\88
C:\temp\MyFolder\1\99
C:\temp\MyFolder\1\_wow
C:\temp\MyFolder\1\55\2
C:\temp\MyFolder\1\55\3
C:\temp\MyFolder\1\55\_wow
C:\temp\MyFolder\1\66\2
C:\temp\MyFolder\1\66\3
C:\temp\MyFolder\1\66\_wow
C:\temp\MyFolder\1\77\_wow
C:\temp\MyFolder\1\88\_wow
C:\temp\MyFolder\1\99\_wow
C:\temp\MyFolder\_wow\MyFile.txt

Now, open the MyOutput.txt file in MS Excel.

Select the column 1, and choose the ribbon and select Conditional Formatting -> High light cells Rules -> Text That Contains.

image

Choose a fill like the following...

image

From the ribbon, click Sort & Filter -> Custom Sort

image

Drop down the Sort on menu and select -> Cell Color
Click on No Cell Color button and select the color.
The output is something like the following...

image

Now, you can select all these colored cells, create a new text file and paste it in there. Let's call it... FindSize.txt.

I have written a small exe, which can count the size of all these files for you.

image

Last but not the least, have a look at the output of the problem which I was talking about earlier...

image

As you can see the gut feeling was not completely wrong... BUT quantity makes it a bit more palpable! Besides, as you can see the size of all the 1647 files combined is just about 25 MB. The performance problem may be due to the number and type of these files, but the high memory consumption is more likely not caused by this. We'll see later! After all, it is taking just 25 MB odd. The problem is not solved yet, but I am sure we will nail it down soon.

Download the utility here if you like!

Until next time Wave
Rahul

Share this post :

CalculateFileSizeByGivingFileNames.zip

Comments

  • Anonymous
    August 24, 2007
    Troubleshooting is about starting with building blocks of proof, but often which blocks you start with are from the guy.  Great blog post!

  • Anonymous
    August 27, 2007
    The comment has been removed

  • Anonymous
    August 28, 2007
    That's good to know Nitin. I am not sure if there is any Windows equivalent to the find.exe. If there is, let us know!

  • Anonymous
    August 30, 2007
    No script or exe needed. Right-click, Search for _wow, Ctrl-A (to select all), right-click, properties.

  • Anonymous
    August 30, 2007
    Thanks for your comments glenh. I really wish this search this worked as This whole problem happened because the Search didn't work in my case. We were seeing the folders, but it was not showing it when we searched for it. So, the search not working was the root cause of why we had to resort to this!

  • Anonymous
    August 30, 2007
    The comment has been removed

  • Anonymous
    August 30, 2007
    Thanks Jas! That will definitely save some of the steps which I did using Excel :-)

  • Anonymous
    August 31, 2007
    I was reading this post from Rahul a few days ago, and wondered if something similar could have been

  • Anonymous
    August 31, 2007
    I was reading this post from Rahul a few days ago, and wondered if something similar could have been