2

Five Points To Improve Your Estimation

Posted by Sameer on March 28, 2008 in Software Engineering, Work Related |
Software Estimation is tricky business.  You are often confronted with complicated technical (or even non-technical) work and asked “how long will it take?” on the spot.  You are given fuzzy requirements (or no requirements) with ambiguous definitions and have to work with a code base that is best described as “chaos”.  How can you give an accurate estimate with such a difficult environment? 

I recently read an excellent book called “Software Estimation: Demystifying The Black Art"

Here are some amazing points that you can use.
 
1)       We should not be pressured to reduce our estimates.  Stick to your estimates – whatever they are.  If management wants to reduce them, that’s fine, but stick to whatever you put in the first place.  Managers trying to reduce estimates is a very silly thing to do, because by looking at the track record of our software industry, you will find that we consistently underestimate, not overestimate!  So if you are pressuring your developers to reduce the estimate, you are asking for trouble.  Better thing to do would be to see if the estimate is well founded, based on task level estimates, or is the estimate based on unjustifiable assumptions, etc. 

2)       Underestimating is MORE expensive than overestimating.  So if you cannot estimate accurately, lean towards overestimating.  Underestimating has an exponential additional cost, whereas overestimating has a potential of a linear extra cost due to Parkinson’s law.  This is because underestimating results in lots of other problems such as having extra meetings to justify why you are behind schedule, having extra meetings to decide which features to cut, added stress on developers trying to meet tight deadlines, code having less quality in order to ‘just finish it’ so that developers can go home, having customers get angry at not-ready releases, having other teams who are waiting for the code to slip their schedules too, etc..  If you overestimate, then Parkinson’s law can kick in and developers will ‘fill in the empty time’, or as they say ‘work expands to fill the available time’

3)       Task level estimates are best done by the developer who will be doing the work.  A task level estimate is defined as a low level estimate for a particular item of work.  For example, for building a workflow engine, a task level estimate is “2 hours to add the save button that will commit the changes to the database”.

4)       With small teams (say under 5 people), estimates are most accurate when done bottom-up (i.e. from the developers).  What this means is your complicated formulas for estimation are not so helpful in such environments. 

5)       There are many other ways to estimate such as ‘estimation by analogy’ (i.e. by looking at a similar project that was built), and so on, but I won’t explain those right now.  The trick in general is to try to quantify as much as possible and leave subjectivity out of the picture.  So if you can find out how similar your project is in terms of number of lines of codes, number of features, etc, you will have a better estimate.  As well, don’t be fooled into thinking that more “estimation knobs” will give you a better result than less “estimation knobs”.  I define an estimation knob as some sort of criteria you are using to estimate, such as “size of team”, “team programmer skills as a percentile of the industry”, “how many burgers they had for lunch”, etc..  It looks like it would give you a better result but in reality the estimate becomes more and more subjective as you add more “knobs”
 
As well, developers are more likely to try to meet their estimates if they are the ones who gave it, rather than if its dictated to them from above.

Lastly, estimation is a skill we can learn and improve over time.  You will find that over the months, your developers will get better at estimating based on past experience.
 
In summary, stick to giving your developers clear requirements, lock them down, and then get them to break down the requirements into task items, and then estimate for those task items. 
 

0

My experiences with pair programming

Posted by Sameer on March 19, 2008 in Work Related |

My initial thoughts on pair programming (definition).

Normally at our shop, we don’t do pair programming.  Each developer is assigned his or her own tasks and they are responsible for completing them.  If they require assistance of explanation from another developer who is more familiar with that system they need to go and request that assistance on their own.  I believe the vast majority of offices work in this manner.

However, yesterday we decided to try pair programming.  We have a complicated CRM application where it takes months to learn the application, and the situation is that another fellow who works on the reporting side of things and I worked together to solve a few bugs in the system.

As we worked together (with a slightly decreased speed than the two of us seperately working) we managed to eliminate two instances of duplicate SQL code in our application and instead used a view that already existed which I was not aware of.  As well, I am very confident you that this resulted in a much higher quality of the application, even if it took longer.  You are far less likely to end up with bugs when you use pair programming.

I am still skeptical if it really was slower or not, because when you work on your own, you can get stuck on a particular problem and waste half a day, whereas when the other person knows it, you can immediately solve the problem and move on.  As well, you are less likely to waste time checking the news, weather, etc, because the other person is sitting and working with you.  Also, keep in mind that "task switching is expensive".  When you are working on a piece of code and you are "in the zone" and then you have to stop to assist someone else, this lowers your productivity, whereas when both of you are concentrating on one issue, you don’t have this issue.

Pair programming can also be used to train or "rough in" new employees, and as well it results in increased code awareness (i.e. both of you are trained on the same piece of code, incase one of you quits or is sick, the other person also has a good idea of the code).

While the jury might still be out on this principle, and maybe its still too "extreme" for most managers to allow it, give it a shot when you have a chance and see how it goes.

 

2

Convert your site from VB.NET to C#

Posted by Sameer on February 12, 2008 in .NET articles |

If you are tired of the headaches of using VB.NET code, and want to convert it to C#.. Here is a nice way to do it and get the comments too. 

Why would you do that?  Well if I have to explain to you, then you probably don’t need to do it.  One big reason for me is that VB.NET does not support multi line strings.  You have to keep putting " & _  at the end of each line making it very difficult to go back and forth from another program with strings.  You could technically create a tool to do this for you but that’s so silly.   Another thing is I really like using ?? operator in C#, for example txtLocation.Text = (GetLocation() ?? "").ToString() will ensure that even if SomeFunction returns null, this will always give you at least empty string..  I like to use this in cases where I am getting null and its a text box I am setting the text for.

Another reason is that VB.NET code doesn’t force you to type your variables properly, so everything is an object unless you specify, or unless you put on Option Strict

One way to do it is to use Reflector + Code generation add-in, but you will lose the comments and maybe even variable names because it uses the intermediate language (IL) from the DLL to re-make the source, so it’s pretty useless to do it this way if you want to keep your code in the long run.

The way I like is to use a site such as http://labs.developerfusion.co.uk/convert/vb-to-csharp.aspx

Keep in mind I know there are other tools available that you can pay for, and some that are free too, but this is a quick and easy way to do it at work without requiring any installation.

Once you perform the conversion, the code will still have some things that need to be manually converted.
One thing is this site does not compile the code, so it does not know if the VB code .Rows(5) is actually a function call called Rows with a parameter 5, or its an array index [5].  So you will need to fix these manually.

Same thing with Tables[], Session[], etc.
Sometimes with VB code you will see .Item being used when no such thing exists in C#, you just use the square brackets to get the item directly such as .Rows.Item(5) in VB.NET is equal to .Rows[5] in C#

Here is how you can search and replace these with a regular expression.

Search: \({(["|a-z|A-Z|,|0-9|\:|\.|_])+}\)

Replace: [\1]

You have to do this manually and confirm each one.. Or make it more smart by searching for keywords such as QueryString, Cookie, Value, etc.. and replace all, then do the rest manually

And again if you want to do them all at once, I wold recommend improving this search to find Rows, Tables, etc.. Also if you have declared an instance of a DataRow object or any other array type object, you will need to search for those as well.
 
Keep in mind its not perfect.  For example I did not put any numbers in my search, so it will miss some of the results.  You might need to tweak it a bit and add |0-9 to the first set of brackets which means OR 0-9

The next thing is that functions such as .InStr are not supported directly from C#, you need to call Microsoft.VisualBasic.Strings.InStr(…)   The code converter will automatically do this, but you need to add using Microsoft.VisualBasic.Strings to the top of your class file.

There are a few more things I could mention, but just try it out and see how it goes.

Here is a few more things
1) "my string" & " is very long" will give an error while converting. Before you run the converter, replace these & in your VB code with + instead.
2) the functions that have handles such as lstItems_SelectedIndexChanged(…) handles lstItems.SelectedIndexChanged need to be taken care of by putting either an event in the ASPX page in the <DropDownList OnSelectedIndexChanged="lstItems_SelectedIndexChanged"

Also, they might be private, so you need to make them protected.  Here’s a regexp to start you off:
find: {(protected|private)} void {(lnk|lst)}
replace with: public void \2

3) Functions that have optional parameters need to be taken care of.  An easy way to take care of this is to create another function that calls the second one that passes the default value with the same name. That easy!

By the way, since I am not regularly writing here, I would suggest you subscribe to my feed so you can get an update when I post a new article.

3

Timing is EVERYTHING

Posted by Sameer on December 23, 2007 in Work Related |

Being a person who believes in continual improvement, and being proactive instead of reactive, I suggested to one of my employers that I worked for that we should implement ELMAH (Exception Logging For .NET) to handle our uncaught exceptions.  They weren’t too interested when I made the suggestion and so I did not push it.   I had found this gem of code here because I spend a bit of time each day doing research and improving my skills. (see point 2)

However, we had this continuing problem where customers would have a problem with the .NET site, maybe they had a crash and we had no way to diagnose the crash other than to remote into their machine and try to reproduce the error.  What made it worse is that they had debug mode off, and we could not even see the exact crash details.  Often we would have to take a risk and modify their file to put debug on (I say take a risk because if you make enough modifications, the application will restart and kick everybody off the system) and this was a live site with real customers.

So once the problem was large enough that it was passed on to the development manager.  When he queried the team for possible solutions to that particular crash, I jumped at the opportunity to suggest ELMAH.  The sale was immediately closed, and within a week we had ELMAH up and running on some of our customers.

Lesson?  Wait for the right time to make your suggestions.  it can be very effective.  Otherwise your advice may be falling on deaf ears.

Also.. this had the same result again when the timing was right we were able to successfully convince our development manager to make the switch from SourceSafe to Subversion.  We had discussed it many times but we had to wait for just the right time and when this time came (since they already knew about it and liked it, but were just waiting for a good time), getting approval from management was easy as pie.

1

How Random is your Random??

Posted by Sameer on December 5, 2007 in .NET articles, Software Engineering |
How random is your random? 

Computers are very deterministic.  What that means is that you put something in, you get something out.  In order to get computers to perform "randomness", it is very difficult.

Why is this important to understand? Because we want to write our code properly, if we depend on the random function for some security purpose, such as for generating passwords, we are actually putting security holes in our application without realizing it.

In .NET Using RNGCryptoServiceProvider would give you much better random results than just a Random.Next()
 
However, in order to truly  randomize your number, you would have to do something like use data from customer mouse movements, or something wierd like that.  Alternatively you can use a hardware random number generator such as the one Intel created that uses thermal noise to generate real random numbers

To realize just how complicated this really is, lets look at this quote from the Pokerstars web site on how they shuffle the cards in their software:
 
 
SHUFFLE
"Anyone who considers arithmetic methods of producing random digits is, of course, in a state of sin." – John von Neumann, 1951
We understand that a use of a fair and unpredictable shuffle algorithm is critical to our software. To ensure this and avoid major problems described in [2], we are using two independent sources of truly random data:
* user input, including summary of mouse movements and events timing, collected from client software
* true hardware random number generator developed by Intel [3], which uses thermal noise as an entropy source
Each of these sources itself generates enough entropy to ensure a fair and unpredictable shuffle.
Shuffle Highlights:
* A deck of 52 cards can be shuffled in 52! ways. 52! is about 2^225 (to be precise, 80,658,175,170,943,878,571,660,636,856,404,000,000,000,000,000 ways). We use 249 random bits from both entropy sources (user input and thermal noise) to achieve an even and unpredictable statistical distribution.
* Furthermore, we apply conservative rules to enforce the required degree of randomness; for instance, if user input does not generate required amount of entropy, we do not start the next hand until we obtain the required amount of entropy from Intel RNG.
* We use the SHA-1 cryptographic hash algorithm to mix the entropy gathered from both sources to provide an extra level of security
* We also maintain a SHA-1-based pseudo-random generator to provide even more security and protection from user data attacks
* To convert random bit stream to random numbers within a required range without bias, we use a simple and reliable algorithm. For example, if we need a random number in the range 0-25:
o we take 5 random bits and convert them to a random number 0-31
o if this number is greater than 25 we just discard all 5 bits and repeat the process
* This method is not affected by biases related to modulus operation for generation of random numbers that are not 2n, n = 1,2,..
* To perform an actual shuffle, we use another simple and reliable algorithm:
o first we draw a random card from the original deck (1 of 52) and place it in a new deck – now original deck contains 51 cards and the new deck contains 1 card
o then we draw another random card from the original deck (1 of 51) and place it on top of the new deck – now original deck contains 50 cards and the new deck contains 2 cards
o we repeat the process until all cards have moved from the original deck to the new deck
* This algorithm does not suffer from "Bad Distribution Of Shuffles" described in [2]
PokerStars shuffle verified by Cigital and BMM International
PokerStars submitted extensive information about the PokerStars random number generator (RNG) to two independent organizations. We asked these two trusted resources to perform an in-depth analysis of the randomness of the output of the RNG, and its implementation in the shuffling of the cards on PokerStars.
Both independent companies were given full access to the source code and confirmed the randomness and security of our shuffle. Visit Online Poker Random Number Generator for more details.
[2] "How We Learned to Cheat at Online Poker: A Study in Software Security" – http://itmanagement.earthweb.com/entdev/article.php/616221
[3] "The Intel Random Number Generator" – http://www.cryptography.com/resources/whitepapers/IntelRNG.pdf"
 
Here is an article about how to shuffle a deck of cards: http://www.codinghorror.com/blog/archives/001008.html?r=31644 and in one of the links it explains a big security hole in their random number generation and how it could have been used to leverage thousands of dollars from players.

Here is a snippet of how to get Cryographically safe random numbers:

 

This will fill in the 8 bytes with a crytographically strong sequence of random values.

byte[] salt = new byte[8];
RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
rng.GetBytes(salt);

0

Understanding Your 10 Days Vacation

Posted by Sameer on November 12, 2007 in Work Related |

This article is based on Canadian vacation laws.

It seems that most people don’t understand that by law we get 4% vacation.  So where does this number of 2 weeks come from?

Well If you calculate 50 weeks * 4%, you will end up with 2 weeks, what that means is in 1 year, you get 2 weeks paid vacation and 50 weeks of work.  So if your employer prorates your vacation for the first year, meaning that after 3 months they give you 3/12 of 10 days (i.e. 2.5 days or 25% of your total) but you need to take off 2 weeks and they refuse to give it to you, you shouldn’t worry about because either it means at the end of the year thats all you took (2.5 days paid), and by law they will pay you for the other 7.5 days or 3.75% of your total salary at the end of the year (or whenever they reconcile their accounts which could be a few months later)

The other case is that you will still get 7.5 days vacation after those 2.5 paid days, so in that case you didn’t lose anything and you ended up getting about 2 weeks extra unpaid vacation. 

So make a decision, and enjoy your vacation!

0

How To Debug Inside The Code Of Your 3rd Party Libraries

Posted by Sameer on November 6, 2007 in .NET articles |

Let’s say you are using a 3rd Party Library like ELMAH in your web site project to handle unhandled exceptions and it’s throwing an exception and you don’t know why.  Since the code is open source, you would LOVE to see what line its crashing on and then submit your changes to the project so that others can benefit from the fix you made.  It gives you that good feeling, right?

Well, unfortunately, you put the compiled DLL in the bin folder and when it crashes, it just crashes and burns.

How can you see the source code?  Simple!  Compile the source code on your machine, and copy the PDB ("Debugging Database") file to your bin folder along with the DLL, and then your web site crashes while in debug mode, it will go to the exact folder on your file system where the DLL was compiled from and show you the exact line of the error.

This was my recent experience with Elmah that it was crashing due to the fact that XMLWriter was misbehaving and throwing an exception when it was supposed to actually replace invalid entities in the XML automatically (at least, according to the MSDN documentation).

Take a look at Issue 43 on Elmah Site for more details.

 

0

Get A Build Process Now!

Posted by Sameer on October 31, 2007 in Software Engineering, Visual SourceSafe |
This article will tell you how to get a build process. It uses CruiseControl.NET to automate the build.  You can use any other tool you like including a .BAT file, but CruiseControl will do just fine.  It’s very easy to set up, it uses an XML configuration file and does most of the difficult job like connecting to your source control, applying labels, and getting your files to build for you.
This software is just great. Its totally free. Really it is a sophisticated batch file (.BAT file). It allows you to set up a build process fairly easily.  It allows you to execute arbitrary processes and check the return codes and then report the results to your developers (or managers, or what not).
 
For example, you set it up to monitor your source control system (say SourceSafe or even a local folder in your file system) every 30 minutes, and then perform a build on it if there is changes. 
It will report who made the last set of changes, what they did (check in, check out, delete, etc) by grabbing that information from your source control and also display the comment from the user that checked it in.  It also labels it in the source control as UNVERIFIED (build process failed), or by incrementing the build number (again, this is configurable).
 
2. Install CCTray (its a little icon that sits in your tray and interfaces to the cruise control server to notify you when the build(s) are broken)
If the build does not compile (according to the procedure you set it up to run such as MsBuild, or Visual Studio, or executing some process) it will turn red and the team will all know that the build is broken and must be fixed. One person can volunteer to fix this build.  Included with CruiseControl.  Or you can just use the web dashboard instead:
 Screenshot from CruiseControl Dashboard
 
3. Perform more advanced setup on it.
You can configure it to do just about anything. Our CruiseControl does a nightly build, runs NDOC (Alpha version, discontinued, you can use SandCastle instead if you like) to get the latest documentation, and also we have a separate project that runs some database unit tests against 5 or 6 QA databases. The NDoc step is a bit tricky if you are running a “Web Site Project” because by default the “Web Site Project” does not create XML documentation files when compiled and requires modifications to your Web.Config. Getting NUnit to run requires some advanced setup, because NUnit requires you to have a copy of the Web.Config in the local folder where the tests are run. You can also run a bunch of fun stuff like Simian (Similarity Analysis, Duplicate Line counter), Fitnesses (another testing framework), NCover (Unit test coverage), etc..  It can also send out emails on each successful build if you like (technically it can do cartwheels if you have an exe file that you can make it call to do the cartwheels :) )
 
4. Involve QA and the rest of the team to start using the build numbers and relying on CruiseControl to verify the site compiles. Also you can start to add more unit tests to your code and strengthening the quality of your product, and then using NCover to verify how much of your site is tested. Not to mention you can also stick FxCop in there! FxCop will analyse your code according to Microsoft Best Standards and create a report for you on bad code that is not according to their naming conventions, or bad code that is not using SQL Parameters (and could have SQL Injection vulnerabilities)

More to come soon. In summary – CruiseControl makes a world of difference. Get it now.

As Jeff Atwood wrote, "The F5 key is not a build process"

0

100 Visitors a Day

Posted by Sameer on October 3, 2007 in Updates |

This site has now grown to 100 visitors a day.. not too shabby :)   But still needs lots of work. I have taken a break from posting for Ramadan but will return soon.

In other news, patch your Subtext 1.5 installation as soon as possible if you haven’t already done so.

4

SourceSafe Branch Recursively

Posted by Sameer on September 24, 2007 in Visual SourceSafe |

Did you know there is no way to branch sourcesafe projects recursively?  Or so you thought!

Here is a work around:

From Visual SourceSafe Explorer, go to View -> Search -> WildCard Search

SourceSafe Wildcard Search

Then search for Wildcard: *
(a single asterisk) with "Search in current project and all subprojects" selected.

Then it will give you a list of ALL of your files in that project.  Simply highlight them all and click on the Branch button.

The only caveat is that you need to make sure the files are not checked out in that project in order to branch them.

And again,.. I would recommend you ditch SourceSafe and go for something better, like Perforce, Subversion, or Vault

 

Copyright © 2007-2012 SharpDeveloper now AgileChai All rights reserved.
This site is using the Desk Mess Mirrored theme, v2.0.2, from BuyNowShop.com.