Software Engineering

Follow The Leader

Sameer Alibhai — Thu, 17 Jul 2008 22:40:22 GMT

In order to succeed as a team, in any sort of team, you have to follow this basic principle, which has been applied and maybe is accepted universally.

The principle is simple.

Appoint a Leader
Leader makes council with the team
Leader makes a decision
Team supports leader in his or her decision

Its that simple. In this way, an organization, a team, a family, or a company can move forward. Every decision that you need to make, is done in this manner. The team will work together for the best solution, but in the end the leader needs to make a decision. Once that leader makes a decision, the team needs to move forward WITH the leader.

This means, the leader doesn’t necessarily dictate, but he or she has gathered input from the team and made a decision. Then they will have to choose a solution and go with it. If the team continues to argue and fight over the decision, progress will be slow. I believe this applies in families too. There has to be a decision maker in the family, for example that is appointed for financial decisions, and then having a discussion or gathering input from the family is great, but in the end one person has to make the decision, and the family needs to be supportive, even if they don’t all agree with it. However, this applies in normal circumstances and there are caveats. There might be some cases where it would be unethical for team members to support a plan if its morally wrong or it goes against everything inside them and they feel it’s a plan headed for disaster.

It pains me that time and time again I see this happen – the leader makes a decision and the team continues to question him at every step of the way – “Why are we doing this”, “Why are we doing this”, and “why are we doing this” instead of understanding that they were appointed for this role (they might be your boss for example) and they are ultimately responsible for the decision, you need to do your job and support them. I have seen some cases where I questioned my manager because I didn't see the wisdom in the decision he made, but in the end it turned out he was right and it was just my foresight wasn't as far as his... So I learned my lesson, be patient, and be a team player.

As a team member, sometimes I have to swallow my feelings and say, “okay I don’t think this is the smartest decision and my idea is actually better, but I will go with you on this”.

From a developers perspective, you can think of this as requirements. Your boss gives you the requirements, you implement it. How you implement is up to you, but you won’t be able to proceed if you start arguing about the requirements. Requirements are requirements, once they are agreed upon, please continue.

So in normal circumstances, if you want to succeed, get a leader, and help him with his decisions, but in the end.. Respect the decision he makes and go with it. (He or she, that is).

The best team isn't necessarily the one with the best players, it's the one that plays together the best.

How to Waste Millions Of Dollars With Outsourcing (or Make Millions)

Sameer Alibhai — Thu, 10 Jul 2008 22:14:17 GMT

To management, the idea of outsourcing sounds very sexy.... The idea of producing the same content (code, or what not) at 1/2 or 1/3^rd the cost is almost a wet dream for management, if I may be so bold. Even though it sounds great in theory, it's actually a very tricky function to master. Here are some things I have learned with my outsourcing experience.

Keep in mind I am not discussing the outsourcing style of passing requirements and getting the end product complete. I am discussing the style of hiring outsiders and working with them on a daily basis.

You have to start by looking at what is the purpose of outsourcing. Is it to save money? Or is it to improve quality? Or is it so that your team can focus on other things? Get this straight first before going any further. My points below are in the context that you are a software company (or at least do some software development) and you are considering outsourcing to save money and cut costs.

From friends, I know that some very popular companies outsource, such as E-Trade Canada, Accenture, and recently the new online T.V. web site Hulu which outsourced its development to China.

Before you start outsourcing, have your process solid - i.e. have regular scrum, know how much code you are generating each week, and so on. It's very important that you have some idea of costs for developing software for your local team. If you have no idea, you won't be able to see if you are really saving money or wasting it.
Know velocities of individual team members so you can measure cost effectiveness of your outsourced work.
Build your estimation skills. Read Joel's article on estimation and his second article on estimation (which is really a promo for his bug tracking software but still worth reading) and Steve McConnell's book on Software Estimation (highly recommended, very easy to understand).
Get smart/able/competent guys. This can make or break your outsourcing project. If you are going to get developers that need baby sitting, then hire a baby sitter on their end to clean up their code, otherwise you are going to waste your valuable resources fixing and re-fixing and re-fixing their code. In this case you might not be actually saving money.
Review their code. Someone on your side is going to have to review their code to make sure that they aren't purposely obfuscating it in order to secure their jobs in the future. I have seen a Flash application that was built in this manner, the team overseas purposely messed up the code in such a way that it would be difficult for others to continue where they left off.
Learn from those who have done it before. If not, you are going to mess up big time, in many ways. Might be worth getting a consultant who has been successful with such projects. Another idea is to find someone who has connections "back home", and go there to see how some of the shops work.
Turnover is really high in India/Bangladesh/ and so on. This is because jobs start at really low salaries (like $200 a month) and go upwards to like $3000 a month (comparable to working in Canada or USA). You will need to find a way to solve this problem. Somehow you will need to get them to commit that guys will not drop like flies. This is so important because there is always an upfront cost to learn an application, and it becomes more as the complexity and lines of code increase.
Consider a cross-cultural learning program, you send some people there for a while, they come over here for a while. A lot of big companies do this. It's almost a must.
For the team overseas, its important to spend your valuable time together in the beginning to ask lots of questions and understand the requirements as much as possible, in case there is a task that you run into questions, then leave it and work on something else.

Hamid, Axosoft CEO claims that Outsourcing is for Dummies. I think this isn't true in all cases, as I have been able to apply outsourcing successfully on some small projects. However, it all depends on the case, and for building complicated software with a (geographically) fragmented team, you may just end up proving his point.

Updating Your Web.Config From Each Build with CruiseControl

Sameer Alibhai — Tue, 17 Jun 2008 23:10:32 GMT

Let's say you have CruiseControl.NET all set up nice and hunky dory. Now you love the fact that it labels your each build, and you want to somehow show that from your project. Here is how you can get your CruiseControl.NET project build number into your Web.Config. This concept can be applied to update any file.

It’s super easy. You will need to add an exec block to your ccnet.config

<exec>

<executable>E:\Userdata\CruiseControl\Tools\MergeWebConfigValues.exe (my custom tool)</executable>

<buildArgs>"E:\Program Files (x86)\CruiseControl.NET\server\Main.state" e:\userdata\cruisecontrol\Dev_Main\Web.config</buildArgs>

<baseDirectory>e:\userdata\cruisecontrol\Dev_Main</baseDirectory>

</exec>

Now all you need to do is write the MergeWebConfigValues.exe

What does it do?
1. It reads the CCNET state file (provided by argument 1) and grabs the last label from there. (The state file is an XML file)
2. It then goes and writes to your Web.Config and updates the build number in there. (or writes to any specified file you like)

Lastly, update your code to read this value by adding some version or about page to read your Web.Config

You now instantly know what version your code is.

Unit Testing on Your Database

Sameer Alibhai — Thu, 22 May 2008 22:09:58 GMT

When unit testing on your database, you will run into a common problem.

Rolling back.

So you want to do some unit tests, and then you want to reset your database back to the nice squeaky clean version that doesn't have half failed unit tests.

So how can you do this?

There are many ways to achieve this.

The best way I found (requires Win XP SP2 or Windows Server 2003) is to use Roy's Unit Testing Rollback Attribute. Simply inherit his class, add a "DataRollback" attribute, and you are good to go. Using some complicated Interception logic and Enterprise Services (COM+) it rolls back all the database work that was done. It's super easy to implement. Here is some sample code that shows you just how easy it is. You just have to download XtUnit (an extension to NUnit) to do this. (Full source code available)

using System;

using System.Data;

using System.Collections.Generic;

using System.Text;

using TeamAgile.ApplicationBlocks.Interception.UnitTestExtensions;

using NUnit.Framework;

using System.Data.SqlClient;

using Microsoft.ApplicationBlocks.Data;

namespace DBTest

{

///<summary>

/// Test roll back functionality.

///</summary>

[TestFixture]

public class RollbackTest : ExtensibleFixture

{

[Test, DataRollBack]

[Category("Database")]

public void TestInsert()

{

//this method will be performed inside a COM+ transaction

//this requires windows XP SP2 or better

//Windows Server 2003 works as well.

string strCnn = "your_conn_string";

Guid random = Guid.NewGuid();

string sqlI = string.Format(@"insert into log4net (message,date,thread,level,logger) values ('{0}',getdate(),'{1}','Debug','Test')", random.ToString(), System.Threading.Thread.CurrentThread.GetHashCode());

SqlHelper.ExecuteNonQuery(strCnn, CommandType.Text, sqlI);

string sqlS = string.Format(@"select count(*) from log4net where message='{0}'", random.ToString());

int rowcount = (int)SqlHelper.ExecuteScalar(strCnn, CommandType.Text, sqlS);

Assert.That(rowcount > 0, "Cannot find {0}", random.ToString());

}

This test passes successfully. What does that mean? The insert and select worked perfectly fine. After that, I did a query and found the database to be clean. So the rollback worked too.

There are also other ways to skin this cat (i.e. to achieve this goal). One is to use Spring Framework and extend their Unit Testing class (AbstractTransactionalSpringContextTests), and they will handle rolling back everything. Here is an article on this topic, but unfortunately I was not able to make it work for me due to some odd reason. Here's hoping you have better luck. If you don't have any transactions, and your code is wired to use Spring, it's still also very easy, you just need to call TransactionManager.Rollback.

You can also try to achieve this using Nested Transactions if you have existing transactions implemented via Spring. But then you have to set up checkpoints and stuff like that.

I have some other ideas on how to achieve this that I will post later, God willing.

Unit test your life!

Sameer Alibhai — Tue, 29 Apr 2008 23:25:53 GMT

If you are not unit testing your code, chances are you are not unit testing in your life.

If you aren’t unit testing, START now! At the very least, do some “manual” unit testing in your code. How can you do this? Well, try running your code on a very basic case. Then try a bit more complicated case. Then another, then another. If you are smart, you are saving these cases using a testing framework like NUnit. If not, well at least you can have some confidence when your manager comes that you tried it comprehensively and that it’s not going to crash on you while you are showing it to him, or even worse, in a demo to the team or to your big boss.

I recently ran into some problems in my life which I managed to solve amazingly well by doing “unit testing”...

First problem, my DVD burner was going awfully slow. I had some complex and messy setup including an external IDE card, two burners, two hard drives, and all I know is that at some point in time something went wrong and it started going really slow. What I don’t know, is how it happened.

Second problem, I was doing some video encoding/rendering, and for some reason it was doing something bizarre and the application VirtualDub kept looping over and over and would never end encoding the file. Again, I don’t know what happened.

How did I solve these problems?

UNIT TESTING!!

For the first problem of the DVD drive. I removed everything from my PC and set up a very basic system which included 1 HD, 1 Burner, etc.. Then when I found this wasn’t working, a quick check online and I resolved the issue which was incorrect DMA settings. It was trying to send all the data through the CPU (PIO mode) instead of directly to the burners, which was causing a massive slowdown. Once this worked, I quickly put together my system again, and checked each case (HD on same IDE channel as Burner, on separate IDE channel, and so on).

With the encoding problem, again, I was very confused, but by unit testing the situation, I was able to resolve it. How did I do that? I tried encoding on a different machine, reinstalled the software, etc, etc, and it was still having problems.

And finally when I started from scratch, I removed the batch encoding, I removed the DiVX processing, and so on, and then made each test pass. Once the test passed, I added another level of complexity, until finally I figured out that VirtualDub was looping infinitely because I had the “segment AVI file” option enabled. I don’t know why this was the problem, but by unit testing, I was able to resolve it.

Lesson to learn? Unit testing (if you can call it that) can really help you solve such issues. Start from the base case, and slowly work back towards what you need. After each case, write down the results.

Microsoft felt strongly enough about Unit Testing that Visual Studio 2008 has built in unit testing (Wahoo!). As well, it integrates nicely even with NUnit or MbUnit (Don’t ask me how, though).

Five Points To Improve Your Estimation

Sameer Alibhai — Fri, 28 Mar 2008 12:32:19 GMT

Software Estimation is tricky business. You are often confronted with complicated technical (or even non-technical) work and asked “how long will it take?” on the spot. You are given fuzzy requirements (or no requirements) with ambiguous definitions and have to work with a code base that is best described as “chaos”. How can you give an accurate estimate with such a difficult environment?

I recently read an excellent book called “Software Estimation: Demystifying The Black Art"

Here are some amazing points that you can use.

1) We should not be pressured to reduce our estimates. Stick to your estimates – whatever they are. If management wants to reduce them, that’s fine, but stick to whatever you put in the first place. Managers trying to reduce estimates is a very silly thing to do, because by looking at the track record of our software industry, you will find that we consistently underestimate, not overestimate! So if you are pressuring your developers to reduce the estimate, you are asking for trouble. Better thing to do would be to see if the estimate is well founded, based on task level estimates, or is the estimate based on unjustifiable assumptions, etc.

2) Underestimating is MORE expensive than overestimating. So if you cannot estimate accurately, lean towards overestimating. Underestimating has an exponential additional cost, whereas overestimating has a potential of a linear extra cost due to Parkinson’s law. This is because underestimating results in lots of other problems such as having extra meetings to justify why you are behind schedule, having extra meetings to decide which features to cut, added stress on developers trying to meet tight deadlines, code having less quality in order to ‘just finish it’ so that developers can go home, having customers get angry at not-ready releases, having other teams who are waiting for the code to slip their schedules too, etc.. If you overestimate, then Parkinson’s law can kick in and developers will ‘fill in the empty time’, or as they say ‘work expands to fill the available time’

3) Task level estimates are best done by the developer who will be doing the work. A task level estimate is defined as a low level estimate for a particular item of work. For example, for building a workflow engine, a task level estimate is “2 hours to add the save button that will commit the changes to the database”.

4) With small teams (say under 5 people), estimates are most accurate when done bottom-up (i.e. from the developers). What this means is your complicated formulas for estimation are not so helpful in such environments.

5) There are many other ways to estimate such as ‘estimation by analogy’ (i.e. by looking at a similar project that was built), and so on, but I won’t explain those right now. The trick in general is to try to quantify as much as possible and leave subjectivity out of the picture. So if you can find out how similar your project is in terms of number of lines of codes, number of features, etc, you will have a better estimate. As well, don’t be fooled into thinking that more “estimation knobs” will give you a better result than less “estimation knobs”. I define an estimation knob as some sort of criteria you are using to estimate, such as “size of team”, “team programmer skills as a percentile of the industry”, “how many burgers they had for lunch”, etc.. It looks like it would give you a better result but in reality the estimate becomes more and more subjective as you add more “knobs”

As well, developers are more likely to try to meet their estimates if they are the ones who gave it, rather than if its dictated to them from above.

Lastly, estimation is a skill we can learn and improve over time. You will find that over the months, your developers will get better at estimating based on past experience.

In summary, stick to giving your developers clear requirements, lock them down, and then get them to break down the requirements into task items, and then estimate for those task items.

How Random is your Random??

Sameer Alibhai — Wed, 05 Dec 2007 13:52:22 GMT

How random is your random?

Computers are very deterministic. What that means is that you put something in, you get something out. In order to get computers to perform "randomness", it is very difficult.

Why is this important to understand? Because we want to write our code properly, if we depend on the random function for some security purpose, such as for generating passwords, we are actually putting security holes in our application without realizing it.

In .NET Using RNGCryptoServiceProvider would give you much better random results than just a Random.Next()

http://msdn2.microsoft.com/en-us/library/system.security.cryptography.rngcryptoserviceprovider.aspx

However, in order to truly randomize your number, you would have to do something like use data from customer mouse movements, or something wierd like that. Alternatively you can use a hardware random number generator such as the one Intel created that uses thermal noise to generate real random numbers

To realize just how complicated this really is, lets look at this quote from the Pokerstars web site on how they shuffle the cards in their software:

SHUFFLE

"Anyone who considers arithmetic methods of producing random digits is, of course, in a state of sin." - John von Neumann, 1951

We understand that a use of a fair and unpredictable shuffle algorithm is critical to our software. To ensure this and avoid major problems described in [2], we are using two independent sources of truly random data:

* user input, including summary of mouse movements and events timing, collected from client software
* true hardware random number generator developed by Intel [3], which uses thermal noise as an entropy source

Each of these sources itself generates enough entropy to ensure a fair and unpredictable shuffle.
Shuffle Highlights:

* A deck of 52 cards can be shuffled in 52! ways. 52! is about 2^225 (to be precise, 80,658,175,170,943,878,571,660,636,856,404,000,000,000,000,000 ways). We use 249 random bits from both entropy sources (user input and thermal noise) to achieve an even and unpredictable statistical distribution.
* Furthermore, we apply conservative rules to enforce the required degree of randomness; for instance, if user input does not generate required amount of entropy, we do not start the next hand until we obtain the required amount of entropy from Intel RNG.
* We use the SHA-1 cryptographic hash algorithm to mix the entropy gathered from both sources to provide an extra level of security
* We also maintain a SHA-1-based pseudo-random generator to provide even more security and protection from user data attacks
* To convert random bit stream to random numbers within a required range without bias, we use a simple and reliable algorithm. For example, if we need a random number in the range 0-25:
o we take 5 random bits and convert them to a random number 0-31
o if this number is greater than 25 we just discard all 5 bits and repeat the process
* This method is not affected by biases related to modulus operation for generation of random numbers that are not 2n, n = 1,2,..
* To perform an actual shuffle, we use another simple and reliable algorithm:
o first we draw a random card from the original deck (1 of 52) and place it in a new deck - now original deck contains 51 cards and the new deck contains 1 card
o then we draw another random card from the original deck (1 of 51) and place it on top of the new deck - now original deck contains 50 cards and the new deck contains 2 cards
o we repeat the process until all cards have moved from the original deck to the new deck
* This algorithm does not suffer from "Bad Distribution Of Shuffles" described in [2]

PokerStars shuffle verified by Cigital and BMM International

PokerStars submitted extensive information about the PokerStars random number generator (RNG) to two independent organizations. We asked these two trusted resources to perform an in-depth analysis of the randomness of the output of the RNG, and its implementation in the shuffling of the cards on PokerStars.

Both independent companies were given full access to the source code and confirmed the randomness and security of our shuffle. Visit Online Poker Random Number Generator for more details.

[2] "How We Learned to Cheat at Online Poker: A Study in Software Security" - http://itmanagement.earthweb.com/entdev/article.php/616221
[3] "The Intel Random Number Generator" - http://www.cryptography.com/resources/whitepapers/IntelRNG.pdf"

Here is an article about how to shuffle a deck of cards: http://www.codinghorror.com/blog/archives/001008.html?r=31644 and in one of the links it explains a big security hole in their random number generation and how it could have been used to leverage thousands of dollars from players.

Here is a snippet of how to get Cryographically safe random numbers:

This will fill in the 8 bytes with a crytographically strong sequence of random values.

byte[] salt = new byte[8];
RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
rng.GetBytes(salt);

Get A Build Process Now!

Sameer Alibhai — Wed, 31 Oct 2007 21:44:11 GMT

This article will tell you how to get a build process. It uses CruiseControl.NET to automate the build. You can use any other tool you like including a .BAT file, but CruiseControl will do just fine. It's very easy to set up, it uses an XML configuration file and does most of the difficult job like connecting to your source control, applying labels, and getting your files to build for you.

1. Install CruiseControl.NET

This software is just great. Its totally free. Really it is a sophisticated batch file (.BAT file). It allows you to set up a build process fairly easily. It allows you to execute arbitrary processes and check the return codes and then report the results to your developers (or managers, or what not).

For example, you set it up to monitor your source control system (say SourceSafe or even a local folder in your file system) every 30 minutes, and then perform a build on it if there is changes.

It will report who made the last set of changes, what they did (check in, check out, delete, etc) by grabbing that information from your source control and also display the comment from the user that checked it in. It also labels it in the source control as UNVERIFIED (build process failed), or by incrementing the build number (again, this is configurable).

2. Install CCTray (its a little icon that sits in your tray and interfaces to the cruise control server to notify you when the build(s) are broken)

If the build does not compile (according to the procedure you set it up to run such as MsBuild, or Visual Studio, or executing some process) it will turn red and the team will all know that the build is broken and must be fixed. One person can volunteer to fix this build. Included with CruiseControl. Or you can just use the web dashboard instead:

3. Perform more advanced setup on it.

You can configure it to do just about anything. Our CruiseControl does a nightly build, runs NDOC (Alpha version, discontinued, you can use SandCastle instead if you like) to get the latest documentation, and also we have a separate project that runs some database unit tests against 5 or 6 QA databases. The NDoc step is a bit tricky if you are running a “Web Site Project” because by default the “Web Site Project” does not create XML documentation files when compiled and requires modifications to your Web.Config. Getting NUnit to run requires some advanced setup, because NUnit requires you to have a copy of the Web.Config in the local folder where the tests are run. You can also run a bunch of fun stuff like Simian (Similarity Analysis, Duplicate Line counter), Fitnesses (another testing framework), NCover (Unit test coverage), etc.. It can also send out emails on each successful build if you like (technically it can do cartwheels if you have an exe file that you can make it call to do the cartwheels :))

4. Involve QA and the rest of the team to start using the build numbers and relying on CruiseControl to verify the site compiles. Also you can start to add more unit tests to your code and strengthening the quality of your product, and then using NCover to verify how much of your site is tested. Not to mention you can also stick FxCop in there! FxCop will analyse your code according to Microsoft Best Standards and create a report for you on bad code that is not according to their naming conventions, or bad code that is not using SQL Parameters (and could have SQL Injection vulnerabilities)

More to come soon. In summary – CruiseControl makes a world of difference. Get it now.

As Jeff Atwood wrote, "The F5 key is not a build process"

How Gmail Records Usage Statistics to Increase Conversions

Sameer Alibhai — Wed, 12 Sep 2007 11:53:34 GMT

Recording statistics are good for any project. You can do it on a macro level or a micro level. Normally, we do it on a macro level (i.e. recording page hits for the entire site.). However, we also sometimes use these same tools - say Google Analytics, to track clicks on certain buttons, etc.

How about building your application with such logic? At a past company, we used this information to record which menu items were most frequently clicked. Just a simple insert into the database on menu item click (web application). This allowed us to find menu items that were infrequently clicked, and we reworded them or moved the more important ones to more visible locations.

The term conversion is used to mean something you are trying to achieve, say a sale, or a signup, or whatever it may be. You can use this idea to monitor the effectiveness of your pages and increase conversions. You can also use this to improve your user interface - say if certain menu items are rarely clicked, it could be a usability issue.

As well, Google applies this trick! They monitor (or so it seems) their "Sign up" button, and they change it multiple times and monitor the effectiveness of each one.

Take a look at the different sign up buttons I have seen on GMail:

This is the simple basic "Sign up for Gmail" link

This is the more fancy "Pick a name" link.

It's sort of like their Google Adwords works, it allows you to set up ads to advertise your site or products, and allows you to monitor the effectiveness of that ad compared to the same ad say worded differently.

The general rule is collect as much statistics as possible, and then decide what you want to do with it. You can always delete it if you don't need it, but most likely it will be an eye opener for you. You can then tweak your UI until you get it exactly how you like. As well, to follow in the footsteps of GMail, you can use it to get more signups (or more generally known as "conversions") for your site!

Growing Pains of a Software Company

Sameer Alibhai — Mon, 16 Jul 2007 12:44:34 GMT

There is a particular small software company that has been in business for more than 10 years that builds a web application similar to say PHPNuke. Here are a few characteristics of this company: They have survived this far using Microsoft Visual SourceSafe for their source control (which is probably okay for small teams but starts to have problems when you start to use it extensively), they have not used any unit testing, they do not have any form of automated testing or regression testing. They have sales in the millions of dollars, and they have about 6-10 developers and 2 QA (as well as many QA overseas). Every time a build is made, the overseas QA team has to "crash test it" (meaning to run the site and click frantically until something breaks). This does not mean there is no test plan, but rather this method of testing is very unreliable and prone to errors. As well the number of customers they have is less than 100, and we can reasonably assume that simultaneously less than 10 users would be logged in for each application. Now that the company is growing each one of these issues is going to bite them in the butt.

The boss might wonder, "if we got along this far without doing any of the following, well... why should we??".. But as a "soldier" on the front lines, you start to see the problems that can come up with this sort of thinking.

Not using unit testing

This means that QA receives a page that is not unit tested. It is ad-hoc tested by the developer, meaning that they try to click and break their new code, but there is no form of automated manner to this testing. As well, every time the code is updated, the developer has to re-test and re-re-test it manually. As well there is no real confidence in the code other than the fact that the QA team has tried their best to crash it and it did not crash. It might even be very time consuming and painful to set up the test each time. Without unit testing, this is a lot of time wasted that could be used in development. Read up on Test Driven Development for something else that you can gain from proper Unit Testing. By the way, what does unit testing software cost you? FREE if you use NUnit. Or like $14,000 if you use IBM Rational Test RealTime. Its up to you. Just Do It.

Not using proper Source Control

Trying to implement branching ,sharing, multiple releases, with Visual SourceSafe! You will find it is very difficult. As the company is growing they needed to be able to release and develop multiple versions concurrently. This is near impossible with SourceSafe, if not very difficult, and they need to move into using a better source control software such as Team System, Subversion, Perforce, etc.. This will give them features such as atomic checkins, shelving (if required), cheap labels, etc..

Not using automated testing

This comes back to the earlier topic of unit testing... Without automated testing, there is no confidence in the code. Each change, even a minor one, can possibly break some unknown piece of code in the system.

To hear a real story, making a change to a general object in the system caused a crash on a Report that was only somewhat related to the general object.

This was not noticed until on a live customer demo, but due to slickness on the part of the presenter, it was overlooked. This is a deadly game of cat and mouse, that can result in lost customer confidence and eventually lost customers. AUTOMATED TESTING is FAR TOO IMPORTANT to not use, especially if your company is growing and growing.

Small number of customers

With a small number of customers, there are some problems that you have in your code that will not be noticed until you start to grow. For example, if your code is not thread safe, and you are using shared/static variables that are not meant to be shared on a specific page, problems that might not show up until you have multiple users loading that page simultaneously. This will happen as your number of customers grow.

As well as load issues. As a small shop, you can throw the most horrible piece of inefficient code and in most cases it will not cause a problem because your server can most likely handle it. As you start to increase in size as a company, and as you have more people hammering your server, your inefficient code is going to start to tear at the seams and you will have to go back and re-organize it. Sometimes code can survive and be in use for more than 5 years so its well worth your time to build it properly. Look at your own code set and see how old it is. If it's been years and years and you are still using it, re-consider that the time you spend to build it properly is well worth it. Tell your boss to make sure he knows who he is hiring, as bad apples can spoil the whole pie.