The Cost of Ineffective Search

Great article in Network World on the Cost of Ineffective Search. The author starts with a strong punch line:

A company that employs 1,000 information workers can expect more than $5 million in annual salary costs to go down the drain because of the time wasted looking for information and not finding it, IDC research found last year.

You think that's bad. think again:

A survey conducted by Accenture this month of 1,000 middle managers found that more than half of the information they find during searches is useless.

• The Butler Group estimates that 10% of a company's salary are wasted on ineffective searches.* Susan Feldman at IDC found that 3.5 hours each week are wasted on search that don't turn up the right information.
• People spend about 9-10 hours a week on average looking for information and don't find the information they are looking for 1/3 to 1/2 of the time.

So why is search so ineffective?

1. Enterprises are not investing much in search
Most companies are under-investing in search compared to other systems such as portals, dashboards, databases and other enterprise systems and are not using the latest search technology. The best search applications use concept searches and very few companies have adopted such technology.

2. 90% of the documents that are created have no useful metadata
Companies for the past 10 years have found it extremely difficult to get employees to add metadata to content.

3. The right data sources are not getting indexed
Companies need to figure out what search is used for (customer service, eDiscovery, etc.) and adapt their search strategy.

The Answer?
The semantic web is viewed as an answer to ineffective search. Recent research from MIT Sloan shows that using semantic search technology will turn up most desired results about 80% of the time compared to 50% of the time with search technology used by most companies today.

Improved metadata through a non-intrusive tagging process will definitely contribute to improved search as well, further indicating the value of social tagging.

Cost benefits of Unit Testing

Many definitions are available for unit testing but, in simple terms, a unit test is a method for testing the correctness of a particular module of source code. Sounds like something most software development organizations would want to do... In the real world, however, the issue of cost benefits often comes up. It is indeed very hard to talk about the benefits of unit testing without knowing the answer to the following question [1], [2]: How many defects did unit tests avoid, how much time was saved and, how much time and defects will be saved in the future?
Over the next few weeks, I will blog on the cost benefits of unit testing and how to quantify such benefits. Commonly quoted benefits of unit testing are [3], [4], [5], [6]:

1. Problems are found early in the development cycle.
2. Code that works now, will work in the future.
3. New features will not break existing functionality.
4. Making change becomes easier, as controls are in place.
5. The development process becomes more flexible.
6. Implementation design is improved as APIs are forced to be more flexible and unit-testable.
7. Bringing new developers on board becomes easier and improve teamwork. Unit tests document the code.
8. The need for manual testing is reduced.
9. The development process becomes more predictable and repeatable

However, quantifying such benefits is often challenging for organizations. Still, everyone agrees that the cost of fixing bugs or changing software increases exponentially the later issues are uncovered in the software life cycle. In a development process where identifying bugs is the responsibility of the quality assurance (QA) group, the QA group risks to run into
bug indigestion. Unit testing is critical to prevent unit-level issues to be uncovered later in the software development life cycle. Software methodologies that rely heavily on developer unit-level testing can therefore achieve a much lower cost of ownership throughout the software development life cycle.

Enforcing unit-level testing throughout the development process should be a major focus of any management team involved in managing the delivery of a software product or solution. Identifying the optimal amount of unit-level testing to maximize the benefits of writing unit tests is hard to measure. Best practices suggest that the ratio of test code to code under test required to achieve at least 90% code coverage is between 2/1 and 4/1. This means that to thoroughly test a 100-line Java class requires 200 to 400 lines of test code. The higher the unit-level test coverage, the better the quality. This suggests a best case scenario and does not necessarily maximizes the return on writing unit tests. My experience suggests that the benefits of unit testing can be achieved much sooner but also suggests that there seems to be a threshold above which the benefits of unit testing can be most felt. Let's take, as an example one of the projects I recently completed. As illustrated below, the initial phases of QA resulted in a flood of bugs. At the same time, levels of unit tests were insufficient. I would draw a first lesson from this observation:

To maximize, the cost benefits of unit testing; start writing tests early. Focus on "quality" assertions where assertions are performed against relevant data.

As testing went on, I believe that we can clearly identify the point of inflexion where the value of unit tests really shows. As illustrated below, as our unit tests level improved, our development team was able to significantly improve our bugs closing rate without introducing new issues and therefore avoid a bugterial infection. Therefore, the second key lessons:

Good unit-level test coverage allows projects to significantly shorten their bug-fixing cycle. Therefore, resulting in direct cost benefits.

Quantifying cost benefits depends on the type of project under scrutiny, however simple empirical data clearly demonstrates such benefits. As a result if you are a developer, you should be writing unit tests today; if you are a manager, you should be driving adoption of unit-level testing practices. The quality of your software and your ability to respond to your customers demand will significantly improve. Be agile, today!