Joel:

You've written several times about why its a bad idea for an organization to do a total rewrite of a project. I've come up with the following justifications/excuses for doing just that.

1) The Tools Don't Match the Task

The emphasis in software tools and literature is clearly on development and not on maintenance. In fact, its hard to find tools or books that are truly helpful for maintaining legacy projects. Its even harder to find college programs that discuss any aspect of software maintenance.

2) The Hardware Died (or Will Die)

This the problem the FAA has been struggling with for many years. Your legacy hardware is wearing out. You're running out of spare parts; even the technology for making new spare parts is gone. Plus the source code has been lost, or if its still there its in languages no one knows any more. See #3.

3) The Tools Are Dead

The legacy project is based on tools that are no longer supported. In fact the company that wrote 'em is no longer even in business. In fact, no one can find the install CD, or the backup install CD. Gee, when we bought it ten years ago, it really looked like that proprietary 4GL language had staying power.

4) New Technology is So Much Better

You've got a 100KLOC pile of C code that is used to process a 20 Gig database in little slices. All the code started life on MS-DOS eight years ago. Its buggy, badly documented, and incredibly fragile. No test suites exist. You know in your heart of hearts that life would be so much better if you could retarget the entire process to an industrial strength RDBMS and replace all that C code with a few hundred lines of SQL.

5) Your Customers Changed

Your code was developed using an immature process, and just maintaining it is all your staff can handle. Yet your customers expectations are changing, and new competitors are coming out with products that are faster, bigger, and closer to your customers latest requirements.

Actually, I think the problem really boils down to two things: the first is source code that was developed in an immature fashion. The old Netscape source might be an example of that. It simply becomes impossible to make significant enhancements to the code.

The second, and perhaps more profound problem is that in our industry maintenance takes a back seat. What we need are tools and processes that can take away a lot of the tedium from maintenance and make it as fun and challenging as real development.

Greg

-- Anonymous, October 03, 2001

Answers

There are definitely sound reasons for rewriting software from scratch. Code analysis can bear this out. Having said that, the press/analyst (and I do actually agree with Joel on this one) community asserting that rewriting code is somehow a magic bullet or first-choice strategy for dealing with a buggy system is silly. As Joel states, defects will be introduced in any new effort (some new defects, and some defects that the old code may have already experienced and had fixed). One must always look at the cost/benefit. Certainly complete rewrites CAN make sense, it's just that they seldom do.

-- Anonymous, October 03, 2001

> The emphasis in software tools and literature is clearly on development and not on maintenance

There's something called "Refactoring" that's a bit of both (development *and* maintenance).

For example, see http://www.amazon.com/exec/obidos/ASIN/0201485672

The software that I'm working on started 10 years ago as a DOS TSR; now it's an NT Service, and not one line of original code remains ... yet throughout those years we had e.g. three releases per year ... and we *never* rewrote *everything* from scratch all at once.

-- Anonymous, October 04, 2001

Mojmir's refactoring laws

Law#1: You could not refactor dead project.

Law#2: Costs of refactoring increase exponentially with each generation not used.

Explanation: Let's call newest generation of particular technology G. Same technology 1 generation older (G-1) can be refactored to G relatively easy. Technology 2 generations behind (G-2) needs much more refactoring (and costs). Technology 3 generations behind cannot be refactored. It is dead and application should be rewritten from scratch.

Example: If your database application uses: -ADO: you are up-to-date -ODBC: minor refactoring -DB-library: major refactoring -ESQL (Embedded SQL): your product is dead -C-ISAM (MySQL): please beam yourself back to 1980's :-)

Law#3: Lack of encapsulation (OOP) throws your app 1 generation behind.

Explanation: Encapsulated ODBC can be refactored to ADO easily, rest of app will not notice at all. ODBC spread across all sources is major refactoring problem.

Law #4: Never refactor more then 1 major component at the same time.

Explanation: You could refactor front-end, or back-end, but not both at the same time.

And now let's apply these laws to never published Netscape Navigator 5.0: -NN 4.x was 1 generation behind IE 4.0 - HTTP 1.1 versus 1.0, HTML 4.0 versus 3.2, etc. -NN 4.x was an un-encapsulated mess

According to my laws above NN4.x needed a major refactoring, but NN5.0 could be done, if developers focused at back-end, refactoring (and encapsulating) low-level protocols (FTP, HTTP) and mid-level (HTML parser). But in such a case Netscape developers couldn't focus on their preferred activity - rewriting standard Windows dialogs :-)))

Mojmir

-- Anonymous, October 13, 2001

Christopher Wells wrote

> There's something called "Refactoring" that's a bit of both > (development *and* maintenance).

Been there, done that. Last year I conducted a book study on Refactoring. Example #4 is not far from my current business life, and when you have a large legacy application that has no test suites at all, refactoring becomes just another buzzword. After all, which sounds better, to write thousands of lines of test suite code, or to develop proper tests in tandem with a total rewrite? More importantly, which project would the average developer want to work on?

-- Anonymous, October 15, 2001

Greg Compestine wrote

> when you have a large legacy application that has no test suites at all, refactoring becomes just another buzzword

Well, I don't think so; like I said, I think that where I work I/we have been 'refactoring' (successfully, in the sense that we're still in business) for about 10 years.

The 'buzzword' comes about because, now that a book has been written about it, I can give 'refactor' plus a book reference as a convenient short-hand description of such a process.

Re. 'test-cases', I think that the book did three things: * promote the idea that refactoring can be used for development and maintenance simultaneously * catalog various refactorings (if that's a noun) * provide a 'formal' method for refactoring, for what that's worth; a benefit of test-cases (if I recall correctly ... I lent the book to someone else, and he hasn't wanted to give it back yet) might be that they're a useful adjunct to machine-proven correctness.

I won't say that I or we don't test at all, but I think I refactor 'by eye' or 'by inspection' instead of 'by testing' (and then only, for example, retest the final build).

In fact the other main type of 'testing' that I always do is use the compiler and linker as a 'test tool': using C++ 'type safety' and similar techniques, with confidence that "if it compiles and links, then it probably runs correctly", etc.

> More importantly, which project would the average developer want to work on?

I really don't understand that question. Who knows what an 'average developer' is? Also I'm paid to do what I think is best for the software (or for the product, or the for customer or company), and not 'what I want to work on' whatever that means.

I'm not saying that your original post (5 reasons to do a total rewrite) is always wrong; just that 'refactoring' may also be useful, and is at least one example of literature re. "maintenance".

I have certainly sometimes done a "total rewrite" ... of a subsystem.

Also, re. "Mojmir's refactoring laws, Law#2": I don't see why the costs of refactoring should increase *exponentially* with each generation not used; perhaps it increases only linearly. :)

Christopher

-- Anonymous, October 15, 2001