1 Review of Test Driven Development

Programmers practicing Test Driven Development (TDD) or Behavior Driven Development (BDD) techniques follow a broadly recognizable, somewhat over-simplified, process of software evolution:

Receive a new requirement (in the form of a bug report, user story, etc.)
Understand what's being asked by writing a writing a unit test. Often the requirement covers some functionality which doesn't exist yet. The programmer covers this situation by coding his test either as though the required functionality already existed, or as he wants the desired functionality to work or appear in source code. (Both methods essentially amount to the same thing.)
The programmer invokes the test, making sure it fails in the expected way. For instance, if the coder uses an undefined procedure, then the program should fail obviously with a symbol undefined error. Otherwise, the desired behavior of the procedure should not manifest.
The programmer writes just enough code to make the test pass. Note that this may involve changing procedure signatures — adding or removing parameters, refactoring a big function into two smaller functions, et. al. When the interface changes, all existing call sites must be updated to use the new interface.
The programmer re-runs the test, making sure it passes this time. If it does not, repeat from step 4.
The programmer then commits his or her changes to the project's code repository for safe-keeping, and marks the requirement or story as completed.

The larger the code-base grows, the more difficult it becomes to update all existing call-sites for a given procedure to use the new interface established after step 4. Even a code-base of moderate size might offer an inconvenient number of call-sites for a single developer to remember to handle. Languages with strong, compile-time type checking, such as Java, Haskell, Modula-2, or Oberon, readily provides a listing of these call-sites, albeit in the form of type-mismatch errors, very nearly always during the compile or link stages.

For example, let's consider the following hypothetical Oberon code to create a new file:

MODULE FileUtils;
IMPORT T := CommonTypes, ...;
(* ... *)
PROCEDURE CreateFile*(fileName: T.String, accessModes: Modes): File;
VAR     aFile: File;
BEGIN   NEW(aFile);
        (* ... *)
        RETURN aFile;
END CreateFile;
(* ... *)
END FileUtils.

Some time later, you receive a new requirement: CreateFile must work without having a filename, in essence creating a temporary file in a RAM-disk. Two basic approaches towards implementing this requirement exists: one backward compatible, and one not. The backward compatible change involves simply checking to see if fileName happens to be NIL. This covers perhaps 99% of the use-cases, but it's not clean — creating a file with a nil name simply isn't the same thing as creating an unnamed file.

The backward incompatible, and more correct, approach involves refactoring the procedure into two smaller, more orthogonal definitions.

PROCEDURE CreateFile*(accessModes: Modes): File;
VAR     aFile: File;
BEGIN   NEW(aFile);
        (* ... *)
        RETURN aFile;
END CreateFile;

PROCEDURE SetFilename*(fileName: T.String; VAR theFile: File);
VAR     diskResidentFile: File;
BEGIN   NEW(diskResidentFile);
        diskResidentFile^.fileName := fileName;
        (* ... *)
        CopyFileContentsFromTo(theFile, diskResidentFile);
        DeleteFile(theFile);
        theFile := diskResidentFile;
END SetFilename;

Now, creating a genuine file involves two calls, not one: first you need to CreateFile, then you slap a label onto it with SetFilename. If we were to make this change in our module, Oberon would fail to link and/or load any client of FileUtils until such time all references to CreateFile have been updated to the new signature. In most cases, it would additionally provide source file and line number information, enabling the programmer to rapidly fix each call-site.

However, there exists a plurality of useful software written in dynamically-typed or even completely type unsafe languages, such as Python or Forth. Can a programmer use the tools available to him or her to acquire the list of unadjusted call-sites in these languages too? Yes.

2 Serial Numbers with Eager Name Resolution

For languages with eager name resolution, such as C or Forth, a program cannot invoke a procedure without knowing the name of that procedure at compile-time and/or link-time. Therefore, simply removing the procedure's definition suffices to coerce an otherwise type unsafe language implementation into divulging the locations of all the procedure call-sites it knows about. I advocate simply appending a serial number to the procedure's symbol name.

Let's suppose we implement CreateFile in Forth instead of Oberon, chosen for its total lack of type safety. We'll start with a sketch of the original code:

: CreateFile ( modes fileName -- aFile )   ... ;

We then refactor our code:

: CreateFile ( modes -- aFile ) ... ;
: SetFilename ( fileName aFile -- aFile' ) ... ;

Because the signature of CreateFile has changed incompatibly, but still uses the old procedure name, compiling out-of-date client modules will result in a run-time program error (typically manifested as a stack overflow error perhaps many seconds, minutes, or even hours after the actual error occurred!). Remember these old modules push two items on the data stack, while the new code consumes only one.

The solution simply involves changing the name:

: CreateFile1 ( modes -- aFile ) ... ;
: SetFilename ( fileName aFile -- aFile' ) ... ;

Now, no definition exists for CreateFile anymore, so when compiling the older modules, Forth will issue a compile-time error with useful location information.

What if your procedure already has a serial number affixed to it? After making the incompatible change, we simply bump the serial number. For example, let's suppose a third requirement comes in asking for CreateFile to support an initial access control list, to help secure the file against unauthorized access. This brings our stack utilization back up to two elements, but the semantics behind those two elements differ significantly from the original definition of CreateFile. Therefore, we adjust our procedure name accordingly:

: CreateFile2 ( modes permissions -- aFile ) ... ;

Now, not only will call-sites using CreateFile fail to compile, but so too will CreateFile1 call-sites!

3 Camel-Caps with Lazy Name Resolution

Some languages, such as Python, do not resolve names at compile-time; doing so would ruin some of the great advantages Python's object model has to offer. At first glace, this suggests we cannot use the serial-number approach, particularly if the procedure or method name changed exists in its own module.

Using a tool such as pychecker may provide one solution to this conundrum. pychecker will parse your Python module and its dependencies, looking for tell-tale signs of problems, including but not limited to misspelled or non-existent procedure or method names. However, pychecker cannot always determine whether it should generate a warning or error. Consider the following piece of Python:

#!/bin/env python

def aProcedure(anObject):
    return anObject.someMethod()

This example demonstrates completely valid Python code, and will work fine provided the object passed in the anObject parameter supports the niladic someMethod method. However, how can pychecker confirm that someMethod exists? Even if we rename someMethod to something else in a local module, no guarantee exists that a module external to our project won't use aProcedure on an object of its own. Hence, because pychecker cannot prove someMethod() doesn't exist, it will not issue an error.

Another approach involves changing the number of parameters passed to the method. If adding parameters, they needn't actually be used. They exist solely to get pychecker to recognize the discrepency. However, again, pychecker remains ignorant of the big picture, and runs into similar problems as before.

Unfortunately, without a pychecker-like tool that recognizes some kind of optional type annotation, you may find that grep remains your best tool for locating out-of-date code. Since grep will happily find substring matches, simply tacking a serial number at the end of the procedure name may not be conveniently sufficient.

In the English language, all letters of the alphabet contain two cases. By default, grep performs a case-sensitive search. Thus, we may embed a procedure's serial number into its name through varying the cases of each of its letters. For example, to re-use the above example, we can generate a sequence of names as follows:

aProcedure (original, official name)
aProcedurE (1st backward-incompatible change)
aProceduRe (2nd such change)
aProceduRE (3rd)
aProcedUre
...etc...

Following this convention preserves the desired characteristic of being able to locate specific call-sites of an earlier procedure interface, while concurrently working well with the grep tool. A symbol of length n has 2ⁿ possible unique case combinations, so even relatively short symbols tend to have sufficient quantity of combinations between code check-ins. You know you've completed retrofitting existing code when the result of a shell command operating over your source code, such as the Linux or BSD command find . -type f | grep .py\$ | xargs grep aProceduRE | wc -l, yields a zero result indicating no lines of code using the old procedure name can be found.

5 Documentation Benefits

You should also run a quick search over your program's documentation to ensure that it's kept up to date. For example, invoking the Linux/BSD command find docs -type f | xargs grep -in aProcedure | grep -v aProcedURE will yield a listing of all references to aProcedure which do not match the current camel-cap embedded or explicitly appended serial number. For each of these occurences, resolve the documentation to take the new functionality into consideration, making sure to alter the procedure name in the documentation to the current name as it appears in the source tree. Do not worry about the camel-cap mutilation or serial number; this will be fixed using the same basic methods as a final step prior to code check-in.

As a footnote, I would even propose altering the name of a procedure even with fully type-safe languages, with the explicit goal of supporting easily discovered references to the procedure in large bodies of documentation. I find keeping documentation up to date incrementally, as a deliverable of the development process like any other, proves significantly easier if you don't have to wade through large bodies of irrelevant or marginally relevant text.

6 Finishing Up

When you find yourself ready to bundle your software for QA testing or for production deployment, you will want to remove the serial number or case mutilation from your procedure's name. This final step effectively resets the state of your symbol back to normal, so that the next round of development can start with a new sequence of serial numbers.

7 Conclusion

I've detailed a very simple method of invalidating all call-sites of a given procedure, such as the plurality of sites found in various unit tests, whose signature has changed in a backward incompatible manner. Since even type-free languages depend on a name to locate which procedure to call, altering the name to a name never used before guarantees compiler errors which can help in locating incompatible code. For languages which lack compile-time type checking and which defer name lookups until run-time, external tools supply the desired pre-run checking.