First reading

The empirical software engineering group is headed by Premkumar Thomas Devenbu. He invited me to sit in on their weekly readings. The paper we discussed yesterday was An experiment about static and dynamic type systems: doubts about the positive impact of static type systems on development time (http://dl.acm.org/citation.cfm?id=1869462). It was fun to participate in the discussion but I found this paper frustrating since I find the conclusion utterly unremarkable.

The experiment had two groups of students create a scanner and parser. One group used a statically typed language while the other used a dynamically typed language. Given the anecdotal evidence, the expectation is that the statically typed language group would achieve superior results.

The starting point of the experiment is the hypothesis underlyingthe work in [23, 7]: the application of static typesystems reduces the development time of programmingtasks.

The conclusion is that the dynamically typed language produced the better results. The conclusion was that static typing did not have a positive or negative affect on development time.

My first issue with the paper is the underlying motivation for the experiement. The experiment was for a task done by a single student for a single result over a period of about 27 hours development time. He states that the motivation is to provide empirical evidence for decision makers who must decide whether to employ static or dynamic typed languages when given a choice. I fail to see how one can seriously argue that the results of this experiment can give you the evidence needed for that decision except to provide a paper to begin to discuss the methodology by which one approaches the gathering of that evidence as we did.

First, my intuition tells me that typing helps the programmer when the type of the variable cannot be known from its name alone and a design choice is made that is incompatible with the actual type of the variable. As I think Earl pointed out, (paraphrasing and elaborating) the greater the distance between the design decision when the variable is first recognized and the use of that variable, there can be some cognitive slip. If it is intended as one thing (say an integer) but later interpreted as a pointer (assuming that is possible), there will be a run time error in a dynamically typed language but a compile time error for a statically typed language. The greater the distance, the higher the likelihood that a type error can occur.

How much distance can be created between the recognition and use of a variable in this experiment? Distance comes in several forms. First, there can be the lexical distance (Earl's point). If you have a module of 6k SLOC (they exist), the first use can be many screens away from the new use. The harder it is to see all uses of a variable the easier it is to depend on memory, instinct or some other clue. The worse case is when the immediate need overwhelms the other evidence or substitutes for that evidence and causes the programmer to remember falsely that the variable is of one type when it is in fact of another. For those who practice a form of bottom up design, this shift in thinking is likely as their understanding of the problem grows. While this is possible in the experiment, how large could the source code have become? If we are generous and assume a productivity of 100 SLOC per hour, we are only talking about 2.7 kSLOC; hardly an overwhelming burden.

Another source of distance occurs when the person who wishes to use the variable is someone different that the one who conceived it. This is the norm for maintenance programming. It would also be true when the development is a team effort and one member creates modules and publishes an API without sufficient (or unused) documentation. It is even possible when the same person approaches the same task after a break and had decided on a change in design and does not completely refactor the code to properly reflect the change. Here the distance is created by a perceptual shift, the perceptual shift being the change due to different programmers or due to a change in thinking related to the passage of time.

Again, how likely is this to occur in this experiment? A single student working over a relatively short period of time cannot create much distance. The most likely cause would be due to the bottom-up method of design that the student is likely to exhibit and for which static typing is helpful. Yet the size of the task make it easy for the student to be continually aware of most of the variables used in their design without reference; they will be able to contain the entire design in their head.

Another way in which dynamic typing can introduce type errors is when there are a large number of types being used. In an object-oriented language, each class should be thought of as a type. The larger the number of classes, the higher the likelihood the programmer will mis-remember the correct class for a variable.

I see how even a shift in data can result in latent run time errors. Consider that a program could come undone when it ordinarily expects to see a string but instead finds a number. The chain of type inference will lead it to conclude that all subsequent references are numeric and not string.