Factors that Affect Software Systems Development Project Outcomes: A Survey of Research

by McLeod & MacDonell from ACM Computing Surveys, Vol 43, No 4, Article 24, Publication date: October 2011

This is the next paper we will discuss and the one that I most viscerally connect to. I'm not sure how we will discuss it since it's scope is so wide. Yet out of this scope I see so many opportunities for further research that I'm overwhelmed. The biggest challenge I'll face will be finding a small enough topic to support my research without it blowing out of control.

Since I don't have enough experience with this group to know how Prem will want to parse this paper I'm at a loss to know how to prepare myself for the meeting when we get together in two weeks. So I'll first pass over this paper contemplating my interest in each of the areas the authors identify. I'll go back afterwards and consider their methodology for finding these papers.

The novel contribution of this paper is the inclusion of a category the authors call People and Action in addition to the categories of Development Processes, Project Content, and Institutional Context. While I am personally in touch with these factors as vectors to be studied in the analysis of project success predictors, I was surprised to see them included in a paper that we are studying since I had thought these might stray too far outside of ESE. Perhaps they do. That is a question I will ask when we meet. Much of this overlaps with management, especially project management (Drucker) or social systems, group dynamics, etc. There is no doubt that all of these can impact project outcome but where will we draw a line between these other disciplines and ESE? What is the tolerance for the journals and conferences for a paper on the impact on project budget due to cultural collisions? That sounds so completely social science to me that I find it hard to see how some of the major venues will accept it. But I have a lot to learn.

The authors call their first category Project Content and consider these properties of the project itself. So the charter, mission, scope, and goals to be Project Content factors. They also include the resources of the project (why do they phrase it as "the resources it attracts"?) To them, this includes the software and hardware used (to me, the tools). To me this would include the IDE, any CASE tools, the languages adopted, the processing power dedicated to this project. For this survey they exclude people since they are included in the new category.

Their second category is Development Processes. It is surprising to me that they choose to place requirements determination in this category while including the goals, and scope in the prior category. I have always understood requirements to be an elaboration of the mission and goals of the project extended into more concrete terms. However I do clearly understand their placing things like the method to be used (methodology) which comes from outside the project. There is clearly some overlap here between processes and external constraints. A methodology may be imposed as an institutional constraint or it could be something that is chosen by the project manager since there is no imposed requirement for a given methodology by the organization. It isn't clear to me how the authors chose to deal with papers that may arguably be within these overlaping areas. But if their intention is simply one of making sure they included all the papers perhaps the categorization is not important.

From a system's perspective I can understand how this is a workable categorization for the authors' purpose but there are some nuances that I think are important for the very large organizations. First, there is not a single layer of projects within organizational contexts. A project may be part of a program which itself may be within the context of a long-term architectural plan for the organization. Also there are various levels of organization which are often involved. The users may belong to a department in a business unit which is part of a subsidiary that is owned by some conglomerate. It is not uncommon for a project manager to strategically chose to ignore some organizational constraint. He may know that there is no enforcement of that requirement, have more power and influence in the organization than the area that is trying to impose that requirement or perhaps by making an appeal to the inapplicability of that constraint on this project. This is analogous to Daryl's paper on ecology where the proper level for the analysis must be found before it becomes meaningful.

One aspect of this hierarchical embedding intrigues me. We are spending more time talking about software systems. I always take this to really mean software-intensive systems since I am hard pressed to think of any software system that provides value in the total absence of hardware. (The very fact that we can now study software divorced from the hardware that it runs on says volumes to me about how far we have come). So there is a software embedded in hardware and it is this combined system that generates some value. However from a change management perspective it was never sufficient to allow this to be the only scope of study and analysis. Any system is contained in some larger context and it is often the qualities in use that are important for analysis and measure. This necessitates including at least the people who directly interface with the system and possibly others. The focus becomes tasks that are done with the assistance of the system. How else can usability be understood if not within this context?

While the organizational scopes add an interesting way of parsing out utility from the system, some qualities are only understood when viewed longitudinally. Modifiability, to name one, can only be measured over a period of time that includes enough transformations of the system to make some observation about how well this system meets the quality requirement it was given.

(more later)

One sentence sticks out to me on page 5 at the bottom. "While generic influences are common to a range of systems development contexts, they manifest themselves differently in specific situations." I suspect I'll need to read the cited paper to understand their point.