Software Art Thou: A summary of the lecture by Glenn Vanderburg from Real Software Engineering

Software Art Thou: Real Software Engineering
A summary of the lecture by Glenn Vanderburg currently VP Engineering at First.io

https://youtu.be/RhdlBHHimeM

Zendesk published a Youtube in January 2018 in which Glenn Vanderburg talks about the nature of Software Engineering and offers some insightful comments. Since many of his points are important to the research I am doing I am creating a summary of this Youtube to assist in citation. Parenthetic remarks are time m

Where SE went wrong and how to fix.
(0:27) Programmers: Stop Calling Yourselves Engineers, The Atlantic. Are SEs engineers?
Software Engineering: An Idea Whose Time Has Come and Gone?, Tom DeMarco, Computer Magazine, ACM? IEEE?
"Calling what we do 'engineering' doesn't make us engineers. It makes us liars, Dave (Pragmatic) Thomas, Extreme Programming mailing list at 2003
(2:04) Arguments against SE as engineering reduce to charge that it is fundamentally different from other engineering or that it is more artistic than engineering which essentially is the same argument.

The term Software Engineering can be traced to a NATO publication from a conference in Garmisch Germany in Oct 1968. And then Rome 1969.

He calls the 1969 the launch of Academic Software Engineering and was analytical, formal and aspirational toward engineering.

All academic software engineering (ASE) modeled process models of large scale engineering projects. Requ

Allister Colbert, was proj successful? how strictly to chosen process? PM differed from coders. Would you use this process again? (mostly no)

good people who cared were motivated to subvert process

in mid 90s, attempt to find new process, Agile
less iterative, experimental, less formal, correctness by testing, rapid adoption.
Now two views: software development should be more like engineering and software development is NOT like engineering.

Both ignore what engineering process really is.

Herbert Simon, The Sciences of the Artificial,
A Rational Model of Software , Parnass

(12:50) Glenn's view:
Software development already is engineering
it always was
the rational model (Parnass) is a caricature of engineering since no engineering is really done that ways and some look nothing like that
We don't need to become engineers but refine the software engineering process which does not have to adhere to the academic software engineering processes that were first suggested.

Remainder of talk:
the errors of ASE, what is engineering?, what is SE?, what's next?

(15:00)
The Errors of Analy\tical software engineering
Brian Randell, newcastle ed 1968
history of software engieering conference 1996
At that history of SE cibnference, he said:
"Unlike the first conference, at which it was fully accepted that the term softare engineering expressed a need rather than a reality, in Rome there was already a slighgt tendency to talk as if the subject already existed. And it became clear during the conference that the organizers had a hidden agenda, namely that of persuading NATO to fund the setting up of an Internal Software Engineering Institute. However things did not go according to their plan. The discussion sessions which were meant to provide evidence of strong and extensive supooret for this proposal were instead marked by considerable scepticism, and led one of the participants, Tom Simpson of IBM, to write a splendid short satire on 'Masterpiece Engineering'. IOt was little surprise to any of the participants in the Rome conference that no attempt was made to continue the NATO conference series, but the software engineering bandwagon began to roll as many people started to use the term to describe their work, to my mind often with very little justification. ... I made a particular point for many years of res\fusing to use the term or be associated with any even that used it."   Brian Randell

Early dissents

Alan Perliss. in 1968
A softwae system can best be designed if the testing is interlaced with the designing instead of being used after the design. (TDD)
A simulation which matches the reqrements contains the cotrol which organizes the design of the system.
Througb successive repetititons of this process of interlaced testing and design the model ultimately becaomes the software system itself. ... in effect the testing and the replacement of simulations with modules that are deeper and more detailed goes on with the simulatin model cotrolling, as it were, the place and order in which these things are done.

Fred Brooks, The Design of Design

Winston Royce agreed that waterfall seems like it should make sense but then went on to criticize it in the same paper. (Managing the Development of Large Software Systems http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf)

"John H. Van Vleck, Nobel-laureate physicist was Dean of Harvard's Division of Engineer and Applied Science when I was a graduate student there, in Aiken's lab. Van Vleck was very concerned that the practice of engineering e put on a firmer scientific basis. He led a vigorous shift of American engineering education away from design toward applied science. The pendulum swung too far; reaction set in; and the teaching of design has been contentious every since. I am grateful that three of my Harvard teachers never lost sight of the importance of design and taught it: Philippe E Le Corbeiller,  Hary R Mimno, and Howard H Aiken, my adviser."
Preface to The Design of Design by Fred Brooks

(19:47)
Devalued the role of code.
What is needed is not classical mathematics, but mathematics. Systems should be built at levels and modules, which form a mathematical structure." Friedrich Bauer

[...] our basic tools are mathematical in nature. -Edsger Dijkstra

David Parnas 2007 oopsla, "In engineering...people design through documentation,", "Code cannot fill that role."

Academic software engineering emphasized correctness over cost.

(23:27)
graph of cost of error correction versus phase of project where error is found. But, Glenn says, emphasis on careful review of all artifacts at all stages increases costs throughout since more reviews, more quality control is needed. "The cure is worse than the disease." Low cost, high impact. Cost of long feedback loops.

(25:00) What is engineering anyway?
"Unlike the extensive analysis of the scientific method, little significant research to date has sought the philosophical foundations of engineering. Library shelves groan under the weight of books by the most scholarly, most respected people analyzing the human activity called science. No equivalent body of research treats the engineering method." -Billy Vaughn Koen

8 good books on engineering process

The sciences of the Artifiicial, Herbert A Simon
The Design of Design, Gordon L Glegg
Definition of the Engineering Method, Billy Vaughn Koen
To Engineer is Human, Henry Petroski
What Engineers Know and How They Know It, Walter G Vincenti
Engineering and the Minds's Eye, Eugene S Ferguson
The Existential Pleasures of Engineer , Samuel C Florman
The Design of Design, Fred Brooks


[Programming] is not some kind of engineering where all we have to do is put something in one end and turn the crank." -Bruce Eckel

(or paraphrased, it is not algorithmic, nor can it be)

Engineering process is inherently creative, exploratory, and will always have false starts, dead ends, and learning processes.

"The Engineering Method is the use of heuristics to cause the best change in a poorly understood or uncertain situation within the available resources." -Billy Vaughn Koen

Different engineering disciplines are different. Process Control Model, defined at one end, empirical at the other.

"The defined process control model requires that every piece of work be completely understood. A defined process can be started and allowed to run until completion, with the same results every time. The empirical process control model provides and exercises control through frequent inspection and adaptation for processes that are imperfectly defined and generate unpredictable and unrepeatable outputs." -Ken Schwaber and mike Beedle, SCRUM

Engineering which has expensive construction requires are defined.

"The conversion of an idea to an artifact, which engages both the designer and the maker, is a complex and subtle process that will always be far closer to art than to science." -Eugene S Ferguson

(31:18)
"At first sight the idea of any rules or principles being superimposed on the creative mind seems more likely to hinder than to help, but this is really quite untrue in practice. Disciplined thinking focuses inspiration rather than blinkers it." -Gordon Glegg

(38:00)
Deflection theory for suspension bridges. Tacoma Narrows bridge. Deck so thin that it had aerodynamic properties that exciting resonant frequency.

Engineers adopted formal methods to save money. Predicting failure point makes it possible to reduce safety factor.

Structural analyses (indeed any engineering calculations) must be employed with caution and judgement, because mathematical models are always less complex than actual structures, processes, or machines." -Eugene S Ferguson

Engineering driven by practitioners but refined by academics.

Robert Hook, 1671
(45:51)
Sagrada Familia

(49:00)What is Software Engineering?
Structural engineering is the science and art of designing and making, with economy and elegance, ... structures so that they can safely resist the forces to which they may be subjected.
-Structural Engineers Association


In structural engineering the engineers create diagrams and specifications which are given to builders. Historically we map this onto SE by thinking of the designers as creating specifications for the coders. But this hasn't worked well. Are coders really builders?

The source code is not the solution. It is a design document. It is what the compiler that constructs the system that actually provides the solution.

For structural engineering the most expensive phase was construction. But for SE the compiling is the cheapest. With continuous integration this phase is effectively free.

Jack Reeves, what is software design? (http://www.developerdotstar.com/mag/articles/PDF/DevDotStar_Reeves_CodeAsDesign.pdf)

Dave Thomas said XP processes tightly coupled.

Kent Beck wrote first XP book

If you look at the 13 processes, they can be arranged along a continuum of the amount of feedback they give and the scale at which it is given. From the processes that provide the most immediate feedback with the smallest scope is pair programming. At the other end we have practices that give valuable feedback but at a large scale of time and effort in short releases.

pair programming
unit testing
system metaphor
continuous integration
on-site customer
collective ownership
acceptance testing
planning game
short releases.

One suggestion: don't use languages that enable mutable state! (ha)

platform/application divide.

Glenn: stop using unsafe languages for applications.

formal methods to prove validity and security of small micro-kernels. formal methods now practical for well restricted domain.

Glenn Vanderburg, First.io, glenn@vanderburg.org or @glv

What Does Software Look Like?

I am reading the book, Engineering in the Mind's Eye by Eugene S Ferguson. The book is an in-depth discussion of the design process as found in many areas of engineering. And the focus appears to be on how those forms come to exist. He frequently cites the many paper models of the project to be created. And he stresses the visual component of how the designer tries to document what the finished product and its components look like. But how do I apply that to software engineering? If I am going to specify a system, what images help? What does software look like?

The difference between the surface appeal of a product and the inner workings is not unique to software of course. Something like a motor presents the user with a largely inscrutable object which turns an armature. But we all know that the inner workings of such an engine, or pump, or even an auto are complex. At the extreme we have the computer which has a relatively trivial surface appearance with a screen and a keyboard. But we all know the enormous complexity that exist under the cover. By analogy it is not much different than having a modest cottage that has all the inner workings of a large modern skyscraper. The appearance belies the complexity.

It is not as if the surface appearance does not matter. Quite the contrary. Working out the human-computer interface is a very important part of the design process. Not even as much because of the importance of that interface for the successful operation of the product, but because it is the way that most users can most easily explain the expected behavior, and the complex inner state transitions that are expected.

Some software product however have no human-interface design at all. Embedded systems have an interface of course. But that interface has no user perceived behavior. It becomes a piece of the infrastructure like the plumbing that we never want to deal with. And as long as it works, we don't need to.

Yet these are all important applications of software technology. And we must find a way to understand how to specify the software we need and communicate that to those who can build that for us. And much is the design literature leads us to believe that this division of designer and implementor is something artificial even while scale may require some specialization by skill.

So in this book I am hoping to find the start of some answers about what software looks like. When I engage in creating a blueprint for a software system, what should that look like? What should be visual? Are UML diagrams the right way to communicate intentions? What must be tabular? or do we need new extensions to computer languages to capture the information that right now seems to imperfectly flow from designer to builder in the processes we use today?

Software Engineering and Society

The Atlantic ran a provocative article a few years ago and I take strong exception to the points it makes. Titled Programmers: Stop Calling Yourselves Engineers (https://www.theatlantic.com/technology/archive/2015/11/programmers-should-not-call-themselves-engineers/414271/) I can agree with the point that much software engineering fails to live up to the aspirational goal of being like structural engineering. But by painting with such broad strokes the article does more to set that aspirational progress back than it does to propel it forward.

First let's consider the basic metaphysics of what a software engineer must do. Advances in steel are small and incremental. Yet changes in hardware design are still following Moore's law many decades after it was first recognized. In addition operating systems have grown in size, complexity and function just as fast as the underlying hardware. How is it possible to imagine the kind of well defined processes we find in structural engineering we see today if those engineers had such dramatic changes in their raw materials? Or if the scale of the structures they built kept pace with the growth of software systems? We would not have the dependability we see today. To suggest that software engineering is failing because it is not on par with structural engineering is a destructive false equivalence.

Next the author talks about how society has regulated some engineering disciplines. Yes, some engineering areas are subject to laws, have well defined apprenticeship tracks and continuing education requirements. But to assume that the mere adoption of those superficial forms would somehow achieve the desired effect of creating fewer software engineering failures is naive at best. Those professions themselves evolved those norms after decades, or even centuries, of practice. Software engineering today bears very little resemblance to the practice of even 30 years ago. We continue to experiment with the processes by which products are created. And the needs of the clients are so disparate that it is obvious the methods we use to created embedded systems for high safety applications bears only the vaguest similarity to the free social media apps the author cites.

Another false equivalence in the article is the criticism of the data breaches that are now getting so much criticism. The way in which we are using computer systems today is far more radical than the way we are using bridges. If society and its institutions continue to use software in novel ways we must expect that we will uncover new problems. Few thinkers in 1968 could have anticipated the kind of cyber espionage and warfare we must prepare ourselves for today. To place the blame for these failures at the feet of the software engineers who were as trusting as the people who commissioned them is hardly fair.

The author's charge of "engineerwash" is not fairly placed against engineers. Like all attempts to manipulate public trust this needs to be assessed against the organizations that profit from the work of those engineers. Engineers are rarely granted the opportunity to design the products they will build without significant input from their clients. Budget, time frames and quality are almost entirely set by the client and not the engineer. As long as we have a capitalist economy, there will be tension between engineers who may be inclined to over-engineer the product and the entrepreneurs who have incentive to under-engineer them. A great deal of legislation was introduced to affect this balance and check the worst impulses of decision makers who profit from the moral hazard.

The author does do a service to the community by pointing out the most obvious way in which engineering software systems is fundamentally different than a discipline of physical rather than information forms. Yes, changes to software alone are almost trivially made compared to structures of steel. And a great deal of rapid advance was enabled by recognizing this in our processes. If we had continued with the kind of waterfall processes that dominated most other engineering processes, many of the largest corporations on the planet today would not yet have been profitable and probably would never have been built. There is clearly a tension in practice between the need to do some level of high level design and yet embrace evolutionary design.

But process is not one-size fits all. The author glibly flits from low risk products such as social media platforms through large enterprise wide systems like airline reservation systems to individual products like spreadsheets to embedded systems with human life and safety concerns. No single process is appropriate for all of these products. The most safety critical benefit from reusing proven designs on stable platforms that have been exhaustively tested. But a great deal of wealth has been created by relaxing those constraints to explore new product categories and experimental systems. The matching of the process to the product goals is something an engineer can influence but is hardly the sole decider. And this leads into some of the most disingenuous parts of this article.

Software engineers are not unique in having clients who wish to employ them to achieve their organizational goals. A necessary feature of all engineering, as opposed to pure science, is the reality of economic constraints. These constraints are mediated by the client and are subject to limited negotiation by the engineer. We have already seen some engineering failures that were predicted to some degree of certainty by engineers such as the Saturn O-ring disaster. Engineering decisions are not made solely by engineers. To blame them for all failures is bordering on scurrilous.

Let's face facts here, Facebook is nothing like the Golden Gate Bridge. A complete failure of that platform would not end lives are throw a region into chaos. To suggest that the same standard of care should be exercised equally over both is polemic and unfair. Perhaps the notion of "move fast and break things" was in retrospect a bit naive. But the entire history of innovation is really no different. If one cannot take chances with a novel and uncritical application like a social network, then when are risks worth taking? As long as the publics health and safety is not placed at risk, the suggestion that heavy handed approach to regulating the software engineering profession betrays an officious attitude toward business.

The author also charges that software engineering has become hermetically insulated from the world. As if rocket science is not?

By page 7, the author has come back to the issue of professional licensure. I am old enough to remember these attempts. But I already pointed out how quickly the professional has changed. I looked at these early attempts with scorn at the superficiality of the certifications. Even green hiring managers could more accurately divine the capability of a potential engineer than these pseudo certifications.

I'll offer one aside on this issue. It is traditional that engineers have a four year degree. But there is scant evidence that the degree, by itself, is a reliable predictor of the engineers innate ability. While it is common that corporations will impose that restriction on hiring, it is usually out of expediency in filtering resumes more than empirical evidence that they make better engineers 4 years after hire. And given the cultural filtering of who can achieve a four years degree, it becomes more of a social filter than a technical one. This argues for the prejudice that Google and Thiel promote and which the author sneers at.

I feel I have addressed the most superficial faults of this article. But there is much more we can learn from some issues raised. One is the lure of entrepreneurship. He is absolutely correct that the ability to create a prototype system by a small group of entrepreneurial software engineers and make a great deal of money by selling this promise changes the very nature of what it means to be a software engineer. These engineers have merged the role of developer and business risk taker just as many scientists have done the same when developing their contributions to basic science. But no, this avenue is NOT turning computing into a type of speculative finance simply because the number of engineers inclined, or in a position, to take those risks is a minority. But he glosses over why this avenue is so attractive. He blithely says that engineering has always been a well-paid profession. Perhaps to a journalist.

Given the explosive growth in the technology, there has always been a severe shortage of talent compared to the appetite of the market. But like so many other labor markets in the US, this market mismatch does not explain the compensation. Consider the different economic outlooks of a hedge fund manager who can act as angel investor compared to the economic outlook of the engineer who creates the product. One can consistently outperform the other by a factor of at least 10, often far more. The basic capitalistic system encourages the use, some can say exploitation, of the engineer to remain in a risk free, but limit compensation role. How is it any wonder that young and talented developers are more inclined to explore being entrepreneurs rather than engineers?

Add to this structural bias toward risk taking the fact that powerful corporations can use selective manipulation of immigration laws to grant themselves economic relief. By keeping salaries low in the US for software engineers, they control the flow of students who choose it for a profession, When the inevitable skill shortage affects them, they claim an exception to immigration laws to allow for more software engineers to enter the US. If this were done fairly by giving them green cards I would accept it as part of the inevitable globalization of labor. But that is not what is done, these corporations go along with restrictions of these foreign workers that limit their ability to respond rationally to the US labor market and go where they can be paid the market rate for their skills. The author misses a great opportunity to make an important point about the power of the engineer in society by ignoring this.

The author cites the difference between many other professional engineers and software engineers and I made some comments about the false equivalence. However I will end by saying that I think the author does touch on something that is actually a valid criticism although I doubt they understand how valid a criticism it is. ABET is the standard by which most university programs are judged. A dirty little secret is that the computer science curriculum has fundamentally changed very little since 1968. And for prestige universities computer science is the program an aspiring software engineer must navigate to have the best chance of securing a position after graduation. Yet as the knowledge in the industry has exploded, the education system has not kept up. Once someone who wanted to study computer science got a math or physics degree. We are long past the time when meaningful distinctions can be made between computer science and software engineering. Why haven't we? Why has the gulf between academia and practice grown? Why are the first two years of a software engineer spent learning things not taught in school? And by this criticism I am not talking about skills training which by necessity is the responsibility of the employer. But instead a more mature approach to the entire software lifecycle than a very superficial mastery of basic coding, if even that.

The author accurately says that software engineering titles are aspirational more than an accurate description of what goes on in the industry or taught by the schools. But on the whole the article comes across as an uninformed hit piece that is designed to get attention by offering half truths about the role of the software engineer today. It missed a great opportunity to make some very valid criticisms of the role of software and the role of software creators in today's world.

Speech Act Theory and It's Role in Software Engineering

John Searle was a disciple of J L Austin who has first defined what we now call Speech Act Theory. Searle's lectures for both the Philosophy of Language and Philosophy of Mind at UC Berkeley were both recorded and are available on Youtube. I want to make a few points about how I believe this linguistic theory informs software engineering and complex software systems.

A key point in speech act theory (SAT) is that some human realities are created with some utterances. When a person has the formal authority to marry or judge, a simple utterance can make a person guilty or married. I extend this concept (or perhaps am anticipating what others have already said) to surmise that language is not only a means of communication among people but is actually a tool by which we construct and maintain social reality. And I see no reason why this would not apply equally to how we use language and software artifacts as language to construct other realities. Here I am trying to work out what that can mean.

What is a computer program? What is software? I take a piece of code to be a blueprint for a machine. When combined with hardware and given agency it transforms that machine in which it is loaded into a realization of that intended machine. The message, utterance if you will, to run the program creates a kind of intentionality that has a world to mind direction of fit, we desire to have that machine be real and make it so with our utterance. This can be a command issued at a command line or the click of a button.

Programming can become arbitrarily complex and progressively more agency can be invested in the machines. Instead of pressing a button, we can invest a machine with the authority to press the button in response to some external stimulus (a mind to world direction of fit in reference to the agent) or on some schedule, which is after all a stimulus of passing time. As we progress with giving more agency to these machines I believe it makes sense to use a theory of mind to talk about the agents and not just humans.

There are two distinct threads in machines today, computation communications. Computation often steals the focus today with the dramatic advances in artificial intelligence and the prevalence of algorithms in social media. But the advances in communications are no less impressive as we progress toward a single pervasive and omnipresent communications network that is used for internet of things as well as for human communications. The growth of computer to computer communications is no less impressive than the ability of any person to be able to converse with any other person on the globe at will. But this intra-computer agent communication must become visible to us at our will as well, especially when it does not serve our needs and we need to understand or change it.

Does it not make sense to look at all network communications as utterances on par with human communications? Do we not preside over a cybernetic realm that is now an extension of our human and physical realm? Do we not have utterances by agents that change our social world as thoroughly as a chaplain or judge? When a municipal system issues a warrant for our arrest, even when done in error, it is a real warrant and will have real world consequences. I want to take this as a starting point and begin to review computer systems through this lens.

***

The origin story for this line of thinking came from a more modest question of how programmers document their code. It is rather obvious that a programmer is creating a unique language that becomes a kind of private language between them and compiler. If the intentionalities of the coder are to be preserved, care must be taken in the naming of the nouns and verbs. For they invariably have some connection, usually indirect, with things in the real world. And in the complexity of the design, this intended relationship is often lost. Even worse is the complexity of the mechanisms by which the surrogates of the real world are manipulated by the algorithms. Anyone trying to pick up the pieces behind a coder must enter into a unique world as envisioned by that coder with the private language they had constructed to communicate their intensions to the compiler.

The connection to speech act came about because of some earlier work I once did on trying to relate communications between a client and a lawyer and codify the conditions of satisfaction that would underlie commitments that were made. But once one understands this linguistic theory, it begins to open new ways of looking at computer languages beyond the standard syntax and semantics that are used.

The Role of Design in Engineering and Software Engineering Education.

A continuing issue I have in the teaching of computer science at university is that I believe it is either delusion or fraud. What the computer science program is used for by students is to prepare them for a job in software engineering yet few schools seem inclined to embrace that label and instead cling to the more traditional notion of computer science. I don't see how we can progress as educators if we do not look honestly at what is going on and how that is not serving the needs of the students. 

Computer science, when it was first acknowledged as a separate academic department apart from applied mathematics, was either organized with mathematics, as a branch of applied math, or in the engineering college, to align with the hardware engineering. According to a an ACM article, the first PhD in computer science was awarded to Richard Wexelblat in 1965 and the second was awarded to Andy van Dam both at the University of Pennsylvania in 1965/1966. (https://cacm.acm.org/blogs/blog-cacm/159591-who-earned-first-computer-science-ph-d/fulltext). The general trend has been to organize the department into the college of engineering but I happen to believe that is wrong and implicitly acknowledges that what is really taught as computer science is truly software engineering under the false label of computer science.

This was easily justified in decades past since there was really not sufficient material to make a distinction. But the amount of material that can be taught, and should be taught, to a budding software engineer has grown and university programs find themselves constantly squeezed to satisfy current market needs by teaching many current techniques while also unwilling to stick to the traditional programs that stressed the more abstract computer science in favor of heavy emphasis on programming. 

There is one particular failing that results from this trend to cling to the science as opposed to the engineering and that is the place of design in the curriculum. And this failure is not limited to only the computer science department but can be seen throughout the college of engineering. For engineering differs from science in one crucial respect, it is inherently tied to economy and to the satisfaction of a client. So why do I believe this is so in today's universities? I believe it is a combination of a few aspects of the culture and history of engineering in the US.
Engineers are invariably quantitatively oriented. And for software engineering in particular, the formal mathematical foundation of the discipline does not admit for much qualitative or humanistic ambiguity. The first uses of computers were for intense calculation and that did not require any design, at least as that concept was understood outside of an engineering discipline where design only meant the choices one made between some relatively well known approaches. Design, and humanities in general, were kept external to the discipline and while that works for most engineering disciplines, it is no longer tenable in software engineering. 

All engineering requires a degree of creativity although the practice is more dominated by the reuse of a few well known designs that are scaled or altered by the application. But even the most routine of engineering disciplines allows for novel design and innovation. Software engineering is exceptional in the growth and innovation that it has experienced. And its high-touch applications in human-computer interaction with its use of visual communications, it has benefited from graphic artists and others outside of the engineering college. 

The other way in which humanities was admitted to the study of software engineering was from the liberal admission of metaphors to architecture. Today major software systems are complex and enormous ensembles that dwarf any physical structures we build today. Yet the way we understood the construction of a software system was heavily influenced by the construction of buildings and other large scale physical artifacts like airplanes or rockets. This had enormous impact on both practice and education as I'll explain below. But for now the point i wish to make is that all of engineering wanted to distance themselves from a discipline that was aligned with the art department, art and architecture. 

A lecture that gives voice to many of the beliefs I hold about software engineering can be found on Youtube and was given by Glenn Vanderburg under the title, Software Art Thou. (https://youtu.be/RhdlBHHimeM)  
This was easily justified in decades past since there was really not sufficient material to make a distinction. But the amount of material that can be taught, and should be taught, to a budding software engineer has grown and university programs find themselves constantly squeezed to satisfy current market needs by teaching many current techniques while also unwilling to stick to the traditional programs that stressed the more abstract computer science in favor of heavy emphasis on programming. 

There is one particular failing that results from this trend to cling to the science as opposed to the engineering and that is the place of design in the curriculum. And this failure is not limited to only the computer science department but can be seen throughout the college of engineering. For engineering differs from science in one crucial respect, it is inherently tied to economy and to the satisfaction of a client. So why do I believe this is so in today's universities? I believe it is a combination of a few aspects of the culture and history of engineering in the US.

Engineers are invariably quantitatively oriented. And for software engineering in particular, the formal mathematical foundation of the discipline does not admit for much qualitative or humanistic ambiguity. The first uses of computers were for intense calculation and that did not require any design, at least as that concept was understood outside of an engineering discipline where design only meant the choices one made between some relatively well known approaches. Design, and humanities in general, were kept external to the discipline and while that works for most engineering disciplines, it is no longer tenable in software engineering. 

All engineering requires a degree of creativity although the practice is more dominated by the reuse of a few well known designs that are scaled or altered by the application. But even the most routine of engineering disciplines allows for novel design and innovation. Software engineering is exceptional in the growth and innovation that it has experienced. And its high-touch applications in human-computer interaction with its use of visual communications, it has benefited from graphic artists and others outside of the engineering college. 

The other way in which humanities was admitted to the study of software engineering was from the liberal admission of metaphors to architecture. Today major software systems are complex and enormous ensembles that dwarf any physical structures we build today. Yet the way we understood the construction of a software system was heavily influenced by the construction of buildings and other large scale physical artifacts like airplanes or rockets. This had enormous impact on both practice and education as I'll explain below. But for now the point i wish to make is that all of engineering wanted to distance themselves from a discipline that was aligned with the art department, art and architecture. 

A lecture that gives voice to many of the beliefs I hold about software engineering can be found on Youtube and was given by Glenn Vanderburg under the title, Software Art Thou. (https://youtu.be/RhdlBHHimeM)  A key item he reports is ideas held by the dean of the Harvard engineering school from 1951 to 1957, Doctor Van Vleck. He desired to place engineering on a more scientific basis. It should be noted that Doctor Van Vleck was a Nobel Prize winning physicist. In the talk Brooks is quoted as having said:

The Vanderburg video rails against the embrace of the waterfall model for earlier academics. There was always a belief that this was the dominant process model for all engineering. 


"Van Vleck was very concerned the the practice of engineering be put on a firm scientific basis. He led a vigorous shift of American engineering education away from design toward applied science. The pendulum swung too far; reactions set in; and the teaching of design has been contentious ever since."
I agree with Brook's comment and I see as a consequence that the heightened need to teach design principles now affects all engineering disciplines but is most visible in the failure of computer science professors to teach even rudimentary design concepts. And it has also shaped the expectations of these students to what the academy believes is the correct process that should be followed in industry.
The needs of students, the needs of future employers and even the material itself all argues for a fundamental rethinking of how students who wish to become software engineers are taught at university today. We must look at the distinction between engineering versus science and also redress the avoidance of good design in their education. It is our job to do the best we can and I believe we are failing to do so.