This link leads to a post on Quora that discusses some thoughts software engineers have shared about the culture at their previous employers. Since this is one of my favorite topics, I can't let it pass without some thought and comment.
One of the common complaints is bureaucratic obstacles to creation. Having experienced this on both sides of the engineering/management divide, I feel their pain. Engineers take great pride in their creations. Given the difficulty of bringing a large, complex project to a successful conclusion, they have every right to be proud of the accomplishment. Nothing is more frustrating than to pour your effort into a project for many months or years only to have it get cancelled when the promise of a successful deployment is within sight. Even worse, there is rarely work immediately available to fill the hole left by the project's cancellation.
While a tad crude, it is not completely inappropriate to consider this a form of coitus interuptus. For an engineer, the job well done is a powerful reward and to withhold that accomplishment is a form of cruel and unusual punishment. While the situation has gotten better through the years, too many companies still do not understand the IT function or the role of software engineers in this function. The churn of CIOs illustrates how difficult it is to be at the intersection of management and technology. Most senior managers today seem to be either incapable or unwilling to make even a two year plan that they can stick to. These changing priorities often lead to the cancellation of projects that have already consumed a significant portion of their project budget.
Another source of failed projects is bad project management. In the world of bespoke software, the challenge to create something completely new is constant. But the more novel, the more risk. Managing that risk while maintaining hubris is a recipe for disaster.
No matter the reason for the cancellation, to the engineer it is all the same. This seems to go to a core value in an engineering culture; you don't work for the money, you work for the intrinsic rewards of the job. I cannot think of a better contrast to this value than that of a sales and marketing professional.
SALES v. ENGINEERING: the eternal struggle
I have grown to think of salespeople as the hunter-gatherers of the corporate world. They live for the kill. Too many times the relationship between sales and engineering is along orthogonal lines. To the salesperson, engineering is there to create the thing to be sold. When that thing is commoditized, there is a comfortable insulation between the two functions. But in the world of software, and especially in the world of bespoke software, the two are close or even combined as you can find in a consulting organization. When these two functions are separate yet forced to coordinate very closely, there is friction.
For operational efficiency, a software intensive product company will attempt to standardize and minimize their internal product line to the extent possible so as to achieve economies of scale. While this internal product line may be expressed in the marketplace as a large array of products, software allows for variation that is needed in the market while minimizing the internal complexity. This is the essence of the product line approach in software architecture.
In contrast the customer often wants a customized product that can rarely be realized from the rationalized product line that exists. Since sales people always want to say yes to the customer, they will accept orders knowing that to satisfy their customer they must ask engineering to make changes. When we are talking about the largest and most complex software intensive products, this was a common occurrence. (Since I've been out of the game for a few years, I can't be certain how prevalent it still is but I'd be very surprised if it still is not an issue for large software vendors of business solutions.) This creates a tension between sales who want to sell a product defined by the customers wants and needs, and engineering and operations who want to sell a standardized and easily maintained product line. Pre-sales support moderates the worst of this, assuming management doesn't, but like the sales people, they are not responsible for delivering on the promise, only the achievement of a signed contract. Pre-sales engineering often participate in some sales compensation commissions, although not as much as the closer, and hence are as motivated to make the sale as the closer. After the order is taken, it is the responsibility of the organization to make the customer happy by delivering the product that was promised.
It is normal for management to focus on its revenue stream. No matter how good the product, a large, complex product must be promoted, explained, negotiated and delivered. For large customizable products, this is no small feat and I have the greatest admiration for those skills. And of course the success of the company depends on those skills. But there must be a balance to the motivation to sell a customized product with no regard for the ability of the organization to deliver on that promise. This leads to the second frustration that most engineers seem to face, the demand for change to the product in an unanticipated way. Sometimes this change even undermines some of the design principles that guided its creation. An engineer will see a hack to a product to make a sale as an affront on the purity of the design, or worse, a huge risk to the reputation they have invested in the product. Since any hack is likely to be required under and unrealistic timetable (from the engineering perspective), there is no upside to the engineer but a significant downside. It is a draw/lose proposition to the engineer since any reward for their heroics will be transitory while any punishment for failure is likely to be severe. Is there any wonder engineers would always prefer to never see a sales person in their area?
SPEED
It does not take too much experience to understand the intoxicating need for speed in the valleys of innovation. From the business perspective, the need to be the first in a new market and maintain a lead is vital to success. I doubt that anyone completely dismisses the association between youth and new skills or the relationship between youth and enthusiasm. Rightly or wrongly, youth is also associated with risk taking. When you put these all together you create a culture that is wired for speed.
One of the Quora comments concerns the excitement and motivation engendered by rapid development cycles. Too many organizations still want to manage risk by using management gates which inherently drive toward a waterfall methodology. The comments clearly show an incompatibility between the management perspective on managing risk and the engineer perspective of reducing or eliminating bureaucratic (to the engineer) obstacles. The first theme raised was empowerment.
Management, especially a command and control structure, values predictability and conformance to established rules. This invariably ascribes a level of management a level of control, their formal authority, to make decisions on behalf of the entire organization. Upper- and mid-level managers must delegate some of this authority to lower levels. Since lower level managers are at the bottom of the hierarchy, are generally the less experienced or less competent managers, their ego often is embodied in their authority. If they are ambitious, they are concerned with pleasing their supervisor and doing an exemplary job so as to better their chances for promotion. For many reasons, they are disinclined to delegate the limited authority they have to line workers. This hesitancy can chafe in an engineering culture which will value rapid management decision making and a resentment of waiting for decisions that appear obvious. It is not uncommon for the delay in a management decision to cause a delay in the critical path of the project and management will both complain about the project delay while deny that they are the cause of the delay. Of course inexperienced project management that does not anticipate some predictable decision points does not help. This is both an example of what is perceived as bureaucratic obstacles and a stop/go or lethargic pace to the project.
Another theme I see in the complaints about speed are the lack of understanding of the risk management is attempting to mitigate. Take for instance the issue of committing changes to production in Google when those changes could potentially lead to bad customer experiences. Anyone in the industry knows how hard it is to build customer goodwill and how quickly it can evaporate. Consider NetFlix. Google is wise to carefully guard this intangible asset. Yet the caution that their management displays is unmotivating for an engineer, especially a young, talented engineer, who is anxious to see the results of their labor. Immaturity and a less risk-averse attitude must surely be a common theme among young, talented software engineers.
One of the many speeches I've heard was a cautionary tale I heard when being prepared for work in the field as a mainline consultant. Remember, I was told, that the people you are working with do not have the same career path you do. As a young consultant, we had reason to believe that we would see a stratospheric soar in our salaries and job prospects due to our work at THE FIRM. Meanwhile the staffs we were working with expected a 9 to 5 job and had made different choices regarding their employment. The obvious message was that we were young, ambitious and typically had no life distracting us from the martial pitch to get the job done. To expect the same motivation of the staffs we worked with was just unrealistic. I'll skip over the smugness and condescension. While the situation is different, many of the same social factors are at work among the software engineers at startups.
I did a stint at Siebel. While I was employee 1500 something and the company had been in business 4 years with most of them profitable, Tom Siebel had the managers referring to it as a start-up and expected everyone to act like it was. It was 1999, after all, and everyone wanted to be at one. Yes, we got stock options and had all the trappings of a mature start-up. But in my opinion it wasn't and I was very dubious that the stock options would ever be worth much. As it turns out I was mostly right, although I did end up making some from the options before they became completely worthless and it made up for the lower than market wage they paid. But it gave me a taste of what I knew must be going on all over the valley. Young, naive but talented software engineers were employed by very savvy, talented guys that had an appreciation for what the technology could deliver. When they were ambitious they found ways of selling this vision to the employees and were willing to allow them to share in the expected rewards with generous stock options. This model has been working well for over a decade but I think it is far more nuanced than the media portrays.
Despite all the variations in the valley success stories, the one theme they all share is the infectious excitement of making something new and different. This type of greenfield development seems to be the ambition of every software engineer I've known. The usual preclude is maintenance programming and is something that most software engineers seek to leave behind them as soon as possible.This eroticism of the new is a theme I hope to come back to someday when i discuss design. But since this is not found in the posting I'll skip over it. However I believe that this desire to do greenfield development is inherent and helps illustrate the excitement.
When developing a product that does not yet have customers and no existing structure to restrict development, it is all promise. Further, most greenfield development, especially large, mission critical systems, enjoy a great deal of management attention. There are a great many positive incentives, fewer obstacles and a shared vision of the future. Of course the later phases are difficult as these realities begin to intrude but the human assets accumulated during the development can be used to see the project to a successful end. Such projects are often feted and people can take great pride in a job well done, or at least done. The group dynamics of the project invariably lead to rapid development.
The comments stress speed specifically in the form of rapid iterations. It isn't completely clear how rapid iterations contribute to engineer motivation so I'll parse out the comments. Iterations stand in opposition to a strict waterfall methodology. Most waterfall methodologies encourage a focus on the phase transition points and then load them with various quality control points. I doubt that each iteration has all the QA points that a waterfall plan would have and even with some QA between iterations, I doubt the total time devoted is greater in an iterative approach and I suspect a great deal less. This alone may be an explanation for an engineer's preference of iterative over waterfall.
The last point I saw in the post concerned continuous deployment. He contends that at Quora the commitment to get changes into production quickly has many positive effects. He wasn't completely clear as to what their continuous deployment infrastructure includes but presumably it includes some automated tests to ensure any change does not completely break the system. In addition, if there is any form of Test Driven Design, the testing includes some demonstration of the new feature/fixed feature that was the subject of the change.
Why would this be something an engineer would enjoy? Speculating, I would say that knowing a layer of control that is meant to shield me from my errors increases the pressure to be right. Nothing can be more humiliating than to commit a change to a code base that breaks the product. You get immediate attention and none of it is the kind you want. You might make this mistake once in your career but you are not allowed to commit the same mistake too many times and remain on a viable career path. Of course this also increases the intimacy between the effort of a software engineer and their contribution to the organization.
AUTOMATION
This poster suggests that attention to the automation of operational tasks is a key to a high-quality culture that will attract engineers. I'm disappointed he didn't connect the dots for me but I'll speculate on what he has in mind.
As I mentioned, maintenance tasks are generally not fun. The more maintenance, the less fun. People want fun. Minimize the maintenance and you increase your fun to drudge factor. Perhaps it is this simple. But I see more.
One of the standard gripes I heard on projects came from the people who would support the system after it was put into production. Generally the first briefing they had of the design was when the planning for transfer began. The project scope and the scope of design rarely included the after-turnover support. It was not uncommon for a new system to increase the burden on the operational staff, require new skills, be incompatible with other systems or any of a number of other bone-headed moves that some forethought could have avoided. Meanwhile the best projects I saw included at least some consideration of the operational environment in the planning, if not the design. Clearly the best projects will do this.
It's too frequently remarked that maintenance is the most expensive phase of a product's lifecycle. I suspect there are a great many hidden costs in operation and support as well. Where this poster gives me hope is that it sounds like there is a greater emphasis on post-implementation support, at least among the engineers. From a business perspective, this will lead to far greater efficiency of operation and contribute to the overall product profitability over the life of the project. In the end, I see this as a move to consider not only the design of the software, or even the software/hardware system, but of the complete human/machine system and the complete product lifecycle.
There was one reference that I found intriguing. http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/ This link expounds on the virtues of a metric driven approach to operation. I have to believe that if I dig into this meme I will find Basili. He goes on to say:
"...open-source monitoring and charting tools like graphite [5] and statsd [6] highlight an important aspect of automation -- that automation must be driven by data and monitoring. Without monitoring and logs to know what, how, or why something is going wrong, automation is difficult. A good follow-up motto would be to "measure anything, measure everything, and automate as much as possible."A key point, I think, about the dotCom revolution and it's allied OSS, is the acceptance of putting out products that are likely to contain bugs. This is a stark break with prior practice in the commercial world. dotComs can get away with it because many of their products are free and in the formative stage and are often not mission critical.
DESIGN (build the right software abstractions)
Again, the poster neglected to tie this back to the main point which is how to build a winning software engineering culture. This is a very important topic to me in ways that go beyond its influence on the culture. However I can riff on this although I may not have the same opinion as the poster.
Who doesn't want to play with winners? One of the most ubiquitous cubicle decorations I've seen is the one that reads "It's hard to soar with eagles when you're surrounded by turkeys." This clearly communicates the idea that greatness is within reach but the individuals around us prevent it. It is never clear who the turkeys are (the co-workers? the bosses?) but the thought is clearly that outside forces hold back personal achievement.
This attitude suggests a cynical attitude and a discounting of any personal contribution to the obstacles ("If you are not part of the solution, you're part of the problem") and even worse, a defeatist attitude toward greatness. A good project manager or line manager should know how to counteract this ennui. But to the point, a highly motivated group with high performing individuals is a rewarding and fulfilling group to be in. How much is the peer relations and how much is the management which has created and maintains this group?
To what may have been the writer's main point however, what part does a good design play in the software engineer's motivation? Is it the fact that a good design suggests talented people? Is it because an engineer appreciates the beauty of the design and is rewarded by getting to know it? or learns new skills because of it? Is it because a good design makes a software engineer's job easier by anticipating the build-out of that design and does not get in the way? After all, nothing is more frustrating than being given a design that is immediately seen to be unworkable in the current context. While not every software engineer may have the skills to be a gifted designer, they certainly learn to recognize good ones. Or perhaps it is the relationship between the designer and the construction crew. Rather than having such a bifurcation of staff responsibilities, perhaps it is because the designers are themselves well respected coders who will remain intimately involved in the design through its implementation.
For me, the gold in this part of the post is its references:
- Prof Daniel Jackson of MIT's SAIL http://people.csail.mit.edu/dnj/,
- MapReduce [http://research.google.com/archive/mapreduce.html ],
- SSTable [http://www.quora.com/What-is-an-SSTable-in-Googles-internal-infrastructure ],
- protocol buffers [http://code.google.com/p/protobuf/ ],
- Thrift [https://thrift.apache.org/ ],
- Scribe [https://github.com/facebook/scribe ], and
- Hive [http://hive.apache.org/ ].
These are all references I'll want to use in a post on software architecture.
STRIVE FOR GREATNESS (Develop a focus on high code quality with code reviews)
This section makes many excellent points but again does not focus on the contributions to culture I am trying to develop. Here's my riff.
Pride in workmanship is part of the culture. A staff that truly buys into code reviews is one that exhibits this quality. For code reviews to work properly, several social phenomenon must exist. First, people's egos are strong enough that they can tolerate the review, and receive recommendations, on their products. This speaks to a certain degree of maturity in the individuals and a level of comfort that has been achieved among peers. When either of these things is lacking, dysfunctions are eventually exposed.
There are a number of quotes that come to mind when I consider this topic. Daniel Burnham, who created a master plan for the city of Chicago, is alleged to have said:
Make no little plans. They have no magic to stir men's blood and probably themselves will not be realized. Make big plans. Aim high in hope and work. Remembering that a noble, logical diagram once recorded will not die.If he said it, I suspect it was a paraphrase of a similar quote from Machiavelli:
Make no small plans for they have no power to stir the soul.Personally I prefer Machiavelli's pithier version although it lacks the point that a good design, once created, can live on independently. But both stress the importance of grand ideas to succeed in the leadership of large, complex projects.
I don't know that I can separate culture from leadership. A leader attracts talent to a project and makes an investment to keep everyone motivated and invested in the project. It seems self-evident that the people will contain some common characteristics even if they are nothing more than some momentary suggestion in the mind of the leader that they will make a positive contribution to the project and that they at least accept the leader and want to participate in the project.
An important aspect of project management is to ensure that everyone maintains a common vision of the goal to be reached. A common problem is miscommunication and the tendency of people to hear not what was said, but what they want to hear. Since a software-intensive project must deal with layers of abstraction and vague ideas that must be reified before any product can be realized, there is enormous room for the internal concepts of what will be delivered to wildly diverge from each other. A well managed project will reconcile these divergent visions at or before the last possible moment before which they lead to inefficiency or error.
Another problem that can infect a software engineering culture is the tendency to allow standards to slip. In my opinion, this is failure of management. In one instance, the engineers bring a decision to management that will concern some trade-off between quality and some other economic metric such as time or money. Management may be forced to accept the quality trade-off but how this is communicated to the staff can dramatically affect the culture. On the one hand, if the management accepts that they have incurred a technical debt that must be paid in the future, the engineering staff will see that management is committed to delivering a quality product but must accept certain commercial imperatives. But if managers begin to fudge, rationalize, or deny that de facto decisions that are made, the message to the engineers is that management is more concerned about reputation, money or something else more than they are in the pride they take in achieving the best possible product. Of course this issue is far more nuanced than I let on but it is sufficient to drop a marker on the interplay between decision making and how that can affect an engineering staff.
One management aspect that the poster and I seem to believe is a major contribution to a healthy and desirable engineering culture is the support of code reviews. I see this as a management contribution simply because without their support, it is impossible to create a viable code review process. But implementing this process still requires some courage and faith on the part of management since it has very measurable cost and an unmeasurable benefit. I have seen a few words written as to why people believe code reviews work and I will not take the time right now to research their opinions. Instead I will provide my own knowing full well that most, if not all of them, are the work of others and not original to me.
Socially the most important aspect of a code review process is that it forces engineers to review and discuss their work products. It is perhaps an unavoidable truth about humans that we are always more critical of the work of others than we are of our own. It is also common that people will be sensitive about comments about their work. Yet if these barriers are overcome, debate and an atmosphere of continuous learning begins to happen. Tiny decisions are made by these review meetings and conflicting points of view are aired. Over time, some consensus arises and the I contend that the work of that staff will begin to converge on a common style and uniformity more effectively than any style manual could hope to achieve. This is particularly important where staff members must back each other up to support production. In my case, the list of people who participated in the code review became a matter of record and any member of the review team could be called upon to deal with a problem in that module when the primary was not available. This potential gave an urgency to the work.
The poster suggests that non FTF code reviews can be done and he is absolute right. However I do not believe that they have the same contribution to the staff culture that FTF meetings have. To be in the same room as someone brings a hightened tension to the need to make critical observations or even question someone about their work product. While the improvement to the work product could perhaps be done via computer mediated communication, it does not build the same social skills and interpersonal relationships that FTF meetings have. If you want a cohesive staff, FTF code reviews is one sure way to build strong morale as well as encourage a high-quality, high-performing environment. In the social science literature, this would be due to group norming.
R-E-S-P-E-C-T (Maintain a respectful work environment)
I hope I made the point in the above section on FTF code reviews that they become a vital tool for allowing groups to overcome the obstacles that exist to their effective team effort. Lacking some serious dysfunction, group will naturally move toward some behavioral norms regarding new people or new revelations about people. But the focus on the work product and the mission of the organization often cause people to reevaluate prejudices and try harder to engage in real interpersonal communication.
The poster uses this heading to also discuss how groups can contribute synergy to a design task, a conclusion I endorse. I suppose the cultural issue here is that if engineers are either not asked for their opinions when they should be, or are not listened to when they are, it is understandable why the engineering culture of that organization may flag.
CODE OWNERSHIP (Build shared ownership of code)
The natural tendency for an engineering organization is to divide responsibility for the code along module, sub-system, system boundaries. While logical in that assembly line way of thinking about specialization, it fails to address the stifling aspect of the job. As I've noted elsewhere, software engineers, especially highly talented and young software engineers, are receiving a great deal of intrinsic reward from their job. Asking them to focus on one piece of code to exclusion of the remainder of the system removes the excitement of newness from the job and leads to the inevitable growth of job dissatisfaction. Engineering organizations should take a page from management training programs that move managers frequently so as to give them the broadest possible exposure to the organization. While the organization may lose some instant recall of the internal workings of the code due to the lack of specialization, over time the engineers should become more capable of recognizing the potential causes of systemic and intermittent errors that cross module boundaries.
At a young start-up, compensating different teams differently will lead to strong boundaries between the teams. The greater the distance from one team to another the greater the distance between the code modules. (Conway's law). If Agile methods are ever to reach their maximum effectiveness, people from different areas must be capable of leaving their functional groups and joining together into a cohesive team. The broader the social comfort each team member has with the broader organization, the faster the norming of the group and the sooner they can begin performing.
The poster references the term "bus number" for a project. The reference was another Quora post and I haven't taken the time to research the origin of this phrase. However it is a reference to the number of people in a project who could have a calamitous event (like being hit with a bus) without impacting the team mission. Too often this is 1. For non-mission critical projects, this may make economic sense however it is worth considering how helpful it is to adopt a journeyman/apprentice approach or even a pair-programming approach.
INVEST IN AUTOMATED TESTING
He's right. However I think the poster stops short here. Software engineering is one of the least capital intensive industries in the US today. (I need to find evidence for this but I did once come across a chart showing capital per worker and I remember it being near the bottom). It would be an interesting empirical study to demonstrate how much this means to an organization over the amortization period of the investment in the testing software. The usual obstacle is that most development is done with project level accounting and the project manager can never demonstrate the payback on the investment. Project level accounting for testing software is just boneheaded. It should be at the enterprise level and few senior level executives are convinced that expensive test suites are worth the money. What does he think? That his staff is too lazy to test the products? That people enjoy the often tedious work of testing? This should be a part of every organization's quality program.
The products delivered now are larger and more complex than their predecessors. The skills needed to test them are more complex both because of the complexity of the structures but also because of the greater subject domain they encompass. To deny professionals the tools they need to do their best work just doesn't make any sense. It is also not in keeping with several other goals for a high-quality engineering culture; rapid deployment and Agile processes.
The current practice is to release a product to testing and use basic system's analysis to ensure that it meets its specifications. Once the process is complete it is released to production. But more recent practices are calling for smaller releases that do not lend themselves to the former waterfall approach of holding back changes until a release. The releases are smaller and more targeted. It is difficult to perform the best possible recursion test in this environment without some automated help. In fact, most organizations do not perform complete recursion testing on releases because of the cost of testing it with people.
I think one of the most compelling reasons for an automated test suite is its support for test driven design. While I am not necessarily a proponent, I see many benefits in this approach. First and foremost, it provides an actionable definition of what is expected in the final product. As soon as a feature or error is found, the test to demonstrate that feature is added to the test suite. If full recursion testing is done, all future releases are assured of passing that test or knowing that the release fails that test.
More broadly, the same obstacles that test suites face with enterprise management are faced by many software engineering tools and training. Ironically, the very biggest enterprise purchases such as a database and customer relationship management have a much easier time in the executive suite. This is due to the far greater cost, the smaller number of decisions asked of the executives and the power of the sales and marketing organizations behind these products. Will it be necessary for software engineering tools to coalesce into a mega-application before they can get this kind of attention?
20% Time
This is one section I will quote rather than paraphrase. Since he speaks from experience and provides good citations, it needs no further commentary:
Gmail found its roots in Paul Buchheit's 20% project, and he hacked together the first version in a single day. [http://paulbuchheit.blogspot.com/2009/01/communicating-with-code.html ] Google News, Google Transit, and Google Suggest also started and launched as 20% projects. I used 20% time while at Google to write a python framework that made it significantly easier to build search page demos. While Google's 20% time may be less productive now than during the early days of the company [http://www.quora.com/Engineering-in-Silicon-Valley/How-does-Googles-Innovation-Time-Off-20-time-work-in-practice ], the notion of letting engineers spend 20% of their time working on something not on their product map remains a cradle of innovation for smaller engineering organizations.BUILD A CULTURE OF LEARNING AND CONTINUOUS IMPROVEMENT
Ooyala didn't officially have 20% time while I was there, but I took some anyway and wrote a command-line build tool for Flex and Actionscript that sped up the team's build times, just as Adobe's Flex Builder tool chain started to degrade, and the tool's still in use today even though the engineering team has nearly tripled in size. Atlassian adopted 20% time after experimenting it for year. [http://www.atlassian.com/company/careers/life ] A variation of 20% time that Facebook's fond of and that Ooyala added later is periodic hackathons -- all-night events where the rule is that you can work on anything except your normal project. [http://gigaom.com/2011/12/16/exclusive-inside-facebooks-final-palo-alto-hackathon/ ]
Top-down approaches to product planning, while necessary for focusing the overall direction of the company, can't account to for the multitude of ideas that might arise from engineers closer to the ground. As long as engineers are accountable for their 20% time and focus on what can be high-impact changes, these projects can lead to large steps forward in progress. Without official 20% time, it's still possible but much more difficult for engineers and designers to try out crazy ideas -- the dedicated ones basically have to find weekends or vacation days to do it.
The poster makes three valid points. First, software engineers, like everyone, find satisfaction when they achieve what Mihaly Csikszentmihalyi calls a state of "flow", second, weekly tech talks, third, a good mentoring program. Let me take these in order.
I agree that flow is a vital to job satisfaction although I do not see how it relates to culture so much as software engineeer psychology. I can see that when a software engineer is comfortable in her environment that some obstacles to flow have been removed. However I don't see how the culture, per se, can make this happen.
The recommendation for tech talks is one I can support from my own experience. Let's explore the cause and effect. Why would a technical meeting lead to improved culture? It shows management support and recognition for the software engineering profession. It takes time away from other tasks (in the eyes of management) and thus represents a perceived cost to the organization. Whether it ultimately does or not would be a matter of empirical research. But the perception from the engineer is likely to be that of encouragement of new skill development, and a connection to the broader profession. Too many organizations that do not have software engineering as their prime revenue generator do not view software engineers as professionals. This denigration of their skills and fear of the business function leads to an isolation of this group from the broader organizational goals and alienation from the other staff. This alienation is often absent when the software products are the chief output of the organization. Here the software engineers are more valued for their expertise although this is not universal. Software consulting organizations are perhaps the best at both recognizing the value of the software engineers and taking an active interest in their continued growth.
I have seen many mentoring programs and I do not believe they work. I say this with great sadness since I think the relationships can make an enormous difference to the individuals and the organization. Any professional who is a minority in an organization will feel some degree of alienation from the other staff. The job function requires considerable education and training, it is introspective by nature and forces a form of interpersonal communication that is unnatural to the non-professional. Having a mentor is one way to overcome alienation since the humanity of that relationship overcomes the inherently unsatisfying relationship to the organization as a whole. (I had one friend who once remarked that all organizations are inherently dysfunctional maternal objects).
So why do I think mentoring programs don't work? At its core is the growth of an interpersonal relationship between two people, one with more power than the other. This power difference creates interpersonal distance that must be overcome before the benefits of mentoring are seen. This cannot be achieved when the relationship is forced on two people and cannot happen when there is distrust on either side. It cannot be someone in the management chain to which this person reports which creates many obstacles for smaller companies. This represents too many barriers and consequently is rarely effective as a program. What companies should do instead is create opportunities for different levels to mix as equals and to work on creating environments where people are comfortable speaking candidly. It is a rare company that does even this.
HIRE THE BEST
This sound completely reasonable but I do not think this maxim holds up to critical analysis. Does this mean that all these high-performing silicon valley organization all employ the best engineers? If so, how many "best" engineers are there? Are we conflating best with competent? There is no doubt that there is a great difference between engineering talent at any level (Sackman's Second Law, [Enders03] ). I would suggest that "the best" isn't much more than a slogan or a bit of organizational jingoism.
Clearly no one should be given responsibility to create a product that is beyond their means. If the new hire has so little experience or talent so as to not be a contributing member of the team, they should not be given the position. I think it is denigrating to dismiss them as not being the best, something lacking, instead of recognizing that they do not fit the requirements. There are too many skill sets in the marketplace now. Some very talented programmers may have a difficult time coming up to speed on a set of technologies that are unfamiliar to them. Does that mean they are not "the best" or just not the best at that set of tasks?
If technical expertise were all that was required to be a productive member of a team a strict meritocracy would evolve. But there is more to being a highly skilled software engineer than being technically proficient in whatever technologies are needed. The ability to both adhere to group norms and to provide leadership is one pair of traits that can be in conflict. Interpersonal skills are vital to being able to navigate the sometimes difficult terrain of corporate social structures. If you doubt this, just imagine someone with no English skills trying to work as a software engineer in a high-performing organization. Human communication plays a much bigger role in competence than is sometimes recognized.
As to the contribution that hiring practices can have, I will share one anecdote. As a manager of a testing group I was saddled with an incompetent employee who had been hired because of political connections and shuttled around until he found an area where he was unlikely to do much harm. It was difficult to get him fired and it took almost a year but in the end I feared that his departure would impact the group morale since he was well liked. I was wrong. Almost immediately after his departure, the morale significantly improved and productivity went up. Conversely, when I hired someone very competent shortly before his departure, there was also a notable uptick in morale and productivity. The conclusion I came to was that the self perception of the staff was helped by the positive improvements and recognition of both competence and incompetence. Incompetent workers not only create technical debt, they also weigh down a group.
THE RESEARCH PERSPECTIVE
This post would be pointless beyond my self-indulgent rant about the problems in the industry if I did not bring it to closure with an improved focus on my research agenda. My first filter for looking at this material is the question of what open source software is bringing us, how it is doing that and some way of looking at its future. There is ample suggestion that software engineers are inclined to freely volunteer significant talents to the creation, nurturing and maintenance of open source software products. Why? This post explored some key themes the Quora post suggests are on the minds of talented software engineers in the field. My own experience supports the posters claims. While suggestive, this is hardly incontrovertible empirical evidence and needs more careful collection and analysis.
Putting aside the flimsy evidence for the moment and making a guess as to what will be found, I can see many other research questions. What first comes up in my mind is the motivation of contributors. Do software engineers who volunteer for open software projects find the work intrinsically rewarding? What are the components of this instrinsic reward? Is it the sense of ownership they derive? Is it the participatory management style of the projects? Is it the reward of having a seat at the decision making table? the ability to scratch their own itch and create the tools they want to work with instead of allowing management to chose their tools and present obstacles for the acquisition of tools that software engineers believe will be helpful? Is it an altruistic impulse to give back to a community that gives them so many software products that would otherwise be economically impossible to use? Is it a self-serving mechanism to impress a future employer? a way to get mentored by experienced professionals? Is it a form of idol worship where people are motivated by the star status or charisma of a project leader? Is this just part of a job that employer is paying for? If so, what is the motivation of the employer? Why does IBM support Eclipse? Why does Oracle support Java and MySQL? What will happen to these motivations over time? Is this a sustainable model for development? If the motivations are completely self-serving, will they poison the motivations that non-employees feel about contributing? I have heard it said that OSS engineers are scratching their own itch. To what extent are contributors only working on products that they themselves use or had used in the past? Is participation in infrastructure products, like DBMS, web servers, IDEs, etc, more prevalent than end-user products like word processors, graphics tools or web browsers? How many work hours are contributed to projects per year? What is the skill set of the contributors?
software architecture,
LAWS OF SOFTWARE ENGINEERING
Glass's Law: Requirement deficiences are the prime source of project failures.
Boehm's First Law: Errors are most frequent during the requirements and design activities and
are the more expensive the later they are removed.
Boehm's Second Law: Prototyping (significantly) reduces requirement and design errors, especially
for user interfaces.
Davis Law: The value of a model depends on the view taken, but none is best for all purposes.
Curtis' Law: Good designs require deep application domain knowledge.
Many more laws can be found in what looks like an excellent text on software engineering mentioned by MG.
(found in [Enders03] )
FURTHER TOPICS
The software engineering of vaporware
"the teeth of time", "an encounter with the ghosts of the past" ~Stephen Greenblatt author of The Swerve
What is the best measure for program length
REFERENCES
[Enders03] Handbook of Software and System's Engineering: Empirical Observations, Laws, and Theory by Albert Endres and Dieter Rombach, 2003, Pearson available at http://dev.co.ua/docs/a-handbook-of-software-and-systems-engineering-empirical-observations-laws-and-theories.9780321154200.29795.pdf)
THE POSTER
Edmond Lau, Quora Engineer
I work as an engineer at Quora, and I'm loving it. Prior to Quora, I led the video analytics and monetization team at Ooyala, worked on query suggestions and UI experiments on the search quality team at Google, and studied at Massachusetts Institute of Technology.
No comments:
Post a Comment