For those of you who met Clippy, I'm sure you're groaning already. But let me explain. While Clippy was a flawed character I will still defend the basic idea that underlied the design and contend that Clippy is out there waiting for his day. In this defense is a tale of the current state of customer service and an insight into why I am dubious of the grand claims for the future of AI.
While living in SF I shopped at the Safeway at Potrero and 16th. One of the notable characteristics of shopping there, and I suspect at all Safeway stores, is how each and every employee would greet you as you passed them in the aisles. It was obviously a requirement on them and I found it annoying since I loose myself in wandering the aisles looking at products I don't know. Each greeting was an unwelcome interruption to my reverie. One day I needed to find a product that could logically have been shelved in one of three areas. After checking all three I decided that I would take them up on their continual offers to help me find something. I got that deer-in-the-headlights look from the clerk and I thought he would start to cry. But he did get me in touch with someone that eventually figured out that the product was in a fourth category that I would never have thought to check. So while there was enforced chiperness in the interest of improving customer experience, it was not backed up by deep knowledge of the merchandise.
Here in Sacramento there is an old line grocery store with an excellent butcher and a deli counter that beats any single shop I knew in the city. The employees are sometimes chatty and friendly, sometimes hurried and focused on their tasks and sometimes even a bit diffident. The variability of their personalities doesn't bother me a bit. I can imagine that some people are intimidated by some of the employees but I also think some people are too sensitive. I say, get over it. What I love about this store is that every time I have a question I get an answer that is almost encyclopedic about the product in question. What I lose in chipperness I get back in competency. In the end, the only thing I want is for them to pay attention to me when I need it and take care of my need in the most efficient way possible. If I want a friend I'll sign onto Facebook.
Once I recognized this I see it all over my life, especially in tech and telephone support. Too many companies give their operators scripts that they tightly enforce and very little deep training in their products. I suspect that this has something to do with the bad reputation that Indian call centers have earned. Perhaps it is the more hierarchical social structures of their history or just the rapid fire growth of the industry there, but the bone headed questions I've gotten from some operators would fill a volume of comedy. ("Hi, my name is Dale, I'm calling about order xyz.", "Yes sir. I'd be glad to help you. But first can I get your name?") What managements have done is substitute rote scripts for true management, supervision and training.
So enter Clippy. Even the name suggests some perky young thing that is eager to help. It's continual dance in the corner of the screen just screamed, "Please, please, let me help you!" But as soon as you ask it something, if you weren't lucky enough to ask it the two things it knew, you got useless information. People blamed the mode of delivery. But what was lacking was the competence. If Clippy was able to have even a fraction of the success at answering a real question about the products it was embedded into, it would not have become the brunt of jokes.
Consider what Clippy had going for it; it was an observer of what you were doing across products. It knew the context. It had access to databases of information. Now we have much better search products that could potentially make Clippy far more capable of intuiting what advice we might be seeking. It could potentially recognize repetitive behavior and offer real help in making our interaction with the product better. Will it? I suspect any employee at Microsoft who suggests anything that sounds like Clippy is immediately taken out back and shot. Too bad. RIP Clippy. I hope you come back in another life. You were much more endearing than the clerks at Safeway.
Tuesday, May 22, 2012
What's the best way to read a file in Java?
The title of this post was an innocuous and seemingly straightforward question in StackOverflow that caught my eye. (http://stackoverflow.com/questions/4716503/best-way-to-read-a-text-file) The poster had noticed that there was more than one way to read an ASCII file and as a noob was curious as to what the "best" way was. There was hardly a consensus in the answer and I believe there is something to be learned in that lack of a rapid consensus.
First some background. I do not profess to be a Java expert. Since my needs in this department are simple I usually mimic what I find in the first text I grab. Since there are different ways in different texts, I'll lay those out first.
From Savitch:
Scanner inputStream = new Scanner (new FileInputStream("stuff.txt"));
From Gaddis:
BufferedReader inputFile = new BufferedReader(new FileReader("stuff.txt"));
So pangea suggests Gaddis' approach as does Knubo for his first suggestion. However Knubo offers an alternative that only uses the FileInputStream rather than encapsulating it into a Scanner object. Jesus Ramos also agrees with the first suggestion. Juan Carlos Kuri Pinto suggests his way is better but it still uses the Gaddis classes.
Peter Lawrey offered a novel approach.
as did Claude
One poster, jzd, referenced a page at Oracle which attempts to provide an overview of the different methods and their differences. (http://docs.oracle.com/javase/tutorial/essential/io/file.html)
So if I were to reply to this poster with all of this information, what would I say? All these suggestions will result in code that will read an ASCII file. What are the differences? What makes this interesting to me is that the differences are not in the gross functionality of the code but rather the non-functional qualities that the resulting system will have.
Let's take a trivial and obvious example, the try-catch block. If we know that all our data is good and want a quick-and-dirty piece of code, we might drop the coding of this block since we want to get the code written quickly and have enough control over the file that the code isn't needed. Here we are trading off coding speed (time-to-market) with robustness. The first exception will cause the code to halt. In many cases we don't care and choose for the quick-and-dirty.
There are two important qualities that vary in these treatments. First, is the relationship between the operating code (the dynamic structures) and time. The buffering feature will provide for a more efficient operation for a static file that is to be read and processed as it might in a batch system. But presumably there is some price to be paid for this feature in terms of processor and memory load. In a modern system it is likely that this is negligible in most systems but for very large, high-performance systems, this cannot always be taken for granted. The quality trade-off may become an architectural consideration if this file reading is in the critical path of a high-performance system or one embedded with limited hardware performance.
Another quality difference between the coding options is maintainability. As jzd observes, Oracle now offers some classes that provide much faster processing but at the cost of readability. This illustrates how quality choices are often trade-offs between two or more different requirements. I suspect in the end, we first code what we know best in the absence of any requirement forcing consideration of other options.
Normally I need to find more complicated examples to illustrate this concept. As the design gets more complex, the design task becomes more complicated
First some background. I do not profess to be a Java expert. Since my needs in this department are simple I usually mimic what I find in the first text I grab. Since there are different ways in different texts, I'll lay those out first.
From Savitch:
Scanner inputStream = new Scanner (new FileInputStream("stuff.txt"));
From Gaddis:
BufferedReader inputFile = new BufferedReader(new FileReader("stuff.txt"));
So pangea suggests Gaddis' approach as does Knubo for his first suggestion. However Knubo offers an alternative that only uses the FileInputStream rather than encapsulating it into a Scanner object. Jesus Ramos also agrees with the first suggestion. Juan Carlos Kuri Pinto suggests his way is better but it still uses the Gaddis classes.
Peter Lawrey offered a novel approach.
for(String line: FileUtils.readLines("my-text-file"))
System.out.println(line);as did Claude
The methods within
apache.commons.io.FileUtils may also be very handy, e.g.:/**
* Reads the contents of a file line by line to a List
* of Strings using the default encoding for the VM.
*/
static List readLines(File file)One poster, jzd, referenced a page at Oracle which attempts to provide an overview of the different methods and their differences. (http://docs.oracle.com/javase/tutorial/essential/io/file.html)
So if I were to reply to this poster with all of this information, what would I say? All these suggestions will result in code that will read an ASCII file. What are the differences? What makes this interesting to me is that the differences are not in the gross functionality of the code but rather the non-functional qualities that the resulting system will have.
Let's take a trivial and obvious example, the try-catch block. If we know that all our data is good and want a quick-and-dirty piece of code, we might drop the coding of this block since we want to get the code written quickly and have enough control over the file that the code isn't needed. Here we are trading off coding speed (time-to-market) with robustness. The first exception will cause the code to halt. In many cases we don't care and choose for the quick-and-dirty.
There are two important qualities that vary in these treatments. First, is the relationship between the operating code (the dynamic structures) and time. The buffering feature will provide for a more efficient operation for a static file that is to be read and processed as it might in a batch system. But presumably there is some price to be paid for this feature in terms of processor and memory load. In a modern system it is likely that this is negligible in most systems but for very large, high-performance systems, this cannot always be taken for granted. The quality trade-off may become an architectural consideration if this file reading is in the critical path of a high-performance system or one embedded with limited hardware performance.
Another quality difference between the coding options is maintainability. As jzd observes, Oracle now offers some classes that provide much faster processing but at the cost of readability. This illustrates how quality choices are often trade-offs between two or more different requirements. I suspect in the end, we first code what we know best in the absence of any requirement forcing consideration of other options.
Normally I need to find more complicated examples to illustrate this concept. As the design gets more complex, the design task becomes more complicated
Friday, May 18, 2012
Research Questions: April 2012
In today's symposium, a comment was tossed to me concerning a question of programmer productivity. In the discussion P drew a diagram showing the economic 1% (Xaxis, income, Yaxis, number of people) and compared it to the number of contributions in a typical open source project (Xaxis, contributions, Yaxis number of people). Why, he asked, should the shape of these two curves be so similar? U suggested that it was because of the lower cost of creation that this 1% experienced; that their ability to crank out code faster gave them an advantage. V suggested that it was due to the distribution of talent in the population. I objected to the suggestion that talent was the primary factor and that led P to suggest that some basic research into the factors that influence the contributions to open source projects. Perhaps he's right.
For anyone who has worked in a software shop, we easily accept the oft made assertion that there is a factor of ten between the productivity of a good programmer and the productivity of a poor programmer. I, like I suspect most people who have written code, believe that there is clearly a talent component in one's ability to perform the kind of abstract reasoning and design needed to create computer code. However I also see people generalize far too quickly and accept these assertions uncritically. I, for one, can suggest many reasons that can explain differences in contributions in addition, possibly instead of, talent.
In my own experience, my periods of highest coding productivity happened when coding was my full time job. It seems pretty self-evident that if all you are doing is coding day-after-day you get pretty good at it. This alone can explain a lot of variation in open source software contribution.
Not everyone who contributes to an open source project may be a full-time coder. (Q: How many hours a week does a contributor spend coding? How many of those are for the open source project?)
Even when a contributor is a full-time coder, there may be differences between the coding environment of the open source project and the coding environment of their other coding work. Unlike a shop where everyone is using a small set of languages, possibly only one, with significant support for that language, an open source project will pull in a wide variety of people using a wide variety of languages at various levels. Where the contributor either works full-time on the open source project or happens to code in the same environment in their other coding work, this may contribute to high-contribution levels. (Q: When not all coding is for the open source project, what is the environment of the open source project? What is the environment for the other coding work? Language, version, IDE, )
For anyone who has worked in a software shop, we easily accept the oft made assertion that there is a factor of ten between the productivity of a good programmer and the productivity of a poor programmer. I, like I suspect most people who have written code, believe that there is clearly a talent component in one's ability to perform the kind of abstract reasoning and design needed to create computer code. However I also see people generalize far too quickly and accept these assertions uncritically. I, for one, can suggest many reasons that can explain differences in contributions in addition, possibly instead of, talent.
In my own experience, my periods of highest coding productivity happened when coding was my full time job. It seems pretty self-evident that if all you are doing is coding day-after-day you get pretty good at it. This alone can explain a lot of variation in open source software contribution.
Not everyone who contributes to an open source project may be a full-time coder. (Q: How many hours a week does a contributor spend coding? How many of those are for the open source project?)
Even when a contributor is a full-time coder, there may be differences between the coding environment of the open source project and the coding environment of their other coding work. Unlike a shop where everyone is using a small set of languages, possibly only one, with significant support for that language, an open source project will pull in a wide variety of people using a wide variety of languages at various levels. Where the contributor either works full-time on the open source project or happens to code in the same environment in their other coding work, this may contribute to high-contribution levels. (Q: When not all coding is for the open source project, what is the environment of the open source project? What is the environment for the other coding work? Language, version, IDE, )
The Cathedral and the Bazaar
Against all odds, I see hits against this blog. So for those of you who may be following me I'd like to offer a few words about my absence. I am teacshing a course at a local JC titled Introduction to Computer Science which has no articulation to any four year CS curriculum. While the book the supervisor of this course has chosen is probably the best in its class, that is very faint praise. These texts fall into one of two camps: the either mirror a first course in computer science which is often a programming methodology or they are computer literacy. I rebel against the thought that this course should fall into either of those two camps. I cannot countenance the thought of making non-cs majors competent at any level of programming to the exclusion of all other topics. Nor do I accept the "dummies" approach to teaching skills that the average high-school student has already mastered. Computer literacy is not something that should earn a college credit, even at a JC. Instead I approach the course as teaching core topics that are resonant with material from other non-CS classes but which also illustrate important concepts, and vocabulary, from CS. I sometimes spend way too much time thinking about this and trying to twist the material to my liking.
Were this all I am doing, I'd still have plenty of time to blog. But as I'm sure I've already mentioned, I start a PhD program in the fall and am looking to ease into that program with the least amount of trauma and stress possible. To that end, I enrolled in the natural language processing course offered online by the Stanford professors. I took the AI class last semester which did not include programming assignments. I looked forward to the programming of this course. What I did not count on was the linguistic background this course seems to take for granted. So while I was getting up to speed with the mechanics of downloading their scaffolding code, I was mildly challenged to complete their assignments in the week they were given. For me, two assignments, admittedly the hardest of the course, just became too time consuming to take seriously. The first was the creation of a probabilistic context free grammar for a restricted lexicon and the other a probabilistic CYK parser that would be trained from a tree bank they provide. I love both the assignments but I find that I am not willing to spend the time right now to get an acceptable solution to these assignments to the exclusion of the other things going on. I expect to complete them after the course is through but no in time to get any credit for them. Not that the credit matters to me anyway.
So I am posting today to give my thoughts on an article I just read that, in part, addresses a question P. posed some weeks ago, "What does an open source project lose by not having a traditional project manager?". I think the question is one that deserves a good answer since if I cannot articulate the differences between open source projects and traditional closed source projects, I am not as familiar with this form of organization as I need to be.
In my research into open source software, I came across a paper by Eric Steven Raymond titled "The Cathedral and the Bazaar" from about 1996. I suspect this is well know to people more familiar with open source software than I am but I have a tendency to enjoy the search for beginnings and this looked like a good place to start. What I find in this paper is the articulation of many core beliefs I have about the right way to develop software in the context of his own experience developing the fetchmail program. I'll review some of his key points and my thoughts about them.
His first lesson: every good work of software starts by scratching a developer's personal itch. I don't agree with this formulation of the thought but see a basic truth expressed here about why open source software can be so powerful. Systems workers use many tools to do their work: operating systems, compilers, integrated development environments, databases, etc. We become intimately familiar with these tools including their shortcomings and strengths. In the world before open source software these were products that needed to be purchased from an organization, mostly for-profit organizations. Prior to the PC, the tools of production were too expensive for the average person to afford and corporations had an incentive to price them for corporations and not people. This placed the cost of production for software outside the hands of a hobbyist or even a garage entrepreneur. But the open source movement spawned by the transparancy brought by UNIX and the dramatic reduction in the cost of hardware changed that. We now have the ability to change what we don't like in our tools and many incentives to do so. When we have an itch to add a new feature, we now have the ability to scratch it. For those who have the requisite skills and motivation, we can craft our own tools by making small modifications to what is already available. This is very empowering and I believe a major motive force behind open source software. Now I just need to find the evidence to support it or refute it.
Where I disagree with this maxim is the suggestion that this broadly applies to all software. Is it reasonable to assume that there will always be a group of people who will have an itch to develop any imaginable piece of software and be able to grow a circle of supports around it? I find this hard to accept. That would suggest that any commercial product out there could eventually fall to an open sourced alternative. I'm not yet confident enough to say that this is unrealistic, yet I believe this is treating open source software as a silver bullet, and we all know how successful those have been in the history of software engineering. Yes, a great many software products are likely to be at least partly open source software in the future but I believe this will be far more graduated than this maxim suggests.
I cannot consider this maxim without also considering Karl Marx. One of the popular conceptions from his is that the workers should own the means of production. Sadly I have not yet studied his work yet and this is something I will want to do to see if this explains some of the open source software movement.
2. "Good programmers know what to write. Great ones know what to rewrite (and reuse). "
On the surface, this maxim acknowledges that great programmers recognize good code and see how to reuse it. What I immediately get out of this is how other programmers cannot quickly see the value of an existing piece of code and can only understand code that they have written. Alternatively other programmers, in hubris, feel they can always do it better and dismiss code written by others. I think this poses a good research question, "What factors are cited by programmers who have access to other similar code when they duplicate functions?". I also believe, without good evidence, that better programmers are better designers and have greater powers of abstraction for seeing how existing components can be reassembled for a novel application.
3. "Plan to throw one away: you will anyhow." from Mythical Man-Month by Brooks
This maxim is often cited but I have not yet seen a good theory for why this should be so. In my opinion, this happens because the design space is novel to the programmer and the process of creating the first version is an exploration of that space. Often early design decisions necessitate later design choices and block others. It is not uncommon for the programmer to realize an alternative version only after a significant amount of effort and the resistance to the refactoring of the code that would be needed. The second time, if starting from a blank page, the programmer will avoid the bad decisions of the first construction while retaining to good decisions made in that design.
4. "If you have the right attitude, interesting problems will find you."
It is really not clear from his essay what this maxim really means to him. I can mean that embracing a collaborative mindset will make you open to seeing possibilities for interesting work. I am not sure this is really a software engineering maxim as it is a philosophy of life. There is so much need for high-quality software that the opportunities are boundless. Yet few people find software development interesting. Even those who do can take a parochial attitude that if they are going to do work, it should be for a company that will pay them for their effort. In software, we are lucky enough to enjoy a lifestyle that both rewards us monetarily while offers us an opportunity to do what we love. But as for artists or philosophers, the show Cabaret tells us what a happens to love for something when there is no money "...and the fat little pastor tells you to love everymore, and the hunger comes a rat-tat-a-tat at your window, and your love flies out the door. Money makes the world go around..."
5. "When you lose interest in a program your last duty to it is to hand it off to a competent successor."
How is this software engineering? This is social consciousness 101 and it applies to any work for public good and is seen in boards all across the country. Many a good organization was built by a competent lead and languished under the leadership, or lack thereof, by subsequent people. If you really love something, you have an obligation that goes beyond your own interests.
6. Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
There may be a software engineering truth here but first let's separate the business truth. No business can be assured of continued customer goodwill if they do not treat their customers with respect. The organization must create value for its customers or else it will be vulnerable. The world is not static and an organization has probably never gotten the function set exactly right in the first place. Either an organization will be responsive to its customers or not. If it is not responsive to its customers, it is vulnerable. Where is the software engineering principle that is separate from the business truth?
Customer intimacy is practiced by organizations that are committed to being responsive. In open source software, the customers are the users who are often the potential developers or at least valued members of the "hallow". The collapsing of these relationships dramatically reduces the communications noise and filtering that would exist in a close source environment. There is no product manager, or any management for that matter, to block or alter a message. Since the communication comes from someone who is more invested in the product, they are willing to give more to see that feature added. This can be extended discussion to work out details of behavior, prototyping or testing. For many reasons, this is often not the case in close-source software and the additional contributions are made at the right place and time to speed the work of the developer.
One aspect that is not discussed in the paper though is the decision making process that a suggestion must go through. In a closed source environment, this is part of the organization processes. I understand there are some standard processes to be found in open source software (OSS) projects but the reality is that someone must decide if this suggestion become acted upon if the requester does not have the skills needed to perform the addition themselves. Earlier I talked about how OSS can differ in the self-selected nature of the people who will work on a project. They are often drawn to work on the tools of their own production. What if the product were a legal database system that served the needs of lawyers? While developers who earned a living consulting to law firms would have a vested interest in the product, the end-users are the lawyers. How would this work in an OSS environment? Would lawyers themselves be posting their suggestions? Would they be articulate enough to provide actionable specifications for the new features they need? Would they remain engaged with the developer long enough to see their suggestion be developed into a product feature? These are all questions in my mind regarding the applicability of OSS outside the domain of system's software.
7. Release early. Release often. And listen to your customers.
I think the listen to your customers is a repeat of a prior point.
This software engineering maxim has proven its worth but I don't think this essay really explains why. Nothing demonstrates an organization's commitment to their customers than an immediate response. OSS projects differ from ordinary organizations in that the submitter is likely to also be the person who submits the proto-solution, whether for a bug-fix or for an enhancement. Whatever the form of leadership, their job is far easier than in a closed source software environment. First, the submitter is motivated to submit a working solution and has most likely demonstrated it in their environment. The OSS project has less of an investment to make. Second, the philosophy of the project is to get the new code in front of as many eyeballs as possible as quickly as possible. There is no expectation of exhaustive quality assurance before the code is seen by a set of users who are inclined to use the test version. The more the code is exercised as a form of black box testing, and examined by other programmers, as a form of white box testing or static analysis, the faster any questionable code will be found. This keeps the process streamlined to production and imposes a caveat emptor on the product pushing more responsibility to test the product against their own acceptance criteria.
While I can see the benefit on the customer side, what I believe is the bigger justification for this maxim is the impact it has on the development side. Long product life-cycles were the norm in the traditional waterfall methodologies.
Were this all I am doing, I'd still have plenty of time to blog. But as I'm sure I've already mentioned, I start a PhD program in the fall and am looking to ease into that program with the least amount of trauma and stress possible. To that end, I enrolled in the natural language processing course offered online by the Stanford professors. I took the AI class last semester which did not include programming assignments. I looked forward to the programming of this course. What I did not count on was the linguistic background this course seems to take for granted. So while I was getting up to speed with the mechanics of downloading their scaffolding code, I was mildly challenged to complete their assignments in the week they were given. For me, two assignments, admittedly the hardest of the course, just became too time consuming to take seriously. The first was the creation of a probabilistic context free grammar for a restricted lexicon and the other a probabilistic CYK parser that would be trained from a tree bank they provide. I love both the assignments but I find that I am not willing to spend the time right now to get an acceptable solution to these assignments to the exclusion of the other things going on. I expect to complete them after the course is through but no in time to get any credit for them. Not that the credit matters to me anyway.
So I am posting today to give my thoughts on an article I just read that, in part, addresses a question P. posed some weeks ago, "What does an open source project lose by not having a traditional project manager?". I think the question is one that deserves a good answer since if I cannot articulate the differences between open source projects and traditional closed source projects, I am not as familiar with this form of organization as I need to be.
In my research into open source software, I came across a paper by Eric Steven Raymond titled "The Cathedral and the Bazaar" from about 1996. I suspect this is well know to people more familiar with open source software than I am but I have a tendency to enjoy the search for beginnings and this looked like a good place to start. What I find in this paper is the articulation of many core beliefs I have about the right way to develop software in the context of his own experience developing the fetchmail program. I'll review some of his key points and my thoughts about them.
His first lesson: every good work of software starts by scratching a developer's personal itch. I don't agree with this formulation of the thought but see a basic truth expressed here about why open source software can be so powerful. Systems workers use many tools to do their work: operating systems, compilers, integrated development environments, databases, etc. We become intimately familiar with these tools including their shortcomings and strengths. In the world before open source software these were products that needed to be purchased from an organization, mostly for-profit organizations. Prior to the PC, the tools of production were too expensive for the average person to afford and corporations had an incentive to price them for corporations and not people. This placed the cost of production for software outside the hands of a hobbyist or even a garage entrepreneur. But the open source movement spawned by the transparancy brought by UNIX and the dramatic reduction in the cost of hardware changed that. We now have the ability to change what we don't like in our tools and many incentives to do so. When we have an itch to add a new feature, we now have the ability to scratch it. For those who have the requisite skills and motivation, we can craft our own tools by making small modifications to what is already available. This is very empowering and I believe a major motive force behind open source software. Now I just need to find the evidence to support it or refute it.
Where I disagree with this maxim is the suggestion that this broadly applies to all software. Is it reasonable to assume that there will always be a group of people who will have an itch to develop any imaginable piece of software and be able to grow a circle of supports around it? I find this hard to accept. That would suggest that any commercial product out there could eventually fall to an open sourced alternative. I'm not yet confident enough to say that this is unrealistic, yet I believe this is treating open source software as a silver bullet, and we all know how successful those have been in the history of software engineering. Yes, a great many software products are likely to be at least partly open source software in the future but I believe this will be far more graduated than this maxim suggests.
I cannot consider this maxim without also considering Karl Marx. One of the popular conceptions from his is that the workers should own the means of production. Sadly I have not yet studied his work yet and this is something I will want to do to see if this explains some of the open source software movement.
2. "Good programmers know what to write. Great ones know what to rewrite (and reuse). "
On the surface, this maxim acknowledges that great programmers recognize good code and see how to reuse it. What I immediately get out of this is how other programmers cannot quickly see the value of an existing piece of code and can only understand code that they have written. Alternatively other programmers, in hubris, feel they can always do it better and dismiss code written by others. I think this poses a good research question, "What factors are cited by programmers who have access to other similar code when they duplicate functions?". I also believe, without good evidence, that better programmers are better designers and have greater powers of abstraction for seeing how existing components can be reassembled for a novel application.
3. "Plan to throw one away: you will anyhow." from Mythical Man-Month by Brooks
This maxim is often cited but I have not yet seen a good theory for why this should be so. In my opinion, this happens because the design space is novel to the programmer and the process of creating the first version is an exploration of that space. Often early design decisions necessitate later design choices and block others. It is not uncommon for the programmer to realize an alternative version only after a significant amount of effort and the resistance to the refactoring of the code that would be needed. The second time, if starting from a blank page, the programmer will avoid the bad decisions of the first construction while retaining to good decisions made in that design.
4. "If you have the right attitude, interesting problems will find you."
It is really not clear from his essay what this maxim really means to him. I can mean that embracing a collaborative mindset will make you open to seeing possibilities for interesting work. I am not sure this is really a software engineering maxim as it is a philosophy of life. There is so much need for high-quality software that the opportunities are boundless. Yet few people find software development interesting. Even those who do can take a parochial attitude that if they are going to do work, it should be for a company that will pay them for their effort. In software, we are lucky enough to enjoy a lifestyle that both rewards us monetarily while offers us an opportunity to do what we love. But as for artists or philosophers, the show Cabaret tells us what a happens to love for something when there is no money "...and the fat little pastor tells you to love everymore, and the hunger comes a rat-tat-a-tat at your window, and your love flies out the door. Money makes the world go around..."
5. "When you lose interest in a program your last duty to it is to hand it off to a competent successor."
How is this software engineering? This is social consciousness 101 and it applies to any work for public good and is seen in boards all across the country. Many a good organization was built by a competent lead and languished under the leadership, or lack thereof, by subsequent people. If you really love something, you have an obligation that goes beyond your own interests.
6. Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
There may be a software engineering truth here but first let's separate the business truth. No business can be assured of continued customer goodwill if they do not treat their customers with respect. The organization must create value for its customers or else it will be vulnerable. The world is not static and an organization has probably never gotten the function set exactly right in the first place. Either an organization will be responsive to its customers or not. If it is not responsive to its customers, it is vulnerable. Where is the software engineering principle that is separate from the business truth?
Customer intimacy is practiced by organizations that are committed to being responsive. In open source software, the customers are the users who are often the potential developers or at least valued members of the "hallow". The collapsing of these relationships dramatically reduces the communications noise and filtering that would exist in a close source environment. There is no product manager, or any management for that matter, to block or alter a message. Since the communication comes from someone who is more invested in the product, they are willing to give more to see that feature added. This can be extended discussion to work out details of behavior, prototyping or testing. For many reasons, this is often not the case in close-source software and the additional contributions are made at the right place and time to speed the work of the developer.
One aspect that is not discussed in the paper though is the decision making process that a suggestion must go through. In a closed source environment, this is part of the organization processes. I understand there are some standard processes to be found in open source software (OSS) projects but the reality is that someone must decide if this suggestion become acted upon if the requester does not have the skills needed to perform the addition themselves. Earlier I talked about how OSS can differ in the self-selected nature of the people who will work on a project. They are often drawn to work on the tools of their own production. What if the product were a legal database system that served the needs of lawyers? While developers who earned a living consulting to law firms would have a vested interest in the product, the end-users are the lawyers. How would this work in an OSS environment? Would lawyers themselves be posting their suggestions? Would they be articulate enough to provide actionable specifications for the new features they need? Would they remain engaged with the developer long enough to see their suggestion be developed into a product feature? These are all questions in my mind regarding the applicability of OSS outside the domain of system's software.
7. Release early. Release often. And listen to your customers.
I think the listen to your customers is a repeat of a prior point.
This software engineering maxim has proven its worth but I don't think this essay really explains why. Nothing demonstrates an organization's commitment to their customers than an immediate response. OSS projects differ from ordinary organizations in that the submitter is likely to also be the person who submits the proto-solution, whether for a bug-fix or for an enhancement. Whatever the form of leadership, their job is far easier than in a closed source software environment. First, the submitter is motivated to submit a working solution and has most likely demonstrated it in their environment. The OSS project has less of an investment to make. Second, the philosophy of the project is to get the new code in front of as many eyeballs as possible as quickly as possible. There is no expectation of exhaustive quality assurance before the code is seen by a set of users who are inclined to use the test version. The more the code is exercised as a form of black box testing, and examined by other programmers, as a form of white box testing or static analysis, the faster any questionable code will be found. This keeps the process streamlined to production and imposes a caveat emptor on the product pushing more responsibility to test the product against their own acceptance criteria.
While I can see the benefit on the customer side, what I believe is the bigger justification for this maxim is the impact it has on the development side. Long product life-cycles were the norm in the traditional waterfall methodologies.
Is computer code really a language?
I've already discussed the paper that is the main topic of this post. (http://www.blogger.com/blogger.g?blogID=8640583147187562680#editor/target=post;postID=7393746261888193647) I am trying to refine that post and setup for further work in this area.
This post title is a complete rip off of a paper I just finished reading (again) titled On the Naturalness of Software by Hindle, Barr, Gabel, Su, Devanbu and has been accepted for ICSE2012. The hypothesis is that code utterances are amenable to the same kind of simple language models that have worked well for NLP over the past decade. The paper suggests that token completion and suggestions in an IDE can be considerably enhanced by the use of a (relatively) simple language model. What catches my interest is that the going-in position is that the "naturalness" of the coders use of the language must be established. I am glad this is being done but I find this a nearly established fact in my mind.
Clearly computer languages and they way they are used are not human languages in the sense that they primarily serve human-to-human communication. The obvious receiver for the messages is the computer. And we know how limited the language capabilities of compilers and interpreters are. But code is written as much for other humans as it is for the computer. The use of white space is mostly compliant with norms in the industry. But when you teach an intro programming class you realize how much of that is a cultural norm and not a requirement of the language. Not only are pretty-print nicities like the vertical alignment of parallel structures but naming conventions, when a temporary result is committed to a variable and the choices someone makes for helper methods are all hallmarks of an individuals style. When that style is divergent from norms and not consistent or clear, reading the code is simply painful. This alone is enough to convince me that human communication is an inherent property of computer code and thereby prove that computer code that offers expressive capabilities is a natural language in this capacity.
My own research interests are less with computer code itself than with the broader context in which computer code is created. In particular I am fascinated with the transformations of language that span the life-cycle: starting with problem recognition; project definition; problem statement; requirements definition; specification; code construction; and all the feedback loops reversing the waterfall. I am all too aware how difficult the research challenges are outside of code and approach it with great caution. I need to start small.
Having asserted that code is still (at least in part) a human language, I am now concerned with asserting another hypothesis: that pre-code artifacts written in a common natural language are in fact more structured than other non-fiction prose. That is, a requirements document can be shown to use a more restricted form of its natural language and may have clues as to how the language can be more narrowly defined so as to improve the subsequent quality of the product to be built. Even this is a challenging assignment.
In preparation for some of the challenges that are inherent in the above two research directions, I am interested in doing a narrow study to see if the text artifacts in an OSS can be mechanically and successfully categorized in a way that makes a reasonable prediction or correlation to some attribute of the later construction. My first stab at this would be to look at the text in bug reports, use a relatively simple language model and try training that corpus with some hand coded bugs using different categories that I come up with through intuition and common practice. As I think of this research it sounds like an exploritory survey of the corpus. I would do this first for one of the larger and well respected OSS projects (Apache?) to see if any categorization can achieve a reasonable prediction using that language model.
This post title is a complete rip off of a paper I just finished reading (again) titled On the Naturalness of Software by Hindle, Barr, Gabel, Su, Devanbu and has been accepted for ICSE2012. The hypothesis is that code utterances are amenable to the same kind of simple language models that have worked well for NLP over the past decade. The paper suggests that token completion and suggestions in an IDE can be considerably enhanced by the use of a (relatively) simple language model. What catches my interest is that the going-in position is that the "naturalness" of the coders use of the language must be established. I am glad this is being done but I find this a nearly established fact in my mind.
Clearly computer languages and they way they are used are not human languages in the sense that they primarily serve human-to-human communication. The obvious receiver for the messages is the computer. And we know how limited the language capabilities of compilers and interpreters are. But code is written as much for other humans as it is for the computer. The use of white space is mostly compliant with norms in the industry. But when you teach an intro programming class you realize how much of that is a cultural norm and not a requirement of the language. Not only are pretty-print nicities like the vertical alignment of parallel structures but naming conventions, when a temporary result is committed to a variable and the choices someone makes for helper methods are all hallmarks of an individuals style. When that style is divergent from norms and not consistent or clear, reading the code is simply painful. This alone is enough to convince me that human communication is an inherent property of computer code and thereby prove that computer code that offers expressive capabilities is a natural language in this capacity.
My own research interests are less with computer code itself than with the broader context in which computer code is created. In particular I am fascinated with the transformations of language that span the life-cycle: starting with problem recognition; project definition; problem statement; requirements definition; specification; code construction; and all the feedback loops reversing the waterfall. I am all too aware how difficult the research challenges are outside of code and approach it with great caution. I need to start small.
Having asserted that code is still (at least in part) a human language, I am now concerned with asserting another hypothesis: that pre-code artifacts written in a common natural language are in fact more structured than other non-fiction prose. That is, a requirements document can be shown to use a more restricted form of its natural language and may have clues as to how the language can be more narrowly defined so as to improve the subsequent quality of the product to be built. Even this is a challenging assignment.
In preparation for some of the challenges that are inherent in the above two research directions, I am interested in doing a narrow study to see if the text artifacts in an OSS can be mechanically and successfully categorized in a way that makes a reasonable prediction or correlation to some attribute of the later construction. My first stab at this would be to look at the text in bug reports, use a relatively simple language model and try training that corpus with some hand coded bugs using different categories that I come up with through intuition and common practice. As I think of this research it sounds like an exploritory survey of the corpus. I would do this first for one of the larger and well respected OSS projects (Apache?) to see if any categorization can achieve a reasonable prediction using that language model.
Saturday, May 12, 2012
My Notes on Qualitative Techniques in Empirical Software Engineering Research - Part 1
I think my current line of study can best be summarized by a paragraph from the book, Guide to Advanced Empirical Software Engineering by Shull, singer, Sjøberg. This paragraph is from chapter two which is a paper by Carolyn Seaman titled Qualitative Methods:
At this time, I am more interested in looking at case studies than the other two techniques. On page 148, section 4.3 is on Planning Case Studies.
While many issues are shared between formal experiments and case studies, the book explores some differences.
Curtis, B., "Measurement and experimentation in software engineering," Proceedings of the IEEE, 68(9), pp. 1144-57, 1980
Basili, V.R., Selby, R.W., and utchens, D.H., "Experimentation in software engineering," IEEE Transactions on Software Engineering, 12(7), pp. 733-43, 1986
Shen, V.Y., Conte S.D., and Dunsmore, H.E., "Software science revisted: A critical analysis of the theory and its empirical support," IEEE Transactions on Software Engineering, 9(2), pp. 155-65, 1983
Swanson, E.B. and Beath, C.M., "The use of cast study data in software management research," Journal of Systems and Software, 8, pp.63-71, 1988
Kitchenham, B., Pickard L., and Pfleeger, S.L., "Case studies for method and tool evaluation," IEEE Software, 12(4), pp. 52-62, 1995
The study of software engineering has always been complex adn difficult. The complexity arises from technical issues, from the awkward intersection of machine and human capabilities, and from the central role of the people performing software engineering tasks. The first two aspects provide more than enough complex problems to keep empirical software engineering researchers busy. ut the last factor, the people themselves, introduces aspects that are especially difficult to capture. However, studies attempting to capture human behavior as it relates to software engineering are increasing and, not surprisingly, are increasingly employing qualitative methods.This post is my first attempt to distill some of what I am learning about qualitative techniques in software engineering research. My first stop is from a text I have from a software metrics class which seems to frame this well for me. In Software Metrics: A Rigorous and Practical Approach by Norman Fenton and Shari Lawrence Pfleeger they state that there are three investigative techniques: survey, case study, and formal experiment. They characterize surveys as research in the large, formal experiments as research in the small, and case studies as research in the typical. They also point out that surveys are most often retrospective while case studies and formal experiments require a decision regarding what will be investigated. They present four principles of investigation that are common to all three types of investigation. The principles of investigation are:
- choosing an investigative technique
- stating the hypothesis
- maintaining control over variables
- making your investigation meaningful
At this time, I am more interested in looking at case studies than the other two techniques. On page 148, section 4.3 is on Planning Case Studies.
While many issues are shared between formal experiments and case studies, the book explores some differences.
A case study usually compares one situation with another: the results of using one method or tool with the results of using another, for example. To avoid bias and make sure that you are testing the relationship you hypothesize, you can organize your study in one of three ways: sister project, baseline, or random selection.
Sister ProjectsAt the end of this chapter, there some suggestions for further reading. The ones that catch my eye are:
Suppose your organization is interested in modifying the way it performs code inspections. You decide to perform a case study to assess the effects of using a new inspection technique. To perform such a study, you select two projects, called sister projects, each of which is typical of the organization and has similar values for the state variables that you have planned to measure. for instance, the projects may be similar in terms of application domain, implementation language, specification technique, and design method. Then, you perform inspections the current way on the first project, and the new way on the second project. By selecting projects that are as similar as possible, you are controlling as much as you can. This situation allows you to attribute any differences in result to the difference in inspection technique.
Baselines
If you are unable to find two projects similar enough to be sister projects, you can compare your new inspection technique with a general baseline. Here, your company or organization gathers data from its various projects, regardless of how different one project is from another. In addition to the variable information mentioned above, the data can include descriptive measures, such as product size, effort expended, number of faults discovered, and so on. then, you can calculate measures of central tendency and dispersion on the data in the database, so you have some idea of the "average" situation that is typical in your company. Your case study involves completing a project using the new inspection technique, and then comparing the results with the baseline. in some cases, you may be able to select from the organization database a subset of projects that is similar to the one using the new inspection technique; again, the subset adds a degree of control to your study, giving you more confidence that any differences in result are caused by the difference in inspection technique.
Random selection
Sometimes, it is possible to partition a single project into parts, where one part uses the new technique while the other does not. Here, the case study resembles a formal experiment, because you are taking advantage of randomization and replication in performing your analysis. It is not a formal experiment, however, because the project was not selected at random from among the others in the company or organization. In this case, you randomly assign the code components to either the old inspection technique or the new. As with an experiment, the randomization helps to reduce the experimental error and balance out the confounding factors.
This type of case study design is particularly useful for situation where the method being studied can take on a variety of values. For example, you may want to determine whether preparation time affects the effectiveness of the inspections. You record the preparation time as well as component size and faults discovered. You can then investigate whether increased preparation time result sin a higher detection rate.
Curtis, B., "Measurement and experimentation in software engineering," Proceedings of the IEEE, 68(9), pp. 1144-57, 1980
Basili, V.R., Selby, R.W., and utchens, D.H., "Experimentation in software engineering," IEEE Transactions on Software Engineering, 12(7), pp. 733-43, 1986
Shen, V.Y., Conte S.D., and Dunsmore, H.E., "Software science revisted: A critical analysis of the theory and its empirical support," IEEE Transactions on Software Engineering, 9(2), pp. 155-65, 1983
Swanson, E.B. and Beath, C.M., "The use of cast study data in software management research," Journal of Systems and Software, 8, pp.63-71, 1988
Kitchenham, B., Pickard L., and Pfleeger, S.L., "Case studies for method and tool evaluation," IEEE Software, 12(4), pp. 52-62, 1995
Friday, May 11, 2012
What Gay Marriage and Sloppy Programming Have In Common
This week marked a high water mark for the progressive agenda in the president's acknowledgement that he does not see a fundamental difference between gay and straight marriage. As today's NYT observed, the speed with which social change occurs seems to be accelerating and they posit that it is the result of media. I'll leave that question to other researchers like Barton Friedland to ponder. But for me this comes the same week our group looked at a paper titled Sloppy Programming by Little, Miller, Chou, Bernstein, Cypher and Lau from MIT's SAIL and IBM's Almaden Research Center. The discussion raised my mojo and seems to hit close to where my thesis may go. I also see a rather significant connection between these two events and this takes some explanation.
The Sloppy Programming referenced by the title of the paper refers to the error that occurs when a coder enters a line of code that fails for lack of syntax or semantics, among other things. They explore several ways in which this error can be handled including Quack, and Eclipse plugin. They attempt to mimic the auto complete functionality that Eclipse offers in Java whereby it offers a reasonable list of things that might come next while the code is being entered.
The existing auto complete function draws its power from the fact that as the line is typed, the possible tokens that can come next becomes severely constrained. Once one types an object and then a period, the set of possible tokens is limited to the methods and attributes of that object. The authors attempt to expand this concept to encompass other constraints in an attempt to interpret tokens that are not syntactically correct as they appear on the line. The existing auto complete cannot function since the context for the typed statements cannot be as easily interpreted and the task is to look at what they COULD mean rather than what they DO mean. They do this using techniques from NLP that basically look at the context for the statement and look for syntactically valid statements that include the typed words in that statement. I don't find the paper itself groundbreaking since this seems like a very straightforward attempt to offer greater support to the coder in the task of creating both a semantically and syntactically correct statement. What fascinates me about the paper is how it was received.
My biggest turnoff in my undergrad experience was the cultural norms of the engineering school I attended. I was completely and utterly an outsider. Coming out at the time did nothing to help me feel comfortable in my own skin or at the institution. In retrospect, it is little wonder that I dropped out and took a very bitter aftertaste of academic life with me.
From my own perspective, engineers culturally have a surprisingly uniform and distinct world view from other disciplines. I often joke that every engineer is a bit autistic but in my heart I'm not so sure there isn't a bit of truth in that joke. Engineers see the beauty in logic and math and have crystalline visions for how systems should work. As long as they are dealing with purely physical systems, they do brilliant work. However most engineers I know begin to act like a misfiring engine when faced with human systems. The management decision processes, the illogical behavior of markets and human behavior are accepted to various degrees but usually grudgingly.
Business people are sometimes diametrically opposed to this world view embracing the humanity of systems and often with little or no true comprehension for the mechanical world they depend upon. I had a very competent lawyer friend who once needed very close handholding on how far to insert a document into a fax auto feeder having no feel for anything so mechanical. The intuition that you put it in until you feel the resistance was something that was utterly foreign to his consciousness even while he had the haptic sensations, he couldn't integrate that into the task. I mention this cultural divide because I think it still exists today albeit at an attenuated level and I see that divide in this paper and the reactions to it.
Engineers are a macho lot on the whole. Software engineers are not immune to this pull. I see it in the pride they take in their mastery of the various arcane skills that are needed to create quality systems. This paper seems to cut against the grain for some of them by offering a "crutch" to help coders by reducing the cognitive load a language may offer. In a spirited debate after the paper, D. and I discussed the value of this work with him questioning whether the empirical evidence could be gathered to show that this innovation would do any good. He also rightly pointed out that there would need to be a better statement of the problem and context before the empirical study could be approached. He is, of course, correct.
But beside the research questions, which I intend to return to, I find it interesting that his knee-jerk reaction to the work was negative. I also heard a similar reference to "imprecision" in a comment by V. in questioning the paper. Most damning is the title itself. Sloppy is pejorative. While the intent is clearly understood, it belies a certain judgement that if the coder does not get the syntax correct, it is some failure to properly learn the language so as to avoid the error in the first place. For my taste, at least, I could have accepted "Fuzzy Programming" as a more accurate way of conveying the same idea.
I hear echos of a very old argument from Dijkstra in this line. He properly observed that "bug" in its origin story in computers is not the correct term for the errors that are found in source code. A bug is beyond human control, almost literally a deus ex machina that cannot be predicted or prevented (OK, maybe putting the screens back in the windows would have prevented them). A modern equivalent would be the overheating of a rack due to the failure of a cooling component in the machine room. But to ascribe the label bug to an error in coding is to disown and distance oneself from the error in logic that was made. In the very bad old days of mainframe computing, professional became amazingly good at deskchecking for syntax errors simply because it was necessary for improving productivity.
The strength in D.'s resistance to the paper stems from another paper he remembers reading that suggested that code crutches delayed the language proficiency rather than accelerated it. Hence his reference to this mechanism as one of a crutch. He saw a difference between this mechanism and the auto complete suggestions of Eclipse Java but we didn't have a chance to drill down into where the distinction exists in his view. I suspect it has to do with the extent to which the mechanism can be ignored by an experienced coder. I want to explore this in a bit more depth before I go on.
D. echoed something I also deeply believe in; computer languages are not so different from human languages in that you do not learn them superficially. He went so far as to suggest that you learn to dream in them and while I don't quite see it that way, I do believe that the constructs of the language go deeper than our linguistic center and allow us to envision constructs of statements in that language and express those constructs in the syntax very quickly once we become proficient in that language. In his discussion, he made a impassioned argument on why committing a language to memory was vital to gaining full mastery of that language. A part of me agrees that maximum productivity in a language will never be achieved if you continually struggle for the syntactically correct way to express a semantic point. Just like a speaker not yet comfortable in conversational English will struggle to find the correct word or improperly conjugate a vowel, maximum communication over that channel cannot be achieved since these errors can impeded understanding.
But where I begin to differ with D. regarding the direction of this paper has to do with the larger context of computer languages and our orientation to them. The authors of this paper started by showing a browser command line interface that would accept more natural language and attempt to interpret user commands relative to the current page. It seems clear that the primary thought for a command line such as this would not be for a keyboard interface but rather for a spoken language interface for the casual user where verbal utterances often use linguistic devices that assume the current context. This is a markedly different context for this feature than the proficient programmer coding in a favored language that D. had objected to. Yet I don't see the bright line between the two that D. does. Rather I take a cue for interpreting this from HCI literature.
In HCI, the user is never taken as a member of a homogeneous group. If the tool has any complexity at all, there will be noobs, intermediate users and experts in the tool. The type and level of support change with the experience and expectations of the user. D.'s reaction was at the level of the expert or those who would want to become an expert. One assumption here is that everyone using a given language intends to become an expert in that language. An expert will value innate knowledge of the syntax and a wide vocabulary of the keywords in that language. Any thing that would stand in the way of that expertise is to be avoided. This leads me to a research question that would be interesting: RQ1: given a support tool such as described by this paper, how is the learning curve moved by its presence or absence over the long-term? I suspect that in the study D. recalls, the cohort was not as motivated to achieve high levels of proficiency in the language under study. I would expect that if the cohort were students who were not committed to using this language over a significant period of time with significant incentives to improve productivity. The kind of effort required to memorize the nuances or quirks of a language represent a significant cognitive investment. Like any investment, motivation will depend upon the payback. For a professional committed to a given language over a career, the payback is easy to see. For a student with no committment to the continued use of the language, the payback is not so clear.
D. questioned the value of the enhancement for the noob. I agree that for someone who has not yet begun to grasp the fundamentals of the language this type of tool would have very limited utility to them in a programming language. But in the broader application of the concept in various contexts, I can see the value of adding additional intelligence into the application to go beyond a simple rejection of the statement as invalid. This will have its greatest use for someone who is at an intermediate level of proficiency but who may occasionally forget or mis-remember a keyword or misspell a variable name while coding. In HCI literature, the point is made that most experts become intermediate users at various points in their use of a product. And this brings me to the main objection I have: a computer language is best viewed as a product and not a language.
While for some purposes it makes sense to accept the metaphor of computer language it is not always the most helpful metaphor. If you are to take a code-monkey who spends more than 4 hours a day writing in a specific language and virtually no other, the metaphor is completely apt. Their familiarity and recall in that language can quickly ramp up to the point where supplementary materials are simply unneeded. This is to be expected as any professional becomes intimately familiar with the tools of their daily trade. But the days when only a few languages existed has long passed. No software engineer today should expect to spend the rest of their career coding in even the most common languages of today 30 years hence. There is simply too much innovation in the field to think that these languages will remain static. Nor should the junior software engineer expect to never need to master new languages in their career. I don't have any data to back it up, but I don't believe the cognitive load on a new software engineer will be less than it was on a senior software engineer (RQ2: over the past 50 years, how many languages have software engineers needed to use over their careers? What is the cognitive load of each language? Has this been consistent over time or is the cognitive load and complexity of the languages been increasing over time?)
Exploring the metaphor of computer language as language, let's compare it to multi-lingualism. It is quite common for people to learn multiple languages in their life. Some become completely fluent in multiple languages. However this is not the norm, especially for languages learned after adolescence. Most people will speak with an accent, a fact I feel we can safely ignore in this metaphor, but more importantly, will stumble on the cultural idioms of that language and will often make mistakes in the conjugation and declension of words or make odd diction choices when expressing themselves. The errors rarely severely degrade communication either in writing or speech due to the dialog nature of verbal communication and the redundancies of the language and our ability to fill in syntactic missteps.
This flexibility in the communications does not extend to communications with a computer at this time. Only in the past few decades have people begun to suggest that human utterances should be met by a less inscrutable host. This one-way communication had been the norm and had been the major stumbling block in the use of computers by non-trained users; ie. experts. The statistical techniques of the 90s and the renewed interest in using natural language is showing how computers can now engage in something that is less like a dialog from human to a computer and moving more toward a dialog. Errors of syntax and semantics are naturally handled in a dialog since the receiver will point out the confusion and wait for the sender to clarify. At its heart, this is what I think this paper is attempting to address. As we must use more languages in our jobs and as people who are not dedicated to full-time use of a language must grapple with expressing their thoughts in that language grow, I think this form of just-in-time end-user support makes more sense than the language model.
The forces of cultural conservatism will naturally resist some of this movement with the expected calls of the loss of discipline among the younger adherents and the we're-all-going-to-hell-in-a-handbasket attitude toward these sops to the sloppy programmers who are too lazy to really learn the languages. I'm clearly not one of them. Rather I have grown frustrated with how little innovation there has been in the tools we use to program computers over my career. I have not yet memorized the correct spelling of the method names for even the most common classes in Java yet and unless I am going to be working in that language for more than 6 months full time I don't see the point. For me the auto complete feature saves me from making a query to lookup the exact spelling. This tool would actually help me even more since if I type what I believe is the correct method name, a feature like this could recognize the small edit distance between what I typed and what was needed and propose it as a correction.
I believe the human truth at work here is the limited speed with which a culture can change. Engineering cultures are far less willing to change than most others and this runs headlong into the rapid innovation in the field. I don't think it is any accident that the most rapid innovation is occurring in the open source marketplace. Open source is highly influenced by the commercial acceptance of new products. The wall between the development lab and the marketplace is virtually transparent there and the culture of business permeates open source and brings a dynamism that drives change. But even progressive cultures have a speed limit. Conservative cultures an even slower speed limit, one that sometimes approaches zero.
This finally brings me to what I see as shared between this movement toward more intelligent and helpful programming tools and the cultural movement toward same-sex marriage. Clearly for many Americans now, it is difficult for them to understand why a relationship built on sexual attraction, shared responsibility and a life-long commitment should be different for two members of the same sex than for two people of different sex. The bonds between reproduction and sex were severed decades ago and the pocketbook issues have nothing to do with any religious tenets. Yet for a majority of Americans today, gay marriage is simply a change they won't accept. I believe that change happens at different speeds for different peoples at different times. It is clear that at some point this will all be old news. But for the moment there is significant resistance to this change.
So too will the engineering culture begin to view programming languages as products which serve as the tools which help us build better software systems and not tests of manhood. The tools we use are not and should not be static but should be fluid objects that grow as our understanding of the tasks at hand grow. Rather than embracing an ethic that says that these tools will change with each new release. In time, it will be common for the tool to suggest improvements in the way we have coded something so as to take advantage of a new feature which we may not yet be familiar with. Is this a bad thing? Will it be wrong if we insist that it accept a deprecated syntax instead of recasting it into the new form? I don't think so but I suspect some contemporary software engineers will just feel they have gone over the hill when they are reduced to this diminished role as coder.
In a perfect world of research, I would propose a longitudinal study to look at the use of language and tools over the natural arc of a software engineer's career. That is clearly beyond the reach of my own research with the possible exception of doing in-depth interviews of senior software engineers of their recollections of the early part of their careers and comparing them to the current generation. Hmm, maybe that isn't too bad an idea.
The Sloppy Programming referenced by the title of the paper refers to the error that occurs when a coder enters a line of code that fails for lack of syntax or semantics, among other things. They explore several ways in which this error can be handled including Quack, and Eclipse plugin. They attempt to mimic the auto complete functionality that Eclipse offers in Java whereby it offers a reasonable list of things that might come next while the code is being entered.
The existing auto complete function draws its power from the fact that as the line is typed, the possible tokens that can come next becomes severely constrained. Once one types an object and then a period, the set of possible tokens is limited to the methods and attributes of that object. The authors attempt to expand this concept to encompass other constraints in an attempt to interpret tokens that are not syntactically correct as they appear on the line. The existing auto complete cannot function since the context for the typed statements cannot be as easily interpreted and the task is to look at what they COULD mean rather than what they DO mean. They do this using techniques from NLP that basically look at the context for the statement and look for syntactically valid statements that include the typed words in that statement. I don't find the paper itself groundbreaking since this seems like a very straightforward attempt to offer greater support to the coder in the task of creating both a semantically and syntactically correct statement. What fascinates me about the paper is how it was received.
My biggest turnoff in my undergrad experience was the cultural norms of the engineering school I attended. I was completely and utterly an outsider. Coming out at the time did nothing to help me feel comfortable in my own skin or at the institution. In retrospect, it is little wonder that I dropped out and took a very bitter aftertaste of academic life with me.
From my own perspective, engineers culturally have a surprisingly uniform and distinct world view from other disciplines. I often joke that every engineer is a bit autistic but in my heart I'm not so sure there isn't a bit of truth in that joke. Engineers see the beauty in logic and math and have crystalline visions for how systems should work. As long as they are dealing with purely physical systems, they do brilliant work. However most engineers I know begin to act like a misfiring engine when faced with human systems. The management decision processes, the illogical behavior of markets and human behavior are accepted to various degrees but usually grudgingly.
Business people are sometimes diametrically opposed to this world view embracing the humanity of systems and often with little or no true comprehension for the mechanical world they depend upon. I had a very competent lawyer friend who once needed very close handholding on how far to insert a document into a fax auto feeder having no feel for anything so mechanical. The intuition that you put it in until you feel the resistance was something that was utterly foreign to his consciousness even while he had the haptic sensations, he couldn't integrate that into the task. I mention this cultural divide because I think it still exists today albeit at an attenuated level and I see that divide in this paper and the reactions to it.
Engineers are a macho lot on the whole. Software engineers are not immune to this pull. I see it in the pride they take in their mastery of the various arcane skills that are needed to create quality systems. This paper seems to cut against the grain for some of them by offering a "crutch" to help coders by reducing the cognitive load a language may offer. In a spirited debate after the paper, D. and I discussed the value of this work with him questioning whether the empirical evidence could be gathered to show that this innovation would do any good. He also rightly pointed out that there would need to be a better statement of the problem and context before the empirical study could be approached. He is, of course, correct.
But beside the research questions, which I intend to return to, I find it interesting that his knee-jerk reaction to the work was negative. I also heard a similar reference to "imprecision" in a comment by V. in questioning the paper. Most damning is the title itself. Sloppy is pejorative. While the intent is clearly understood, it belies a certain judgement that if the coder does not get the syntax correct, it is some failure to properly learn the language so as to avoid the error in the first place. For my taste, at least, I could have accepted "Fuzzy Programming" as a more accurate way of conveying the same idea.
I hear echos of a very old argument from Dijkstra in this line. He properly observed that "bug" in its origin story in computers is not the correct term for the errors that are found in source code. A bug is beyond human control, almost literally a deus ex machina that cannot be predicted or prevented (OK, maybe putting the screens back in the windows would have prevented them). A modern equivalent would be the overheating of a rack due to the failure of a cooling component in the machine room. But to ascribe the label bug to an error in coding is to disown and distance oneself from the error in logic that was made. In the very bad old days of mainframe computing, professional became amazingly good at deskchecking for syntax errors simply because it was necessary for improving productivity.
The strength in D.'s resistance to the paper stems from another paper he remembers reading that suggested that code crutches delayed the language proficiency rather than accelerated it. Hence his reference to this mechanism as one of a crutch. He saw a difference between this mechanism and the auto complete suggestions of Eclipse Java but we didn't have a chance to drill down into where the distinction exists in his view. I suspect it has to do with the extent to which the mechanism can be ignored by an experienced coder. I want to explore this in a bit more depth before I go on.
D. echoed something I also deeply believe in; computer languages are not so different from human languages in that you do not learn them superficially. He went so far as to suggest that you learn to dream in them and while I don't quite see it that way, I do believe that the constructs of the language go deeper than our linguistic center and allow us to envision constructs of statements in that language and express those constructs in the syntax very quickly once we become proficient in that language. In his discussion, he made a impassioned argument on why committing a language to memory was vital to gaining full mastery of that language. A part of me agrees that maximum productivity in a language will never be achieved if you continually struggle for the syntactically correct way to express a semantic point. Just like a speaker not yet comfortable in conversational English will struggle to find the correct word or improperly conjugate a vowel, maximum communication over that channel cannot be achieved since these errors can impeded understanding.
But where I begin to differ with D. regarding the direction of this paper has to do with the larger context of computer languages and our orientation to them. The authors of this paper started by showing a browser command line interface that would accept more natural language and attempt to interpret user commands relative to the current page. It seems clear that the primary thought for a command line such as this would not be for a keyboard interface but rather for a spoken language interface for the casual user where verbal utterances often use linguistic devices that assume the current context. This is a markedly different context for this feature than the proficient programmer coding in a favored language that D. had objected to. Yet I don't see the bright line between the two that D. does. Rather I take a cue for interpreting this from HCI literature.
In HCI, the user is never taken as a member of a homogeneous group. If the tool has any complexity at all, there will be noobs, intermediate users and experts in the tool. The type and level of support change with the experience and expectations of the user. D.'s reaction was at the level of the expert or those who would want to become an expert. One assumption here is that everyone using a given language intends to become an expert in that language. An expert will value innate knowledge of the syntax and a wide vocabulary of the keywords in that language. Any thing that would stand in the way of that expertise is to be avoided. This leads me to a research question that would be interesting: RQ1: given a support tool such as described by this paper, how is the learning curve moved by its presence or absence over the long-term? I suspect that in the study D. recalls, the cohort was not as motivated to achieve high levels of proficiency in the language under study. I would expect that if the cohort were students who were not committed to using this language over a significant period of time with significant incentives to improve productivity. The kind of effort required to memorize the nuances or quirks of a language represent a significant cognitive investment. Like any investment, motivation will depend upon the payback. For a professional committed to a given language over a career, the payback is easy to see. For a student with no committment to the continued use of the language, the payback is not so clear.
D. questioned the value of the enhancement for the noob. I agree that for someone who has not yet begun to grasp the fundamentals of the language this type of tool would have very limited utility to them in a programming language. But in the broader application of the concept in various contexts, I can see the value of adding additional intelligence into the application to go beyond a simple rejection of the statement as invalid. This will have its greatest use for someone who is at an intermediate level of proficiency but who may occasionally forget or mis-remember a keyword or misspell a variable name while coding. In HCI literature, the point is made that most experts become intermediate users at various points in their use of a product. And this brings me to the main objection I have: a computer language is best viewed as a product and not a language.
While for some purposes it makes sense to accept the metaphor of computer language it is not always the most helpful metaphor. If you are to take a code-monkey who spends more than 4 hours a day writing in a specific language and virtually no other, the metaphor is completely apt. Their familiarity and recall in that language can quickly ramp up to the point where supplementary materials are simply unneeded. This is to be expected as any professional becomes intimately familiar with the tools of their daily trade. But the days when only a few languages existed has long passed. No software engineer today should expect to spend the rest of their career coding in even the most common languages of today 30 years hence. There is simply too much innovation in the field to think that these languages will remain static. Nor should the junior software engineer expect to never need to master new languages in their career. I don't have any data to back it up, but I don't believe the cognitive load on a new software engineer will be less than it was on a senior software engineer (RQ2: over the past 50 years, how many languages have software engineers needed to use over their careers? What is the cognitive load of each language? Has this been consistent over time or is the cognitive load and complexity of the languages been increasing over time?)
Exploring the metaphor of computer language as language, let's compare it to multi-lingualism. It is quite common for people to learn multiple languages in their life. Some become completely fluent in multiple languages. However this is not the norm, especially for languages learned after adolescence. Most people will speak with an accent, a fact I feel we can safely ignore in this metaphor, but more importantly, will stumble on the cultural idioms of that language and will often make mistakes in the conjugation and declension of words or make odd diction choices when expressing themselves. The errors rarely severely degrade communication either in writing or speech due to the dialog nature of verbal communication and the redundancies of the language and our ability to fill in syntactic missteps.
This flexibility in the communications does not extend to communications with a computer at this time. Only in the past few decades have people begun to suggest that human utterances should be met by a less inscrutable host. This one-way communication had been the norm and had been the major stumbling block in the use of computers by non-trained users; ie. experts. The statistical techniques of the 90s and the renewed interest in using natural language is showing how computers can now engage in something that is less like a dialog from human to a computer and moving more toward a dialog. Errors of syntax and semantics are naturally handled in a dialog since the receiver will point out the confusion and wait for the sender to clarify. At its heart, this is what I think this paper is attempting to address. As we must use more languages in our jobs and as people who are not dedicated to full-time use of a language must grapple with expressing their thoughts in that language grow, I think this form of just-in-time end-user support makes more sense than the language model.
The forces of cultural conservatism will naturally resist some of this movement with the expected calls of the loss of discipline among the younger adherents and the we're-all-going-to-hell-in-a-handbasket attitude toward these sops to the sloppy programmers who are too lazy to really learn the languages. I'm clearly not one of them. Rather I have grown frustrated with how little innovation there has been in the tools we use to program computers over my career. I have not yet memorized the correct spelling of the method names for even the most common classes in Java yet and unless I am going to be working in that language for more than 6 months full time I don't see the point. For me the auto complete feature saves me from making a query to lookup the exact spelling. This tool would actually help me even more since if I type what I believe is the correct method name, a feature like this could recognize the small edit distance between what I typed and what was needed and propose it as a correction.
I believe the human truth at work here is the limited speed with which a culture can change. Engineering cultures are far less willing to change than most others and this runs headlong into the rapid innovation in the field. I don't think it is any accident that the most rapid innovation is occurring in the open source marketplace. Open source is highly influenced by the commercial acceptance of new products. The wall between the development lab and the marketplace is virtually transparent there and the culture of business permeates open source and brings a dynamism that drives change. But even progressive cultures have a speed limit. Conservative cultures an even slower speed limit, one that sometimes approaches zero.
This finally brings me to what I see as shared between this movement toward more intelligent and helpful programming tools and the cultural movement toward same-sex marriage. Clearly for many Americans now, it is difficult for them to understand why a relationship built on sexual attraction, shared responsibility and a life-long commitment should be different for two members of the same sex than for two people of different sex. The bonds between reproduction and sex were severed decades ago and the pocketbook issues have nothing to do with any religious tenets. Yet for a majority of Americans today, gay marriage is simply a change they won't accept. I believe that change happens at different speeds for different peoples at different times. It is clear that at some point this will all be old news. But for the moment there is significant resistance to this change.
So too will the engineering culture begin to view programming languages as products which serve as the tools which help us build better software systems and not tests of manhood. The tools we use are not and should not be static but should be fluid objects that grow as our understanding of the tasks at hand grow. Rather than embracing an ethic that says that these tools will change with each new release. In time, it will be common for the tool to suggest improvements in the way we have coded something so as to take advantage of a new feature which we may not yet be familiar with. Is this a bad thing? Will it be wrong if we insist that it accept a deprecated syntax instead of recasting it into the new form? I don't think so but I suspect some contemporary software engineers will just feel they have gone over the hill when they are reduced to this diminished role as coder.
In a perfect world of research, I would propose a longitudinal study to look at the use of language and tools over the natural arc of a software engineer's career. That is clearly beyond the reach of my own research with the possible exception of doing in-depth interviews of senior software engineers of their recollections of the early part of their careers and comparing them to the current generation. Hmm, maybe that isn't too bad an idea.
Subscribe to:
Posts (Atom)