Monthly Archives: April 2013

Physical Reasons Why Languages Fail – 1

The next few blogs are excerpts from a paper that I presented at the JavaOne conference back in 2011 titled, “Re-thinking How We Think About Parallel Programming.”

black-ant-illustration_432x432The brains of all animals are generally comprised of multiple cooperating neurological subsystems. The lowest lifeforms don’t have much – starfish don’t even have what we normally think of as a brain – it’s more like an advanced nervous system but decentralized. Here’s a short table comparing the approximate number of neurons in the brains of different animals:

Animal Number of Neurons
Zebra fish 10K
Fruit fly 100k
Ant 250k
Cockroach 1M
Frog 16M
Octopus 300M
Human 100B

frog-1Fish and frogs and higher life forms have systems that automatically control breathing and other systems for vision and others that control evading predators (with varying degrees of success). All of these systems operate in parallel, semi-autonomously, without any controls or awareness. Everything they do is  essentially done unconsciously, without conscious thought. They are purely responsive systems triggered by external stimuli. And without higher order thinking or planning, they can respond very quickly.

Consciousness is primarily the domain of humanity. Only human beings (and a few other species, such as dolphins and elephants) have enough raw brainpower to have conscious thought or to be self-aware. Very few animals have enough computing power between their ears to be able to formulate thoughts and develop plans.

Physical Reasons Why Languages Fail – 2

Conscious thought and reason are based on the Cerebral Cortex, which is a sheet of neural tissue that plays a key role in memory, attention, awareness, thought, language and consciousness.

dogIn general, the more neurons in the cerebral cortex, the better, but a lot depends on the efficiency of the “wiring” of the brain. Birds have a different brain structure than mammals and lack a cerebral cortex. And yet, they have fewer neurons than mammals but still are capable of solving surprisingly difficult problems that mammals cannot solve.

Here’s a short table of the approximate number of neurons in the cerebral cortex for different types of non-avian animals.

Animal Neurons in Cerebral Cortex
Mouse 4M
Rat 21M
Dog 160M
Cat 300M
Dolphin 5.8B
Chimpanzee 6.2B
Human 11B

The cerebral cortex sits on “top” of the lower portions of the brain (cerebellum, limbic system, and brain stem). These lower portions of the brain handle the automatic systems (respiration, heartbeat, blood pressure, movement, balance, etc) without the conscious control or supervision of the cerebral cortex.

Here’s the thing about consciousness – it is single threaded.

While the other portions of the brain are merrily performing multiple activities simultaneously, the cerebral cortex can only process one stream of consciousness at a time. Our consciousness can’t do multiple tasks simultaneously.

Thinking, logic, reason; all of these activities take place under the tightly focused spotlight of consciousness.  Anything outside of the spotlight essentially disappears from our consciousness and we are incapable of thinking about it until it once again reappears under the spotlight.

Physical Reasons Why Languages Fail – 3

Mutli-tasking vs Time-slicing

We like to think that we can do several things at once but we can’t. Instead we time-slice. When we’re younger we can do time-slice effortlessly, seamlessly, allowing us to believe that we are performing multiple tasks concurrently. But in fact, we’re not. Don’t believe me, try these simple experiments.

Duck-Rabbit_illusion

Do you see a rabbit (facing right) or a duck (facing left)? Can you see them both at the same time? (This image from J. Jastrow, from an article published in Popular Science in 1899, titled “The Mind’s Eye”.)

 

 

candlesticklg

Do you see faces at the sides of the picture or a candlestick in the center of the picture? Here’s the important thing. Can you see them both at the same time?

These pictures are examples of bi-stable images, images that our consciousness can interpret in one of two ways but not in both ways at the same time. Our brains simply aren’t wired to be able to perceive both images simultaneously.

Not convinced? Numerous studies of people who attend meeting AND do emails at the same time do both activities poorly. Their contribution to the meeting is mediocre and their email responses are ambiguous and/or inaccurate and/or off-target and/or lacking in useful content. Go back and re-read some of your emails that you sent while you were in a meeting – are they really as good as they should have been, or just sufficient.

Still not convinced? The accident rate for people talking on cell phones while driving is approximately the same as for people who have had 1 drink. The accident rate for people who are texting and driving is tragically high; their minds are distracted for too long with their eyes off the road, resulting in too many accidents and fatalities.

Still not convinced? Read two books at the same time. I don’t mean read a chapter in one and then a chapter in the other, or a page in one and a page in the other, or even a paragraph or a sentence in one and then the other. Read both books simultaneously, word for word. Let me know how that works for you. I already know my limitations.

Physical Reasons Why Languages Fail – 4

Multiple Perspectives, Single-Threaded Minds

So what do rabbit/duck images have to do with programming? Surely programming doesn’t require thinking in parallel. Or does it?

Consider this little flowchart, probably typical of an impromptu design sessions. Click on it to view it full size.
FlowDiagram1Notice that every time your eyes focus in on a box to read the words, the rest of the flowchart “disappears”. If you read another box, you can no longer read the first box. To go back to the spotlight analogy, when we read a box, we put the spotlight on that box to the exclusion of the other boxes. And then to understand the relationship between the two boxes, you have to “pull back” the spotlight so you can see both boxes simultaneously. But when you pull back far enough to see the relationship, the box details may be too dim to read any more.

Another example would be that moment when you are considering adding a variable to a method or function. Should it be a string or a double or an int? What is the scope of the variable? Will the information contained in that variable be useful in other parts of the program? Is the information held in that variable related to other information and should the related information be held by a single object instead of being multiple independent variables? Based on these global-level assessments of the role of the variable, we switch to the detail perspective and implement that variable, making sure we meet all of the language requirements.

These are examples of how we switch back and forth between detail perspective(s) and the global perspective when we are programming. The normal programming process is to write a little code (details), mentally verify that the code matches the desired program flow (global), write a little more code (details), mentally verify that the code matches the desired program flow (global), and so on, mentally switching back and forth between the detail perspective and the global perspective because of the single-threaded limitations of our brains.

IF we could think in parallel, we wouldn’t have this perspective switching issue. We would be able to design and develop both in the global perspective and detail perspective simultaneously. We would never need to switch perspectives because we would always be able to fully consider both perspectives all the time.

Instead, we are forced to limp along with our single-threaded consciousness, time-slicing our attention between detail and global perspectives.

Physical Reasons Why Languages Fail – 5

Conditional Statements

Our single-threaded consciousness also struggles to correctly handle conditional statements in a program. To successfully process that “simple” IF statement, we have to correctly write the code that handles both the true and the false outcomes of that IF statement. And since we can’t think about and write both the true and false responses simultaneously, we have to time-slice our attention between on case and the other.

Typically, we’ll start by writing the code to handle one condition and do nothing if the other condition exists (IF {some special condition} exists THEN {perform some special procedure} ).

And then during testing we find that the other condition needs to receive special processing (IF {some special condition} exists THEN {perform some special procedure} ELSE {perform some different special procedure} ).

And before we know it, we find other special conditions (or combinations of conditions) that need to be handled and end up with a bunch of nested conditionals with each branch requiring special processing to get the right results in all situations.

Unfortunately, we have to focus all of our attention on one branch at a time because of the way our brains work. We can’t code the sequence of actions in two or three branches simultaneously. Instead, we have to mentally switch the spotlight back and forth between the various branches to try to get them all to work together, just like we did when we were switching perspectives. For more … Programming Languages Are Not Our Friends

Unit testing programs encourage testing every branch of every conditional statement to ensure that the right outcomes are achieved. Unit testing can be extremely effective but until this tool or similar tools are used 100%, we are limited by the single-threadedness of our own minds.

Code Complexity

Counting the number of lines of code in a program is one way of estimating the “difficultness” or complexity of that program. TJ McCabe, back in 1976, suggested an alternative measure of code complexity would be the number of conditional statements in the code (“A Complexity Measure”. IEEE Transactions on Software Engineering: 308–320). McCabe developed a method of estimating the complexity of a program, referred to as its Cyclomatic Complexity.

And things don’t seem to be getting any better, despite the passage of time and the “improvements” in programming languages. In 2008, Les Hatton stated as part of his keynote speech (Keynote at TAIC-PART 2008, Windsor, UK, Sept 2008) that McCabe Cyclomatic Complexity has the same prediction ability as lines of code.

I find it astounding that despite the rise and eventual fall of BASIC, Pascal, Ada, Modula, c, c++, Java, and all of the less popular languages, we’re really no better at managing the complexity of our programs than we were almost 40 years ago. In other words, the languages that have come and gone (or still remain) don’t address the limitations that our single-threaded consciousness impose upon us. And until we provide a system that intentionally compensates for our physical limitations, we shall probably not get any better at developing computer programs.

Physical Reasons Why Languages Fail – 6

Writer vs Editor

Program developers have to fill two roles: writer and editor. Unfortunately, developers cannot fill both roles simultaneously so they have to mentally time-slice between the two roles.

For example, experience (usually bad) teaches developers to switch back and forth between their inner-optimist that generates new code and their inner-pessimist that applies defensive code. The Writer (optimist) describes the basics of what needs to be done, but then we mentally shift to become the Editor (pessimist) that reviews the code for weaknesses and rewrites the code to make it more resilient to input errors or unexpected return values. From Pooh’s perspective in the 40 acre woods, we need to be either Tigger and Eor but we can’t be both of them at the same time so we have to switch back and forth.

The Writer is also responsible for inserting the punctuation needed in the program. The end-of-statement punctuation marks (; . , etc) aren’t too tough, but the punctuation that comes in pairs () {} [] “”, etc, have to be manually managed. This means that periodically we have to shift from Writer into Editor and start counting parentheses and/or curly braces and/or square braces.

IDE’s and smart editors try to help, but they aren’t smart enough to know what we intend a series of statements to do, so they can’t punctuate them correctly. Which means we again have to mentally switch back and forth between Writer and Editor to get the right number of closing curly braces at the end of our function. Because we can’t split our consciousness between Writer and Editor, we have to time-slice between them.

The Writer is also responsible for spelling everything correctly but we must mentally switch to the Editor to find any spelling errors. Again, IDE’s and smart editors help spot spelling errors, but they usually just highlight words that don’t match words previously spelled, interrupting the Writer and urging the Editor to mentally seize control of our single-threaded consciousness. We have to switch back and forth between Writer and Editor, time-slicing our attention.

Writer vs Debugger

Every developer must also perform the roles of Writer and of Debugger, but again we can’t do them both at the same time. As the program starts becoming runnable, the developer has to switch between Writer and Debugger, diagnosing errors and error conditions as the Debugger, and adding new code and fixing existing code as the Writer.

It would be incredibly more efficient and productive if these two roles could be filled simultaneously but it is unlikely that our single-threaded consciousness will ever be able to do them both at the same time.

For more on this . . .  Programming Languages Are Not Our Friends

Physical Reasons Why Languages Fail – 7

Mental Capacity

In 1956, George A. Miller published “The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information” in Psychological Review. This paper argues that the number of objects an average human can hold in working memory is 7 +/- 2. This paper also discusses memory span, which is the longest list of items (digits, letters, words) that a person can repeat back immediately after presentation in correct order on 50% of trials. Again, Miller notes that the number is 7 +/- 2.

More recent research (Cowan – 2001) has put the limit of the “focus of memory” (working memory) at 4 “chunks.” Oberauer in 2002 refined Cowan’s model by saying we have a more narrow focus of attention, a one-element focus that is embedded in the four-element focus that can be selected for processing (which sounds suspiciously like single-threaded consciousness). There are numerous other models of the organization of memory; interested readers should research “working memory” on the internet. The “Useful Resources” page on this blog also contains other resources.

Which means that when we are developing a program using programming languages, we start running out of chunks somewhere around punctuation and spelling. Even worse, if each perspective consumes 1 chunk (Global perspective = 1, Detail perspective = 1, Positive branch = 1, Negative branch = 1 for a total of 4 chunks) then we’ve already used up all of the available chunks of memory when we start processing our first IF statement. The issues our brains must hurdle with when using programming languages are listed below:

    1. Global vs detail perspective
    2. Conditional Statements (Positive Branch vs Negative Branch)
    3. Punctuation
    4. Spelling
    5. Defensive Programming (Optimist vs Pessimist)
    6. Writer vs Debugger

No wonder it is so difficult to develop computer programs using programming languages; our brains have too much to keep track of and end up having to dump one chunk to bring a different chunk into working memory, like a computer system that has exceeded its RAM and has to access its virtual memory on disk. Worse, our biological RAM is time limited; unused items are either partially remembered or completely forgotten. At least electronic RAM can do parity checking and alert us when the memory contents are corrupted or lost.

It is time that we admit that our single-threaded brains simply can’t simultaneously perform all of the activities that programming languages require of them. And it goes a long ways towards explaining why, despite many decades of experience using programming languages, we’re not that much more efficient or productive at developing software than we were in the beginning.

Physical Reasons Why Languages Fail – Summary

In this series of blogs, we have looked at how the human brain’s capabilities fail us when we try to develop computer programs using programming languages. The topics covered so far are:

Impedance Mismatch

Programming languages are not inherently evil. Nor are they pure good. The preceding blogs were written to highlight how the physical characteristics and capabilities of our brains detrimentally affect our development performance when using programming languages.

squarePegThe point is not that programming languages are bad or that humans are incompetent; instead the point is that there is an impedance mismatch between human thinking and programming languages, like the proverbial square peg in a round hole.

What is really needed is a different (non-language) methodology that plays to the strengths of the human brain and minimizing it’s weaknesses. If the next big “program development technology” is programming language based, it will have about the results as previous “savior languages” and we will continue to experience the same slow and error-prone program development cycles that we have always experienced.

We Can’t Change Human Thinking

I know that there are some of you thinking, “We can’t change the way that people think, so we should just buckle down, work harder and do better.” And honestly, if that approach would work, wouldn’t it have worked by now? We’ve had some brilliant people writing programs since the dawn of computers; shouldn’t we have had some shining examples of programming perfection, of projects that are released that are 100% error free and do everything exactly the way the user wants done. Or at least projects that finished ahead of schedule and under budget?

Our mediocre past performance speaks for itself. Most development projects are overdue and undertested. Many projects are so late and so far over budget that they are canceled instead of completed. And the projects that are delivered rarely do exactly what the user wants in the way that the user wants them done. Not until version 3.1 (or later).

We Can Do Better

  • When we wanted to travel faster than our legs could carry us, we domesticated horses.
  • When we wanted to travel faster than horses, we invented steam locomotives and then automobiles.
  • When we wanted to travel faster than automobiles, we invented airplanes to let us fly.

History is full of examples where a physical limitation of humanity has been overcome by developing new tools and technologies. The same situation exists with programming languages; we can only develop programs as quickly as our horse-and-buggy programming-language technology will allow.

What we need is the next tool, the one that removes the impedance mismatch and allows us to develop programs efficiently. The next blog sequence, Programming Languages Are Not Our Friends, will look in more detail at the problems and consequences of using programming languages.