Time to own up to my Arty side – Darling!

So given the recent posts I have been producing I thought it only proper that I own up to some of the other activities in which I indulge.

You will already know about the gliding, but there is a more ‘closet’ side to my nature. 3 to 4 years ago I started learning to dance Argentinian Tango and have to say it is one of the most wonderful dances I have ever come across. It is so totally engrossing that you will rarely see the dancing couple having a chat!

I have added a page of VIDEOS where I have put links to some of my favourites. Here is one I love for the colours!

More recently, like in only the last few months, I have also taken up watercolour painting to balance out the techie side of my nature. So I have taken my heart in my hands and published a page of my fledgling ART. Please be gentle!

Painting from a sculpture.

Painting from a sculpture.

There now. I feel a lot better getting that of my chest.
But now back to the Phenomenal Software.
Or is it the Software Phenomenon?
Thanks for listening.

Phenomenal Software Development : Delicate Empiricism in Software Development

I am going to start this section with a discussion about doubt, or unknowing. This was not in the original talk since it would not fit within the time allocated, but the response I received when I included it in a subsequent version of the talk makes me realise that it is a key idea we need to consider.

CloudscapeThe Place of Doubt or Unknowing

If we take the Cartesian view that we cannot really know the ‘things in themselves’, then in terms of creating new knowledge we have two possibilities:

  • Run Away
    We can decide that since we cannot get to know the Things, then we should not even try. This is the path chosen by those that wish to shun everything technological and go back to the old pre-modern ways. But as Ken Wilber points out in The Marriage of Sense & Soul(p44), I don’t think many folk today would want to go back there if they really knew what it was like.
  • Over-Hypothesize
    Here we decide that we can be ‘Aggressively Empirical’. Don’t waste too much time with observation, just come up with hypotheses and test these against reality. This approach has no consideration for the embeddedness of the perceiver within the system being perceived, the classic dualistic bind, and can lead to an experiment reinforcing your expectations about a phenomenon.

Note that both of these approaches attempt to push the unknowing away.

Unknowing does not feel comfortable so a typical human reaction is to move away from the discomfort. But in this area it is a mistake. So what can we do?

  • We need to develop ourselves and stay with the unknowing. This is the Delicately Empirical approach. We should consciously stay with the doubt and keep observing, hold back assumptions until our ideas and thinking mature in line with our observations.

It is this latter approach that requires a stronger sense of self in order to stay longer with the unknowing and the ability to hold this state is a core skill when working in a technical environment.

ThermallingKnowledge or Preconceptions?

The first point at which I got an inkling that the field of software development could benefit from the insights of Delicate Empiricism was during a workshop at the OT2001 conference when I attended a session called “Tracer Bullets” hosted by Paul Simmons and Tom Ayerst. They initially showed a short film about the difference between the Russian and American approaches during the space race of the 1960s. The Russians were much more trial-and-error based whereas the Americans had a very strong quality assurance programme. [I notice there was a re-imagining of the session at SPA2013]

The workshop part of the session involved splitting up into small teams of about 4 or 5 people. Each team had a range of materials they could use to try and identify what item was hidden within a tall kitchen bin placed on a stool so you could not look down into it. Each team had to create tools to find out what was at the bottom of the bin. There was a limitation in that each team only had 8 tokens which would allow them to go up to the bin for 1 minute per token. So they only had 8 minutes of ‘research’ time up at the bin.

Having already been familiar with Goethe’s approach of Delicate Empiricism, I argued in my team to use the tokens one at a time and only go forward slowly. Luckily the other folks agreed with the approach and we set off. What surprised me, and provided the impetus for this research into phenomenology, was seeing the approaches taken by the other teams. Whereas our team simply went up with some long balsa wood ‘pokers’ and spent our first minute just stabbing around to get an initial sense of what was at the bottom of the bin, I saw other teams creating elaborate measuring tools which already pre-supposed certain attributes about what they expected to find. A classic case of over-hypothesizing.

Before using each token our team decided to identify what it was that we wanted to learn next, i.e being clear about the boundary of our knowledge, and devising a tool to move that boundary forward. Then once we got the extra data, we would reflect on it and adjust the model of what we thought was at the bottom of the bin. Then we would discuss what it was we wanted to know next, making sure that our hypothesizing did not rush too far ahead of our current knowledge.

My observation of this contrast between our approach and that of the other teams gave me the impetus to delve further into this subject.

The primary question to ask here is just how aware we are about our boundaries of knowledge, and do we know how best we should proceed when at those boundaries? These questions are absolutely crucial when it comes to debugging a wayward software application.

From the workshop I identified the following steps in the process:

  1. Ask: What do we already know?
    We need to be clear about the knowledge we already have and need to consciously check the limits of that knowledge. This knowledge then forms the foundation for expanding what we know. In certain cases we might decide that we need to move into learning about a completely different area.
  2. Ask: What raw data do we need next?
    What is the next piece of raw data we need to help us expand our knowledge? Note that I differentiate here between raw data and knowledge. The raw data we collect relates to the percept. It will need our reflection to find what concept fits the data. Done properly this enables us to create a mental model that properly fits the phenomenon and is not abstract.
  3. Ask: What tool do we need to get this information without overly disturbing the phenomenon?
    Here we need to focus on creating the best tool for the job of finding the next piece of information we need, but without disturbing the system. This latter consideration is particularly important when debugging real-time multi-threaded systems. I find I frequently fall back into the ‘old school’ tradition of printing out the data if I can because usually a debugger is too invasive. (see ‘Staying Free’ below)
  4. Research: Use the tool to run the experiment and collect the raw data.
    So at last we get to do what feels like ‘real’ research. In actuality the ‘real’ research starts with our thinking. Frequently programmers love to get to the keyboard too early because they (and their managers) mistake typing for software development. It feels safer because one feels like one is making demonstrable progress. But this is an illusion and is at the core of Robert Glass’ fact about the ‘disconnect’ between management and programmers. In arguing this I have found David Bohm’s insight that the experiment is purely an external manifestation of the thought concept useful.
  5. Reflect: Expand my knowledge by reflecting on the collected data.
    This is where we need to look at the data we have collected (the percept) and start developing concepts that match it. By using a disciplined imagination we can check our concepts against the data and when we match them (the Aha! experience) we have expanded our knowledge. So now we can go back to stage 1.

Crash1Staying free – Knowing your technology

As mentioned above programmers will likely use debuggers for helping them investigate a problem. This can work well when you are trying to find a fault that is relatively easy to pin down. This is because a debugger will have to modify the running code in order for it to be able to function. For instance it may allow the program to stop at user specified points which it can only do if it modifies the running software. It may also completely modify the memory footprint by putting guard areas around the ‘real’ areas of data in order to catch ‘illegal’ memory accesses outside of what was expected.

Now that systems and debuggers have become more complicated, it is not a given that a programmer will know what the system is doing underneath the hood to provide this helpful functionality. This is where we touch on the nature of freedom. If you do not know what the system is doing – you cannot be free, i.e. you cannot know exactly what is going on, and so your assumptions can be faulty. I will come back to this concept in a later post since it relates to ANY technological use and should start some alarm bells ringing.

But back to the developer. As you can imagine, if you are working on a system that has real-time constraints (I work on video systems which need to play out faultlessly for hours) it just is not feasible to use a debugger to help find some of the nastier problems. Hence my comments above about the ‘old school’ techniques, i.e. printing out the data. With older systems it was not even possible to do this without disturbing the real-time operation because the process was too slow. Nowadays you can generally get away with it, but there will still be the occasional problem where printing out a single number could cause a change to the timing of the system and thus the fault will not manifest. Thus you need to know your system and then know what is the best ‘instrument’ – code modification – that can collect the data you need to find the fault.

Computer as Thought Mirror

Pity the novice or  journeyman programmer who has not yet realised that his or her process of debugging is flawed. All too frequently I see less experienced programmers (a) jumping to an early conclusion about the cause of a fault and then (b) modifying the code to ‘fix’ the problem based on this conclusion. Jumping to the wrong conclusion too early happens time and time again and you will see it happening many times a day in most software companies.

It can cost them a lot of money because they may now ship this software proudly announcing the fix of this specific bug only to find that it has re-appeared in a slightly different guise. What has usually happened is that the timing of the system was changed so the original fault became hidden, but the actual cause, i.e. the bug, was still there. I have found from bitter experience that it can take many many years – a decade is usual – for a programmer to sufficiently hone their debugging skills to the point that they do not fall into the trap of making false assumptions. It truly requires a great deal of willpower and self-awareness. Of course, better processes can help but they will still rely upon discerning human judgement.

The developer is having to learn the boundaries of their knowledge the hard way and the computer is mirroring back the quality of their thought process. Hence my calling it a ‘Thought Mirror’. Generally the novice starts out having great faith in their ability to understand the effects of their software creation, possibly being surprised that it is going wrong. If they are truly growing through the course of their career this will change to assuming they are wrong and possibly not even trusting that the code is correct even though the system is functioning perfectly. You will also find that many seasoned programmers will not buy the latest and greatest software releases – unless they really really have to!

I hope that gives you some sense of how convinced I am that anyone good at debugging will be using the technique of Goethe’s Delicate Empiricism without even realizing it.

Have we unconsciously created a technology that pushes us to the limits of knowledge just so we can come to know ourselves better?

To paraphrase Jeff Carreira, in software development ‘philosophy is not a luxury’.

Next time I shall take a deeper look at the issue of disciplined imagination, Exact Sensorial Imagination to be precise, and how it relates to software development.

BOUNDARY STORIES 6 : Imagine

“Your world has been Split,
But can you see it as one?
And feel the stream of Wholeness,
Wash the hurt and conflict away.”

Since his visit to his mother, Edwin’s internal war had intensified as though the revelations about her had caused his whole inner world to fall into a deeper chaotic state.

Edwin was falling apart. He could physically feel the Split inside him.

And then another note turned up just at the point when he felt most vulnerable. It was a dull Monday morning – again. Why did so many of these notes turn up on Mondays? He felt awful and crumpled the note in his hands as feelings he thought were under control hit him like a mobile brick wall. He collapsed against the front door and the tears streamed down his face as his defences gave way.

In a quiet inner space he could actually FEEL the imagination of the truth of the words and the tears echoed his sadness about not feeling Whole for many a year. Not since he was a small child.

After a while a loud left-brain voice berated him, telling him that this must STOP!

And then the doorbell rang.

“Hello!” A hand opened the letterbox. “Is anyone there?”

It was a girl’s voice. Edwin mumbled something approaching an apology and wiped his nose with the back of his hand, dried his eyes with the ever present kitchen tissue he always had, and opened the door. “Yes? What do you want?”

“Err. I live across the way and this letter was delivered to my house by mistake.” A pause as she took in his appearance. “Are you alright?”

Edwin found himself looking into a pretty face with blue eyes and blond hair parted in the middle, which faded to brown at the roots. He stood there mute, managing to blurt out the hastily created phrase “Sorry. Just got some bad news. – Family” to cover his awkwardness.

“Oh. Right. Sorry to hear that. I hope everything will be alright. Well – here is your letter. Sorry – it looks like a bill. I’ll be on my way.” And with that she handed the letter to him, turned and started walking away.

Girls – Oh Hell. Or rather, Oh HELP!

He was useless with girls.

The adolescent voices of derision from the past bloomed in his mind like a crowd of paparazzi appearing out of nowhere. He summoned all his courage, fought back, ignoring them and blurted out “Would you like a quick cuppa?”

YES! He had won!

This was the first time he had ever been able to find some words, any words, to try and extend a connection with a girl. She stopped and looked over her shoulder, raising one eyebrow as she considered the offer, checked her watch, then shrugged and smiled saying “Yes, as long as you have Earl Grey.”

Edwin dashed back into the house, ransacked the kitchen and found some old Earl Grey teabags in a black box at the back of the cupboard. He ran back to the front door and almost knocked the girl down in his eagerness to prove that, yes, he had Earl Grey tea, and yes, this would justify her staying.

He invited her in with a lighter heart and in that time honoured English fashion put the kettle on.

© Charles Tolman 2013.

Phenomenal Software Development : Philosophical Interlude

Please note that I am not an academic philosopher (as evidenced by my use of the word ‘Dudes’ in the style of Bill & Ted’s Excellent Adventure!) In preparing this material I have drawn heavily on Henri Bortoft’s excellent précis of philosophical history in his book “Taking Appearance Seriously” which he also relates to current phenomenological thinking.

MothSideViewPhilosopher Dudes from History

  • Bacon (1561-1626)
    This is the man who used binary notation to ‘encrypt’ messages transferred around his network of contacts. He concluded that mathematics was the path to certainty and believed in the mastery of Man over Nature, both conclusions that Goethe subsequently disagreed with.
  • Descartes (1596-1650)
    One of the thinkers that has had the most impact on the current time. The founder of the subject/object dualistic way of thinking about things. I shall talk more about Descartes below.
  • Newton (1642-1727)
    The man who gave us those equations of motion I remember having to learn by rote at school. He used a disciplined imagination to propose mathematical models of reality. This then allowed scientists to solve problems within the mathematical realm and subsequently translate the solutions into a physical situation. This link between mathematics and reality is now something we take for granted and is a powerful tool without which we would not have the world we know today.
  • Goethe (1749-1832)
    Although perhaps known more for his artistic endeavours, Goethe was a natural scientist as well. Even back in the 1800s he was raising a warning flag about the scientific method and the problem of over-hypothesising and imposing these hypotheses upon reality. Although in everyday parlance this is the problem of jumping to conclusions, in science this concern has become clearest in the field of quantum physics. The most useful idea I find in Goethe is the metaphor of ‘conversation’ for any scientific research with phenomena. In this Goethe prefigured the coming phenomenological school of thought.
  • Husserl (1859-1938)
    The founder of phenomenology who highlighted the importance of focusing on the process of a thing appearing to us as opposed to the final result of the thing itself. A tricky distinction which I will come back to later.
  • Gadamer (1900-2002)
    Before reading Bortoft’s book I was not familiar with Gadamer’s work and the whole realm of the philosophy of meaning – hermeneutics. Apart from having to rush for the dictionary to check out these new words I relate strongly to the importance of meaning for us. Giving life meaning is something we do all the time, frequently without realizing it. My view is that as we begin to impose our meanings on the world we need to become aware of this aspect of our cognition. (Simon Robinson has produced a great review of Gadamer’s book Truth and Method)

That is the brief overview. Now I will contrast Descartes and Goethe to highlight how these points fit into the discipline of software development.

Descartes and Dualism

Descartes lived between 1596 and 1650. He wanted to be sure of what we could really know uncluttered by the input from our senses. The foundation that he found was the ‘rock’ that is our thinking – something we can be sure of. Then given that we knew we were thinking we could be sure that we were thinking with something, an element of our body.

This is how he came to identify this Dualism of the mind and the body, but it is important to realise that he also wanted to fit in with the church’s views at that time which held that the human being as composed of body and soul. Descartes wanted the church to accept the primacy of thinking. He succeeded in doing so which provided an acceptable foundation for the mathematics that became the basis for the Scientific Revolution. This Scientific Revolution then led to the Industrial Revolution which has created the world we know today.

An interesting point here is the language he used to describe the mind and body. The minds was ‘res cogitans’ – thinking, a verb. An active principle. The body was ‘ res extensa’ – extension, the body, a noun. A passive principle. This further consolidated the view of Francis Bacon that man’s mind was to be master over nature, the world’s body. However this point of view has significant negative consequences as we now know since it has led to many of the ecological problems we have today.

Descartes ideas and thought are firmly entrenched in software development, because software development is indeed an exercise in applied mathematics.

Goethe and Delicate Empiricism

Goethe was a natural scientist who was around at the time of the Industrial Revolution and he was against the Baconian approach of the separation of man from the natural world and the mastery of man over nature. He warned of the danger of over-hypothesising and recognised the need to focus on the phenomenon and not on abstract ideas. He realised that over-abstracting away from the phenomenon under observation could cause errors in understanding. He counselled against moving too far and too early into the realm of abstract ideas and coined the term Delicate Empiricism.

Delicate Empiricism is where the Observer needs to be aware of their own process of observation and how it affects their final conclusions. He also coined the term Exact Sensorial Imagination. Rather than indicating an ungrounded fantasy this is a disciplined process in understanding any phenomenon and how it exists, hence the used of the word ‘Exact’.

In software development we use this process all the time. Whenever I have to imagine how my software is going to work, or am trying to understand how it may be going wrong, I have to use this disciplined process of imagination to ‘run’ the software’s structures (concepts) in my head. If I am not exact then I will come to incorrect conclusions.

However Goethe only dealt with the natural world, not something that links well into software development. It was Rudolf Steiner (1861-1925), a Goethean scholar, who realised that the same methods could be used for the perception of our own thinking, i.e. treating our thought as a phenomenon in the same way.

At the point where we become aware of our own thinking processes when doing software development – something I consider to be crucial – we are treating our thought as a phenomenon. Thus as we observe our own thinking we are able to improve it and become better developers. Needless to say this does not just apply to software development, but this domain makes it easier to see the link to the phenomenological thinkers.

20111119_1643_SunsetCropped

Phenomenology

Phenomenology focuses on the process of the ‘coming into being’ or ‘Appearance of a Thing’, rather than just focusing on ‘Thing’ itself. This is a very difficult concept to understand but I believe this focus on the process of how we see or know anything is key to healing the destructive consequences of the Cartesian subject object split that are so prevalent in our current world.

It has been said that you cannot actually teach this particular way of seeing things but must embark on a process of just trying to do so. It does represent a change of worldview when we come to look at our environment. It means that we need to realise and understand that the ‘Appearance’ and the ‘Thing’ are ONE item. This can also be understood in terms of reconciling the function of the brain hemispheres.

One of the insights about hemispheric brain function that has endured is that the right brain deals with the immediacy of lived experience, i.e. the Appearance of a thing, and the left brain deals with re-presenting what is lived in order to know something, i.e. the Thing itself.

In the words of Iain McGilchrist, the author of ‘The Master and His Emissary‘, the left brain deals with ‘Static, Isolated, Fixed, Decontextualised, Denotative issues‘. It deals with closed systems and perfection. The right brain, however, deals with ‘Individual, Changing, Evolving, Interconnected, Living ideas‘. It handles things that are never fully graspable, never perfectly known.

This we can see when we are looking at something completely new that we do not understand, or have the concepts to fit. We have the right brain perceiving it and the left trying to make sense of what we are perceiving. When we get it (or ‘grok’ it as Robert Henlein would say) we have that Aha! experience which means that we have identified the concept that fits the perception. It is only at this point that we can know something about what we are seeing.

3dBoxA Perception Exercise

It is difficult to put over this idea of the unity of the thing and the perceptive process but perhaps I can give you a sense of how our concepts affect what we see.

Next to this text you can see an image of lines that could be a cube. There are 2 ways to see the image as a cube, (a) one where the front face is at the lower right of the image, and (b) one where the front face is at the top left of the image.

So now just play with switching your perception over from state (a) to state (b) and be aware that nothing is changing in the external world. All you are doing is adjusting your conceptual filter that you are applying to the external percept.

Now can you see another way of seeing it? Try looking it to see a third way before reading further…

It should be possible to see it just as a flat image with lines on it. Not easy once you have seen the cube. If you have a laptop then perhaps rotate the display 45 degrees and that will make it easier.

If you spend some time with this you might be getting more awareness of what is happening inside you as you change your concept filters.

I am going to leave it there for now.
In the next post I will spend more time going deeper into the links between software development and phenomenology.