ACCU2016: Talk on Software Architecture Design 7: Perceiving Organising Principles

Perceiving Organising Principles requires us to develop a living and mobile thinking perception.

Unfortunately, as programmers, we are at a disadvantage.

We work in a domain where a lot of our thinking needs to be fixed into a rule-based structure in order to imagine how a computer program will function. This can have unwanted side effects of making it difficult to think in a mobile, living way. Multi-threaded programming demands this mobile thinking and is why it is so difficult.

At a personal level if we want to develop this other way of seeing we need to engage in some activities that foster that mobile mode of cognition. Perceiving livingness, almost by definition, requires that we need to handle ambiguity. This is what is required when we are working in the ‘gap’, or whenever we are dealing with human situations.

Logical thinking can cope with known and static issues, but as programmers we need to be very aware of the boundaries of our knowledge, more so than the lay person due to the inherent fixity of the domain of computer programming.

Alexander Revisited
At this point it is useful to look at some of Christopher Alexander’s ideas about the perception of beauty and links to what I have been saying about the idea of Cognitive Feeling.

Alexander started with defining a Pattern Language to help foster good architectural design – what he called Living Structure. This metamorphosed into his masterwork, The Nature of Order where he tried to get a better understanding of why we find certain structures beautiful.

In the Nature of Order, Volume 1 Chapter 5, he identified the following 15 properties of Living Structure:

  • Levels Of Scale
  • Strong Centres
  • Boundaries
  • Alternating Repetitions
  • Positive Space
  • Good Shape
  • Local Symmetries
  • Deep Interlock And Ambiguity
  • Contrast
  • Gradients
  • Roughness
  • Echoes
  • The Void
  • Simplicity And Inner Calm
  • Not-Separateness

If you just look at this as a list of items, it can be difficult to understand how these may be useful in design, apart from as heuristic guidelines. Although useful, if we look at them in the light of the dynamic concept of the Organising Principle, they make a lot more sense.

A major point is Alexander’s use of the word: Living. As I point out, this implies ambiguity. Therefore these 15 Properties can be seen instead as Organising Principles and when we try and ‘bring them down’ into a more fixed understanding we will only be seeing one way of looking at each one.

Perceiving the Organising Principle as a Disciplined Artistic Process.
In order to develop a mobile dynamic cognition that can better perceive Organising Principles, my thesis is that we need to take up some artistic pursuit in a disciplined and self-aware way. Do whatever appeals to you. For me I find painting and dance work well.

Lets look at how the practice of these pursuits parallels software development, or indeed any technical effort.

The following image is a watercolour painting of my daughter.

Princess

Freehand painting based on photo of my daughter.

This was one of my first forays into the world of painting and like the good novice artist I was, I decided to draw the picture first, using a photograph as a reference.

It took me 3 hours!

The first effort took 2 hours. The next took 1 hour and the last two each took half an hour, with the final result intended as the basis for the final painting. Being the worried novice that I was I decided to perform a ‘colour check’ painting freehand before doing the final version. In the end this became the final painting I have included here as I found that when I tried to paint into the final drawing it did not have the same life as the freehand painting.

This is an example of the difference between the ‘master’ freehand approach as compared to the ‘journeyman’ drawn approach. Of course I do not consider myself to be a master painter, but this example illustrates the self-developmental dynamic inherent in the artistic process.

We can also see here the need to do the foundational, ‘analytic’ work, in this case the drawing; followed by the ‘gap’ of putting the drawing away and using the freehand skill to come up with the ‘solution idea’.

The following is a painting by Jim Spencer and is for me an example of how less is more and illustrates how such minimalism is an essential aspect of any mastery. In this case Jim began learning art just after the second world war. (Also see my post Minimalist Mastery)

RedSkyAtNightSmall

The third example of and artistic pursuit is that of dance, in this case Argentine Tango. This particular dance is a form strongly founded on being far more conscious about what is a primary human activity: walking. (See my post on Dance as True Movement)

Here there is a need for structure, and a mobile process of interpretation and improvisation, both founded on a disciplined form of the dance. It can take years to learn how to ‘walk’ again but if followed in a disciplined manner can lead to sublime experiences of ‘Living Structure’ as the ‘team’ of two people dance from a common centre of balance.

In conclusion I hope you have been able to see the implicit link between Art and Technology and the value of balancing ourselves up as human beings.

Thank you for your attention.

In response to my statement about dancing John Lakos (author of Large Scale C++ Software Design) asked for some tango teaching at the end of the talk! The picture was taken by Mogens Hansen.

CharlesAndJohnTangoSmall

ACCU2016: Talk on Software Architecture Design 6: Organising Principles

ACCU2016: Talk on Software Architecture Design 6: Organising Principles

I now want to try and explain the idea of an Organising Principle more clearly.

As an introduction they have a number of characteristics. They:

  • Embody a Living Wholeness.
  • Have a high degree of Ambiguity.
  • Are never Static.
  • Lie Behind the Parts.

ElanWoods

Embedded within a set of requirements is something that we need to find and embody in our implementation. As with high level schematics, there is a high degree of ambiguity surrounding this ‘something’.

During the talk we have moved from the high level schematics where I can hand wave to my hearts content about structure and architecture; down to diving into the depths of the actual implementation.

You may remember Andrei Alexandrescu’s keynote where he was talking about the details of a sort algorithm – first thing in the morning. Difficult after a late night before!

When trying to understand what he was talking about we are need to think at multiple levels:

  • Listen to the words he was saying.
  • Read through the code examples in his slides.
  • Imagine the dynamic behaviour of these code extracts.

In order to comprehend how this algorithm works dynamically, the listener needs to use their imagination at the same time as trying to understand what Andrei is saying. During this particular talk you could palpably feel the intense concentration in the whole audience.

During deep technical discussions listeners will work at these two levels of understanding:

  • The static code structure.
  • The dynamic behaviour of the static construct.

In addition one will only be looking at a single possible instantiation of a solution. There could be alternative ways of doing the same thing.

The term I like to use here is that during this process we are trying to ‘Livingly Think’ the Organising Principle.

An Organising Principle is NOT a static thing. It cannot be written down on a piece of paper. Although it informs the implementation it cannot be put into the code. If you fix it: You Haven’t Got It. Remember that phrase because getting it is the real challenge. It lives behind the parts of any concrete implementation.

I am explicitly using this separation of Whole and Parts, a discipline called Mereology. How can we understand the world in terms of wholeness and yet also see how the parts work therein?

Experienced people will have a sense of the whole and yet will be able to see where the likely risk points are going to be. They simultaneously see the whole picture and also know the essence of what needs to happen in the parts. This is what you pay for when you employ an expert. In my design example, for instance, an expert will know to ask about your allocation strategies and if they see some allocations going out of order they will point you at where you need to change the system.

Requirements implicitly contain Organising Principles.

They implicitly, not explicitly, embody dynamic specifications of what needs to happen. Much of my job when I talk to any customer is to try and understand:

  • What they want to do.
  • What I think can be built
  • How I can bring those together.

I need to understand what the core dynamic principles are that are embedded in their requirements. By definition if you are doing a useful piece of software the customer wont know what they actually NEED. They can talk about what they WANT, but that is based in their present experience. The conversation between the user and the developer will be generating new knowledge about future experience.

[Software development involves moving knowledge from the Complex to the Complicated domain as Cynefin defines it. This means we are dealing with emergent knowledge.]

Architecture references the Organising Principle.

If we take the point that an Organising Principle is not a fixed, discrete idea then we can see that it is much like comparing a film of an activity with its reality. The film is just a set of discrete frames, but the reality is continuous.

With software, the different architectures and implementations are different views of a specific Organising Principle. This is very hard to get our heads around and needs a far more mobile thinking than we normally use. This is the core of why good software development is so difficult.

Code is the precipitate of the Organising Principle.

If we manage to perceive this dynamic principle, referencing it by designing a software architecture that we embody into a specific code implementation, we are embarking on a process of precipitation. If we DON’T do this then – indeed – we only have a “Hopeful Arrangement of Bytes” as this talk title suggests.

Of course this may be a valid thing to do sometimes as a piece of ‘software research’ just to see how the code works out. This is especially relevant if there is a blank sheet, or ‘green field’ site where you just have to start somewhere. In this case you need to look at the part of the requirements that represent the riskiest aspects and just implement some proof of concept solution to check your understanding.

This is where it may not be an idea for a novice or journeyman programmer to follow the example of an experienced one.

Be warned: It is possible that a master programmer may be able to work at the keyboard and seemingly design as they go creating from internalized mature ideas. This is something that is definitely NOT recommended for inexperienced developers who will need to spend much more time maturing and externalizing their ideas first before those ideas become part of their ‘cognitive muscle memory’.

In the next and final post of the transcript of this talk I will deal with how to improve our perception of Organising Principles…

ACCU2016: Talk on Software Architecture Design 7: Perceiving Organising Principles
ACCU2016: Talk on Software Architecture Design 5: Active Design Ideas

STUDY DIARIES: Dances with Cars

Since 1970 I, along with an old university friend, have wondered just why some people love speed.

In my younger years I used to race karts and until recently my conclusion had been that it was the experience of mastery – the wonderful feeling when you managed to power drift through a corner on just the right line, or that oh so elusive relaxed attention when a lap came out just right at a faster time than before.

But now I think I have found the answer, and surprisingly it has links to dance.

In a previous post I talked about how dancing, at its best, uses conscious movement to express our ‘true’ movement of thinking. Notice the use of the use of the word thinkING instead of thought, which could be construed as a fixed item rather than the mobile and dynamic activity that I am describing.

This ‘true’ movement is something some people want to express and dance is one form.

The other can also be driving or flying well.

In line with the previous post, to me it makes sense that we have an inner experience of our thinking being able to move instantaneously, and we feel good when we can manage to express it physically, whether it be with our bodies, or through a technological construct such as a car – or an aircraft. Indeed Ayrton Senna has been described as someone who could dance with the car.

So now I have a far better explanation of why I am such a petrol head, dancer and lover of flying!

But why is this post in the section of the study diaries?

If you read Scaligero or Steiner, their wisdom is predicated on developing a true experience of such a living, mobile, dynamic thinking. It is not something you can really put into words, which are fixed entities.

However it IS something we can experience.

The difficulty is that any characterization of it in fixed form, whether it be in words, pictures, or a materialistic science, will always – always – miss the point. Such expressions can dimly point to the living idea but the listener or spectator will always need to be active. They will have to re-enliven such fixed and dead forms with their own thinking in order to reproduce the living experience.

As I have said before, this is why Steiner and such authors are so hard to understand. In their writing they are purposely trying to short-circuit your analytical brain, which likes fixed constructs,  in order to try and help you move into the living experience of the idea.

BOUNDARY STORIES 8: By Chants

“There is more to being human than meets the eye
We think we know who we are
But if we fix the image
We will never see our truth”

Edwin received the next message on his doorstep on a grey rainy English day that matched his mood.

In truth, he liked this sort of weather. It felt cozier than any summer’s day with a bright blue sky. A point he viewed with perplexed amusement.

In the light of being solo again, he had taken himself off to a meditation retreat where he had learnt some Tai Chi, practiced Zen meditation and enjoyed American Indian chanting as he had braved a hot sweat-lodge ceremony. A smorgasbord of hippy-ness that fitted his variable mood.

He had forgotten about the anonymous messages, despite the fact that it was they that had provided the original impetus to attend the retreat. Something very different for him that reflected a new found drive to better understand himself.

Two days later he received a call that his mother’s condition had deteriorated and that she was not likely to last the day. After some frenzied packing he drove to the home where she lived, reflecting gratefully that he was glad he had gone to see her more in the recent months.

When he arrived an attendant showed him to his mother’s room and commented that the day had been a difficult one for her. His mother had become unconscious and was having trouble with her breathing. Edwin’s heart went out to her because he knew her worst fear was of drowning and not being able to breathe. It was as if at the gate of death such fears had a life of their own and were purposely confronting her, attracted by such a weakness.

As he sat by the bed his mother, seemingly sensing a change, became more panicked, grasping at her breath as though clinging to the last straw of life. It was then that he received as a gift – he could think of no other word for it – an intuition that he should sing to her. And so he began a chant that he had learnt on the retreat – pitching his voice soft and low, timing his chanting to match the rhythm of her breathing.

It was if a switch had been thrown, providing a light in the darkness. She immediately relaxed and began to breathe calmly as he kept up his chanting. He brushed a stray hair back from her forehead, feeling the wonder of how life changed and how it was that the child became the comforter to the parent.

It was only a short time later that, as if his mother had been waiting all day for him to arrive, her breathing slowly became more shallow and reduced. As he carried on chanting her breaths became less frequent and finally…

Naturally…

Stopped.

He carried on chanting for a few minutes more – rather for himself than for his mother.

When he stopped, his tears came – quietly – matching the peace of her death. They were not angry or upset tears but rather – fulfilled – rightfully placed.

After seeing to the details he walked back to the car feeling somehow more whole, as though this was a moment he had been meant for, a seminal point in his life. He had carried his mother over a threshold, gently and lovingly laying her soul to rest in a safe place where he knew that whatever followed, it would be right for her.

Despite all his shortcomings and self-doubts this was one thing he knew he had done well.

TANGO THOUGHTS: True Movement

My thoughts recently have turned to teaching beginners Argentine Tango at TLC in Southampton.

Afternoon Milonga

Afternoon Milonga by Pat Murray

In preparing for this I experienced an interesting case of synchronicity as I was reading a book by Massimo Scaligero called ‘The Secrets of Space and Time’. In this book Scaligero talks about ‘true’ movement, an idea that smacked me right in the forehead as I saw its relation to the essence of dance!

Most of the time we move normally without much thought. We think functionally: I want to pick this object up; I want to move my hand to here etc, and our bodies do this for us.

If you wish to truly understand just how much is automatic go and talk to someone with Parkinsons, a disease which afflicted by father. Even now I remember his tears as this once amazing dancer tried to will his body to do the simplest tasks.

During this unconscious movement our bodies move us. But there is a different form of movement where we can learn to be more conscious as WE move our bodies. The former is something we cannot take responsibility for, and cannot truly individually own. The latter is where we can truly express ourselves.

This second form of movement is not purely a physical phenomenon – it has its source in our thinking. True conscious movement is the physical expression of our Thinking – capital T.

Try the following exercise and you might just be able to experience how different this movement is from our normal unconscious mode:

1: Stand up in a space where you have room to stretch your arms straight above you without hitting any ceiling. Stand with you arms down your sides.

2: Raise your arms to the horizontal position as you would normally do – unconsciously, without giving it much thought. You might experience this as a pushing feeling. You should be able to feel that you do not own the movement, just its result.

3: Now return your hands to your sides and imagine that there are infinitely long threads (and I mean infinite!) extending straight out from the ends of your fingertips. Imagine that these threads are raising your hands as they pull outwards and upwards. With your thinking imagine how, as those strings traverse space, you are cutting the universe in half out from your centre to infinity. You may need to do this quite slowly and might experience it as more of a pulling feeling.

This takes conscious practice and requires a disciplined – yet relaxed – imagination. Properly done it will feel totally different to step 2. It will be as if your Thought is moving you, as if the body need do nothing but attend to the Thinking movement you are trying to realize.

This is the essence of the movement of a good dancer

They will be centred in themselves.
They will be poised in space.
They will seem to move with a graceful lack of effort.

They are doing nothing less than touching the infinite with their own Thinking movement as it expresses their individuality into their bodily movement.

They will be collapsing that infinitude to a singularity in their thought and thus, given the relationship between time and space, they can bring us to a timeless point of awareness. According to Scaligero this is the true awareness of space and time and it will take some practice.

Tango Sketch 2

Tango dancers sketch by Pat Murray

Bringing this back to dance, and Argentine Tango in particular where we learn to consciously walk with a partner, it is noticeable how difficult it is to re-learn something we normally unconsciously do. In this case something we learnt in our formative toddler years before our ‘I’ became present (usually around 3 years old).

If we can individually achieve this consciousness and attention it can be very enlightening and life enhancing.

If we can do this with our dance partner then it becomes a truly creative, artistic and sublime experience.

Happy dancing!

ACCU2016: Talk on Software Architecture Design 5: Active Design Ideas

In the last post I highlighted some specific design problems and associated solutions. Now I want to look at these solutions a little more deeply.

To refresh our memory the solutions were as follows:

  1. Separating Mutex Concerns.
  2. Sequential Resource Allocation.
  3. Global Command Identification.

I want to characterise these differently because these names sound a little like pattern titles. Although we as a software community have had success using the idea of patterns I think we have fixed the concept rather more than Christopher Alexander may have intended.

I want to rename the solutions as shown below in order to expressly highlight their dynamic behavioural aspect:

  1. Access Separation.
  2. Sequential Allocation.
  3. Operation Filtering.

You might have noticed in the third example the original concept of “Global Command Identification” represents just one possible way to implement the dynamic issue of filtering operations. Something it has in common with much of the published design pattern work where specific example solutions are mentioned. To me design patterns represent a more fixed idea that is closer to the actual implementation.

Others may come up with a better renaming, but I am just trying to get to a more mobile and dynamic definition of the solutions. Looking at the issues in this light starts to get to the core of the issue of why it is so hard to develop an architectural awareness.

If you can truly understand, or ‘grok‘, the core concept of this characterisation, regardless of the actual words, you will see that they do not really represent design patterns – not in the way we have them at the moment.

This is where there is a difference between the architecture of buildings – where design patterns originated – and the architecture of software. Although both deal with the design of fixed constructs, whether it be the building or the code, the programmer has to worry far more about the dynamic behaviour of the fixed construct (their code). Yes – a building architect does have to worry about the dynamic behaviour of people inhabiting their design, but software is an innately active artefact.

Let me recap the debugging and design fixing process in terms of the following actions that are carried out in order:

1: Delicately Empirically Collect the Data.
Here we have to be very aware of the boundaries of our knowledge and collect information in a way that does not disturb the phenomenon we are looking at. Awareness of our own thinking process is vital here.

2: Imagine into the Problem Behaviour.
We have to imagine ourselves into the current behaviour that the system is exhibiting. (This is the hard bit when you are under pressure and is what requires a strong focus in order to understand what the existing design is doing)

3: Imagine into the Required Behaviour.
We need to imagine into what the required behaviour of the system NEEDS to be and it is here that we start to meet the ‘gap’ between problem and solution. It may indeed only need a one line fix, but quite likely there is a deeper design problem. Again here is a point where our self-awareness is important. Do we have the discipline to make ourselves stop and think more carefully and widely about the presenting problem?

4: THE GAP. Cognitively Feeling for the best Solution Concept.
In this stage there is a very fine “Cognitive Feeling” in action to decide what is a good fit to the problem. For the experienced programmer this is more than just a question of “Does this solution fit the requirement?”

There is the consideration of whether the proposed solution idea is going to be a sustainable fix during the future lifetime of the project.

This question is much like asking myself if I will still find
this painting beautiful in 10 years time.

YachtClubSmall

There is a current widely held belief that the best procedure for coming up with a design solution is to produce many possible alternatives and evaluate them in order to choose the best one. In practice I have found that this very rarely – if ever – happens.

I usually arrive at a single design solution by trying out the multiplicity of possible solutions while in the ‘gap’ where I am considering various alternatives – imagining each of them in operation, possibly ‘drawing’ the thoughts out on a whiteboard as I think.

In this part of the process the more experienced programmer will slow things down to the extent of even putting in a provisional simple solution that gives them some breathing, or thinking, space. This is the idea of provisionality mentioned by Marian Petre, because this mode of design thinking requires time and reduced pressure.

It is amazing how often this happens in the shower!

Of course this is predicated on the fact that I have done the required detailed groundwork, but as I mentioned in the poem, our logical thinking can only take us to the boundary of what we know. Trying to push to go faster results in inadequate and buggy designs that are based on immature thinking.

This is the central conundrum of software development. The more we dive down into detailed analysis, the more we encounter these ‘softer’, heuristic elements.

5: Implementation.
Finally we get to the implementation. As you will have seen it is far too easy to jump into “premature implementation”. It is hard, if not impossible, to teach people just how small a part the coding is of the whole process. It needs to be experienced. Until you have seen how a good design triggers an amazing collapse in code complexity, the importance of taking the time to search for that great design is not an obvious conclusion. This is a fundamental eye of the needle that all programmers need to go through.

This is the main reason I like programming:

I get less code.
I get something I can reason about.
I get something that does the job!

Beautiful!

In the next post I am going to show how the dynamic design solution ideas and the human analysis process link to what I will call the “Organising Principle”, a term I have borrowed from Rudolf Steiner’s lexicon.

ACCU2016: Talk on Software Architecture Design 6: Organising Principles
ACCU2016: Talk on Software Architecture Design 4: A Design Example

Asking the Wrong Question?

Maybe this EU Referendum is asking the wrong question.

One week to go and I still don’t know what vote will be best. Though I am probably more of an “innie” than an “outie”.

There do seem to be some common concerns on both sides about not getting done in by faceless bureaucrats or greedy business leaders.

So I am realizing that perhaps we need answers to some other questions:

  • Why are there no politicians whom I can trust further than I can throw them?
  • Why should we have to fight SO hard to protect our society against business interests?

And maybe:

  • Are we ever going to stop voting with our wallets and realise its not all about economics?

Hmmm. Good Luck – we might need it.