Blindly manipulating symbols

This is an interesting presentation from Bret Victor, with some great ideas for how we can support new kinds of data-driven visualisation in software design (with many implications for the design of other ‘programming’ tools):

The whole video is interesting, but the thing that really stood out for me was his statement that “programming is blindly manipulating symbols”.

Blindly manipulating symbols. What he means, of course, is that programming is an abstract (or indirect) activity. When you draw, you put a pen to paper, where you place the pen is where the line appears: direct manipulation. When you program to ‘draw a line’, you are writing a set of instructions that must be interpreted by both software and hardware in order for a line to appear. What you manipulate (the code—symbols) is not what you produce (the image—a line). Programmers don’t know (they can surmise based on experience, but can never be sure) what the output of their programs will be.

For Victor, this is a problematic situation, in that it removes the directness of thought process that is manifest when working directly, in a material way, with the final artefacts of your production. (This was further elaborated in his earlier presentation ‘Inventing on Principle‘). I agree with him on this point, that in most cases, indirect manipulation is a hindrance to idea generation. 

What I want to say about the design that Victor presents here though is that while his work does make ‘direct manipulation’ easier, it hides a code/generation process behind the interface. Although direct manipulation is made more available for those who don’t need to learn to code, it is still only ‘direct’ within the technological limitations and UI affordances imposed by the designer of the software.

This is, of course, not to criticise Victor’s work, I think it’s incredibly valuable (and as he says, it’s just a starting point to get ‘tool makers’ thinking along these lines). Now, I’ve spent a lot of time over the years blindly manipulating symbols. And there are times where such blind manipulation can be valuable, especially if what you are interested in discovering is the  limits of the particular software/code/hardware system that you are working with. Behind Victor’s interface, the ‘blind manipulation’ is still going on, it’s just hidden from the users of the tool. (This is of course normal for software, hardly any users of software ever interact with the code in any real way). 

Writing this has sparked this incomplete thought: I wonder if there is some point in between what Victor presents here, somewhere between ‘blind manipulation of symbols’ and ‘direct manipulation’ that could allow for a level of experimentation at both a direct-drawing level and a code level? Hypercard?