Learning a new language

There is an common saying in software development circles: learn at least one new language every year. This was made famous when being published as the very first learning goal in "The Pragmatic Programmer":

Learn at least one new language every year. Different languages solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut. [...]

I've tried to stick to this ideal, although I admit that in the last few years I haven't kept up. This year, however, I've started by trying to learn a new spoken language: German.

I landed in Germany last year for some consulting work with ThoughtWorks and I started picking up a few phrases here and there to get me through the day. Pretty soon I could do all my most basic things in German: tell someone I didn't know german, say sorry, say hello/goodbye, order at a restaurant, get my bill, go to the store, ask for directions. Of course, I had no idea what the syntactic structure of the phrases were: I just knew that they worked and I got what I wanted. And after a few months, I found that I'd learnt a lot. And so, since February this year, I've have been in Berlin taking intensive German classes to consolidate on that foundation.

The experience has been fascinating, mostly because of what it's taught me about myself.

The first thing I had to get over was discovering that my brain doesn't seem to be as good at learning spoken languages. I can remember the most arcane software keywords after a single use, but spoken vocabulary seems to vanish from my brain as quickly as it arrives.

They say learning multiple languages at a young age is crucial to easy retention of new languages. I've never learnt another language before (I was schooled in rural Australia, so I was lucky enough to learn decent English). So I'm going to posit that this has somehow stunted my growth in this area. Conversely, I learnt multiple software languages before the age of 12. So perhaps this is why learning these has never been problematic.

(Side note: Europe is amazing. I'm the only person in my class who speaks less than 3 languages.)

I've tried all sorts of tricks to try and remember vocabulary faster. But it seems the only thing that has worked for me is time. After nearly 2 months, and many, many hours looking up the same words and writing them down again, I'm just now starting to be able to rapidly recall the meaning of the more interesting ones.

I've also started to notice just how much I tend to think in English. When thinking through a problem, or trying to convince myself to go for that run I've been putting off, I've been doing it in English. I've never really noticed that before, and I hate to think of the amount of time I may have wasted constructing sentences in my head that nobody ever heard. (This may be a symptom of the communication age, as we must think out our words clearly and at the rate at which we can type, lest our brain get too far ahead of our hands). I've started feeding my brain german replacements for the common phrases I've noticed I use ("ich muss aufstehen!"). This seems to be working.

Lastly, I'm fascinated by the different approach needed for learning syntactic structure. German is full of rules, but with so many variations. And, unlike software, it is not enough to learn some simple ones and then just start to apply them achieve your goal in a naive fashion. One must also understand someone elses "code" to engage in conversation. Even understanding takes on a different meaning: it's not translation to something you already know (e.g. "this Ruby structure is the same as doing blah in Java"), but often something more innate because, for some things, there is no direct comparison. There is no underlying, regular system that a language "compiles" down to. What I've experienced so far is a continual set of "ah huh" moments, where suddenly something that was incomprehensible to me before is suddenly clear, obvious and no longer requires mental gymnastics to comprehend.

I'm looking forward to continuing this new journey, which I imagine will take many years. But it's worth it: simply having the opportunity to understand the way your own mind works, and to make it do something entirely new, is worthwhile and entirely fascinating.

JsMockito - Simple & Better Javascript Mocking

I recently put my developer hat back on and spent a few weeks doing some heavy Javascript development on a ThoughtWorks project in our Pune, India office. As well as writing code, I was also encouraging more and better unit testing of the Javascript codebase - but I was constantly frustrated by the lack of a good mocking library that let me do the sort of things I wanted to do (like mock callback functions).

So in my spare time I put together JsMockito, a Javascript mocking library heavily inspired by the awesome Mockito mocking library for Java.

Give it a go and tell me what you think.

http://jsmockito.org

Why drink it?

JsMockito is a JavaScript stub/mock framework heavily inspired by Mockito. To quote the mockito website:

"Mockito is a mocking framework that tastes really good. It lets you write beautiful tests with a clean & simple API. Mockito doesn't give you a hangover because the tests are very readable and they produce clean verification errors."

JsMockito aims to try and reproduce the clean & simple API, with a JavaScript twist. And why not add some variation to your drinking habits?

What do you serve it with?

JsMockito must be served with JsHamcrest. Not only do they go well together, it's essential to avoid a very nasty hangover.

How to drink it?

To use JsMockito with a JavaScript unit test framework, follow the usual installation/configuration instructions for the framework and plug JsMockito into it. If you're integrating with Screw.Unit (and why wouldn't you?) then you just need to make the following calls:

    JsHamcrest.Integration.screwunit();
    JsMockito.Integration.screwunit();

Once installed, you can verify with interactions:

    var mockedObject = mock(Array);

    // -- start code under test --
    mockedObject.push("one");
    // -- end code under test --

    verify(mockedObject).push("one");

Or you can stub method calls:

    var mockedObject = mock(Array);

    when(mockedObject).get(1).thenReturn("hello world");

    // -- start code under test --
    alert(mockedObject.get(1));

    // the following alerts 'true' as get(99) was not stubbed
    alert(typeof (mockedObject.get(99)) === 'undefined');
    // -- end code under test --

For a JavaScript twist, you can also mock functions:

    mockFunc = mockFunction();
    when(mockFunc).call(this, anything()).then(function(arg) {
      return "foo " + arg;
    });

    // -- start code under test --
    mockFunc("bar");
    // -- end code under test --

    verify(mockFunc)(anything());

    // or if you want to verify the scope it was called with, use:
    verify(mockFunc).call(this, anything())

Mockitos are also good for spys

Real super spies don't drink martinis - they go for mockitos. And just like Mockito, JsMockito supports 'spying' on real functions and objects and verifing how they were interacted with.

An example with functions:

    realFunc = function(msg) { alert(msg) };
    mockFunc = spy(realFunc);

    // -- start code under test --
    // the following alerts 'hello world'
    mockFunc("hello world");
    // -- end code under test --

    verify(mockFunc).call(anything(), "hello world");

or with objects:

    realObj = new Array();
    mockedObj = spy(realObj);

    when(mockedObj).pop().thenReturn("bar");

    // -- start code under test --
    mockedObject.push("foo");
    alert(realObj.length); // alerts '1'

    alert(mockedObject.pop()); // alerts 'bar'
    alert(realObj.length); // still alerts '1'
    // -- end code under test --

    verify(mockedObject).push("foo");
    verify(mockedObject).pop();

Who is your bartender?

This variation is served to you by Chris Leishman and friends. Also a big thumbs up and thanks to the Mockito authors for the inspiration!

Also thanks to the JsHamcrest authors, who made this easy.

Bailing out the automakers: are finished goods assets?

So the big autos are going to get bailouts. But there is a condition attached - they must have a plan to be "viable" by the end of 2009.

My suggestion? Use this crisis to make fundamental changes to they way they manage and report the bottom line. My (limited) understanding is that currently most of the big autos treat finished goods as assets, which looks great on the balance sheet and makes investors happy. But are they really assets?

TheAge newspaper just published an interesting article on the mounting costs of new cars that are sitting idle in storage yards as a result of the downturn. It includes an estimate that the australian auto industry losing approximately $2M a week just in storage costs, let alone in the depreciating value as those vehicles age and become less attractive to customers.

These cars are not assets. They are liabilities. They are non-liquid, have high "operational costs" (storage) and are losing value. A very bad investment indeed.

I think the balance sheets of the big autos need to be changed to reflect this. I would suggest instead focusing on ROI, as calculated in a throughput accounting model as:

ROI = (throughput - operational expense) / inventory
where throughput = net sales - totally variable costs

In this model, the bigger the inventory, the proportionally smaller the ROI (the same goes for the operational costs, although this often has less impact). Looking at things this way immediately shows the ridiculousness of treating these cars as valuable assets.

My understanding is that the Japanese auto makers report in exactly this way. Revenue is not recognised until the car is sold to a customer - not when it rolls of the production line to sit in a big finished goods yard. In the past, however, this sort of change would probably have been suicide; any firm making this change would take a massive hit to the balance sheet that would have decimated the share price. But given they now are already in a huge mess, perhaps this is exactly the time to be making big changes like this.

The future of proof

One of my favourite authors and bloggers, Bruce Schneier, recently posted an interesting blog on "The Future of Ephemeral Conversation". Essentially, the premise is that conversation, as it increasingly occurs over electronic mediums, is becoming less ephemeral, meaning that it is now lasting for much longer periods of time than we have previously experienced. This has complex ramifications on privacy as, due to the nature of storage and retrieval that electronic mediums provide, "private" conversations can potentially come back to haunt one years later.

There is another part to this. Electronic mediums provide not only a longer lifetime for communications, but also a wider spread. We no longer just experience direct conversation with our friends or colleagues, but we also experience the conversation they are having with their other friends or colleagues. It is now much more common to 'carbon-copy' communications, to post notes or details on (semi-)public and aggregated mediums like blogs, facebook, myspace or twitter, such that our conversations reach a much wider audience than in the past.

The effect of this is facinating and concerning. Firstly, it leads to the "information overload" that I believe our society is currently grappling with. There is much more information available, especially of a more personal nature involving our friends and colleagues, and we need to be more rapid in our consumption of it. There are many ramifications of this, which are all worthy of their own discussions, but I believe one direct result is that we have, or will, become increasingly less critical and more trusting. We will no longer critique each item of communication for it's accuracy or it's authenticity. This is result of necessity, as we will need to be more efficient at processing information and reduce wasted time. With so much more information available, and the barrier to dissemination of it so much lower, the vast majority of the communications we are privy to will be both accurate and authentic, so the assertion of this naturally become a task to compromise on.

The ramifications of this on our systems of law will be profound.

One ramification will be on the nature of "proof". Currently, a person cannot be held responsible for a written letter unless it is proven in some way that they wrote it. In an electronic world, this proof is almost impossible and yet we can already see that we no longer look for it or think of it as required. There are many legal cases already that are premised on emails sent and yet email is both trivial to forge and has no identifying traits like a writing style or signature to authenticate on. It is incredibly hard to prove who was sitting at the keyboard, or even which keyboard. But our increasing implicit trust will (if it hasn't already) sway people away from this sort of critical thinking. If the overwhelmingly majority of communications they have been privy to are authentic, then they will likely a have greatly reduced level of scepticism when it comes to proof.

The second ramification will be on due process and the burden of proof. Under our existing process, it is required that reasonable proof be supplied before a person is considered responsible for their words or actions. In addition to lowering the expectation for proof, the rapid and wide dissemination of information will result in a great potential for consequence to be enacted before proof has been offered. We are already starting to see this in media, when reporting influences the opinion of the public on an issue before it has been determined in a court. Whilst our existing system has mechanisms to prevent this perversion by the official media, there is no such mechanisms in place for the myriad of alternative information channels that are now becoming available. Combining rapid, wide and personalised dissemination of information with an increased trust in the accuracy of it creates a risky new landscape in which we all tread.

I do believe that electronic communications, with all their ramifications, are overwhelmingly a good thing for our society. The value they provide is massive. But we must consider the consequences in terms of (reduced) privacy, increased visibility into the past opinions of an individual that may no longer be correct or even relevant, the nature and burden of proof and legal due process. We just have to consider the effect these consequences will have on our systems and adjust them accordingly. Ignoring the changing environment and assuming our systems are, and remain, perfect would be a failing for all of us.

XDR-TB: A preventable pandemic

[This blog post will be off-topic, but I feel inspired to write it.]

It saddens me to see that there is so much more we could achieve in this world, if we weren't confounded regularly by politics and greed. And it makes me angry when I see real problems, the sort that could destroy our world as we know it, that could easily be stopped if we put politics and greed aside and co-operated as a race of human beings.

I recently learnt, courtesy of a TED talk, about Extremely Drug Resistant Tuberculosis. TB is a horrific and dangerous disease, but one that we barely give a real thought to in the modern western world - because it's totally treatable and preventable. However in the rest of the world it still kills 4400 people every day.

This, unfortunately, is not the real problem with the current state of TB. Due to lack of funding and co-operation, especially with poorer countries, TB has been mistreated on a large scale - usually by supplying insufficient courses in the antibiotics that treat the disease. Because of this under-treatment, TB has evolved into a new form that is resistant to the antibiotics. This new form of human-made TB, known as Extremely Drug Resistant Tuberculosis (or XDR-TB) is not preventable or treatable with most current drugs.

It's heart breaking to see how easily the west can ignore poverty and other problems in poorer nations. In this case, however, the "problem" isn't going to stay out of sight forever. There is absolutely no reason for this disease to have been created, other than through politics, greed and ignorance. And it needs to be stopped now, or a global pandemic will threaten the population of the world.

Product development is not a good analogy for software delivery

In a quick break from my current obsession with Throughput I thought I'd talk about something I often hear from folk theorising about software development, especially those running Agile software development teams - that the process of delivering software bears more similarity to industrial product design and development than it does to industrial manufacturing.

I believe this to be true for shrink-wrap software product delivery. However the majority of software development is not shrink-wrap software products. The majority are software systems that support or enable a business function - things like websites, transaction management systems, scheduling systems, etc. In these cases I believe that the analogy to industrial product design is misleading.

The ultimate goal of all business software delivery is to make money for the business. In the shrink-wrap product case it's to create revenue through sale of the product. For the rest it is to create revenue by enabling or improving a business function. The software delivery process is a business process improvement mechanism; and the software is simply a side effect.

The last point is important - software is a side-effect. If the business could achieve the same improvement/enablement goal, by some cheaper or more effective mechanism than producing software, then that's exactly what they should do. Luckily for us in the software industry there are many cases where software is the best solution!

So given that we're usually not producing software to sell as a product and instead producing it as side-effect of improving a business function, it makes sense to try and provide any improvements as rapidly and efficiently as possible. In this sense, software delivery is less like the design of a product and more like a continuous manufacturing process for discrete business benefits.

Of course, there are differences between delivery of business process improvements through software and traditional manufacturing. The most obvious one is the substantially larger degree of uncertainty in the effort that will be required to deliver a given improvement (benefit). Manufacturing can often predict how long it takes to build a product down to minutes or seconds. We don't have that luxury. As a result software delivery suffers from much larger statistical variation in each stage of the process, which has large ramifications on the way we must manage the process (hopefully I'll discuss this in further detail in future posts on throughput). On the positive side, software delivery often has a simpler process than many manufacturing processes - in many cases it's quite linear with a single chain of clear and well understood dependent stages.

The presence of uncertainty and variation is what leads people to think of software delivery as a product development process. I can really see the attraction of this thinking because it captures the creative nature of much of the work that occurs and thus makes the larger degree of uncertainty and variation implicit. However I think applying this thinking too widely has detrimental ramifications to how we understand and improve our value to businesses. It leads us to think about the end software system exclusively and to create software 'projects' that attempt to build the whole system, rather than thinking about how we can deliver each individual business process improvement separately (and rapidly). This form of batch thinking can severely limit the flexibility and the value software delivery offers businesses. In turn, I believe this to be a clear reason why many CxOs are afraid of IT and look for ways to avoid involving IT in new business initiatives. If everytime the CxO wants to make an improvement or change to their business and asks IT for help, then gets told "we'll need a new XYZ system", they'll quickly learn to stop talking to IT.

In summary, whilst I believe that the design and development of discrete changes or additions to software systems to be a "product design" activity, I consider the software delivery process, which provides for continuous and on-going addition of value (through business process improvement), to be more analogous to manufacturing. It's about moving peices of work (a software system change/addition) through a continuously improving pipeline.

Defining Throughput

In this second part to my series on Throughput Accounting, I'm going to explore the concept of Throughput, starting with a definition.

Every system has a goal it is trying to achieve. For most business processes this goal is revenue - money given to the organisation by customers due to sales of product or services. Given this, 'Throughput' is simply defined as a measure of the rate at which a system is achieving its goal. It is not a measure of profit or any indicator of efficiency, it is just a measure of flow.

Throughput is often expressed in terms of 'goal units'. In for-profit organisations, these units may be dollars of revenue. For non-profit organisations it may be expressed by another measure, such as patients successfully treated in a hospital emergency ward.

When dealing with monetary goal units, Throughput (T) is often measured as gross revenue from sale of a product (S) minus totally variable costs (TVC), which are only the costs that vary directly with the system output. In a manufacturing setting, TVC is often just the cost of raw materials. Thus T = S - TVC.

In the example from my last post, a company was producing 800 widgets per month. Their TVC, for materials, was $40 per widget. If those widgets were being sold at $50, the throughput per widget would be $10 and the system throughput is $8000 per month, or around $0.18 per minute.

In calculation, this is identical to the cost accounting metric 'contribution margin'. The contribution margin is used to determine the fraction of sales that contributes to offsetting fixed costs and can be used to disambiguate some decisions, such as the purchase price example from my previous post. As Throughput Accounting manages fixed costs separately, it is not required for this purpose - but has a much broader role.

It is important to note that Throughput is measured only as the ultimate achievement of a system, for example it may be the final sale of a product or service. It is not measured in terms of production of assets that will then be processed further to achieve the ultimate goal (eg. assets sold to achieve revenue). This is inventory - which I will discuss in a later post.

A good question is how to define Throughput in the context of a software delivery engagement. Ultimately the software system supports a business process which is in turn producing revenue - so it could be defined in terms of dollars made by the business. Pragmatically, however, it is more useful to define Throughput as the rate at which software additions and/or modifications are delivered into an environment where they can be utilised by the business process.

It is tempting to measure this rate in terms of 'business benefit' (possibly also in dollars), which is the ultimate goal unit of the entire business process. This is an important thing to focus on as often the best improvements can involve organisational changes as well as software changes. However, when the software production process is the organisation's constraint (as it often is) then we are focussed on just this process alone. In which case we need to use a 'goal unit' that is reflective of the amount of the work done to produce some change in software.

In Agile software delivery methodologies there is the concept of 'story point', which is simply a relative measure of the effort required to deliver a particular requirement. Given its relative nature, we can measure it over an arbitrary time period to determine the rate of delivery of story points, which we can use as Throughput. This is effectively the same as the Agile term 'velocity' and this comparison is useful for applying some TA concepts to software delivery. I'll return to this point in future posts...