NLP Modelling, Criticism and Scale

NLP modelling

Recently, I’ve seen an increasing amount of people who don’t understand something important about NLP modelling.

The first group struggle to know whether a model is complete – whether it has enough information or too much. So many (otherwise promising) models languish in a desk drawer, or in a neglected folder on the hard-drive for just this reason.

Perhaps that is why there are so few published NLP models.

So add “How will I know that the model is complete?” to your set of questions preceding modelling.

Bear in mind that I’ve heard a lot of glib answers to that question – answers that don’t hold up very well when modelling moves from pure theory to practical activity. And it really doesn’t matter what your favourite NLPer says about modelling unless you understand it well enough to actually create a useful model.

A second wave, composed mainly of armchair experts, criticise the usefulness of models that others do produce. Common criticisms are:

  1. the model doesn’t do X
  2. the model only works in a certain context/situation
  3. the model only works with a certain type of person
  4. the model is ‘too simple’
  5. the model is ‘too complex’

Some of these criticisms may be valid. However, it’s much more common that the critic has misunderstood something about the nature of the model.

There is one idea which will help with both these problems.

That idea is ‘scale‘.

Let’s use maps as an analogy for an NLP model.

Maps are a simple way of representing a complex physical space. Different types of map represent that physical space in different ways. They do that for a variety of reasons because they fulfil a variety of purposes.

Mostly, the critical principle is to omit information that does not support the purpose of the map.

For example, a road map does not feature individual buildings. It includes no information about power cables or drainage. There are no contours, or information about height on a normal road map. Even some (extremely minor) roads may be omitted.

My point is that you know when a road map is complete when it is fit for purpose. If you focus your mind on the uses it will support, you will more easily understand what territory it must cover, what needs to be included and what can safely be omitted. This is true of any model. So this principle of scale (map detail) can be used to answer concerns relating to completion of the model.

The second group – the critics – have a different set of problems. When you look at the five commonly used criticisms I have listed above, some answers now become clear.

For the first three criticisms, the answer is “so what?” Yes, it’s valid to slam a road map if it doesn’t guide you from A to B by road. However, it is ridiculous to criticise a road map if you try and use it for a walk in the woods. That’s not what it was made for. Few people would get upset when their toaster can’t be used to boil water, so why complain when a telesales model doesn’t work well through the medium of email?

For the last two criticisms, the answer is “too simple/complex for what?” The scale of the model depends on the purpose for which it was created. A model for selling lawnmowers in person will not necessarily work fully for selling lawnmowers by phone. It probably won’t work for selling coffee either. A general ‘model of selling’ should work for selling all these things and will likely embrace greater complexity than a more specific sales model. In short, scale matters.

These are principles of design and are easy to learn. Perhaps that would be a useful place for NLP modellers to begin – with good design in mind. It would certainly give more people the confidence to finish – and publish – their work.