Origin of consciousness

Here’s a story about the origin of consciousness. Don’t take it too literally.

You decide to write a little simulation program, just for fun. You code a world that contains resources and machines. The resources sit around waiting to be used; the machines run around using the resources for whatever goals you give them. So you need to write some code for the machines, to decide what they’ll do. That code needs to contain a model of the resources, or the machines wouldn’t interact with them constructively. But that’s not too hard, and you get everything running pretty quickly.

After a while, you realise that machines can behave like resources, too. A machine might try to co-operate with other machines, for example, or exploit them, or even disassemble them, and use their components as  resources. So you upgrade the machine code to include a simple model of the machines themselves, alongside its model of the resources. You know you can’t build a perfect model of a machine, inside that machine, but an imperfect model will be OK.

Then you see a way to improve your machines again. You realise that the machines’ model of each other doesn’t include the fact that they can model each other. That’s because when you upgraded the machine code to version two, you only had version one to work with.

So you decide to write version three of the machine code, which will allow for the fact that they can model each other. But of course you run into the same problem – you wrote the third version when you only had the second version. So the machines can model the fact that other machines can model other machines, but not the fact that other machines can model that fact, too.

It looks like you’re in for an endless sequence of upgrades. But you go away and do something else, and come back with a cunning plan. You let machines construct their model of other machines using reflection. Each machine starts from the assumption that other machines are similar to itself. It then examines itself, builds a model of itself, and applies that model to other machines. This isn’t so easy to code, but you get it done.

And there’s a bonus: if you upgrade the machines again, you won’t need to worry about a parallel upgrade to the machines’ models of each other.