He looked at me with incomprehension. "But what else can we do?" he said.
At the time, I thought this just meant that - after 10,000 new central government targets promulgated in the previous four years - I had hit on something important. I still think I had. But the question keeps coming back to me.
We know that our increasing reduction of almost everything to numbers, often in the guise of 'evidence' (which it usually isn't), is undermining our ability to control the world around us. It is hollowing out our knowledge just as it is hollowing out our institutions. It makes us believe that these human structures are giant, humming machines which can be programmed and left to run our lives, when this is the great utilitarian fantasy that still grips us.
Worst of all, it is beginning to hollow out our language. When once we knew that a concept like wealth or health were multi-dimensional, whereas the numbers are busily reducing them to pale shadows - meaningless, one-dimensional ghosts of their former selves.
I did write the book. It was called The Tyranny of Numbers (or The Sum of Our Discontent in the USA), but I have puzzled over the question ever since - what do you do instead?
If you abandon the numbers, then the media will impose them on you. It is difficult.
But I'm glad to reveal that I now have a very effective ally in this struggle. The Italian/South African political scientist Lorenzo Fioramonti launched his book How Numbers Rule the World in London yesterday evening. It is a highly readable challenge to the hegemony of numbers - and it takes the argument further.
Because, as Lorenzo says, that humming machine has now meshed itself into the economic system, with devastating and still uncounted consequences.
So I asked him what we can do about it. He said it is all about reclaiming the idea of governance - realising we can't let these giant number-driven machines take over. We have to put in the time, very locally, to run them ourselves.
"This is why the public sphere is so important. All those soft elements of social life, from mutual respect to solidarity, which systematically escape our obsession with measurement, are ultimately much more important than what is integrated into the numerical models driving contemporary governance... Numbers will not save us. We will need to do it ourselves."
Lorenzo comes at this from a different direction to me. We do need numbers to check our impressions, to take us by surprise. But we also need to retain our intuition to check the numbers, if we don't want to sink into a mechanical half-life.
When we believe the league tables and never meet the headteacher - or believe the graphs without asking the people who are being measured - we chip away at the foundations of what makes us human.
But it's difficult. I remember the chief executive of a well-known charity told me he had repeated the Scottish proverb that I ended my book with ("You don't make sheep any fatter by weighing them") during his annual grant negotiations with the Home Office.
He told me he watched the look of rage and incomprehension spreading over the face of the civil servant in front of him.
That was a decade ago. I'm not sure that things are any better yet.
Subscribe to this blog on email; send me a message with the word subscribe to firstname.lastname@example.org. When you want to stop, you can email me the word unsubscribe.