What’s the use of benchmarks…

May 4, 2010

The full title of this would be – “what’s the use of benchmarks if you don’t change anything?” This is stimulated by a post by Steven Clift on his newswire in response to the latest Pew Internet results.

Steve asks questions about the result to a Pew question, which is:

“Overall, when you have a question, problem, or task that requires contact with your local, state or federal government, which method of contact do you prefer most?…Calling on the phone, visiting in person, writing a letter, visiting a website, sending email [ Q.14 ]

Today              –                         Aug 2003

35% Calling on the phone – 38%

20% Visiting in person – 15%

11% Writing a letter – 15%

10% Visiting a website – 17%

18% Sending email – 9%

1% Some other way (Vol.) – 1%

4% Never contact government (Vol.) – 4%

1% Don’t know – 1%

*% Refused

Note from seven years ago that the most preferred way to contact government has sending an e-mail up 8% and visiting a web site down 7%. Very interesting. So for those governments and elected officials who have deleted their e-mail address from their website and replaced it with only a web form, please take note. Also interesting is a 5% increase in those who prefer to visit government in-person. Must be the free coffee.”

The issue I see with the Pew Internet results is that whilst they show the channel shift, they don’t help to explain it. They show usage of the web site dropping, whilst face-to-face and email increase. Without satisfaction ratings to reinforce the data, I’d assume that this was as a result of web site delivery failure and citizens falling back on email and face-to-face to get a service completed. But, that can only ever be an assumption without some data to support it.

The figures shown are for the USA, which I personally find quite shocking if UK e-government is modelled on that, it potentially shows a near-complete failure of e-government with a drift away from web to existing channels. Email might as well be face-to-face, it requires a lot of manual handling.

I suspect the US need to consider something like the Socitm Customer Access Improvement Service or one of the other channel comparison systems I list in my Company table V8.


Improving service

January 12, 2010

The Socitm Customer Access Improvement Service has published its latest (December 2009) report, which is Issue 3. It has received a great detail of reportage for its emphasis on poorly performing council web sites. I’m not sure that quite so much can be read from the cumulative data, and a bit like National Indicator 14 “avoidable contact”  believe these analyses need to take place at a more granular level and thing some of the assumptions are very subjective!

I also have a concern that a document from Socitm is making statements like the one on page 3 “The country cannot afford the current scale of the public sector.” This is a broad brush attack on all government, so includes local authorities and health trusts. This is not a decision for an IT managers organization, it’s one for the electorate since some countries, as we know, have a much higher scale of expenditure. What really matters is the quality being delivered for that expenditure, if its too high a quality or too low, the public have to decide. If too much is being spent they have to decide what services are no longer required, or whether services they can do without are being delivered. Ultimately this is the value of applications such as that used by GovMetric or the others named on my list (see below) – they give the public an opportunity to comment on the value of services delivered.

According to the report there are now 56 councils providing GovMetric data but of these only one is acknowledged to be recording data across the three major channels in one directorate or service only, which is not ideally what we should be achieving if we are to understand channel shift or manage channels at all.

Big things continue to be made about South Tyneside’s apparent channel shift around waste management, which they achieved by developing their web site as a result of feedback through the service, I would argue that all channels need to be improved and this is an end-to-end reform of services, since channels are only the presentation layer. We have a lot more experience with the face-to-face and telephone channels and have obviously some experience at delivering them, but the web is the new kid on the block, it can’t at the moment be interactive in the sense of the Turing machine.

I believe getting feedback from citizens is the way forward but I have doubts about making too much of it from the higher level generalizations that Socitm makes and I must say that the one promoted by Socitm is not the only solution – have a look at the list – Company table V8.

UPDATE – I’ve been asked by Alex Chapman of GovMetric to update on a few possible inaccuracies between my reading of the Socitm CAIS report and the state of play with GovMetric, which I am posting below –

  • “There are currently 59 authorities signed up to GovMetric with a further 9 housing associations; so, there are just under 70 users in total
  • More importantly, almost all of these are using GovMetric in a multi-channel approach measuring customer feedback and performance across at least 3 channels (F2F, phone and web) and across typically 8 services
  • An increasing number are also linking this feedback data to E&D and customer segmentation groups as well to increase their insight about what customers needs are, their experiences and their channel preferences.


I agree with you whole heartedly that, “if we are to understand channel shift or manage channels at all”, we do need to go beyond one service or even one channel; this is not the case with GovMetric, neither in concept nor in practice.  From a GovMetric perspective, customer feedback is not the only thing that matters, but being able to understand service demand by service, by channel, as well.”

How many visitors?

June 9, 2009

A quite recent piece (March/April 2009) in the M.I.T. Technology Review considers some of the issues around web site visitor statistics. In “But Who’s Counting” Jason Pontin considers the problems with the techniques employed by such as ComScore and Nielsen Online which include employing panels as they do for TV, along with log analysis.

Unfortunately, the pretty crude alternative to this seems to involve tagging by third parties to allow them to analyse the result. More unfortunately, despite venture capital investments there are clearly no bright ideas yet.

Having supported the work by the Socitm/GovMetric CAIS on visitors, I’m still surprised at how many visitors council web sites are purported to get in comparison with other channels, and this is obviously the case in the private sector with businesses wanting to gain accurate feedback on their web investments.

In the case of council sites I suspect its just as important as the private sector to employ qualitative feedback to improve the sites. The numbers, however counted, will increase.

What shall we do?

June 7, 2009

It was an interesting week (w/c 1 June 2009). On Wednesday morning I met up with Vicky Sargent of Socitm who was presenting the latest consolidated feedback from the GovMetric/Socitm Customer Access Improvement Service. Vicky and I are old acquaintances and so had time before and afterwards to compare notes, and I am pleased to say CAIS supports what I have been saying on this blog that we (government or any service provider) need to collect and compare feedback across all channels and use it to improve services across them all in a coordinated manner.

On Thursday morning I was travelling around the beautiful northern reaches of the very rural district that I work in, checking on polling stations. The fact that at one I managed to get a mobile signal by standing in the middle of the village green next to the tall steel maypole might indicate the limited coverage. If an emergency had occurred it would have meant looking for one of the BT K6 phoneboxes that are still around these ‘chocolate box’ villages! So, what about datacomms in these areas? What about modernising elections in these rural village and church hall ballot stations, many of which don’t have disabled access let alone Internet?

Friday saw me at the Yorkshire International Business Convention discussing broadband services in between listening to excellent presentations from John Cleese, Dave Stewart, Tim Sanders, Matt Pritchett, Tim Smit and Tracy Edwards about creativity, innovation and inspiration. One message was to be positive and stay that way.

The deep thoughts left me thinking about what should change, what could change, and how we change it.

My current conclusion is to use the above described multi-channel feedback and remodel services around it. When technology permits other things will be facilitated.