A recent posting on the W3C egovernment discussion group was on the topic of IT procurement for eGov/transparency case studies and one of the members revealed that a project she was working on at the moment, involved mapping, with documentation (as documentary evidence), for each eGovernment initiative, how the IT resources were obtained. However when she “consulted the UK government last year, including issuing FOI requests, about the procurement, of CKAN for example, particularly wanted to learn what was the original spec /requirements for CKAN, and what kind of funding was granted, based on what agreement/tender. however, [she] was unable to access any document available, including no contract between the UK government and OKFn for the development of CKAN, although there is some evidence of moneys having been exchanged between the UK Government and OKFn for the purpose of funding CKAN development. ”
She then asked “Does anyone have any info?”
I suggested that “One would hope PDM could extract the information from the CKAN’s mouth itself – http://data.gov.uk/dataset/public-sector-procurement-spend” and suggested Paola look there…
However, Paola’s response was “I was not capable to answer the questions using the website http://data.gov.uk/. It looks to me that the website is designed especially to avoid making transparent the information they are looking for (for example show me the contract between a and b, or how was decision x achieved?) This is what I mean that a service should be designed/driven with usage in mind, and not just stick the word usage as a popular trendy label ‘ (for example, to answer specific questions such as ‘what contractual agreement drives the development of an open data platform such as CKAN?) But if you can drill further with a better understanding of the data.gov platform, please assist! its possible that I am missing something”.
So if the technically proficient are having problems extracting procurement information from the UK government, why are there still claims to be open, transparent etc?
The ongoing conversation is on the W3C site. Can anybody help Paola? Is this transparency or not?
The Great E-mancipator 
When Data.gov.uk was set up under the previous Government it was mainly intended to be a central portal highlighting the availability of data sets held by the UK public sector. The previous Government was responding to calls from journalists and others in civil society for more “open data” release of public sector information.
Data.gov.uk was not really designed to support the current Government’s more heavily politicised “transparency” agenda, which has moved the policy focus away from release of technical and core reference data (i.e. data actually used by the public sector in its work) and now places greater emphasis on release of data for monitoring spending and performance of public sector bodies.
Although there are all sorts of other criticisms that could be made about Data.gov.uk, I don’t think it’s really fair to say that it is designed especially to avoid transparency on specific questions such as the one Paola is trying to answer. Answering those questions (at least directly) just isn’t what Data.gov.uk was intended to do.
The Cabinet Office has recently produced guidance on transparency in public sector procurement and contracting:
http://www.cabinetoffice.gov.uk/content/policy-and-guidance-transparency-public-sector-procurement-and-contracting
However this focuses on new procurement opportunities rather than on access to information on contracts and arrangements already in place.
Ideally it would be good if the Government openly published the documents behind all of its existing IT procurement arrangements, but I don’t think that is very likely to happen. Even if the Government were to commit to that level of transparency, many of the contractors involved would object on the basis of their commercial sensivities.
The best route to obtain the information Paolo requires is still the Freedom of Information process. I note that Paolo has already issued some FOI requests; it would be useful to know to which bodies those requests were made and the responses received. It may be the information does not exist, but alternatively it may be the requests were not correctly framed or the release of information was denied on the basis of FOI exemptions that Paola could challenge.
Owen
thank you very much for the thoughtful reply.
Let me state that I do not wish to attack the UK Gov in specific,
but that
a) I do have an axe to grind
b) the UK Gov on this occasion provides a great example of what is an interesting paradox: on the one hand, they are at the forefront and champions of open data and transparency initiatives, on the other hand, ultimately a very closed club which looks inwards, rather than outwards, and makes it very difficult for a member of the public, and of the press (I have a press card and I am accredited with the UK government press office) to ignore me.NOTE They never reply to my questions, sent directly to the head of Press, never invite me to any of the events organised by the UK governments in relation to open data and transparency , despite my explicit request.
They simply have a team of media mercenaries and cheerleaders they deal with, and ignore anyone asking too many questions. Everyone knows that politicians are the biggest cheats on earth, and I do not personally wish to interact with any of them, other than to strictly access information that they hide when this becomes necessary.
(I happen to be long term UK resident, and although I do not live there now, i still consider it a great country and largely home, despite the local government mafias I occasionally have to deal with there, like anywhere else)
You provide some useful background info, may I ask how do you know the history, do you work for ‘them’? Please disclose if you are paid/hired by the UK government or any PR company they may directly or indirectly hire to wash their dirty laundry, or any other affiliation if any.
You say
“Answering those questions (at least directly) just isn’t what Data.gov.uk was intended to do”
Why not?
Open data repositories should be designed to allow citizens answer questions they may have,
How the data schemas are designed the ‘logical schema’ can make it easy or difficult to find information. Looks to me that in this case it is difficult-impossible,
I maintain, and reiterate, that open data sets around Europe are all being published in a way that is deliberately preventing critical information (say business deals, etc) to come through, and they have teams of cheerleaders to make so much noise around the vacuous efforts, so that really fundamental issues are pushed at the back of the agenda, or left out altogether.
Poor design is one of the big challenges the ‘opendata usage’ debate is facing today: is the data we publish useful to answer questions and provide information citizens may want to ask?
yet it does not seem to be in the WG agenda.
Lastly, on the wiki page linked, repasted
http://www.w3.org/egov/wiki/EGov_Procurement
is the copy of the FOI reply I received, loaded on Google docs.
I appreciate the interest. can it be viewed?
https://docs.google.com/file/d/1Qc2uxKSl8aC7Rz-w52583P5B7Qc57zhHQLG3p6-zv8z1eD1VXk-k5sLdKZ-_/edit
(at my end it says anyone with the link can view, no sign in required)
Let me explain the case better: there was some ‘evidence’ that the Government paid money to OKF, this came up in various conversations, and eventually a blog post, written by the main CKAN developer (in 2010).
http://www.groovy.net/ww/2011/07/misinformation.txt
at the bottom it says:
“Looking at the data from the Cabinet Office (COI), at this time OKF has billed about 500k GBP and has been paid 400k GBP”
(Note the developer in question is no longer working for OKFn, around about the time the questions were publicly raised, and I have not maintained personal correspondence, although at least another phd researcher issued a FOI request and got no answer)
I thought at this stage I should go straight to the source, and issued a FOI request to the Cabinet in April. In August, while not having received a reply, I complained to the Commissioner, and received a letter that said the Cabinet had written to me in July (well beyond the 20 days response time). The one linked above in the google doc which I hope is visible.
I became seriously disgusted with open data, the cabinet, and everyone who is continuing to put up with the transparency and open government farce in the UK, as well as in most EU countries.
You are right, I should continue to chase an answer but in the meantime i got busy, finished a Phd (recently ) where I integrate FOI requests as part of evidence gathering/research method which is quite innovative, and specifically focus on some aspects of open access, and usability and the pragmatic approach
I am currently on sabbatical, taking time off, thinking what is the best way ahead, or maybe i should finally get away from it all.
I am tired (and somewhat disillusioned) of dealing with dumb blind technocrats, the political mafias, their incompetence and lies.
Suggestions welcome
PDM
Paola,
Many thanks for your reply.
I did not mean to defend Data.gov.uk particularly, only to point out that it was not designed conceptually to assist with this type of question. I quite agree that the UK Government should have much better processes in place to respond to serious enquiries from members of the public and the press.
I can assure you that I do not work for the UK Government or any PR company, and my opinions are entirely my own. My professional background is as a geographic risk analyst in the insurance industry, and my main area of interest is in sociodemographic and environmental data.
My knowledge on the subject of open data comes only from following the development of UK policy over the past several years, and from my own discussions and correspondence with UK civil servants during battles to obtain data and information.
I find your comments (and your anger) very refreshing. In my view UK “data journalists” and other users of open data are often too willing to engage in uncritical boosterism for transparency initiatives, without bothering to study the details, and are overly grateful for whatever data the Government decides to release.
I have read the FOI reply and the blog post. I think it might be worthwhile to write to the Cabinet Office again, if you are not too demoralised to do so.
I have looked at some of the spending data for the Central Office of Information (COI) that is available here:
http://data.gov.uk/dataset/financial-transactions-data-co-coi
The returns for April, May, June and July 2011 include six payments to Open Knowledge Foundation Limited, totalling £211,625. Two of the transaction records include a contract number: 114606. (I have not looked at previous years but if the OKFN reports from Jason Kitcat are correct there may be some additional earlier payments.)
The COI was a part of the Cabinet Office from 2005 until it was closed in March 2012. It is therefore quite clear that the Cabinet Office did have a contractual relationship with the Open Knowledge Foundation, and that the FOI reply you received was incorrect.
I hope that is useful. I quite sympathise with your feelings of frustration. If you think there is any help or further information I can provide please do not hesitate to let me know.
Best regards,
— Owen Boswarva
Thank you very much Owen, lack of results and always going against the grain have actually drained me somewhat, my energy to address this is slowing a bit. Thanks a lot for taking a look at the figures with a fresh eye, I shall use your input to drive a further request. Will reference this exchange in the request.
Yes, you are right that the contracts may not be accessible due to commercial sensitivity, but information, at least some information about the contracts should be available. How can the public assess if a contract was awarded fairly – a fair process – or is the result of, say, some bribe? (entirely possible supposition, you will admit)
I understand why you say the current lack of usefulness of data.gov.uk may not be intentional, but accidental.
I research how ‘deviations’ occur, and have established with some confidence that if a correct process is not followed, the chance of a correct outcome is diminished.
In this case, if data.gov.uk development is excluding and selectively weeding out expert contributions, by allowing only the cheerleaders to provide input and lead development, and as you say, disabling the critical input, then it is likely that the current effort is limited /flawed as the result of an intentional process of weeding out critical input and narrowing the clan to a small group of people who actually seem to cherish rubbing shoulders with each other..
A ship can be designed to float, but can also be designed to sink.
If a ship is designed to sink, I d be tempted to think its not by accident, certainly not in a country like the UK
That is what I meant. Thanks a lot again for taking an interest, and
hope to exchange again
will update this thread with progress on the issue. cheers
PDM