The Evaluation is Late
The text of my FOI Request last month was as follows:
Dear Department for Science, Innovation and Technology,
The University Enterprise Zones pilot projects were set up in 2014 in response to the Witty Review with a ten year evaluation plan to provide learning and feedback into the process of funding innovation in relation to universities.
An Interim Evaluation (dated July 2018) was published a year late following my FOI request.
The contract for "Final Evaluation" should have ended on March 2024, but its publication has not appeared on the Department for Science, Innovation and Technology website.
* Under the Freedom of Information Act, please can I have a copy of this Final Evaluation Report into the UEZ pilot projects "promptly" (Section 10).
The reason for this urgency is that Sensor City, the project in run by Liverpool City Council, LJMU and Liverpool University closed due to failure with no lessons learnt, while at the same time an almost identical project is being moved forward by the same three bodies without a reasonable expectation of a different outcome.
The response was:
The final evaluation of the UEZ is still being completed. This is because it has taken longer to access business data for econometric analysis than anticipated. We have extended the timeline to ensure the report provides a fuller picture of impact. We are committed to publishing the final evaluation to the DSIT website. Currently, we plan to finalise the report by the end of the year
I thought: What possible “business data” could they have any problem accessing?
There’s a clue in the Evaluation Plan:
The best methods for measuring impact are by using experimental or quasi experimental methods. These allow the "treated" group, or those engaged in the intervention, with those University Enterprise Zones Pilot Evaluation – Outline Evaluation Plan and Baseline that are not. Broadly, comparing these groups will provide evidence of impact. (p31)
and
One of the possible variants of the 'difference-in differences' approach appears to be suitable: comparing the change in some measure of business performance over the period of implementation of the programme for treated and untreated firms. The question is whether such an approach is feasible in practice and likely to be able to detect the scale of impact that the programme might plausibly have.
The number of firms in each case has to be ‘sufficient’ to achieve a sufficiently low standard error... at least 50-100 firms in each category (treated and untreated) would be needed. (p36)
In other words, we want to compare the job creation numbers and profit margins of the companies who took advantage of the high rents in our swanky Sensor City building to imaginary similar companies who didn’t waste their time and money on it, in an effort to prove that this £15million pile of routine concrete and steel was a net gain to the economy in some numerical sense.
What they are absolutely not going to do is ask people why they did or did not move into the building they built for them having not bothered to ask if anyone actually wanted it, or report back with their honest answers, like: “Because it was a stupid idea and we told you not to do it.”
I’ve had to make a new FOI request:
Dear Department for Science, Innovation and Technology,
On 24 June 2024 I received an FOI response [as summarized above].
The original contract had been due to run from 23 March 2021 to 29 March 2024 at a value of £169,050.
This contract notice contained valuable information as to the scope of the study. In particular it detailed how there would be surveys only of the commercial tenants, without any analysis of the markets in which these pilot projects were to intending to enter on the basis that "there is a market failure that needs to be addressed."
Please can I have the following information:
(1) The request(s) on behalf of Technopolis Ltd setting out all the reasons and justifications for a contract extension. (eg what "business data" were they not able to obtain?)
(2) The response(s) and variation form(s) setting out the terms of the contract extension, including notices of any further fees.
(3) Copies of the "annual short presentations at BEIS" "updating on emerging findings, progress and policy implications" as listed in Part 3 (Deliverables), page 18 of Appendix A of the Letter of Appointment dated 23 March 2021, order number CR20101. [contract]
My intuition is that business data and opinions from enterprises who did *not* participate in a failed venture such as Sensor City would be a whole lot easier to obtain and provide a fuller picture of the situation than these efforts to obtain data from what little business had been undertaken within the confines of the project. Unfortunately, there was no moment of consultation in the whole process in drafting the evaluation plans and methodologies where such obvious issues could be raised and answered.
There is a new Government scheduled to take office next week. One of its missions is to Grow the Economy.
However, there’s every indication that they will be just as much a pack of amateurs as the current regime, and will continue to apply these half-baked cargo cult kindergarten notions of how to invest in high tech without any interest in the experimental results from the previous debacles.