At some point in your interactions with our co-operative, you might have heard us talking about CoVE. But what exactly is CoVE, and how do we use it?
In short, if CoVE didn’t exist our work would be much more difficult. CoVE (which stands for Convert, Validate and Explore) is a utility that sits at the heart of our co-operative. It started as a way to transform data between JSON and CSV files — but has since grown to include many different features that helps us to help others to publish and use open data. It’s also open source, so anyone is free to build on it.
A short history of CoVE
Initially, CoVE was built to work with the 360 Giving and Open Contracting Partnership data standards. As we wanted to offer similar functionality to both users, and the standards are structurally similar, we figured they would be able to share most of a codebase that would enable data conversion (aside from a few bits that needed tweaking to each standard).
We’ve previously written about why we think sharing code strengthens the open data ecosystem. In short, sharing a codebase made developing multiple instances of CoVE easier — after time, upgrades to the Open Contracting version of CoVE became available to 360 Giving, and vice versa.
Since those early days, CoVE has matured beyond these initial aims of converting data alone — for example, we now use it to test the validity of data, and explore it from different angles. We’ve also built in additional checks to ensure data quality — for example, checking if the start date of a contract or a grant is before the end date.
Re-using code means the overheads of making a new instance of CoVE when we start working with a data standard are much lower. As we learn more and make useful tools to speed up basic processes, CoVE allows us to spend more time on the interesting and useful stuff. As a worker-owned cooperative, our intention is to improve efficiency to make our work more engaging.
When we began to build a version of CoVE for organisations that publish data using the International Aid Transparency Initiative (IATI), we introduced XML for the first time. As a result, we refactored the code to make all three standards more independent. Now, the shared functionality is in its own core library — with each individual use of CoVE building on that.
Alongside Open Contracting, 360 Giving and IATI, we now run instances of CoVE for the Beneficial Ownership Data Standard and Open Contracting for Infrastructure Data Standard OC4IDS.
What’s next?
The story of CoVE is about more than the tool itself — it’s a story of working in the open, collaboration, and how our thinking in the co-operative has evolved. It’s also a story of how the open data ecosystem has matured, and learning how to help us help others to publish open data.
We’re beginning to think of CoVE as broader than a utility that allows us to Convert, Validate and Explore data, towards how it helps us Test, Transform, and gain Insights and Analysis (although admittedly TTIA doesn’t make for such a readable acronym).
As CoVE continues to evolve, we’ll be sharing more about our underlying thinking, and how others have incorporated CoVE into their tools.