Profiting from free and underused data

Data is rightly seen as a valuable asset that companies should guard carefully. Having access to unique data sets and possessing the ability to use advanced analytics techniques to process them can separate winners from losers in the 21^st century – just ask Google or Facebook about how profitable that can be.

However, many firms are sitting on data assets that are not being utilized effectively or even used at all. There is also an abundance of freely accessible data made available by public bodies. McKinsey & Co. recently estimated that telecom operators could generate incremental revenues of 5% to 15% as well as reduce costs by 15% to 35% through the better use of existing data assets.

In the US it has been estimated that more than $220 billion was generated by firms that rely on data collected and made freely available by government agencies and other public bodies. Within the EU, legislation las been passed to make member states provide greater access to public information for commercial re-use by the private sector. The economic value of making this data freely available was put at €52 billion in 2018, rising to €194 billion by 2030.

Hiding in plain sight

Through analysis of internal files and documents and the application of a “digital assistant” to answer common queries from clients and staff, legal firm Fenwick and West estimates it saved almost $200,000 worth of staff time in 2018. As machine learning and analytics software is applied to data and records generated within an organisation from CRM, ERP and other processes it opens up a wealth of possibilities for operational improvements.

A key challenge for CIO’s and CDO’s is being able to extract actionable insights from the data. As Doug Laney, a VP analyst at Gartner, recently noted, “Data and analytics will become the centerpiece of enterprise strategy, focus and investment.”

Scraping value from the web

With more than 2 billion websites currently online, there have never been so many opportunities for companies to find potentially valuable data. However, identifying the right websites to investigate and then extracting the data that is required can be difficult. Web scraping solutions have been around since the dawn of the web, but they have now reached a level of sophistication that allow real value to be extracted in near real-time.

Of course, there are legal limits to what can be scraped and how the data can be used but it may be a process your firm should consider. If you want to know what is being said about your company online, what products your competitors are offering and the prices they are charging or how emerging fashion and consumer trends may affect your business then a web data extraction solution could be the answer. Firms such as California-based import.io offer data extraction services from the public web which can then be integrated via APIs into their client’s workflows.

Public data, private profit

Alluding to the non-rivalrous nature of information, Thomas Jefferson, third President of the United States and the first patent examiner stated:

“He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me.”

Unlike many other assets which when used are no longer available for others to benefit from, data can be re-used by multiple players for multiple purposes. National and local governments are waking up to this characteristic in the context of the data they generate and the value it has to businesses and individuals.

In the US, the data.gov website offers more than 200,000 data sets for re-use by any individual or firm while the UK equivalent provides over 40,000 sets. Most of this data can be used in commercial data offerings by third parties and a growing and vibrant set of companies are building businesses around it. According to a recent report from Deloitte, companies are using public weather data to predict seasonal sales patterns, mapping data to improve logistics and satellite imagery to estimate retail footfall and sales.

The lessons to be learned here are that clever use of data and analytics is not just for larger corporations that have access to large internal data sets. You may find your organisation already has valuable data sitting in silos that needs liberating or can access the data it needs through web scraping, data brokers or public sector repositories.