There’s a lot of press given to Artificial Intelligence and Data these days, and justifiably so. It’s seeping into every corner of industry, disrupting incumbents and re-inventing the unimaginable.
But there’s a lot of businesses that are still yet to understand the value their unstructured data could have on their business.
80% of the data available is found in unstructured form and it’s out of reach for businesses that are still using traditional technologies without correct processes in place to capture it appropriately.
Once captured and used properly unstructured data can enhance your view of the world, an industry or a client. It will reinforce your business decisions by giving you a deeper understanding of customer behaviours and opportunities. With unstructured data, you will be able to carefully listen to your customers instead of guessing their feelings around buying decisions. You will be able to identify opportunities to innovate your products and to fill gaps with new features your competitors are yet to produce.
A lot of the work we do is to help clients understand how data could be used more effectively in their business and then implement that change. So many companies are throwing away useful or insightful information because they just don’t know what to do with it. However, with some foresight and strategic planning, the value becomes apparent. Whether it’s building back-office automation or new product development. By capturing and structuring data appropriately you’ll become a more empowered business that can do more with its data.
“Companies that make use of data science are living the high life, reporting more revenue growth, company growth, bottom-line profits, and market-leading performance.”Forrester Research
We’ve spent the last 5 years building a powerful NLP tool, the Skim Engine™ that takes unstructured data and creates structured machine-readable information. Why? Because unstructured data is the bedrock of every business, you just don’t know it yet.
Unstructured vs Structured data
To understand how valuable data can be, it’s important to first understand its different forms. There’s more than one type of data but there’s a big difference between them. As the name implies unstructured data is chaotic, messy, and hard for a machine to understand. Whereas structured data is clean, tagged and stored easily in a database.
Some examples of unstructured data in a business:
- Social Media
- Surveillance videos
- Chat Applications
All of these unstructured data types exist outside of a relational database. The thing that would ordinarily give a ‘name’, or a ‘phone number’ meaning for a machine to use.
We assume that if something is readable by a human on a webpage, that it must also be readable by a machine. But without context and structure, the data is useless.
Unlocking the hidden value of unstructured data
Turning unstructured data into structured machine-readable information is what unlocks the hidden value for companies by enabling them to innovate, analyse and outperform competitors. However, the effort involved in bringing structure to the data is no mean feat.
It requires clear strategic analysis, to understand what parts of the business could be improved by putting this data to use in order to validate the work required to start using it. Understanding access to the data, is it internal or external? Is it licensed or regulated? How accurate, and reliable is the data? How expensive would it be to store or process? These are just some of the questions to take into consideration before starting a project.
Working with companies like Skim Technologies, a feasibility study can be carried out that addresses all of these questions and looks at how effective this data can be in your organisation. Speak to one of our data experts today to learn more.
Once a clear understanding of what data should be used (internal, external, unstructured or structured), and how you’re going to store and access it, then you can go about planning how to use it.
We’ve given a few use cases below to inspire you, but there are so many more. To learn about specific applications for unstructured data within your industry, visit our solutions page.
Use cases for Unstructured Data
Back Office Automation using Unstructured Data
Robotic Process Automation (RPA) has brought great efficiencies to large enterprises over the last 5 years, with still much more to offer in the form of back-office automation. Turning day long labour intensive tasks such as the generation and processing of Volvo’s 2000 invoices, into a few minutes task for a robot.
However, an RPA system is limited to only process structured data. Therefore the data already has to be input to a system, such as an ERP, or always in the same place on a document, before the automation can take place. In order to move beyond these restrictions, back office automation needs to include processing of unstructured data.
For example, in the HR department at a large supermarket chain, two staff spend a day a week processing thousands of holiday and time-off requests sent via email. An unstructured data source.
In order to automatically process this, the requests should be classified into four categories (holiday, sick leave, absent, training). The perfect task for Natural Language Processing (NLP) and our Skim Engine™.
Once a classifier has been trained, the next task is to extract the employees’ name using Named Entity extraction, then Name Matching to an internal employee record. In order to update the system. There’s further NLP work to do with capturing the dates in the email and deducting the correct salary or holiday amounts. But the end result is saving two staff 52 days per year of work, which is a 20% saving per staff member. Well worth the effort of processing this unstructured data, email.
Supply Chain Risk Monitoring Using Unstructured Data
Fragmented supply chains with multi-tiered vendors make monitoring risks near impossible without the help of machines. Most of the data used to analyse these risks are contained within unstructured sources which are also just as fragmented. These can include Weather, News, Port Documents and Satellite to name a few. Mainly external sources, and all unstructured.
All of these data sources need to be aggregated into an Agent-Based Model that can be queried with various scenarios in order to plan ahead of a disaster. One of the hardest sources to process is News.
Using our Skim Engine™we can process this news information and look for ‘Risk’ events based on time, entities (company, person, port etc…) and geolocation. By aggregating local, national and global news stories from millions of sources.
When an event is detected, it’s automatically fed into a Supply Chain Risk Management (SCRM) system via an API, alerting a manager to the event and its location.
This solution saves companies like H&M with over 238 suppliers in Bangladesh alone, millions of pounds in avoiding delayed, damaged or lost goods.
Competitor Price Monitoring Using Unstructured Data
Let’s take the product manager of an FMCG brand, who’s interested in monitoring her competitors’ prices, she would either have to go and check every week to see if there’s been a change or she could use web scraping to capture that information.
Web scraping has been around for a long time. It accesses data from one of the largest unstructured data sets in the world, the web. However, even this age-old technology is flawed in its approach. It uses rules, and therefore, similar to an RPA system, can only extract data from the same section of a site. What if that site changes? The price is no longer at the top of the page, but the bottom. Therefore the rule-based web scraper can no longer find the correct data.
We’ve trained our Skim Engine™ to understand the meaning of price, not just the position on a page. Therefore, when the SaaS website is updated, the Skim Engine™ recognises that change and still extracts the correct price. The product manager saves hours having to go back and search competitor sites and is also able to adjust her pricing accordingly saving both time and lost revenue.
To learn about specific applications for unstructured data within your industry, visit our solutions page or contact us via firstname.lastname@example.org