What Role Does External Data Play In A Post-Pandemic Market?

This article original appeared on the Forbes website.

There have been a number of pain points uncovered by the Covid-19 pandemic. Across all sectors, businesses, researchers, governments, and citizens have come face-to-face with a hard reality that when push came to shove the information and resources, they needed in order to function properly weren’t guaranteed.

Data in particular has been a glaring problem. While there was no shortage of data producers who were eager to provide relevant Covid-19 related information to the public (WHO, Johns Hopkins, CDC), the suddenness and rapid evolution of the crisis meant that these data sources were changing daily, sometimes hourly. The WHO, ostensibly the primary source of truth for infection rates, released their information in daily situation reports. These PDFs were critical data points to understand how the pandemic was changing over time, but were released in a way that made it effectively impossible to use – it’s nice to see the information, but you can’t model, spot trends, and predict outcomes with PDFs.

Faced with this problem (lots of data available, very little of it accessible) organizations are making do with what they have, tying into data sources when possible, piecing together spreadsheets when not.

In the first four months of the pandemic, these primary sources of data were important in order to understand where Covid-19 was and where it was going. Now, as organizations look towards returning to “business-as-usual,” many people are considering how they can use external data more generally to help them navigate the uncertain economy. I recently connected with Bryan Smith, CEO of ThinkData Works, about how businesses can streamline their data infrastructure without creating a lot of additional overhead.

Gary Drenik: As we head into the sixth month of working from home, what do you think is the primary business objective for organizations that are trying to get back on track?

Bryan Smith: I think we’re going to see a dramatic shift in business priorities from innovation to optimization. Data science divisions have been overburdened for years and that needs to change, fast. The janitorial work of connecting to external data is a huge problem, and it’s burning 80% of a data scientist’s time. Optimizing this process should be priority one for any business that wants to become “data first.”

Drenik: There’s a lot of data out there. How do you define “external” data?

Smith: It’s a good question. Our company started out in the open data space, and over the years we’ve watched the conversation shift a lot. To us, external data is anything you didn’t create, whether it’s coming from government sources, other businesses, or Joan in accounting. If you didn’t make the data, it’s external, and you’re probably working harder than you should to get it. There’s an assumption that the enterprise has this figured out, but if you scratch the surface of most data pipelines, there’s a lot of painstaking and manual work being done to use even just a fraction of what’s available. That’s why we have teamed up with Prosper to bring a fresh perspective on high quality external data that goes beyond stats of past behaviors and provides factual insights and data about how consumers are feeling, what they are doing , why they are doing it and what they plan on doing in the future. This is especially timely for today’s world where disruption seems to be the norm and left many wondering about what the future now being shaped by new consumer behaviors will look like.

Prosper Consumer Spending Forecast
Prosper Consumer Spending Forecast PROSPER INSIGHTS & ANALYTICS

Drenik: What role do you think external data plays for businesses that are trying to map out their game plan for next year?

Smith: Traditional models are going to have to be enhanced with new information. For many companies, finding these sources of data is still a big headache. For us, partnering with organizations like Prosper gives us and our network the ability to find new sources of data from a central clearinghouse, which lets them plug into analysis-ready consumer data for economic forecasting and predictive intelligence. We used to assume that most organizations had figured out a way to pull in data from statistical agencies like the Census Bureau and Statistics Canada, but what we’re hearing these days is that there’s a lot more manual work being done to gather this data than we previously thought. If you’re down in the mines trying to get basic demographic indicators from government sources, you probably don’t have the bandwidth to pull in signal-rich data from third-party data providers. The problem is that these days you really need both. Figuring out a scalable way to automate the flow of public data and connect to new sources of data should be top of mind for anyone who’s trying to get predictive.

Drenik: What do you say to a company that’s invested in data management solutions but isn’t seeing a big lift in overall analysis and insight?

Smith: You’ve solved one piece of the puzzle. Figuring out a way to get data into your environment is a big hurdle. A good ETL gives you a pipeline of raw information, but at the end of the day your data scientists are still managing a raw resource, and it’s a misuse of their time. Refining this data into decision-grade products lets them perform actual data science.

Drenik: What do you think is the biggest blind spot facing businesses that use external data? How should they overcome it?

Smith: I think there’s a big disconnect between business priorities and data science realities. At the end of the day organizations need to stream more data, reduce overhead, and add confidence to the entire process. To do this, the first step is to understand where you’re at. Now is the right time to perform a data audit across all divisions to see what data you’re pulling in from where and eliminate as much overhead as possible. A lot of data providers have made a lot of money selling the same product to different divisions within the same company. As businesses tighten their budgets the best way to free up some spend on new data is to eliminate the redundancies that have cropped up over time.

Drenik: How should businesses change their economic models to manage the fluctuations we’re seeing in the market?

Smith: It’s not about starting from square one but adding to what you already have. Obviously, data from the OECD and World Bank is still going to be useful, but what a lot of people are struggling with right now is the latency of these traditional sources. Getting GDP stats from July in September is fine as a baseline, but you need to also grab data that’s up to date. This is where getting data from third-party providers can give you a better overall picture of the economy. Products like the ones you’ve designed at Prosper – which help you understand behavioral trends, consumer sentiment, impulsivity – aren’t necessarily traditional signals, but they’re increasingly important right now.

Prosper Impulsivity Score
Prosper Impulsivity Score PROSPER INSIGHTS & ANALYTICS

Drenik: Between CCPA and GDPR there’s a lot of new regulations around data use. How can businesses ensure they’re being compliant when the landscape is changing?

Smith: Flexibility is important. Companies need to develop a strategy to manage and audit the flow of data through their environment, and this strategy needs to accommodate new regulations as they’re passed down. It’s not enough to have a good policy framework, you also need to back that up with infrastructure that supports the rules you’ve set up. Having mechanisms in place that monitor who’s accessing data, how it’s changing over time, and how it’s shared are all technical requirements wrapped in policy questions.

Drenik: What’s the future of data in the enterprise?

Smith: Every company needs to become data first in order to survive. If you look at Amazon and Apple, who have always focused on data as a core feature of how they do business, it’s clear that the market will be defined by the organizations that figure out how to streamline the flow of data into their everyday processes. Optimizing external data use doesn’t sound as sexy as innovation, but it’s the prerequisite for unlocking the value of data for your business.

Drenik: Thanks Bryan for your astute insights on the value of quality external data in today’s data centric world and the need for businesses to sync data science with business priorities in order to achieve organizational data success.

Complimentary Coronavirus/Covid-19 findings are available at AWS Data Exchange. To learn more, click here: Strategic Insights: Coronavirus Covid-19 Consumer

Comments ( 0 )

    Leave A Comment

    Your email address will not be published. Required fields are marked *



    OTTAWA, ON, June 15, 2020 – Kinaxis® Inc. (TSX: KXS), the authority in driving agility for fast, confident decision-making in an unpredictable world, has signed a definitive agreement to acquire Toronto-based Rubikloud, a disruptive, emerging provider of AI solutions that automate supply chain prescriptive analytics and decision-making in the retail and consumer packaged goods (CPG) industries.

    Globally-recognized retailers and CPG manufacturers in the health and beauty, household and grocery segments use Rubikloud’s AI-based products today. Their offerings include demand forecasting and automation to manage and optimize trade promotions, pricing and assortment to drive product demand and dramatically improve financial results. Kinaxis will enhance RapidResponse’s demand planning capabilities with the Rubikloud offerings, anticipating initial opportunities in the company’s rapidly-growing CPG customer base and over time for other industries such as life sciences. The acquisition also offers Kinaxis a springboard into the enterprise retail industry.

    “Rubikloud has capabilities and value that we can offer our CPG customers today, leads us into the retail industry with some bellwether accounts, and adds a group of approximately 80 people to an already-impressive AI and machine learning (ML) team here at Kinaxis. Over time, this enhanced group will contribute to new and existing AI-powered capabilities across the full Kinaxis RapidResponse® platform and applications,” said John Sicard, President and CEO of Kinaxis. “This acquisition reflects the growing importance of AI and ML to power intelligent automation and augment human decision-making to better deliver on customer promises, remove waste and increase resiliency for effective risk management.”

    Rubikloud’s SaaS-based ML offerings empower retail and CPG manufacturers to transform their core operations by improving and automating complex, profit-generating decisions. Rubikloud’s proven AI capabilities and intuitive tools enable users to leverage disparate data sources to improve forecast accuracy, site-level allocations, inventory availability and promotion plans by allowing users to run boundless simulations in real time.

    “We founded Rubikloud with the belief that purpose-built AI could be used to solve some of the most complex industry problems and we have spent the last seven years building a fantastic product that receives validation from global customers every day,” said Kerry Liu, CEO, Rubikloud. “We’re excited at the prospect of joining Kinaxis, which helps us bring our innovations to a much broader customer base at a faster pace than on our own. Not only that, being two strong Canadian companies we see great cultural synergy and look forward to working on the complex problems we know RapidResponse and concurrent planning can solve for customers.”

    Terms of Agreement
    Kinaxis will acquire Rubikloud for US$60 million in an all-cash transaction that is expected to close within 60 days. Based on Rubikloud’s current revenue and expense profile, the company’s fiscal 2020 revenue and Adjusted EBITDA guidance, as reiterated in its May 6, 2020 news release, remains unchanged. The transaction is subject to customary closing conditions.

    About Kinaxis Inc. 
    Everyday volatility and uncertainty demand quick action. Kinaxis® delivers the agility to make fast, confident decisions across integrated business planning and the digital supply chain. People can plan better, live better and change the world. Trusted by innovative brands, we combine human intelligence with AI and concurrent planning to help companies plan for any future, monitor risks and opportunities and respond at the pace of change. Powered by an extensible, cloud-based platform, Kinaxis delivers industry-proven applications so everyone can know sooner, act faster and remove waste. For more Kinaxis news, follow us on LinkedIn or Twitter.