Jul 16, 2014
Babson College professor Tom Davenport has been researching and writing about Big Data since the late 1990s.
His work long predates the term “Big Data,” which gained traction in the 2010s.
Tom has tremendous insight on the future of Big Data - even if he doesn’t like calling it that.
* * *
AUTOMATED INSIGHTS: Do you like the term Big Data, or would you use a different term if you could?
TOM DAVENPORT: I do not like it all. I think the more you know, the less you like it. Most of the people that I talk to in companies that work with it don’t like it much either.
It’s kind of an umbrella term for a lot of different attributes of data, and size is the least important. We can usually deal with large-sized data quite easily.
It’s the lack of structure that’s usually the problem, and in some cases the fast-moving nature of it.
Since lack of structure is the biggest problem that organizations typically have to deal with, if I had to pick something, I’d probably call it unstructured data or low-structure data.
None of it trips off the tongue quite so easily as Big Data.
I do think we’re stuck with it. I use it in my book – I just couldn’t really think of any good alternative.
How much further can the analytics/Big Data revolution go?
A long way, in that we analyze such small fraction of the amount of data. There was an IDC report [from December 2012] suggesting that we analyze only half of one percent of the data out there. So that suggests there’s still lots more opportunity.
We still have a lot of managers and executives who aren’t really comfortable with the idea of analytical decision making, so we’ll probably need some generational changes in that regard.
The university programs to churn out people to deal with these kinds of issues are just getting cranked up. So I think we’ve got a good run left.
One of the things that’s also very interesting to me is this whole machine learning movement, which if nothing else can drastically improve the productivity of these people doing quantitative analysis.
But we have to figure out how do we get people to understand it and interpret it, since right now that’s a little difficult.
What is the human role when it comes to Big Data?
There are two primary sets of roles.
Some people are producers of analytics and insights.
I wrote about data scientists, it was subtitled “The Sexiest Job of the 21st Century.”
There’s a big need for them in various flavors: some really hardcore statistician types, some data wrangler types, some fairly capable semi-professional analysts – people who know their way around a spreadsheet and a regression equation and so on.
But then there are all the consumers. That’s a much larger number and there are different classes of those as well.
There are managers. If analytics are taking over their business in terms of power and influence, they need to have some idea what’s going on, and what the assumptions are behind the models that they’re using.
Then there are the front-line people whose jobs may be impacted by this – I call them analytical amateurs. I suppose you could argue that a financial or sports reporter is in a way an analytical amateur.
In a book I’m contemplating, I think my argument is going to be that you will be much more successful if you have an idea of what the strengths and weaknesses are of the automated and analytical tools so you can both check up on them and make sure they’re doing what they’re supposed to and also so you can do something else.
What’s something that makes you optimistic about the future of analytics and data?
The organizations that exploit them generally do quite well. They tend to be the leaders in their industries, they’re profitable, and they have high growth.
I’ve done a couple of studies and a number of other organizations and people have done studies relating analytics and analytical capabilities to performance and they’ve all shown a positive relationship.
So that’s good news.
* * *
The Automated Insights Wordsmith platform analyzes data and presents key insights in plain English. Request a demo to learn more.
Follow @AInsights for updates on our company, our technology, and other cool stuff from the future.
Stay tuned for more interviews, and don’t miss our posts on “waste education” and the digital media revolution.
The above conversation has been edited for length and clarity.
Jul 15, 2014
There’s more evidence that robots can write like humans.
Earlier this year, a study in Journalism Practice found that automated content was “considered to be objective although not necessarily discernible from content written by journalists.”
Now, Yahoo Tech columnist Rob Walker has put together a quiz: “Can You Tell Human Writing from Robot Writing?”
Rob tells me that at that the moment, the average score is just 50%.
Might as well flip a coin.
Take the quiz for yourself:
How well did you score?
Jul 14, 2014
The Open Interconnect Consortium, a group of big tech companies including Intel, Samsung, and Dell, announced last week their intention to create and open-source a specification for connecting the billions of things that make up the Internet of Things.
As you might imagine, this announcement went out without a ton of fanfare. In comparison to March headlines touting a bot breaking the story of an earthquake, this was a blip.
I can see why. This is rocket science, or rather, robot science. And unless you’re talking about the Terminator or Robocop or some other machine putting a human out of a job (and/or eventually killing them), people tend to want to get back to enjoying their slow news day.
But this story has much bigger implications for automated content than the template-driven ramblings of an earthquake sensor.
Automated Insights will publish over a billion automated stories in 2014, reporting on everything from finance to sports to fantasy football and beyond. We’ve got some name-droppable partners working with us now, including the AP and Samsung, and if we get standards for web-enabled devices that allow us to more efficiently merge disparate new sets of data, you can slap at least a couple more zeros on the number of stories we’ll create in 2015.
Over the last four years, we’ve evolved automated content from its Mail Merge origins into a fully algorithmic code-churning platform, capable of cranking out professional-sounding, insight-packed articles at the rate of up to 1600 per second. Studies have shown that our automated content is not only indistinguishable from human-authored content, but in most cases it’s viewed as more trustworthy.
If the Robot Writer is already fully-baked, the Robot Reporter is on the horizon, represented by all these devices and sensors tracking and measuring things they previously could not. These sensors are becoming ubiquitous, and not just in sports and finance and traffic and personal fitness, but nearly every walk of life.
The standards that are being proposed by the Open Interconnect Consortium could be perceived as a stepping-stone to a universal Robot Language. This will explode the new science of automated content by creating countless new verified sources.
But while the goal of a universal language might be noble, it’s also somewhat flawed.
From the Old Testament story of the Tower of Babel to the century-old experiment with Esperanto, which today is only remembered for the William Shatner horror vehicle Incubus, to the yet-to-be fully adopted metric system (although to be fair, it’s really just America, and we’re not budging on this), the quest for a common language has always been seen as a path towards efficiency and, thus, innovation. But that quest has always been sandbagged by cautionary tales and a lackluster adoption rate.
Standards, on the other hand, are a much more accepted way of creating, and more importantly, sustaining, the uniformity necessary to get many hands working toward the same outcome.
In technology, standards are crucial. Philosophically, computer languages are really just translation engines from human to machine, English to Binary. And once you get enough software and hardware components working together with at least some compatibility, there’s a lot less reinventing of the wheel, freeing up time for innovation.
These days we have development and compatibility standards for PCs, mobile phones, operating systems, and various applications and application types (i.e. web browsers). Yet even those standards are splintered among various players (Apple vs. Microsoft) and even across devices (iPhone vs. iPad).
It should also be noted that there is another Internet of Things standardization effort, the AllSeen Alliance, which includes companies like Qualcomm, Microsoft, and LG, and it’s basically trying to achieve the same (but different) goal. Anyone who remembers HD-DVD vs. Blu-Ray or the People’s Front of Judea vs. the Judean People’s Front has an idea of how this might turn out.
And it probably isn’t good for the people, Judean or otherwise. Furthermore, Google and Apple are pretty much still out there on their own. At this point, a standards initiative probably seems like a net negative to them.
Look, I applaud the effort to standardize the Internet of Things, and if it gets done, it will be huge not just for automated content, but for anyone who can benefit from a smart anything. I just hope we’ll remember the lessons learned from standardization efforts past, and make sure we’re doing this for the data, not for the branding.
Jul 10, 2014
Ready for some mind-blowing thoughts on the future?
Listen to our interview with Dutch futurist Marcel Bullinga, who says that in the future, some education will be a waste of time.
If reading is more your speed, here’s a slightly modified transcript:
* * *
Different pieces of our current education will become obsolete.
The fact that interfaces will become much easier - we can build, fabricate, use, repair basically anything with little, very little knowledge.
We are capable of doing more with less knowledge in the future.
If you want to repair a car, you want to become a car mechanic - you will have to know all the models of cars and how the breaks work and stuff, etc., like that. It takes a huge amount of time. [CLAP]
It’s all useless, it’s all waste time.
So in the future we’ll have waste education, you’ll learn something and it’s totally irrelevant in a split second.
You can be trendy in your education and try to be future proof.
So for example you want to teach your children Chinese because China is an upcoming economy. And then in five years or so we don’t need to learn any Chinese anymore because it’s inside your contact lens.
It’s an incredibly inefficient way to train your brain.
Maybe it’s enough to train your brain in your own natural language plus one extra language but all other languages become irrelevant.
All the data that used to be owned by big institutions and the government and the military etc., and that took months and ages and lots of professional people to analyze it and interpret it - [CLAP].
It’s going back to the individual level where I can use this data very cheaply for improving my own health, improving my own decisions.
Do I need to be a data scientist in order to interpret my life? No, I don’t. I just - boop - click the app and there it is.
It empowers me without the need to [learn] lots of difficult stuff.
* * *
Turn your data into stories.
Jul 9, 2014
The Associated Press recently announced a partnership with Automated Insights to automate corporate earnings stories.
Here’s what happened next.
Jul 8, 2014
We think a lot about the future of media at Automated Insights.
In fact, The Associated Press just announced it will use our technology to automate corporate earnings stories.
Mark Glaser also thinks a lot about the future of media.
He’s the executive editor of PBS MediaShift, which describes itself as “the premier destination for insight and analysis at the intersection of media and technology.” I caught up with him recently to discuss what’s next.
AUTOMATED INSIGHTS: The tagline for MediaShift is “Your guide to the digital media revolution.” At this point, isn’t the revolution over? Is there anything left for digital media to do to complete the revolution?
MARK GLASER: It’s easy for us to say that the revolution is over and digital media has won, but we represent a minority of people who work in the industries. I can even look at PBS as one example. PBS comes from a history around broadcast. They’ve done a lot in digital, but they remain a broadcast entity.
If you look at the leadership of all these traditional media organizations, whether it’s print, whether it’s broadcast, whether it’s radio, whatever it is - people who remain at the top of those organizations still have very little experience in digital. So to me, it still hasn’t reached the level where it will, where we can say this is over, we won, the digital landscape has swamped the traditional landscape.
In places like music it’s definitely changed a lot, in books, it’s starting to change, but in education, it’s at the very early stages.
Is the future of media about viral traffic, link bait, and listicles? In other words, is it about figuring out different ways of gaming the algorithms through which we receive our media?
I don’t think that mass traffic is going to be a business model in and of itself. I still believe that delivering the content that people want, where they want it, when they want it, will provide value.
And connecting them to relevant services, advertisers, and message makes a lot more sense than this attempt to create a mass audience that is clicking on things just to click on them and then clicking away after a brief millisecond. That just doesn’t have a lot of value.
I think it’s really about creating the experience that people want to get on a regular basis, that they feel like they have affinity with, that they care about, that they’re passionate about. And serving them what they want, when they want it, content that they can trust.
What’s next? What questions about digital media keep you up at night?
If anything, I’m more excited about things that are going on. The only thing that would keep me up at night would be, “well, did I miss something that happened?”
I’m always on the lookout for someone else who can give me a real good filter on things. My bigger concern is, can someone filter things even better than all the filters I already have, and really personalize what I want and what I need? That’s what I’m looking for, and I think that’s what we’re all going to move towards and what we’re all going to get.
How does a filter combine my known demands with my unknown demands, things I don’t know that I want?
I don’t know that the filter will be completely an algorithm. I think it probably will be a mix of humans and algorithm. I still believe that’s the sweet spot — a mix of algorithm filters and human subjectivity that can really bring interesting things your way.
* * *
Request a demo of the Automated Insights Wordsmith platform to learn how our technology creates articles personalized for individual users.
Stay tuned for more interviews with futurist experts, business leaders, and cutting-edge journalists. In the meantime, enjoy these recent interviews:
Invisible Tracking, Unknown Potential: The Future of Health Data
Scooped by a Scandal-Seeking Machine
The above conversation has been edited for length and clarity.
Jul 6, 2014
Our CEO Robbie Allen recently spoke at Data Driven NYC.
Watch this video to learn why and how Automated Insights turns data into narratives.
Jul 1, 2014
What a week for Automated Insights!
We announced the close of our $5.5 million Series B round, including investments from Samsung, Steve Case, and The Associated Press.
The Associated Press announced it will leverage our technology to automate corporate earnings stories.
And our big news caught the attention of the media:
* * *
A leap forward in quarterly earnings stories
The New York Times
The A.P. Plans to Automate Quarterly Earnings Articles
The AP Is Using Robots To Write Earnings Reports
Wall Street Journal
Automation or Augmentation for Business Reporters?
New York Magazine
Robots Are Invading the News Business, and It’s Great for Journalists
How robots will write earnings stories for the AP
The Associated Press welcomes its robot journalist overlords
The Prose of the Machines
AP will use robots to write some business stories
AP on robot reporters: ‘I can’t have journalists spending a ton of time data processing’
AP’s robot-written stories have arrived
Associated Press taps story-writing software
The Associated Press Now Automates Earnings Stories, No Humans Needed
Need to Write 5 Million Stories a Week? Robot Reporters to the Rescue
Associated Press backs Automated Insights to automate boring earnings reports
How to teach a robot to write
The AP’s newest business reporter is an algorithm
You May Be Getting Your Business News From Robots Soon
This Is What A News Story Written By Robots Looks Like
The Era of Robot-Generated Reports Has Begun
Can A Robot Report Better Than A Human Journalist?
$5.5M and The Associated Press: The Next Chapter of Automated Insights
News & Observer
Automated Insights raises $5.5 million, lands deal with AP
Durham start-up’s technology to generate AP financial stories
* * *
If you want to see for yourself what AP, Samsung, and Steve Case have already discovered, request a demo of our Wordsmith technology platform.
There are more stories to come! Stay tuned!
Last updated July 21, 2014.
Jun 29, 2014
Osage, Samsung, The Associated Press and Steve Case Headline Strategic Growth Round
(Durham NC) – Automated Insights (Ai) , the world leader in producing personalized narrative content from Big Data, today announced that it closed a $5.5 million Series B financing round led by Osage Venture Partners.
The round included participation from Samsung Venture Investment Corporation, The Associated Press, AOL co-founder Steve Case, former SevOne CEO Mike Phelan, Court Square Ventures, OCA Ventures, IDEA Fund Partners and other existing investors.
“Automated Insights is an exciting company and we look forward to expanding our partnership,” said Sang Ahn, Managing Director, Samsung Open Innovation Center.
Ai’s patented Wordsmith technology is a natural language generation platform that spots patterns, correlations, and insights in large data sets and then describes them in plain English via narrative reports, just like a data scientist would.
“Automated Insights has innovative and patented artificial intelligence software in a very exciting space, and we believe their natural language generation platform will be a game changer in multiple markets,” said David Drahms of Osage Venture Partners. “Ai’s growth and the feedback from its strong list of customers are impressive. We are thrilled to team up with such an experienced management team and exciting strategic investors.”
As part of this financing, Drahms and the company’s COO, Scott Frederick, have joined Ai’s board of directors.
“Today, companies rely on business intelligence tools or in-house solutions that force users to decipher complex dashboards,” said Robbie Allen, Ai’s CEO and Founder. “The problem is that charts and graphs don’t tell a story, words do. Wordsmith enables any company to communicate a unique and actionable message to each employee or customer.”
The company has been growing rapidly, with dozens of new customers in a wide range of verticals.
“We have hit a tremendous untapped market by helping people unlock the stories hidden in their data,” said Allen. “We will publish over a billion stories and reports this year for partners like Yahoo!, Edmunds.com, and soon The Associated Press.”
The Associated Press announced Monday it would leverage Ai’s Wordsmith technology to produce quarterly earnings reports, based on the reported financial data of publicly traded companies.
“The potential for Ai’s technology is evident in the strategic investors in this financing round,” said Jim Kennedy, AP’s senior vice president for Strategy and Enterprise Development.
About Automated Insights, Inc.
Automated Insights (Ai) transforms Big Data into written reports with the depth of analysis, personality and variability of a human writer. In 2014, Ai and its patented Wordsmith platform will produce over 1 billion personalized reports for clients like Yahoo!, The Associated Press, the NFL, and Edmunds.com.
The Wordsmith platform uses artificial intelligence to dynamically spot patterns and trends in raw data and then describe those findings in plain English. Wordsmith authors insightful, personalized reports around individual user data at unprecedented scale and in real-time.
Automated Insights also offers applications that run on its Wordsmith platform, including the recently launched Wordsmith for Marketing, which enables marketing agencies to automate reporting for clients. Learn more at http://automatedinsights.com.
Jun 27, 2014
Today’s digital health apps and devices are generating massive amounts of useful data.
However, this data comes with at least two major challenges: access and understanding.
At Automated Insights, we tackle the second challenge, helping users understand their health and fitness data by deriving insights from the data and explaining them with individually tailored narratives.
Meanwhile, just a few blocks from our office, another cutting-edge is focused on health data access. Validic connects patient health data from apps and devices to healthcare professionals. Ai is data’s presentation layer; Validic, according to CTO Drew Schiller, is “the conduit through which health data moves.”
I spoke with Drew about the increasingly invisible future of personal health tracking and why improved access to patient data could lead to new healthcare breakthroughs.
AUTOMATED INSIGHTS: What kind of patient activities can be turned into data?
DREW SCHILLER: With digital diabetes blood glucose meters, digital weight scales or digital blood pressure monitors, data is being recorded and typically saved on a device. But it’s not been simple to get the data off the device and back to the healthcare professional.
In the last three to five years, with the release of the iPhone and the Android platforms, a lot more of these devices have been connected to smartphones to deliver the data up into a cloud.
Data could come through a personal tracker like a Fitbit or a Jawbone UP, for example, or through a blood pressure monitor like an Omron or A&D device.
Patients actually already have a fair amount of patient-generated data. In fact, there’s a Pew Research study that shows that seven out of ten US adults currently track their health or the health of their loved one in some way. A growing number of those are with digital health products.
But the problem is that only one out of ten US adults actually share that data with a clinician.
In the next five or ten years, what kind of monitoring tools will come into the market?
There’s a device coming out in the fall called Spire that acts as not only a step tracker but also as a continuous spirometer - so it’s going to be able to detect your stress levels as well as potentially give you insights into conditions like COPD or asthma.
As we look out into the future, there are companies creating skin patches that will effectively look like a Band-Aid or maybe even more translucent that will monitor your steps, your calories and your biometrics like your heart rate, blood pressure, diabetes, and blood glucose.
These big clunky wearable devices are actually going to become invisible.
Honestly, the biggest tracker is something we already all have, which is our our phone.
Recently Apple, Android, and Google have come out with three different platforms, and HTC came out with a partnership with Fitbit. So now all of this data is going to be tracked on your phone without you even having to wear anything.
Can you give me an example of how tracking data makes a difference for a healthcare provider or patient?
This isn’t a Validic instance, but I can tell you a story about an 11 year old girl in Cincinnati who has Crohn’s disease. We’ll call her Sarah.
Sarah is a patient at the Cincinnati Children’s Hospital and her physician was concerned because, like many people with Crohn’s disease, Sarah wakes up about every hour with severe gastrointestinal pain.
He gave her a mechanism to track the number of times that she gets up in the middle of the night. After a couple months, he pulled up the data on his computer and saw a seven day period with almost no incidences.
When he asked her what had happened, she couldn’t remember, so he looked back through her medical history. He found was that she had started a course of antibiotics for a completely unrelated condition at the beginning of that seven day period.
This led to a new line of research investigating whether non-digestible antibiotics can be used to effectively remove or reduce the symptoms of Crohn’s disease. That has implications for millions of patients with Crohn’s disease around the world.
That all happened because Sarah was tracking her data and her physician was able to access her data. Sarah was not a Validic patient, so to speak. However, that is the type of thing that we can power.
One of the questions we get asked a lot is, “What can we do with this data?” But if that physician had decided not to track Sarah’s sleeping data because he didn’t know how he would use it, he never would have come up with that breakthrough.
Until you can do some analytics on your individuals and your populations, it’s difficult to necessarily know the full benefit of patient-generated data.
* * *
Analyzing data and presenting key insights is what the Automated Insights Wordsmith platform does best. Request a demo to learn more and follow @AInsights for updates on how Big Data is shaping the future.
To learn more about Validic’s industry leading digital health platform, follow them on Twitter at @validic or contact them at validic.com/contact.
Stay tuned for more interviews with futurist experts, business leaders, and cutting-edge journalists. And don’t miss our recent interviews about employment in an automated economy and the future of journalism.
The above conversation has been edited for length and clarity.