Mar 6, 2014
A study published in the 2014 issue of Journalism Practice proved that not only was Automated Insights machine-generated content indistinguishable from journalist-created content, but that our automated content was viewed as more informative and more credible.
Christer Clerwall, from the Department of Media and Communication Studies at Karstad University in Sweden, conducted a pilot. For the test, 46 students in media and communications studies were given either a professionally-written NFL game recap from the L.A. Times or an automated recap of the same game from Automated Insights. They were asked to assess their article on both quality and credibility. They were also asked whether the article was written by a journalist or our engine.
Robot or Human
This certainly isn’t the first time Automated Insights has been directly or indirectly involved with a robot vs. human test. It’s something we always do in-house as a part of our normal QA process. Considering we’ve been at this for almost four years, we weren’t surprised at the results.
From the study:
"Of the 27 respondents who read the software-generated text, 10 thought a journalist wrote it and 17 thought it was software-generated. For the 18 respondents in the “journalist group”, 8 perceived it as having been written by a journalist, but 10 thought software wrote it. Using a Mann–Whitney test for significance, we can conclude that there is no significant difference (U = 225, r = −0.07, significance = 0.623) between how the groups have perceived the texts."
Remember, the test subjects were media and communications students, people who should have a good eye for this sort of thing. But spotting the difference isn’t as easy as it sounds, especially today. Our automated content engine can do things now that were seemingly impossible in automated content just a year ago.
Referencing non-statistical facts, for instance, like game-impacting injuries or blown official calls, is no longer a deal-breaker. In fact, we’re working on a project at this very moment with the ability to include “off-field” non-statistical information for an upcoming major sporting event.
Our very first attempts at college basketball articles, over three years ago, were able to determine “fan satisfaction” with an algorithm that tracked momentum vs. expectations for each team.
Clerwall is convinced that the robots sound human:
"As far as this study is concerned, the readers are not able to discern automated content from content written by a human. Some aspects of quality, such as being clear and being pleasant to read, received a slightly higher score for human-written content, but others, such as trustworthiness, informative, and objective, were higher for the automated content."
Credibility: Where the Machines Win
As I said, verification of the Automated Insights engine’s natural-sounding language is nice, but what’s even more satisfying is the notion at the end of Clerwall’s statement, in which he reveals that trustworthiness, objectivity, and the informative value of automated content received much higher scores than content written by a human.
"The software-generated content can be said to score higher on descriptors typically pertaining to the notion for credibility."
"…the text generated by software is perceived as more descriptive, more informative, more boring, but also more accurate, trustworthy, and objective."
This is the true power of the Automated Insights engine. We’re comparable to a human in writing about events and outcomes that humans are already writing about, like a pro football game. However, when it comes to personalized automated content and the ability to suggest and recommend strategic improvements to the reader, the possibilities for Automated Insights are unlimited.
A great example of this can be found in one of our products, SiteAi, in which we unlock the key hidden insights in your website’s Google Analytics data and deliver the results in plain English via a daily or weekly email. These are articles that are impossible for a human to write, let alone at scale, and contain actionable suggestions based on the reader’s unique goals.
That’s a game-changer.
In January, I wrote an article on the Quantified Self, in which I noted that it’s this focus and credibility that ultimately provides value:
“The beauty of quantification is that we can finally start using the web to discover and better ourselves instead of telling it what we think we want. Quantification doesn’t lie, nor is it susceptible to moods or bias. It’s almost like our own personal suite of business software.”
Automated Advisement from a Trusted Source
There’s not much personalization to be had in a pro football game. You either follow the team or you don’t. At best, we might be able to tell you when to put away the face paint until next year. However, when we apply our engine’s informative, accurate, and objective analysis to the data, whether it be a company’s sales data, a website’s analytics data, or your own personal workout data, we can make actionable and unbiased suggestions and recommendations that human writers can’t, especially on a massive scale.
That also happens to be, for lack of a better term, where the money is.
"If journalistic content produced by a piece of software is not (or is barely) discernible from content produced by a journalist, and/or if it is just a bit more boring and less pleasant to read, then why should news organizations allocate resources to human writers? Perhaps the speed, an important factor in adopting new technologies (cf. Örnebring 2010), will make up for the loss of “pleasantness”? If the audience can get automated content cheaper than content produced by journalists, with “less pleasant to read” as the main drawback—why would they want to pay?”
He’s absolutely on target. But take the corollary of that conclusion. If Automated Insights content is seen as more credible, then it stands to reason that the insights it produces, in human-sounding plain English and delivered from a trusted, unbiased source, are extremely valuable.
Jan 13, 2014
When people hear about how Automated Insights turns Big Data into narratives, they’re often incredulous. They ask whether automated content is as good as what a human can create. The answer is an emphatic “Yes!”
After all, even though machines create the articles, there are human experts teaching the machine what to write about. From our Yahoo! Fantasy Football match recaps to personal fitness devices, we’re proving that you can bottle up human expertise and replicate it to create millions of unique analyses that explain data, not just report it.
Now, we’re going to do the same thing for web analytics. We’ve had our SiteAi product available for several months now, and people love it. And today, we just announced that we’re going to tackle the biggest pain point for digital marketing agencies: creating monthly reports for their clients.
Head over to the SiteAi blog to read the full announcement and see how we’re going to save agencies $12,000 a month.
Dec 30, 2013
2013 was a great year for Automated Insights. We published over 300 million personalized stories (that’s 9.5 stories for EVERY SECOND of the year!). We worked with great clients like Yahoo!, the NFL, Turner, Gannett, the Associated Press and Microsoft. And, we were recognized as the “Best Place to Work” and “Most Innovative” company in North Carolina.
Our own Brian Sewell nailed it in his video retrospective.
While 2013 was great, we are looking forward to an even more exciting 2014. 300 million personalized stories in a single year sounds like a lot (more than all the major media companies combined). However, based on some upcoming implementations of our platform we expect to create close to 1 billion stories in 2014.
But before we turn the page on 2013, we thought it would be good to look back at the great times we had while breaking new ground in personalized content.
New Verticals and Clients
We launched Site Ai, which applies the power of personalized reporting to web analytics. Site Ai helps website owners and marketing agencies make sense of their web traffic through daily, weekly and (coming soon) monthly reports which put the results of Google Analytics into actionable, personalized narratives. 2014 will bring big new developments for Site Ai, so stay tuned.
We more than tripled our revenue while launching ground-breaking personalized content initiatives with over 20 new clients in a variety of industries and verticals.
In finance we worked with MSN, Datatrac, and others to create innovative content around stocks, portfolios and bank-related analysis.
In quantified self and personal fitness we launched multiple projects where users will soon experience how personalized narratives can provide dramatically improved understanding of their health through their sleep and exercise patterns.
In sales and internal reporting we kicked off a partnership with one of the largest insurance companies in the world to provide their sales team fully-personalized, action-oriented reports.
In fantasy sports we expanded our relationship with Yahoo! to include multiple personalized fantasy articles (draft recaps, match up recaps, league-wide recaps), as well as interactive trivia and multiple different articles around all the major sports leagues. We also launched a strong partnership with the NFL.
We kicked off new exciting projects with groups like the Associated Press.
We published tens of thousands of articles a week covering real estate markets across the country for clients like HomeSnap
We were a launch partner for the Windows 8 mobile app marketplace with over 900 applications live for Windows 8 phones, tablets and PC’s.
We demonstrated the power of our real-time engine using 120 years of MLB data. Real-time is an area we are doing more and more work in and will have much more to show in 2014.
And that’s just what we can share…
Announcements and Awards
Our Patent on “Dynamically Generating and Presenting Narrative Content” was issued in August. This is the first patent filed of its kind and provides an exciting start to our growing patent portfolio.
We were fortunate to have a lot of press coverage this year, but our favorite was Alexander Eule’s feature story on our work in Barron’s. We believe he nailed the power of personalization through intelligent automation in this piece on our work with Yahoo!
We were selected as one of six “Big Data and High-Performance Computing” Semi-Finalists by Amazon Web Services in their Global Start-up Challenge.
Our CEO, Robbie Allen, spoke at a number of conferences and was named in the “50 to Watch in Business”
We were ranked #1 out of 464 Business Intelligence companies by SIGNL as part of their Momentum Rankings.
We were honored to be named the “Most Innovative Company” by the North Carolina Technology Association.
And last but not least, we were thrilled to be named the overall Best Place to Work in NC by the Triangle Business Journal. It was the second year in a row we were named as a finalist and our first overall win. This is a special one for us because it was voted on by all the nominated companies’ employees. We believe the right environment and teamwork are critical to getting the best work done and this award shows we are building just that environment for our team.
The Future is Bright
At Ai we believe in a future where everyone — from an individual line employee, to a store manager, to a fitness band user, to a fantasy sports GM, to a patient getting their lab results, to an end user looking for information they care about — has the ability to unlock the power of their own data. A future where individuals can get insights about the key things they need to know in real time. A future where the patterns and trends in an individual’s data are put into context, benchmarked against the aggregate population, and made actionable. A future where everyone can be their own personal data scientist and make better decisions, more often.
The best part is that with our technology platform, the future is now.
2014, here we come.
by Adam Smith @adambsmith
Nov 11, 2013
We are excited to have received two awards over the past month. The first was for Most Innovative New Media Company from the North Carolina Technology Association and the second was the overall award for Best Place to Work in the Triangle by the Triangle Business Journal.
We were up against some amazing companies for both awards and we are truly honored to have been named the winner for each. We are excited to see our team receive recognition for building a one-of-a-kind technology that is changing the world of personalized content, while working together to build a culture that will sustain ongoing innovation as we continue to grow. It is an exciting time at Ai, and we look forward to seeing what the future has in store for us!
Thanks again to NCTA, the TBJ and to all the voters.
Oct 2, 2013
We talk a lot about driving action from personalization at scale. However, the best way to really see the potential impact of unlocking big data through high quality automation is by seeing it in action. One of the most public implementations of our technology is in fantasy sports where we are publishing multiple different types of articles for some of the leading providers in the space.
Alexander Eule at Barron’s noticed our work and drew the connection of the power of personalization to create sticky, hyper-relevant content in any market in his article on our work with Yahoo!.
The article was in the print edition on September 2nd, is online here, and is included in full below.
By ALEXANDER EULE
Yahoo’s fantasy football reports show how big data can personalize information and advertising.
Every once in a while, technology still manages to surprise me.
Just minutes after finishing my Yahoo -sponsored fantasy football draft last week, the Web giant sent me a very personal e-mail. Yahoo (ticker: YHOO) had quickly analyzed my draft picks and prepared a grade. I got a B-, in what amounted to my worst showing since college Spanish. I remember contesting that Spanish grade, but it’s hard to argue with Yahoo. The mark was accompanied by hard data, wrapped in a smartly written report.
It turns out that my team, TommyJohn, is the oldest in the 10-team league I share with friends. My players have spent an average of 7.3 years playing in the National Football League, according to the analysis. (Not an ideal scenario when you consider that the average NFL career is less than seven years.) Yahoo summarized my draft like this: “Despite selecting in the top half of the first round, TommyJohn looks a little soft on paper.” Ouch.
If fantasy football sounds trivial, here’s where it gets more meaningful: The report cards are a product of Big Data, the much-hyped but obscure concept dominating conversations in Silicon Valley. For most fantasy players, the report card is probably their first real Big Data experience, which remains largely theoretical in day-to-day life.
Big Data, of course, has far bigger applications than fantasy football. Eventually, our location—determined by mobile phone GPS—will be combined with social-networking profiles and other personal data, all in real time, to create a kind of marketing nirvana.
"Big data really becomes actionable and obvious to the consumer when who you are, what you’re doing, and where you are are all being used," says Jordan Rohan, who covers Internet stocks including Yahoo for Stifel Nicolaus. "We’re on the cusp of it."
In the meantime, sports is already ripe for a Big Data-type incursion. Hoards of statistics—new and old—can be culled into new and meaningful insights. And fantasy football owners, a rather obsessive sort, have always looked for any edge in improving their teams.
"They have an almost insatiable appetite for information, research and detail," says Ken Fuchs, the head of Yahoo Sports, which counts some 13 million fantasy players that spend an average of 29 hours a year on their teams.
Yahoo is using a start-up called Automated Insights to create its report cards. The Durham, N.C.-based firm looks for patterns, trends, and oddities in massive data packages and translates it all into plain English.
"Terms get bantered about and hyped," says Scott Frederick, who’s the chief operating officer of Automated Insights. "Now it’s big data and the cloud, but people struggle to understand what it really means. How does it bring value? What I love about our technology is the ability to do personalization at the data level and turn it into narratives," Frederick told me.
The firm began working with Yahoo’s fantasy group last year, but this is the first time they’re preparing draft reports, along with weekly game previews and recaps, all personalized for every fantasy owner. The company says it will deliver over 250 million personalized reports this season for Yahoo and other fantasy football partners. “We are able to generate and auto-publish over 1,000 a second,” Frederick says.
The processing power comes from the cloud. Automated Insights uses Amazon Web Services. “A few years ago this wouldn’t have been possible because the processing power that you need to generate these recaps is significant. But what we can do is spin it up for an hour or two and then spin it back down,” Frederick says, referring to Amazon’s rented computer power.
I spent at least 10 minutes on Yahoo’s site combing through my own fantasy report card, which also recommended still-available players who could plug my team’s now-apparent holes. That’s a long time with any personal e-mail, let alone one generated by a computer. Engagement, of course, is a key metric for advertisers. That’s surely not lost on Yahoo or its CEO, Marissa Mayer.
YAHOO SHARES ARE UP 73% since July 2012, when Mayer left Google to take on Yahoo’s challenging turnaround. Much of the praise for Mayer centers on things like improved apps, and her $1 billion acquisition of Tumblr, the social blogging site. Yahoo’s new Weather app is beautiful, no question, but I would argue that it’s a wave of content, like in the fantasy football report cards, that speaks to an even greater level of innovation.
In fact, Mayer outlined that thinking earlier this year at a Goldman Sachs investor conference. “I actually think that all of our experiences across desktop and mobile will ultimately become more personalized because at the core of Yahoo’s business is the ability to personalize content, bringing people the right information. But inside that content is also advertising.”
"That the ads and the content work together to really create a fantastic experience is what we’re really focused on," Mayer added. "And that’s where I think the future lies." Sure enough, my report card featured a Toyota ad that embedded the logo of a league-mate with the best draft performance.
Frederick, the COO of Automated Insights, concedes we’re still in the early first inning when it comes to Big Data’s real impact on consumers. “Data for data’s sake doesn’t create any value,” he says. “You need the insights and ability to decipher the insights.”
Automated Insights, with its natural language reports, is trying to be “that final mile of value for big data,” Frederick says. The firm was founded by a former Cisco engineer, Robbie Allen in 2007, as a new way to cover sports. Allen, who’s still CEO, originally called the company StatSheet. When Frederick arrived from a venture-capital firm in 2011, he worked with Allen to broaden the scope. “We realized we could do so much more than sports.” Today, the company is also providing personalized reports covering finance, real estate, health and fitness, and even corporate communications.
The start-up has raised a little over $5 million, thus far, and the payroll ranges from 20 to 30 employees. The small staff has its sights on big things. “We’ll produce basically more on a +Tuesday morning than ESPN, CBS and FOX— the largest traditional media companies—will in a year, on sheer volume of stories,” Frederick says.
As both a journalist and someone struggling to win his fantasy football league, I’ll be paying careful attention.
Sep 18, 2013
Our CEO, Robbie Allen, is in Orlando today speaking at the Business Analytics Strategies Summit.
Robbie will be speaking about Ai’s potential to turn data into action by moving past dashboards and helping people and businesses make the right decisions through the power of personalized narratives.
At Ai, we firmly believe that dashboards aren’t the answer. Dashboards don’t provide insights or deliver knowledge about data - in fact, they actually require interpretation from users. They force users to try to extract meaning from visualizations they did not design and once extracted they require those users to make inferences as to what to do next.
Computers are ideally suited to discover insights out of data, and leveraging our patented platform, we can fulfill the promise of automating the role of Data Analyst—i.e. analyze data and deliver insights about the data in plain English. However, unlike a human analyst, we can do it in real time and at scale.
The beauty of a narrative or a report written just for that individual user is that the content is always relevant, sticky and capable of driving informed action.
And with informed action, companies can finally unlock the power of their big data assets.
by Adam Smith @adambsmith
Aug 16, 2013
We are excited and honored to be selected by the Triangle Business Journal as one of the Best Places to Work in Raleigh-Durham area for the second straight year.
There are a lot of great companies here and, of all the awards we have won, nothing makes us more proud than one that celebrates our culture. We do our best to make our environment awesome because our people are what allow us to give data a voice.
So thanks to TBJ for the recognition and, more importantly, thanks to our people for making this possible!
By Adam Smith, @adambsmith
Jul 30, 2013
Here at Ai, we have a lot of exciting projects going on that require everyone to put in extra time. After all, it takes a lot of work to publish over 300 million pieces of fully personalized content each year.
We ask a lot of our team, but we try to keep things fun by building in a lot of “play time.” We find that a lot of the best ideas come when you periodically let your mind focus on something else for a few minutes. From ping-pong, to foosball, to basketball, to darts, to office putt-putt, there is no shortage of activities to do when you need a break.
And along those lines, every hackathon and work late night has a competitive tournament of some kind attached to it (it is one of the things that makes us one of the best places to work). Unfortunately for our waistlines, these work events often have a food theme as well.
The Ai Fitness Challenge
We love our people and while we enjoy a dip-a-thon greatly, we also want to make sure our employees are taking time to take care of themselves physically.
In an effort to capitalize on our team’s competitive spirit, we have instituted our first Ai Fitness Challenge. The idea is simple: anyone who increases their activity level by at least 25% for two months gets a nice prize and company-wide recognition.
As part of the challenge, we bought every employee a wearable fitness-tracking device that records their steps each day. We tracked their activity for a couple months to get a baseline and on September 1st we will see how well they did versus that baseline.
We are already seeing more people going on runs or playing pickup basketball. We don’t have a lot of internal meetings, but many of the ones we do have are turning into walking discussions.
The hope is that our folks will be more active, have more energy and ultimately be able to perform at their peak mentally. It’s an experiment, but the results so far are encouraging. Creating great products starts with an engaged and excited team and from the early results, we believe the Ai Fitness Challenge is helping us do just that.
Have you tried a similar thing at your company? If so, we’d love to hear about it.
By Adam Smith, @adambsmith
Jun 28, 2013
Turning data into timely and desirable actions from your employees and customers is the goal of any investment in big data.
Companies want to leverage that data to get customers to click the all-important “sign up,” “add to cart” “or “buy now” buttons. Similarly, they want to leverage big data to help turn all their employees into top performers - making the right decisions at the right times. Even individuals are leveraging big data to learn about themselves and to hopefully make better choices about their portfolios, health and lives.
This is the promise of big data: the chance to better understand our companies and ourselves and to make continuously improving decisions through that knowledge.
To date unfortunately, it has been an unfulfilled promise.
We should not blame big data. Turning data into understanding is not an easy task, and driving optimal actions is even harder. Not only do you have to scour massive data sets, you need to be able to separate the signal from the noise and spot the patterns and trends that have meaning. And what’s even more difficult, you must also be able to express that meaning in a way that means something to each user and that drives the desired actions. Oh, and by the way, in most cases you need to deliver that insight in real-time or it loses its value.
Dashboards, Infographics and Data Scientists
From dashboards to infographics to the rise of the “data scientist,” companies have invested billions of dollars to understand and communicate learnings from big data.
The problem is that none of those solutions have been able to scale in delivering real-time, individualized and consistently actionable advice.
Infographics have grown in popularity as a “solution” – but if you can avoid being distracted by the attractive veneer, you realize infographics are rarely able to deliver understanding and actionable insight. All too frequently, the consumers of infographics are forced to wrestle with “chart junk” which is filled with data packaged more for its looks than its relevance or action-ability. In those cases the reader may end up with some “did you know” style facts, but he or she will fall well short of understanding, and it is hard to argue that their behavior or decisions will have been optimized.
Dashboards suffer from similar shortcomings. A well-executed dashboard can effectively convey limited information, but that is the problem – it is limited information. By their nature, dashboards can only focus on a handful of variables. Unfortunately, modern day business decisions require multi-variable and multi-dimensional analysis. To meet that demand, the proponents of dashboards generally suggest additional or expanded dashboards and before you know it, the end-user is forced to process multiple and often conflicting dashboards and charts, each of which is competing for his or her attention.
To go along with the dashboards, data scientists are often employed to analyze the data and communicate what individuals need to do to produce better actions. Having a person sit down and communicate one-on-one what the key learnings are from a big data set is a great solution, if that individual has the skill and is armed with the tools to find the right meaning in that data and only needs to communicate with a handful of people. The problem is that the one-on-one data scientist model does not scale as companies get bigger. Instead, it typically results in complex reports that the individual store manager or line employee has difficulty getting value from.
As with dashboards, when the company notices the employees are not changing their behavior or making better decisions, the response is typically to increase the size of the reports or add charts or tables to give the manager even more data. However, instead of empowering those managers, they overwhelm them, hiding the key information they need to know in a maze of numbers. And what is worse, these reports consume ever-growing budgets only to come out less and less frequently, often ending up as another binder on a shelf.
At Automated Insights we don’t believe that infographics, dashboards or complicated reports are the answer. And creating more of each to try to compensate for their inherent shortcomings is certainly not the answer.
Ai As Your Personalized Advisor
The bottom line is that dashboards, infographics and chart-filled reports require interpretation from users. They force users to try to extract meaning and once extracted they require those users to make inferences as to what to do next. Whether trying to deliver insights to employees or drive action from customers, companies are communicating to a diverse set of users, all with different backgrounds, skill-sets and learning styles. One infographic or dashboard will never fit all.
Fortunately, at Ai we’ve been working on a next-generation technology platform that can help. Our platform can scour a big data set and find the hidden meaning, and even more importantly, can go the last mile and deliver actionable advice to the user via a bullet, alert or full-length story authored in plain English.
And because our software can explain what is happening in the data via the written word, we can create fully personalized stories and reports directly from data - the result is expert analysis that is written in a manner indistinguishable from that written by a human analyst. However, where it might take a human analyst several hours to write a customized report for each end-user, we can codify that expertise and provide multi-variable, multi-dimensional analysis for millions of users in almost real-time. And the beauty of a narrative or a report is that where a graph must be interpreted, words can simply be read.
In essence, we are able to give our customers’ employees and users their own personal data scientists.
Keeping the Promise
At Ai, our mission is to supply every user with an automated advisor who knows them and who helps them unlock the power of big data to improve their performance and hit their goals.
And by so doing, we can give data a voice and help big data make good on that unfulfilled promise.
by Adam Smith @adambsmith
May 23, 2013
Here at Automated Insights we have a talented team of engineers and data experts that are building an innovative platform that enables us to create diverse content such as real-time sports insights, real-estate market updates, fantasy sports recaps, web analytics summaries, and much more. We’ll approach creating over 300 million pieces of content this year alone.
Demand for our technology is accelerating and as a result, we need more people. Because we take hiring so seriously, we want to incentivize everyone to help us find talented people. To do that we are implementing internal and external referral bonuses.
We are going to pay $1,500 to any Ai employee that refers a candidate that results in a full-time hire that is with the company at least 90 days.
We are offering $1,000 to ANYONE (not employed by Ai) that refers a candidate that we hire full-time and is with the company at least 90 days. We’ll also throw in one of our coveted Automated Insights t-shirts!