Why CrowdCloud Pivoted to Filter Social Media Data – and Why Managing Big Data Is the Next Big Thing

Monika Jansen Jun 15, 2012 Tweet

If you’re on social media (and I am assuming all of you reading this are), then you know how much junk is posted all day every day.  Stupid status updates – “Gosh, it’s hot out today” – and tweets that are even more inane – “Just landed at SFO.”  Makes you just want to give up Facebook and Twitter forever.

CrowdCloud to the rescue. This free iPhone and Android app filters through all of the social media data out there to deliver the best results of what’s happening in your current location. You can also share photos and information tagged by location.

But the app didn’t initially sift through the crap to deliver only high quality results.  It was originally intended to simply broadcast local information on what was happening now.  Created by Adiant, “the innovation leader in digital media technology for publishers and advertisers,” the CrowdCloud team ran into two problems.

One: all user-generated apps face the problem of content creation.  Josh Detweiler, Director of Operations – “though we don’t really use titles, but that is basically what I do” – explained:

“We designed the app to be powered exclusively by users, just like a lot of social media platforms – think Foursquare.  Problem with that model is the chicken and egg dilemma – if you open up the app and you don’t see any activity, you’re not gonna use it.”

So they got creative.  To solve the problem, they hired 100 high-profile people in 10 major cities to post 5-10 things per day.  “We thought if we could generate 100 posts per city per day, we’d have the perfect solution to the chicken and egg problem.”

Two: Even people paid to post updates on what was happening generated crappy data.  “We realized the number one problem we had to deal with is quality.  So we started sorting through social media data to see what is most interesting.”

As a result of this experience, CrowdCloud pivoted.  Now it searches through 100 million Facebook posts every day, analyzing data and scoring it by quality to post only useful, relevant information on what is happening.  Whatever gets a high score gets pushed to the app.  As a result, they filter out more than 99% of all data.

“A high quality post is not really about details,” explained Detweiler.  “It is more about actionable data – what can a viewer do with this data and how is it applicable to their life?”

Detweiler was on a roll at this point, and he said the area of big data is a huge opportunity for entrepreneurs right now.  “A few years ago, Twitter was doing virtually no volume – now they’re at 500 million tweets per day, and on Facebook there are 1 billion posts per day.  Dealing with this tsunami of data and self-broadcasting obsession is dealing with the data …. Today, everyone can share news.  How do you find what’s important and useful?  How do you find a trusted source?  Even trusted sources tweet junk.

“This is a bold prediction, but I think that one of the next Bill Gates-type people who gets it and does it extremely well is the person or people who figure out how to handle this user-generated data.  There is enormous power in this data.  If you could classify and analyze it – we’d find out some amazing things about humanity. You’d see some amazing trends happening in real-time.”

Entrepreneurs, take Detweiler’s comments as a serious call to action: Figure out to deal with big data, and you’ll have struck a gold mine.