Big Data and the Case for Optimism

Futurist Peter Diamandis gave an inspiring TED-talk in 2012, making the case for optimism in our world — that we’ll harness technology and continue to invent, innovate and create ways to solve the challenges that loom over us. If you’re not familiar with the technological singularity (aka, the singularity), it’s a theoretical moment in time when artificial intelligence will reach the point of surpassing the intelligence of the collective human species. Supposedly this will radically change human civilization, and “perhaps even human nature itself.”

To expand a little bit further on how the singularity might come about, take Moore’s Law into consideration. Moore’s Law is the observation that the number of transistors on integrated circuits doubles about every two years. In laymen’s terms, technology is getting better and more powerful at a staggering exponential rate, which leads some people believe there will be a period where progress in technology occurs almost instantly.

singularity graphic Big Data and the Case for Optimism

How does all this factor into the future of Big Data and manufacturing? Technology is becoming cheaper and more accessible. Faster computers build faster computers, bringing in more data through the process. Basically, we’re not moving backwards. Manufacturers are realizing that making decisions on gut instincts simply won’t get the job done in the most efficient way possible. Because of technology, businesses can measure operations, interactions with customers, human resources, supply chain relationships, and more with complete accuracy.

But what good is all this meaningful data without a way to harness it and use it to your advantage? The nature of Big Data itself is just that: big data (or data too large to process through traditional methods) — but the fact is organizations that use Big Data to replace guesswork are those that become significantly more profitable than their competitors.

Stay tuned for part two of this post series, where we’ll go into a specific case study on how leveraging Big Data worked to a company’s advantage. (Sneak peek: How Big Data Transformed the Dairy Industry! Moo.)

Rhea Somaney is the community manager at Infochimps, a CSC Big Data Business, and the newest chimp to join the team. She has followed her passion for technology throughout her career, working previously for tech startups like BlackLocus and Main Street Hub. When she’s not working, you can probably find Rhea watching movies, exploring the Austin food scene, or trying to finish one of the many books on her To-Read list.

Image source:

6fefa857 2e95 4742 9684 869168ac7099 Big Data and the Case for Optimism

Movies + Charts = Nerdy Creativity

I love movies. I love charts. I have to say that FlowingData did it again – this is brilliant:

AFI movie quotes Movies + Charts = Nerdy Creativity


In celebration of their 100-year anniversary, the American Film Institute selected the 100 most memorable quotes from American cinema. FlowingData took those quotes and created the 100 most memorable quotes in chart form.

See the chart in bigger detail, here. >>

As always, thank you FlowingData for providing interesting posts for us data nerds.

3527b357 2038 47ae a163 deda4a8c5176 Movies + Charts = Nerdy Creativity

Becoming a Believer in Artificial Intelligence

Big Think originally published this transcript of Eric Siegel’s own words. The article relates to his book, Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die.

Why I Became a Believer in Artificial Intelligence

I’ve been asked periodically for a couple of decades whether I think artificial intelligence is possible.  And I taught the artificial intelligence course at Columbia University.  I’ve always been fascinated by the concept of intelligence.  It’s a subjective word.  I’ve always been very skeptical. And I am only now newly a believer.

Now this is subjective: my opinion is that IBM’s Watson computer is able to answer questions, and so, in my subjective view, that qualifies as intelligence.  I spent six years in graduate school working on two things.  One is machine learning and that’s the core to prediction – learning from data how to predict.  That’s also known as predictive modeling. And the other is natural language processing or computational linguistics.

Working with human language really ties into the way we think and what we’re capable of doing and that does turn out to be extremely hard for computers to do.  Now playing the TV quiz show Jeopardy means you’re answering questions – quiz show questions.  The questions on that game show are really complex grammatically.  And it turns out that in order to answer them Watson looks at huge amounts of text, for example, a snapshot of all the English speaking Wikipedia articles.  And it has to process text not only to look at the question it’s trying to answer but to retrieve the answers themselves.  Now at the core of this it turns out it’s using predictive modeling.  Now it’s not predicting the future but it’s predicting the answer to the question.

The core technology is the same.  In both cases it involves learning from examples.  In the case of Watson playing the TV show Jeopardy it takes hundreds of thousands of previous Jeopardy questions from the TV show having gone on for decades and learns from them.  And what it’s learning to do is predict whether this candidate answer to this question is likely to be the correct answer.  So it’s going to come up with a whole bunch of candidate answers, hundreds of candidate answers, for the one question at hand at any given point in time.  And then amongst all these candidate answers it’s going to score each one.  How likely is it to be the right answer?  And, of course, the one that gets the highest score as the highest vote of confidence – that’s ultimately the one answer it’s going to give.

READ 300x80 Becoming a Believer in Artificial Intelligence



Eric Siegel, Ph.D., founder of Predictive Analytics World and Text Analytics World, and Executive Editor of the Predictive Analytics Times, makes the how and why of predictive analytics understandable and captivating. In addition to being the author of Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, Eric is a former Columbia University professor, and a renowned speaker, educator, and leader in the field. 

406b489e b14e 4684 bbd3 c316b533aea8 Becoming a Believer in Artificial Intelligence

Strata Santa Clara – 25% Discount + Early Price Special

strata logo 500px 300x219 Strata Santa Clara   25% Discount + Early Price SpecialLast October, at Strata + Hadoop World, Infochimps announced Application Reference Designs and had a booth full of t-shirt giving chimps. This February, at Strata Santa Clara, Infochimps will be back with the same enthusiastic team (and t-shirts, of course) eager to talk about Big Data.

The O’Reilly Strata Conference always delivers. They bring together the brightest minds in data science and Big Data: decision makers using data to drive business strategy, as well as practitioners who collect, analyze, and manipulate it. Join Infochimps as we mingle with over 150 of the leading data experts, network with our peers, and hear about the latest (and emerging) data tools, technologies, and best practices.

Infochimps will be exhibiting at Booth # 740 so be sure to stop by and grab our famous Infochimps t-shirt, chat with exhibiting team members, or set up a 1:1 meeting, we’d love to chat!

CONTACT 300x78 Strata Santa Clara   25% Discount + Early Price Special



Not registered? Register today and save 25% with discount code: INCH1 on top of the early price special ending January 9th. Strata sells out every conference, so register today!

119efc1b cf09 4f4f 9085 057e76e0464c Strata Santa Clara   25% Discount + Early Price Special

CSC 2014 Forecast: Big Data Trends (+ Bonus Predictions)

Big Data infrastructure was so last year. In 2013, many companies reported success in bringing together their legacy mainframe infrastructure with new Big Data infrastructure. In 2014, we’ll see companies shift their attention to putting that infrastructure investment to use.

See what Andy Walker, vice president and general manager of Big Data & Analytics at CSC, expects to see happening in enterprise Big Data in 2014. Watch the following video for 3 Big Data predictions for 2014:

For 3 bonus predictions

READ 300x80 CSC 2014 Forecast: Big Data Trends (+ Bonus Predictions)

b0bae296 90b0 4bfe 8177 b5ac72be71c6 CSC 2014 Forecast: Big Data Trends (+ Bonus Predictions)

Jim Kaskade’s Big Data Top 10

Big Data 300x298 Jim Kaskades Big Data Top 10

What do you get when you combine Big Data technologies….like Pig and Hive? A flying pig?

This opening question asked by Infochimps CEO, Jim Kaskade, sets the stage in his newest blog post, “Big Data Top Ten“. The following excerpt highlights his general prediction – leading to his top 10 Big Data predictions in 2014.

Big Data Top Ten
— My general prediction is that Cloudera and Hortonworks are both aggressively moving to fulfilling a vision which looks a lot like Gartner’s “Logical Data Warehouse”….namely, “the next-generation data warehouse that improves agility, enables innovation and responds more efficiently to changing business requirements.”
In 2012, Infochimps (now CSC) leveraged its early use of stream processing, NoSQLs, and Hadoop to create a design pattern which combined real-time, ad-hoc, and batch analytics. This concept of combining the best-in-breed Big Data technologies will continue to advance across the industry until the entire legacy (and proprietary) data infrastructure stack will be replaced with a new (and open) one.

For Jim Kaskade’s Top 10 Predictions, read the full article, here. >>

6fefa857 2e95 4742 9684 869168ac7099 Jim Kaskades Big Data Top 10

[Infographic] Big Data is Changing the Rules…Adapt.

Check out our new infographic, 5 Ways To Become Extinct As Big Data Evolves, highlighting the 5 maladaptive practices that could lead to your own company’s extinction.
5 ways to become extince as big data evolves 528bb2660a7ac w587 [Infographic] Big Data is Changing the Rules...Adapt.

Explore more infographics like this one on the web’s largest information design community – Visually.


[Guest Blog] Jim Kaskade on Requirements for Effective Analytics Projects

Data Informed 300x62 [Guest Blog] Jim Kaskade on Requirements for Effective Analytics ProjectsInfochimps CEO, Jim Kaskade, recently guest blogged for Data Informed, the leading resource for business and IT professionals looking for news, in-depth case studies, expert insight, best practices, and product analyses to plan and implement their data analytics and management strategies.

In his blog post, “Effective Analytics Projects Require Blend of 3 Approaches, in Concert“, he mentions “arranging technologies to address a particular use case is a little bit like arranging instruments in a symphony”.

“Each has particular qualities and plays a different role in the end product, but that end result should be a seamless meshing of the individual elements to achieve the desired effect such that the unique qualities of each one fade into the background. The same is true of Big Data use cases.”

Read the full article, here. >>

Thank you Data Informed for giving decision makers perspective on how they can apply Big Data concepts and technologies to their business needs.

406b489e b14e 4684 bbd3 c316b533aea8 [Guest Blog] Jim Kaskade on Requirements for Effective Analytics Projects

[New Video] Agile Big Data: Getting to Insights Faster

Webinar Resource [New Video] Agile Big Data: Getting to Insights Faster

2 weeks ago, Infochimps Director of Product, Tim Gasper, discussed how Infochimps tackles business problems for customers by deploying a comprehensive Big Data infrastructure in days; sometimes in just hours, in his Getting to Insights Faster: A Framework for Agile Big Data webinar.

If you missed Tim’s webinar, check out the recorded video and view the slide deck on SlideShare.  During this video, you will learn:

  • How agile Big Data application development differs from traditional development approaches
  • What our agile delivery framework looks like for planning Big Data projects and architecting customer solutions
  • What App Reference Designs are, and how they accelerate customer use cases
  • Real life case studies of business problems that have benefited from our agile approach
  • A technology deep dive into a customer example

Get to insights faster by utilizing a new approach to allow agile, iterative development for successful insights in as soon as 30 days.

a9935ad0 3a68 44de 8b48 8fab1afa4b04 [New Video] Agile Big Data: Getting to Insights Faster

5 Ways To Become Extinct As Big Data Evolves

The enterprise is currently in a ‘Big Data Limbo’, where leadership has begun, or is preparing to invest in analytics – but lacks clear direction with where and how to implement.  The technology is powerful, and like the moniker suggests, “Big Data” is massive, leaving some executives with the impression that for these projects to be successful, IT departments must ready themselves to boil the proverbial ocean.

Infographic 300x148 5 Ways To Become Extinct As Big Data Evolves

Check out 5 Ways To Become Extinct As Big Data Evolves, Infochimps’ Infographic on CloudTweaks’ blog highlighting the 5 maladaptive practices that could lead to your own company’s extinction.

Established in 2009, is one of the fastest-growing cloud computing resources on the web. connects brands to an influential audience made of business owners and managers of growing businesses, entrepreneurs and early adopters. Their growing niche community is made of technology professionals, representatives of government agencies, financial institutions, technology firms and Fortune 500 organizations.

a9935ad0 3a68 44de 8b48 8fab1afa4b04 5 Ways To Become Extinct As Big Data Evolves