Big Data News

Tell the White House What You Think About Their Use of Big Data

When you think of the US government and big data, it’s almost impossible not to think about the issue of privacy and the growing concerns surrounding it. In light of multiple controversies surrounding this topic including Edward Snowden, the NSA and general US spying, the government has (perhaps belatedly) opened conversations on the future of privacy — and you can join in.

obamaprivacy 300x148 Tell the White House What You Think About Their Use of Big Data

Back in January the White House launched a 90-day comprehensive review on the use of big data and its impact on the future of privacy. Part of this review includes a short, public survey where you can voice your opinions on who you trust (or don’t) with your data. The survey asks a series of questions like how different types of data collection concern you (with options ranging from “not at all” to “very concerned”), and there’s a free-form section where you can share everything on your mind. Headed by President Obama’s counselor John Podesta, the review should offer insight and eventually lead to an action plan on how the government uses big data.

We know you have lots to say about this so go on, tell the prez what you think HERE. While you’re at it, let us know what you think about this initiative in the comments too — we’re curious!

6fefa857 2e95 4742 9684 869168ac7099 Tell the White House What You Think About Their Use of Big Data

Big Data and the Case for Optimism

Futurist Peter Diamandis gave an inspiring TED-talk in 2012, making the case for optimism in our world — that we’ll harness technology and continue to invent, innovate and create ways to solve the challenges that loom over us. If you’re not familiar with the technological singularity (aka, the singularity), it’s a theoretical moment in time when artificial intelligence will reach the point of surpassing the intelligence of the collective human species. Supposedly this will radically change human civilization, and “perhaps even human nature itself.”

To expand a little bit further on how the singularity might come about, take Moore’s Law into consideration. Moore’s Law is the observation that the number of transistors on integrated circuits doubles about every two years. In laymen’s terms, technology is getting better and more powerful at a staggering exponential rate, which leads some people believe there will be a period where progress in technology occurs almost instantly.

singularity graphic Big Data and the Case for Optimism

How does all this factor into the future of Big Data and manufacturing? Technology is becoming cheaper and more accessible. Faster computers build faster computers, bringing in more data through the process. Basically, we’re not moving backwards. Manufacturers are realizing that making decisions on gut instincts simply won’t get the job done in the most efficient way possible. Because of technology, businesses can measure operations, interactions with customers, human resources, supply chain relationships, and more with complete accuracy.

But what good is all this meaningful data without a way to harness it and use it to your advantage? The nature of Big Data itself is just that: big data (or data too large to process through traditional methods) — but the fact is organizations that use Big Data to replace guesswork are those that become significantly more profitable than their competitors.

Stay tuned for part two of this post series, where we’ll go into a specific case study on how leveraging Big Data worked to a company’s advantage. (Sneak peek: How Big Data Transformed the Dairy Industry! Moo.)

Rhea Somaney is the community manager at Infochimps, a CSC Big Data Business, and the newest chimp to join the team. She has followed her passion for technology throughout her career, working previously for tech startups like BlackLocus and Main Street Hub. When she’s not working, you can probably find Rhea watching movies, exploring the Austin food scene, or trying to finish one of the many books on her To-Read list.

Image source:

6fefa857 2e95 4742 9684 869168ac7099 Big Data and the Case for Optimism

CSC 2014 Forecast: Big Data Trends (+ Bonus Predictions)

Big Data infrastructure was so last year. In 2013, many companies reported success in bringing together their legacy mainframe infrastructure with new Big Data infrastructure. In 2014, we’ll see companies shift their attention to putting that infrastructure investment to use.

See what Andy Walker, vice president and general manager of Big Data & Analytics at CSC, expects to see happening in enterprise Big Data in 2014. Watch the following video for 3 Big Data predictions for 2014:

For 3 bonus predictions

READ 300x80 CSC 2014 Forecast: Big Data Trends (+ Bonus Predictions)

b0bae296 90b0 4bfe 8177 b5ac72be71c6 CSC 2014 Forecast: Big Data Trends (+ Bonus Predictions)

Jim Kaskade’s Big Data Top 10

Big Data 300x298 Jim Kaskades Big Data Top 10

What do you get when you combine Big Data technologies….like Pig and Hive? A flying pig?

This opening question asked by Infochimps CEO, Jim Kaskade, sets the stage in his newest blog post, “Big Data Top Ten“. The following excerpt highlights his general prediction – leading to his top 10 Big Data predictions in 2014.

Big Data Top Ten
— My general prediction is that Cloudera and Hortonworks are both aggressively moving to fulfilling a vision which looks a lot like Gartner’s “Logical Data Warehouse”….namely, “the next-generation data warehouse that improves agility, enables innovation and responds more efficiently to changing business requirements.”
In 2012, Infochimps (now CSC) leveraged its early use of stream processing, NoSQLs, and Hadoop to create a design pattern which combined real-time, ad-hoc, and batch analytics. This concept of combining the best-in-breed Big Data technologies will continue to advance across the industry until the entire legacy (and proprietary) data infrastructure stack will be replaced with a new (and open) one.

For Jim Kaskade’s Top 10 Predictions, read the full article, here. >>

6fefa857 2e95 4742 9684 869168ac7099 Jim Kaskades Big Data Top 10

5 Ways To Become Extinct As Big Data Evolves

The enterprise is currently in a ‘Big Data Limbo’, where leadership has begun, or is preparing to invest in analytics – but lacks clear direction with where and how to implement.  The technology is powerful, and like the moniker suggests, “Big Data” is massive, leaving some executives with the impression that for these projects to be successful, IT departments must ready themselves to boil the proverbial ocean.

Infographic 300x148 5 Ways To Become Extinct As Big Data Evolves

Check out 5 Ways To Become Extinct As Big Data Evolves, Infochimps’ Infographic on CloudTweaks’ blog highlighting the 5 maladaptive practices that could lead to your own company’s extinction.

Established in 2009, is one of the fastest-growing cloud computing resources on the web. connects brands to an influential audience made of business owners and managers of growing businesses, entrepreneurs and early adopters. Their growing niche community is made of technology professionals, representatives of government agencies, financial institutions, technology firms and Fortune 500 organizations.

a9935ad0 3a68 44de 8b48 8fab1afa4b04 5 Ways To Become Extinct As Big Data Evolves

Overcoming the Data Scientist Shortage

Data scientists who can make business decisions are certainly not a dime a dozen. Today’s data professionals are tasked with driving bottom-line success for their companies by using business solutions to make actionable decisions based on customer and market insights. It takes more than a number cruncher to do that; it requires business acumen – an ability to make sense out of massive volumes of data coming from various silos.

Now consider just how much data is at the fingertips of companies today. According to IBM, 90 percent of the world’s data was created in the last two years. With such a large amount of new data, there is huge potential for multiple industries to dig through and extract insights. The only problem is that this has created a heavy demand for data scientists, a role that universities haven’t traditionally built curriculum around and companies haven’t necessarily heavily recruited for. Needless to say, there is a small pool of candidates to pick from.

In the video interview below, Michael Koploy, who researches business intelligence software solutions at Software Advice, talks with icrunchdata Co-Founder Todd Nevins, to discuss the increasing demand for Big Data jobs. They cover which specializations in the Big Data field, from data science to market analytics, are most sought-after, as well as how companies are circumventing the shortage of data science candidates to acquire top talent.

 Overcoming the Data Scientist Shortage

6e6c46da 2b08 4559 8c27 e09f1e4df781 Overcoming the Data Scientist Shortage

Announcing Application Reference Designs

Today at the Strata NY + Hadoop World Conference, we announced a new key component to our business analytics offerings, which empowers enterprises with agile development and rapid deployment of scalable Big Data applications.

Designed with the expertise gained from experience with our customers in ad tech, manufacturing, healthcare, financial services, and with use cases involving social media, and customer service, these pre-packaged frameworks for the development of Big Data applications enable businesses to quickly execute targeted and agile analytics strategies tailored to the individual needs of an organization.

CxOs can not afford to wait 24 months for their Big Data application to launch before they start making mission-critical course corrections to their business. Our customers need to deliver value years ahead of their competition.

Today I’m pleased to announce the launch of a disruptive suite of Application Reference Designs, fueling a new era of analytic application development.

App Reference Designs Release Image 2 300x123 Announcing Application Reference Designs

Read the Full Press Release Here >

Request a Demo >

406b489e b14e 4684 bbd3 c316b533aea8 Announcing Application Reference Designs

Part 1: The Truth – We Failed, We Made Mistakes

announcement 240x240 Part 1: The Truth   We Failed, We Made Mistakes

As I’m sure most of you have heard, Infochimps was recently acquired by CSC, giving us the resources and mandate to build the Big Data platform of the future. This is a perfect landing for the company and our vision, and we couldn’t be more excited.

The great acquisition stories I’m familiar with have a few commonalities: the companies share a mission and vision; the acquired team works to focus their product and integrate it with the parent company’s offering; and the parent company gives them the resources to succeed without changing what enabled the acquiring team to excel.

A perfect example of this was when Apple bought Siri. At that time, Siri was a cute little iphone app, built on amazing technology and with a highly-respected engineering team behind it. Married to Apple’s powerhouse strengths and global network, the result has transformed the way people interact with machines and is a centerpiece advantage of Apple’s product. Our goal is for nothing less than a similar story within CSC.

CSC is a global corporation that provides information technology (IT) services and professional services. They employ 95,000 people globally, who create a $16B revenue stream serving governments and large enterprise. Our challenge, and we embrace it, is to provide a significant positive return even against that massive background.

We think that we can do so (as do many analysts) because the acquisition marries the signal strengths of Infochimps and CSC:

We live in the future:

  • Proven Big Data expertise and perspective on the technical landscape
  • An indelible culture and a crazy-awesome team
  • Solid open-source citizenship, as contributors to the projects we build on and stewards of well-adopted projects we’ve written

CSC lives at enterprise-scale:

  • 50+ years of expertise in big enterprise and security
  • Passion for building customer solutions and support
  • The resources a $16B revenue stream provides

CSC’s strengths address our biggest weaknesses, letting us focus on what we do best. There are no changes to the team, the culture, our Austin location, our open-source contributions, our development approach, our irreverence, our hiring standards, or our mission to make the world smarter. We’ll continue to operate independently, continue buying lunch for the office every day, and continue open-sourcing the majority of code we write.

So this is a huge win for our team, our customers, our investors, and CSC. I could finish the post right here, and all anyone would remember is that we persevered and reached this milestone through tenacious hard work and great ideas.

Well here’s the truth: The actual history of our company is one of failure after failure, costly mistakes, and multiple near-death experiences. The only reason we’ve “succeeded” is through a preposterous series of lucky breaks and kind acts. Trying to list all the people behind that hard work and those lucky breaks would be foolish. There are too many, and I’ll just offend some by omission. But if you’re reading this, you’re probably one of them; so thank you.Success Failure Part 1: The Truth   We Failed, We Made Mistakes

Now for the real story; the story you probably haven’t heard. The story to show how large the number of people making sizable investments of time, energy, money, and kindness is required to make successes out of failures and how a small favor can change the world. It’s a thank you note to those who have helped us and a love letter to other startups figuring it out as they go. It’s a reminder that this is just another chapter of Infochimps’ book, and we’re nowhere near the resolution.

Thanks and love from the co-founders and whole Infochimps team,
Flip Kromer
Infochimps Co-Founder and CTO

*Update* Flip continued this blog series with Part 2: The Lucky Break Scoreboard, where he explains “with every failure, a smaller opportunity opened: one that was sharper; one that was more real; one that brought us closer to the right leverage point for changing the world.” Read Part 2 >>

Philip (Flip) Kromer is co-founder and CTO of Infochimps where he built scalable architecture that allows app programmers and statisticians to quickly and confidently manipulate data streams at arbitrary scale. He holds a B.S. in Physics and Computer Science from Cornell University and attended graduate school in Physics at the University of Texas at Austin. He authored the O’Reilly book on data science in practice, and has spoken at South by Southwest, Hadoop World, Strata, and CloudCon. Email Flip at or follow him on Twitter at @mrflip.

b0bae296 90b0 4bfe 8177 b5ac72be71c6 Part 1: The Truth   We Failed, We Made Mistakes

Splice Data Scientist DNA Into Your Existing Team

IT World Splice Data Scientist DNA Into Your Existing TeamAs organizations continue to grapple with Big Data demands, they may find that business managers who understand data may meet their “data scientist” needs better than the hard core data technologists

There’s little doubt that data-derived insight will be a key differentiator in business success, and even less doubt that those who produce such insight are going to be in very high demand. Harvard Business Review called “data scientist” the “sexiest” job of the 21st century, and McKinsey predicts a shortfall of about 140,000 by 2018. Yet most companies are still clueless as to how they’re going to meet this shortfall.

Unfortunately, the job description for a data scientist has become quite lofty. Unless your company is Google-level cool, you’re going to struggle to hire your Big Data dream team (well, at least right now), and few firms out there could recruit them for you. Ultimately, most organizations will need to enlist the support of existing staff to achieve their data-driven goals, and train them to become data scientists. To accomplish this, you must determine the basic elements of data scientist “DNA” and strategically splice it into the right people.

READ 300x80 Splice Data Scientist DNA Into Your Existing Team



Serial entrepreneur Jim Kaskade, CEO of Infochimps, the company that is bringing Big Data to the cloud, has been leading startups from their founding to acquisition for more than ten years of his 25 years in technology. Prior to Infochimps, Jim was an Entrepreneur-in-Residence at PARC, a Xerox company, where he established PARC’s Big Data program, and helped build its Private Cloud platform. Jim also served as the SVP, General Manager and Chief of Cloud at SIOS Technology, where he led global cloud strategy. Jim started his analytics and data-warehousing career working at Teradata for 10 years, where he initiated the company’s in-database analytics and data mining programs.

229fa9b4 2ea6 4535 8a80 e041d110204c Splice Data Scientist DNA Into Your Existing Team

Cloud. It’s More Than Just Price

It’s not about price, as GigaOM recently posted an article that discusses shifting motivations for adopting cloud.  Sure, adopting cloud will in some cases be a smaller total cost of ownership (TCO), as well as representing a variable (OpEx) expenditure instead of one big upfront investment (CapEx). Despite cloud vendor focuses on cost, customers note that time to value is the top motivation.

everest group cloud chart Cloud. Its More Than Just Price

Barb Darrow of GigaOM notes:

What’s interesting to me is that this debate is evolving much like the discussion around Software as a Service (SaaS) did a decade or so ago. Initially, when was coming into its own, most of the sales pitch was around price. Salesforce was so much cheaper than Siebel Systems.

Of course, when Microsoft started rolling out its own cloud-based CRM, that price-based argument dissipated. […] Then Salesforce’s benefits became that it freed companies from the tedium and expense of on-site server and software upgrades. You could focus on business and leave the IT heavy lifting to your provider.

Customers want to build out applications or see a return on investment as fast as possible regardless of the project; cloud enables faster iteration and agility. No need to worry about operational headaches — particularly around complex systems like streaming data pipelines or Hadoop clusters. This is a primary reasons why Infochimps’ customers choose our managed, cloud services approach to Big Data.

An even more concrete analysis is performed by Virtual Geek, with some key quotes:

[…] it’s not about being “cheaper than IT”, it’s about:

  • Being more agile than traditional IT.
  • Being more elastic economically than traditional IT.
  • Being more more price transparent than traditional IT.
  • Being more “frictionless” than traditional IT.

[…] The place for traditional IT?   IMO – Internal IT are shifting to be more of “IT services brokers”, and less about “operators”.

[…] This isn’t about technology, and the COST is not the benefit of the IaaS model of AWS EC2, it’s that the OPERATING MODEL that is the benefit.

Business units are demanding more insights and delivery on projects that IT has never had to tackle before, such as:

  • Managing terabytes and sometimes petabytes of data
  • Capturing and analyzing social media, ad impressions, website clickstreams, stock prices, and other fast moving data
  • Producing predictive insights, machines learning, statistical modeling, and interactive visualizations and dashboards

IT organizations are discovering that these complex projects don’t have to become the bane of existence and frustrate them for the next several years. These initiatives can be de-risked by embracing “cloud” to iterate more quickly – build faster, fail faster, learn faster, win faster. Cloud empowers the IT team to focus on proving out projects, not just on herding the fundamental systems.

Tim Gasper is the Director of Product for Infochimps. He was previously co-founder and CMO at Keepstream, a social media curation and analytics company. He graduated from Case Western Reserve University with dual degrees in Economics and Management and originally from Cleveland, Ohio.

229fa9b4 2ea6 4535 8a80 e041d110204c Cloud. Its More Than Just Price

Image Source: GigaOM – Everest Group – Cloud Connect 2012 Enterprise Cloud Adoption Survey