The other day I asked my Twitter followers if they had any questions or problems for me to check out. Don Steele, replied with this great thought monkey.

@donsteele : @digitalsavant “can one build enterprise database in social era”

As I started to contemplate the question, it occurred to me that this breaks down into three buckets. Technology, marketing and philosphy.

First, you have the technology challenge of collecting and storing the information. The enormous amounts of data surpass the capacity of most traditional systems.

Mid to large scale corporations generally utilize enterprise data systems to manage their information. Google has multiple data centers throughout the world entirely dedicated to the archiving and storing of information.

Fox’s, MySpace.com has multi-hundred terabyte databases, and Facebook utilizes the open source Hadoop/Hive system which “ingests 15 terabytes of new data per day“. The amount of data coming in, telling details like favorite music, locations, books, thoughts and behavioral tendencies, is ginormous.

The social scape provides mountains of data, thus needing technology back-end solutions that not only have security, but the capacity to scale and perform. These mountains of data present a new problem for IT managers, and more so for marketers. The vasts amount of information become useless without the ability to sort, and find what your looking for.

In a conversation with Kelly Phillips, CIO of Backcountry.com, he made the point of identifying the new layers needed to bring value to the data, as well as the “the increased value of indexing.”

…the social era is the providence of the enterprise database.

Some months back I had written an article about how Twitter had jumped the shark. In comments that came from site readers, a brilliant point came up. Twitters value would be cemented by indexing capability. One reader coined it, “the signal to noise ratio“.

Months later Twitter launched  live search, essentially real time indexing, squelch control. The ability to do this not so small feat revolves around a robust enterprise database and indexing solution, in Twitters case, a MYSQL backup database with a massive real time caching system. So from a technology perspective, the social era is the providence of the enterprise database.

As for marketers, the men and women who make use of this information. The question of the enterprise database in a social era takes on a new dimension. Marketers first must answer, who owns the data?

Some camps believe that businesses should own the data, other camps believe consumers should. Chances are more then likely that at some point in the near future there will be an update to the current Federal oversight laws on data security, privacy and data mining. With the advent of Google and Facebook, this question becomes even more important to solve before we proceed into darker waters.

Enterprise systems have the ability to collect every measurable detail. With the dawn of social networks, online video, streaming radio, and geo-aware tools like Google Maps, it won’t be long before companies can gather a complete web to world archetype of consumers.

Pointing to the obvious pickle, the biggest issue of enterprise databases in the social era is that of privacy. Interestingly enough, users can’t seem to give away their personal information fast enough.

Companies such as Google and Facebook have brilliant products that require users to give their private data in trade for usage and access. While Facebook still offers users the ability to control their data, the reality is that most users are unaware of who they’re sharing their info with, and what those “who people” can do with it.

Interestingly, a small group within Google has started down the path of giving users some level of control and understanding over their information.

The Data Liberation Front, a group of engineers within Google, have created a site that give users as much information, as is available, on importing and exporting data within the Googleverse. As well, most users are unaware of the Google Accounts dashboard feature, which allows users to see what information Google is collecting on them.

These are all interesting points, but still don’t bring to focus the original question. ‘Does the enterprise database have a place in the context of a social era’?

I’d say it does, it’s a matter of permission and usage. As pointed out to me by my colleague, Arnold Sookram, marketers stand to gain much through the creation of relevant messaging that speaks to the consumer. The problem being, asking for permission. As Arnold said, “It’s a fine line, depending on the industry/business, you may get a lot of opt-outs (e.g. work from home companies) or a lot of opt-ins (e.g. apple).”

While some call it ‘snake oil“, others call it gold.

Making the point that permission matters, Arnold brought to light the other main issue that has been haunting marketers worldwide. How do we fully harness social media, and is it still too nascent to leverage?

The reality is, marketers can certainly use the data to make messages more meaningful. Brands can use social media to manage and track product launches as well as collect consumer feedback. Publishers can leverage social to seed their content, but what’s missing is most important of all.

The ever illusive, influencer algorithm.

The potential for enterprise to tie together seemingly unconnected threads, is the promise of digital. While marketers can use tools such as Friendly Feed Facts to build a basic understanding of consumer archetypes, the reality is that there’s still no system connecting it all. The data hidden in relationships, community and communications is still not fully understood.

The future of enterprise in a social era is the connecting of disparate systems, bringing context to communication, behavior and content. Perhaps the best place to see where the world of digital marketing is going is none other then the CIA.

In-Q-Tel, the venture capital arm of the CIA, invests greatly in behavioral sciences, indexing technologies and analytics. Trolling through their investments, you’d think they’re a marketing technology investment fund. It doesn’t take long to see that the future of the enterprise database social scape lives in close proximity to the abilities developed via big brother investments.

With massive amounts of data, it’s important that we focus on finding the meaningful connections through the assimilation of varied data sets. But, before we get there, we must figure out the gray area of ownership and transparent permission.

Even beyond the Orwellian investments of the CIA, there are many areas where datasets of disparate systems are waiting for someone to bridge the divide.

The U.S. Government released the site, Data.Gov, allowing the public to use rich sets of information, in just about any way they want. With all this data, the point originally made by Kelly Phillips becomes more transparent. The data is useless with indexing and context.

Thoughtfully and thankfully, people like Hans Rosling have worked to bring context to the world of information floating about. GapMinder.org, create moving works of mathematic art utilizing the Trendalyzer tool. The tool was so useful that Google bought it.

This trend of free data visualization tools and communities continue with sites like ManyEyes and Swivel.com.

Unfortunate for marketers, but good for consumers, the sharing of enterprise data sets between organizations is not as prevalent as it soon will be. As implied, I don’t believe it will be long until government organizations, corporations and other private entities seek to connect the vast amounts of information that are created daily through social media sites.

Facebook API’s and Twitter API’s used to harvest data are not implausible, and most likely probable. While this is great for those looking to make the most of their messages, I can’t help but feel that its a negative association for consumers.

So here we sit, wielding the proverbial double edged sword. We can learn more, do more, and create more through the evolution of enterprise database marketing in the social web, but we loose more when we do.

If the corporations can do it, so can anybody. Do we really want that?

Cheers,

B