2011: A Big Year for Big Data

Posted by Ramy Ghaly February 03, 2011

During the 2011 NFL playoff TV broadcasts, amid commercials featuring Anheuser-Busch Clydesdales and auto-racing driver Danica Patrick, one ad features an IBM researcher talking about data analytics.

With data volumes moving past terabytes to tens of petabytes and more, business and IT leaders across the board face significant opportunities and challenges from big data. For a large company, big data may be in the petabytes or more; for a small or mid-size enterprise, data volumes that grow into tens of terabytes become "big data". 

In the financial services industry, Fidelity National Information Services (FIS), which sells risk management and fraud detection services to credit card issuers, uses big data analytics to better detect credit card fraud.

Nonetheless, 2011 is shaping up to be a big year for big data.

This summary was made possible by ctrl-news Via The New York Times

Does Social Media Sets “New World Order” in building Revolutions throughout the Middle East?

BY consultramy

While the uprising started in Tunisia requesting from a long dictator to step down, Egypt is following suite of the recent Tunisian's Jasmine Revolution seeking reform, freedom, and respect by using social media to organize, communicate, and express their feelings not only to the world, but to say to their dictators "this is the end of your rule, and now is our time to make the change weather you like it or not".

The young catalysts of Twitter, Facebook, and social media are behind the driving force of the revolution not only in Tunisia and Egypt, but also in Jordan and Yemen where the role of social media has proven a new world order in digital social communications to bring down totalitarian regimes all over the Middle East. Nonstop-able young embryonic youth, the educated elite of the Arab world are speaking out seeking freedom. Yes, freedom is a high commodity especially in the Arab world where no real democracies exists; yet decades of totalitarian regimes that took the power of their people and turned them into modern slaves. Moreover, the blocked elite of Egypt are speaking out after thirty years of unprecedented selfish rule that stole their dignities by keeping them away from engaging in decision making, take responsibilities of their future are now fighting for their pride in setting the road map to gain freedom, the right to speak out, needless to say, gain respect by re-writing history that makes them proud citizens of their forced change to set a better example not only to Egypt, but to other dictators in the Middle East and the world. Now, social media set fear in dictatorships throughout the Arab world where Egypt's dictatorships shut down all social networks and communications to put more pressure on the young youth to stop their uprising, but that not only turned against the totalitarian regime, instead, it sparked more anger and gave the youth more reason not to give up that easily; yet embrace the fight for freedom to the end while Mr. Mubarak is witnessing his last moments of the 30 years old rule.

Going social in the digital world is building a new world order by driving change and bringing down dictators at least in the Arab world. The young blocked elite of Egypt are seeking change from regime reform, free elections, and the ability to choose their own leaders, to requesting their basic civil rights to be heard and respected; yet achieving it is even more challenging when chaos takes over in the streets and sets a whole new ball game. Social media is playing a vital role to make change possible and give these young rebels a tool that it was unprecedented before having the ability to bring down dictators and set a new world order in social digital communications.

Important facts about Egypt according to CIA FactBook's latest figures: 2010

Population: 80 Million

Median age: 24 years

GDP: Approximately $216.8 billion (2009)

Per capita income: $6,200 (GDP/year-PPP 2010)

Unemployment: 9.7% (est.)

Poverty: 40%

Strategic interests to the world:

• Israeli/Egyptian Peace Treaty signed In 1979.
• It is estimated that 10 percent of the global crude oil demand passes through the Suez Canal.
• The biggest population in the Middle East
• Egypt receives nearly $2-$3 billion in aid per year from the United States.
• Egypt holds ancient treasures and artifacts.
• The Pyramids are one of the top wonders of the world.
• A global destination for tourists making it one of the top tourist markets in the world.

Popular Hash Tags used for the "uprising in Egypt" on Twitter:

#liberation technology

#social freedom

#twitter revolution

#SM revolution

#Egypt

#jan25

#freedom

#Tahrir

#Cairo

#reform

#protest

#democracy

#support

#liberty

#civil liberty

#uprising

#Mona Altahawy

Latest Sentiment analysis of Egypt's uprising since it started six days ago on January 25, 2011

Link here: Egypt's Uprising Sentiment Analysis

About the Author:

Ramy Ghaly is a Marketing Strategist with more than ten years in international markets experience. He held professional and managerial positions in multiple global markets in various industries ranging from retail, wholesale, consumer goods, to technology product management with concentration in channel development. In addition, He holds a degree in International Marketing Management with a minor in International Relations and Middle Eastern studies from Daytona State College. He is interested in social media developments, next generation search technologies, semantic search engines, and text analytics; needless to say, strategies in geopolitics, Middle Eastern Studies, and Environmental factors that affect global business growth are general interests that keen to always monitor and encourage writing about.

What are the most challenging issues in Sentiment Analysis(opinion mining)?

Ramy Ghaly January 28, 2011

Hossein Said:

Opinion Mining/Sentiment Analysis is a somewhat recent subtask of Natural Language processing.Some compare it to text classification,some take a more deep stance towards it. What do you think about the most challenging issues in Sentiment Analysis(opinion mining)? Can you name a few?

 

Hightechrider Said:

The key challenges for sentiment analysis are:-

1) Named Entity Recognition - What is the person actually talking about, e.g. is 300 Spartans a group of Greeks or a movie?

2) Anaphora Resolution - the problem of resolving what a pronoun, or a noun phrase refers to. "We watched the movie and went to dinner; it was awful." What does "It" refer to?

3) Parsing - What is the subject and object of the sentence, which one does the verb and/or adjective actually refer to?

4) Sarcasm - If you don't know the author you have no idea whether 'bad' means bad or good.

5) Twitter - abbreviations, lack of capitals, poor spelling, poor punctuation, poor grammar, ...

 

ealdent Said:

I agree with Hightechrider that those are areas where Sentiment Analysis accuracy can see improvement. I would also add that sentiment analysis tends to be done on closed-domain text for the most part. Attempts to do it on open domain text usually winds up having very bad accuracy/F1 measure/what have you or else it is pseudo-open-domain because it only looks at certain grammatical constructions. So I would say topic-sensitive sentiment analysis that can identify context and make decisions based on that is an exciting area for research (and industry products).

I'd also expand his 5th point from Twitter to other social media sites (e.g. Facebook, Youtube), where short, ungrammatical utterances are commonplace.

 

Skarab Said:

I think the answer is the language complexity, mistakes in grammar, and spelling. There is vast of ways people expresses there opinions, e.g., sarcasms could be wrongly interpreted as extremely positive sentiment.

 

What do you think? Do you agree? Would you like to ask a question and get an answer? Try out: Q&A for professional and enthusiast programmers 

 

 

Social commerce to surge

NEW YORK: Social commerce sales will rise dramatically during the next five years, encouraging brands and retailers to enhance their presence on sites like Facebook, Booz & Co has argued.

In a new report, the consultancy stated marketers must shift the terms of engagement with consumers using Web 2.0 properties from "like" to "buy".

"The market for social commerce has been embryonic to date, but that will change over the next five years as companies race to establish stores," it said.

"Trendsetting companies are focused on products and services that benefit from the unique characteristics of social media, including the opportunity to get quick feedback from multiple friends and family members."

The study praised 1-800-Flowers, which boasts a fully-functioning Facebook store allowing customers to complete purchases without leaving the network's pages.

It has also implemented other innovative strategies, for example linking Facebook's calendar and "group gifting" features to its Mother's Day campaign.

"We are going to continue to invest in certain areas to help drive future growth," Bill Shea, 1-800-Flowers' chief financial officer, said in late 2010.

"Whether it be franchising efforts for both the consumer floral and our food group, investments in mobile and social commerce [or] floral supply chain in Celebrations.com, we are going to continue to invest."

Dell was cited as another pioneering early-adopter, having earned millions of dollars in revenue through Twitter.

The IT specialist is becoming increasingly active in the smartphone and tablet segments, which the organisation believes will transform the retail sector.

"It used to be 'We're going to tell you how you're going to experience our store,'" said Brian Slaughter, Dell's director, end-user solutions, large enterprises.

"Now the consumer is walking in and saying: 'No, I'm going to tell you how I'm going to use your store to give me more information.' The tools they have at their disposal are very cool."

Similarly, Quidsi, owned by Amazon, recently set up Facebook outlets for its Soap.com and Diapers.com platforms, although the ability to make purchases is limited to members of these two portals.

"No one has yet cracked the nut on Facebook e-commerce," said Josh Himwich, Quidsi's vp, ecommerce.

Overall, Booz estimated sales of physical goods via social channels would hit $5bn (€3.7bn; £3.1bn) globally in 2011, with the US contributing 20% of this total.

Revenues were pegged to reach $9bn by the close of 2012, incorporating $3bn generated by American internet users.

Such figures should achieve $14bn and $5bn respectively for 2013, while US customers deliver nearly half of the $20bn returns yielded in 2014.

By 2015, the worldwide expenditure attributable to this medium is anticipated to come in at $30bn, housing $14bn from the world's biggest economy.

A previous Booz survey of netizens dedicating one hour or more a month to social networks, and who bought at least one product online in the last year, found 20% proved willing to pay for items through these sites.

Elsewhere, 10% suggested this spending would be incremental to their current outlay, but 71% added "liking" a brand on Facebook did not improve the probability of buying it.

The consultancy predicted that social media will have the greatest impact on consideration, conversion, loyalty and customer service.

Facebook's chief executive Mark Zuckerburg certainly supported such as optimistic reading when rolling out the Places geo-location system last year.

"If I had to guess, social commerce is the next area to really blow up," he said in August. 

Data sourced from Booz & Co, Seeking Alpha, Daily Finance; additional content by Warc staff, 21 January 2011

Via: WARC

Enterprise application software market in Middle East and North Africa to rebound from significant slowdown, says IDC

United Arab Emirates: Sunday, January 16 - 2011 

The Middle East and North Africa (MENA) enterprise application software (EAS) market is expected to return to double-digit growth rates from 2010 after suffering a considerable slowdown in growth in 2009 due to the impact of the global economic crisis on the region. According to a recent report by market research company IDC, the region is predicted to expand at an average annual growth rate of 12.8% by 2014.

Widespread regional liquidity difficulties and the delay or cancellation of EAS projects by organizations that were forced to revisit their spending plans proved particularly troublesome, although the market will soon rebound to former heights," says Dhiraj Daryani, senior analyst for the software market at IDC Middle East, Africa, and Turkey. 

As the global economic recovery takes shape, and as governments and businesses in the MENA region proceed with IT modernization efforts and application transformation, high-growth vertical segments like education and healthcare will continue to remain the fastest growth areas for EAS solutions, while other segments that saw a marked reduction in spending in 2009, like business services and telecommunications, will forge ahead with EAS investments. 

Process and discrete manufacturing will continue to be critical drivers of EAS spending, but their share of the total market will gradually slow through 2014, while the finance vertical will rebound strongly from the pounding it took during the crisis. Saudi Arabia will remain the MENA region's largest EAS market, and will also be its fastest growing, while Egypt is also tipped for strong growth as large numbers of businesses migrate to modern EAS suites from the installed base of legacy applications. 

Global giants SAP, Oracle, and Microsoft Dynamics dominate the MENA region's EAS market. However, the leading vendors must not rest on their laurels as businesses and governments begin to re-examine their wait-and-see approach to investing in innovative IT applications. "Customers are no longer content with just implementing solutions from global vendors," says Daryani. "They also want to see the quantifiable value they derive from adopting such solutions.

As such, vendors need to bring in world-class expertise and resources if they are to be seen as long-term players in local markets. Vendors that can both demonstrate their commitment to individual country markets and clearly convey the value of their solutions to customers stand to gain market share." 

IDC's Arab Middle East and North Africa Enterprise Application Software 2010-2014 Forecast and 2009 Vendor Shares (IDC #ZR01S) provides a detailed overview of the region's market for integrated EAS suites. It includes detailed qualitative and quantitative information, analysis, and forecasts that help vendors answer key questions regarding market size, segmentation, market shares, and major economic and political factors affecting the Arab MENA EAS market.

Via: AMEinfo.com

Google Acquires eBook Technologies

By Thomas Claburn ,  InformationWeek 
January 13, 2011 02:23 PM

The deal strengthens Google's digital content distribution capabilities and diminishes its vulnerability to potential patent lawsuits. 

Having acquired 25 companies in 2010, Google is continuing its buying spree in the new year with the purchase of eBook Technologies, a supplier of e-book reading devices and content distribution technology.

"eBook Technologies, Inc. is excited to announce that we have been acquired by Google," said the La Jolla, Calif.-based company on its Web site. "Working together with Google will further our commitment to providing a first-class reading experience on emerging tablets, e-readers and other portable devices."

No price for the deal was disclosed.

 

Google declined to provide specific details about its future product plans. "We are happy to welcome eBook Technologies' team to Google," said a spokesperson in an e-mailed statement. "Together, we hope to deliver richer reading experiences on tablets, electronic readers and other portable devices."

In December, Google launched its digital bookselling platform,Google eBooks, the culmination of years of legal wrangling and book scanning. Having entered into competition with Amazon and Apple in the process, Google has tried to differentiate itself by characterizing its ecosystem as more open than what's offered by its rivals.

"Open" however doesn't mean open in the sense of content without digital locks. It means open in the sense of allowing partners to have a meaningful role. In fact, Google's interest in eBook Technologies appears to be in securing e-book content and protecting itself against potential patent lawsuits.

eBook Technologies licenses e-book technology from companies hailing from dot com boom at the turn of the second millennium: SoftBook Press, NuvoMedia, and Gemstar. It also appears to have rights related to some relevant patents, such as one titled "Electronic Display Device Event Tracking," which lists eBook Technologies co-founder and president Garth Conboy among the inventors.

Indeed, the company boasts about its intellectual property portfolio on its Web site, or at least it did until these pages were removed in conjunction with the acquisition. "Patented areas of the eBook technology suite cover the unique designs, features and functions of the entire eBook publishing system," the company states on its old Web site. "Intellectual property includes: the eBook system and features, cryptography, user interface elements, industrial design and manufacturing processes."

Google's interest in eBook Technologies may also have an enterprise angle: eBook Technologies has developed a comprehensive network architecture for e-book sales and distribution that includes a component called eBook Express Manager. This is an application that "allows enterprise customers to centrally manage the delivery, access, audit, and updating of enterprise content for groups of e-book users."

Read More Via: InformationWeek 

Enterprise Trends: Contrarians and Other Wise Forecasters

By Lynda Moulton on January 13, 2011

The gradual upturn from the worst economic conditions in decades is reason for hope. A growing economy coupled with continued adoption of enterprise software, in spite of the tough economic climate, keep me tuned to what is transpiring in this industry. Rather than being cajoled into believing that "search" has become commodity software, which it hasn't, I want to comment on the wisdom of Jill Dyché and her Anti-predictions for 2011 in a recent Information Management Blog. There are important lessons here for enterprise search professionals, whether you have already implemented or plan to soon.

Taking her points out of order, I offer a bit of commentary on those that have a direct relationship to enterprise search. Based on past experience, Ms. Dyché predicts some negative outcomes but with a clear challenge for readers to prove her wrong. As noted, enterprise search offers some solutions to meet the challenges:

1. No one will be willing to shine a bright light on the fact that the data on their enterprise data warehouse isn't integrated. It isn't just the data warehouse that lacks integration among assets, but among all applications housing critical structured and unstructured content. This does not have to be the case. Several state-of-the-art enterprise search products that are not tied to a specific platform or suite of products do a fine job of federating indexing of disparate content repositories. In a matter of weeks or few months, a search solution can be deployed to crawl, index and search multiple sources of content. Furthermore, newer search applications are being offered for pre-purchase testing for out-of-the-box suitability in pilot or proof-of-concept (POC) projects. Organizations that are serious about integrating content silos have no excuse for not taking advantage of easier to deploy search products.


2. Even if they are presented with proof of value, management will be reluctant to invest in data governance. Combat this entrenched bias with a strategy to overcome lack of governance; a cost cutting argument is unlikely to change minds. However, risk is an argument that will resonate, particularly when bolstered with examples. Include instances when customers were lost due to poor performance or failure to deliver adequate support services, sales were lost because answers to qualifying questions could not be answered or were not timely, legal or contract issues could not be defended due to inaccessibility of critical supporting documents, or when maintenance revenue was lost due to incomplete, inaccurate or late renewal information getting out to clients. One simple example is the consequences of not sustaining a concordance of customer name, contact, and address changes. The inability of content repositories to talk to each other or aggregate related information in a search because a Customer labeled as Marion University at one address is the same as the Customer labeled University of Marion at another address will be embarrassing in communications and, even worse, costly. Governance of processes like naming conventions and standardized labeling enhances the value and performance of every enterprise system including search.


3. Executives won't approve new master data management or business intelligence funding without an ROI analysis. This ties in with the first item because many enterprise search applications include excellent tools for performing business intelligence, analytics, and advanced functions to track and evaluate content resource use. The latter is an excellent way to understand who is searching, for what types of data, and the language used to search. These supporting functions are being built into applications for enterprise search and do not add additional cost to product licenses or implementation. Look for enterprise search applications that are delivered with tools that can be employed on an ad hoc basis by any business manager.


4. Developers won't track their time in any meaningful way. This is probably true because many managers are poorly equipped to evaluate what goes into software development. However, in this era of adoption of open source, particularly for enterprise search, organizations that commit to using Lucene or Solr (open source search) must be clear on the cost of building these tools into functioning systems for their specialized purposes. Whether development will be done internally or by a third party, it is essential to place strong boundaries around each project and deployment, with specifications that stage development, milestones and change orders. "Free" open source software is not free or even cost effective when an open meter for "time and materials" exists.


5. Companies that don't characteristically invest in IT infrastructure won't change any time soon. So, the silo-ed projects will beget more silo-ed data...Because the adoption rate for new content management applications is so high, and the ease for deploying them encourages replication like rabbits, it is probably futile to try to staunch their proliferation. This is an important area for governance to be employed, to detect redundancy, perform analytics across silos, and call attention to obvious waste and duplication of content and effort. Newer search applications that can crawl and index a multitude of formats and repositories will easily support efforts to monitor and evaluate what is being discovered in search results. Given a little encouragement to report redundancy and replicated content, every user becomes a governor over waste. Play on the natural inclination for people to complain when they feel overwhelmed by messy search results, by setting up a simple (click a button) reporting mechanism to automatically issue a report or set a flag in a log file when a search reveals a problem.

It is time to stop treating enterprise search like a failed experiment and instead, leverage it to address some long-standing technology elephants roaming around our enterprises.

 


To follow other search trends for the coming year, you may want to attend a forthcoming webinar, 11 Trends in Enterprise Search for 2011, which I will be moderating on January 25th. These two blogs also have interesting perspectives on what is in store for enterprise applications: CSI Info-Mgmt: Profiling Predictors 2011, by Jim Ericson and The Hottest BPM Trends You Must Embrace In 2011!, by Clay Richardson. Also, some of Ms. Dyché's commentary aligns nicely with "best practices" offered in this recent beacon, Establishing a Successful Enterprise Search Program: Five Best Practices


Via: http://gilbane.com/search_blog/2011/01/enterprise_trends_contrarians_and_other_wise_forecasters.html#ixzz1B08RuXOW