What are the most challenging issues in Sentiment Analysis(opinion mining)?

Ramy Ghaly January 28, 2011

Hossein Said:

Opinion Mining/Sentiment Analysis is a somewhat recent subtask of Natural Language processing.Some compare it to text classification,some take a more deep stance towards it. What do you think about the most challenging issues in Sentiment Analysis(opinion mining)? Can you name a few?

 

Hightechrider Said:

The key challenges for sentiment analysis are:-

1) Named Entity Recognition - What is the person actually talking about, e.g. is 300 Spartans a group of Greeks or a movie?

2) Anaphora Resolution - the problem of resolving what a pronoun, or a noun phrase refers to. "We watched the movie and went to dinner; it was awful." What does "It" refer to?

3) Parsing - What is the subject and object of the sentence, which one does the verb and/or adjective actually refer to?

4) Sarcasm - If you don't know the author you have no idea whether 'bad' means bad or good.

5) Twitter - abbreviations, lack of capitals, poor spelling, poor punctuation, poor grammar, ...

 

ealdent Said:

I agree with Hightechrider that those are areas where Sentiment Analysis accuracy can see improvement. I would also add that sentiment analysis tends to be done on closed-domain text for the most part. Attempts to do it on open domain text usually winds up having very bad accuracy/F1 measure/what have you or else it is pseudo-open-domain because it only looks at certain grammatical constructions. So I would say topic-sensitive sentiment analysis that can identify context and make decisions based on that is an exciting area for research (and industry products).

I'd also expand his 5th point from Twitter to other social media sites (e.g. Facebook, Youtube), where short, ungrammatical utterances are commonplace.

 

Skarab Said:

I think the answer is the language complexity, mistakes in grammar, and spelling. There is vast of ways people expresses there opinions, e.g., sarcasms could be wrongly interpreted as extremely positive sentiment.

 

What do you think? Do you agree? Would you like to ask a question and get an answer? Try out: Q&A for professional and enthusiast programmers 

 

 

Social commerce to surge

NEW YORK: Social commerce sales will rise dramatically during the next five years, encouraging brands and retailers to enhance their presence on sites like Facebook, Booz & Co has argued.

In a new report, the consultancy stated marketers must shift the terms of engagement with consumers using Web 2.0 properties from "like" to "buy".

"The market for social commerce has been embryonic to date, but that will change over the next five years as companies race to establish stores," it said.

"Trendsetting companies are focused on products and services that benefit from the unique characteristics of social media, including the opportunity to get quick feedback from multiple friends and family members."

The study praised 1-800-Flowers, which boasts a fully-functioning Facebook store allowing customers to complete purchases without leaving the network's pages.

It has also implemented other innovative strategies, for example linking Facebook's calendar and "group gifting" features to its Mother's Day campaign.

"We are going to continue to invest in certain areas to help drive future growth," Bill Shea, 1-800-Flowers' chief financial officer, said in late 2010.

"Whether it be franchising efforts for both the consumer floral and our food group, investments in mobile and social commerce [or] floral supply chain in Celebrations.com, we are going to continue to invest."

Dell was cited as another pioneering early-adopter, having earned millions of dollars in revenue through Twitter.

The IT specialist is becoming increasingly active in the smartphone and tablet segments, which the organisation believes will transform the retail sector.

"It used to be 'We're going to tell you how you're going to experience our store,'" said Brian Slaughter, Dell's director, end-user solutions, large enterprises.

"Now the consumer is walking in and saying: 'No, I'm going to tell you how I'm going to use your store to give me more information.' The tools they have at their disposal are very cool."

Similarly, Quidsi, owned by Amazon, recently set up Facebook outlets for its Soap.com and Diapers.com platforms, although the ability to make purchases is limited to members of these two portals.

"No one has yet cracked the nut on Facebook e-commerce," said Josh Himwich, Quidsi's vp, ecommerce.

Overall, Booz estimated sales of physical goods via social channels would hit $5bn (€3.7bn; £3.1bn) globally in 2011, with the US contributing 20% of this total.

Revenues were pegged to reach $9bn by the close of 2012, incorporating $3bn generated by American internet users.

Such figures should achieve $14bn and $5bn respectively for 2013, while US customers deliver nearly half of the $20bn returns yielded in 2014.

By 2015, the worldwide expenditure attributable to this medium is anticipated to come in at $30bn, housing $14bn from the world's biggest economy.

A previous Booz survey of netizens dedicating one hour or more a month to social networks, and who bought at least one product online in the last year, found 20% proved willing to pay for items through these sites.

Elsewhere, 10% suggested this spending would be incremental to their current outlay, but 71% added "liking" a brand on Facebook did not improve the probability of buying it.

The consultancy predicted that social media will have the greatest impact on consideration, conversion, loyalty and customer service.

Facebook's chief executive Mark Zuckerburg certainly supported such as optimistic reading when rolling out the Places geo-location system last year.

"If I had to guess, social commerce is the next area to really blow up," he said in August. 

Data sourced from Booz & Co, Seeking Alpha, Daily Finance; additional content by Warc staff, 21 January 2011

Via: WARC

2011 Digital Trends #6 - Data and artificial intelligence "algorithm"

January 2011

This is a series of posts on my take on the 11 digital trends for 2011. This sixth trend is on data and AI. The trends are not in order of importance.

In the January 2011 edition of Wired Magazine, the feature The AI Revolution Is On says “By using probability-based algorithms to derive meaning from huge amounts of data, researchers discovered that they didn’t need to teach a computer how to accomplish a task; they could just show it what people did and let the machine figure out how to emulate that behavior under similar circumstances.”

On the web, artificial intelligence is already part of many websites we visit daily: Google’s search, Facebook recommendation of friends and the top news feed, Amazon’s recommendation engine, Pandora’s and Last.fm’s recommendation engine. These AI (some call it algorithm, which I think isn’t correct) derive their “intelligence” from massive amount of data generated by the huge user base, rather than mimicking human intelligence.

AI is not analytics like Google Analytics or Facebook Insights. The goal of analytics is to offer you data for analysis and conclusions, while AI is to offer your users something actionable or tangible like a recommendation.

How is this relevant to marketing? Recommendation and personalization engine is the first in mind. However, it is clearly very costly and difficult, and you may not have a huge data set to begin with. I think the way to go is to rely on third party services, via API as discussed on Trend #5.

The Facebook Recommendations plugin is one of the first to offer such service. It uses data from user interaction with the target website and the user’s friends’ interaction. I believe it takes into account the user Social Graph as well.

Google has a whole library of APIs for its various services, and a few have AI features. Use at your own risk though. Surprisingly I find Google APIs quite unstable because there’s so many and they seem to be created by different teams (they are, actually). If you are using Google AdSense, the AdSence API allows you to generate codes that display contextual ads. The YouTube Data API has some form of recommendation API as well that you can query for “related videos”, and the Google Language API deals with translation.

My 2011 Watch List for data and AI:

• Facebook Social Graph – I hope that the next step is to provide a set of APIs that give third parties AI capabilities such as recommendations based on the data from Facebook itself and the third party sites.
• Foursquare - I am interested to see how Foursquare’s location data can be used for recommendations and personalization beyond the “Trending Now” feature. Or does it need to be integrated with Facebook Social Graph?

Via: http://blog.campaignasia.com/yonghwee/2011-digital-trends-6-data-and-artificial-intelligence-algorithm/