[busydrive]

Wonderful thread. Thanks for starting this. I think this is probably one of the very first times I am seeing first hand algorithm processed information on Harrier. This shows not only people's views but also gives TATA a big chance to utilize the feedback effectively.

Do you think it's possible to present a view of what are the areas to be looked into from a manufacturer's point? Can we extend the algorithm to present a recommendation as well?
Great stuff nonetheless.

[ihrishi]

Originally Posted by ihrishi

Mods: Is it ok if I post subsequent improvements and seek feedback for the fine tuning of the model?

I refined the model further to reduce the noise - i.e., filtered out all those words with neutral sentiments. So all you can see is all those entities with Positive and Negative sentiments only. This is the final version of the entities sentiment analysis.

PS: For context - all of these words are in the context of the Tata Harrier thread since the very beginning till date. Hence you will see things like steering, sun roof etc both as positive and negative.

POSITIVE ENTITIES - by frequency:



NEGATIVE ENTITIES - by frequency:



Coming up next is the following:

• Post count by Name
• Most Thanked person
• Top commentators
• Average comment Sentiment by Name
• Sentiment of comment by the commentator.

Please let me know if anyone would like to see any other insights!

[ihrishi]

Originally Posted by busydrive

Can we extend the algorithm to present a recommendation as well?
Great stuff nonetheless.

Can you elaborate on what kind of recommendations? Will give me a better idea if it is possible or not.

My original vision was - is it possible to predict the sales of a vehicle based just on the comments in TeamBhp.
But what I realized is we don't have enough data to 'train' a model and hence its on the back burner for now.

Till then will go step by step to get more insights and work my way from there.

[landcruiser123]

Hi irishi,
This is excellent. I'm sure many manufacturers are using such techniques and extracting Team-BHP data. If they aren't, they should hire you.

I'm going to try the NLP API and perform a similar analysis on the XUV500. Was this NLP API free? Been a while since I put my ML and Python skills to a real test - let us try collaborating on GitHub in the future.

Cheers,
LC123

[nainan]

Originally Posted by ihrishi

Please let me know if anyone would like to see any other insights!

Can you combine price & pricing. I want to see how big a factor is it combined. Also some stats about both clouds to see what is the ratio of positive to negative sentiment

[PetrolheadRup]

Great work ihrishi I found the analysis very interesting.

Originally Posted by ihrishi

PS: For context - all of these words are in the context of the Tata Harrier thread since the very beginning till date. Hence you will see things like steering, sun roof etc both as positive and negative.

What about taking the difference and ratio of positive and negative sentiment for each entity? You can put those with +ve difference under positive category and same for negative.
This would prevent a single entity from being in both lists. Also currently terms like price and quality are having significant presence in both clouds so it can't be determined if majority of members are finding the car to be well priced or not, this can be avoided by the method suggested above.

[ihrishi]

Originally Posted by landcruiser123

Hi irishi,
This is excellent. I'm sure many manufacturers are using such techniques and extracting Team-BHP data. If they aren't, they should hire you.

I'm going to try the NLP API and perform a similar analysis on the XUV500. Was this NLP API free? Been a while since I put my ML and Python skills to a real test - let us try collaborating on GitHub in the future.

Thank you! I am sure they are already doing this in some shape and form. If not, well, hire me :-D
The way in which I wrote the code - it can be applied on top of any thread in TeamBhp with minimal changes. Can share the code if you are interested.
The NLP API isn't free technically, but Google does give you 300$ credit which is more than enough to play around with it.

Originally Posted by nainan

Can you combine price & pricing. I want to see how big a factor is it combined. Also some stats about both clouds to see what is the ratio of positive to negative sentiment

Yes - I can combine these two. Will add this to the backlog. I have already lemmatized, but obviously, it isn't very effective!

Originally Posted by PetrolheadRup

Great work ihrishi I found the analysis very interesting.
This would prevent a single entity from being in both lists. Also currently terms like price and quality are having significant presence in both clouds so it can't be determined if majority of members are finding the car to be well priced or not, this can be avoided by the method suggested above.

Well, what I feel is - the word cloud is reflecting exactly what people are talking about. So the same entity can have a positive or negative sentiment which is very person dependent. Also - things like NVH, Steering, sunroof - appear in both positive and negative - since Tata did take that feedback and fix it. And since the data is from the first page of the thread to the last page; it not only includes the brick bats for these but also the appreciation and hence they show up in both.

Also - from an ML perspective - any NLP sentiment analysis is going to be indicative. It cannot be perfect. It is always going to be an approximation of what the computer thinks we want to say!

[AirbusCapt]

The big problem I see in any data analysis from our forum keywords is: the herd mentality. Like it or not, lot of emotions expressed here may be, because the influencers said so. The influencers could be the society, the online/real world people one looks up to, the perceived image one builds up within oneself etc. Even the keywords used will be a reflection of such sentiments.

[Shubhendra]

Originally Posted by ihrishi

I will be looking at extracting more information from the text - including the sentiments of each of the above entities. It's slow work and I work on this over the weekends so kindly bear with me. Also let me know if you would like to see any more insights from the data! Will give it a go.

Good initiative and a great analysis.
One suggestion, if you can refine your list of words by removing few more non-relevant word, it will be a fantastic analysis.
I am sure you might have done sentiment analysis of a current topic on twitter through a package twitteR in R programming, its an awesome timepass.

[ACM]

Originally Posted by ihrishi

Well, what I feel is - the word cloud is reflecting exactly what people are talking about. So the same entity can have a positive or negative sentiment which is very person dependent. Also - things like NVH, Steering, sunroof - appear in both positive and negative - since Tata did take that feedback and fix it. And since the data is from the first page of the thread to the last page; it not only includes the brick bats for these but also the appreciation and hence they show up in both.

You could update this to CURRENT Sentiment, by considering only the period post the update. Much smaller sample but much more relevant maybe. Then Steering, NVH, Sunroof may just be significant in 1 section.

[handsofsteel]

Originally Posted by CEF_Beasts

Yup, only keeping the USB slotted in without any device attached it drained. I have observed this in my Creta as well.

I do not think that's the issue here. In my car, the USB has been plugged in 3 years back (serious). Never faced an issue with the battery.
The USB would be an open circuit if not connected to a device, how would it drain the battery?
In fact the security system etc would drain more as they are 'kept alive' on a trickle charge (notice intermittent red flashes coming from the dash on some parked cars?)

[AZ911]

The IT way to look at things.

It would be great to compare other cars in the same segment and see what the word cloud comes up with. Then we can do a cloud vs cloud comparison.

I'd love to see the cloud for the ever finicky XUV500.

[CEF_Beasts]

Originally Posted by handsofsteel

I do not think that's the issue here. In my car, the USB has been plugged in 3 years back (serious). Never faced an issue with the battery.
The USB would be an open circuit if not connected to a device, how would it drain the battery?
In fact the security system etc would drain more as they are 'kept alive' on a trickle charge (notice intermittent red flashes coming from the dash on some parked cars?)

Sir at least in my EcoSport when I had left the cable connected for 2-3 weeks it had drained and the car was only 6 months old that time. I have tried this in my Creta as well, after switching off the engine and then connecting the cable it still started charging my phone. Whereas if I follow the same thing in my Fortuner it doesn't work. IMO it varies car to car.

[Zinda]

I just loved this idea. Outcome is impressive too.
Are you using Python's built in library for Logistic Regression model (I believe that's the algorithm in play here)?

[John]

Originally Posted by CEF_Beasts

Sir at least in my EcoSport when I had left the cable connected for 2-3 weeks it had drained and the car was only 6 months old that time. I have tried this in my Creta as well, after switching off the engine and then connecting the cable it still started charging my phone. Whereas if I follow the same thing in my Fortuner it doesn't work. IMO it varies car to car.

Checked this out on my Grand i10. Phone does not charge with ignition turned off. On my Seltos, on the other hand even with ignition turned off, the phone charges. However I don’t see how a battery could discharge if no device is connected to the cable