Finding Top Keywords with the Social Analytics API

 There are over 50 Entities and Service Operations available in the Social Analytics Lab API. Over the next few posts we will provide details on how the Entities can be used to accomplish some basic scenarios in the Social Analytics lab.

The first scenario we’ll look at identifies top keywords for one of the lab datasets. To accomplish this, you only need two Entities in our API (pictured below):

Keywords

The Keywords entity includes every keyword that has been defined or observed by our curation processes. After two weeks of activity on the Bill Gates Lab, we already have an inventory of over 10,000 keywords. New keywords observed in the Social Stream are marked as New. These new keywords are useful to consider when defining filters (coming in a future version) to curate relevant conversations.

With the following LINQ Statement in LINQPad, we can get a sample of the active keywords in the Bill Gates Lab:

  (
 from k in Keywords
 where k.ItemStatusId == 10 //Active
 select new
 {
 k.Name,
 }
 ).Take(10)

 

Name

Vaccinations

Stanford

Philanthropic

Germany

BillGates

Global Health

Bill Clinton

UN Foundation

GatesFoundation

March of Dimes

 

KeywordReferenceEntities

KeywordReferenceEntities are the record of what Keywords are referenced in Content Items and Message Threads. In our taxonomy, a Content Item is a post of any kind, ranging from gestures such as Facebook likes to a Tweet to a Blog Post. A Message Thread is a summary of an original post in a feed and all responses to that post in the same feed.

With the following LINQ Statement in LINQPad, we can get a sample of Keyword References:

  (
 from k in KeywordReferenceEntities
 select new
 {
 KeywordName = k.Keyword.Name,
 k.MessageThread.LastReplyContentItem.Title,
 k.MessageThread.LastReplyContentItem.HtmlUrl,
 }
 ).Take(10)
 

KeywordName	Title	HtmlUrl
Bill Gates	"Ô Bill Gates, vamos dar um jeito na internet aqui do Brasil, hein?" https://t.co/flCjtGrx	https://twitter.com/13_levi/statuses/126703589409304576
Bill Gates	Em palestra paga por um banco americano Lula se supera: “Sem um dedo, fiz mais que Bill Gates e Steve Jobs”.	https://twitter.com/Thais_Grimald/statuses/126703689510551552
Bill Gates	Rapaz......Isso não era pra hoje não viu....Até agora sem almoçar! Se pudesse, mandava Bill Gates para junto de Jobs!	https://twitter.com/mbatistasilva/statuses/126703767683993600
Bill Gates	RT @GuidoFawkes: Nobody loves Bill Gates, but if he has funded a Malaria vaccine, it makes up for everything and is kind of more signifi ...	https://twitter.com/TadasLabudis/statuses/128649702337032192
Bill Gates	RT @GuidoFawkes: Nobody loves Bill Gates, but if he has funded a Malaria vaccine, it makes up for everything and is kind of more signifi ...	https://twitter.com/TadasLabudis/statuses/128649702337032192
BillGates	BillGates Bill GatesFor those of us lucky enough to get to work with Steve, it’s been an insanely great honor. I will miss Steve immensely.	https://twitter.com/Carolina_S_O/statuses/126704102003572736
Bill Gates	BillGates Bill GatesFor those of us lucky enough to get to work with Steve, it’s been an insanely great honor. I will miss Steve immensely.	https://twitter.com/Carolina_S_O/statuses/126704102003572736
endmalaria	RT @PATHtweets: Blog: Read commentary on 2011 @gatesfoundation Malaria Summit from our Malaria Control Program team. https://t.co/DSdlUoO ...	https://twitter.com/EVERY_ONE_CAN/statuses/126704433638805504
GatesFoundation	RT @PATHtweets: Blog: Read commentary on 2011 @gatesfoundation Malaria Summit from our Malaria Control Program team. https://t.co/DSdlUoO ...	https://twitter.com/EVERY_ONE_CAN/statuses/126704433638805504
Bill & Melinda Gates Foundation Malaria Forum	Using information systems to track a killer parasite	https://macepadiary.wordpress.com/2011/10/18/using-information-systems-to-track-a-killer-parasite/

 

Top Keywords for the Last Week

Using these two Entities together, we can calculate the top 5 Keywords referenced in the last week for Bill Gates. Here’s the code:

  (
 from k in Keywords.Expand("KeywordReferences").AsEnumerable()
 where k.ItemStatusId == 10
 && k.KeywordReferences.Max(kr => kr.LastUpdatedOn) > DateTime.Now.AddDays(-7)
 orderby k.KeywordReferences.Count descending
 select new
 {
 k.Id,
 k.Name,
 References = k.KeywordReferences.Count
 }
 ).Take(5)

and the results:

Id	Name	References
9acbefab-156f-4039-98b9-0a56c2d359d6	Global Health	108
0db05e2f-233f-4155-93db-01a07c693017	philanthropic	102
d268a214-0033-4f89-b297-051f2fdf7029	Germany	91
f41b31fb-63ce-47bc-ab72-016a4e43803a	Stanford	68
a65d7774-59d7-42d5-a224-013b226cdcae	Vaccinations	35

 

Check out our blog later this week to find out more about what you can do with the entities in the Social Analytics Lab.

Also, please let us know what questions you have or other interesting discoveries you find in the labs.