Learning from Big Data: 40 Million Entities in Context

When someone mentions Mercury, are they talking about the planet, the god, the car, the element, Freddie, or one of some 89 other possibilities? This problem is called disambiguation (a word that is itself ambiguous), and while it’s necessary for communication, and humans are amazingly good at it (when was the last time you confused a fruit with a giant tech company?), computers need help.

To provide that help, we are releasing the Wikilinks Corpus: 40 million total disambiguated mentions within over 10 million web pages -- over 100 times bigger than the next largest corpus (about 100,000 documents, see the table below for mention and entity counts). The mentions are found by looking for links to Wikipedia pages where the anchor text of the link closely matches the title of the target Wikipedia page. If we think of each page on Wikipedia as an entity (an idea we’ve discussed before), then the anchor text can be thought of as a mention of the corresponding entity.

Dataset	Number of Mentions	Number of Entities
Bentivogli et al. (data) (2008)	43,704	709
Day et al. (2008)	less than 55,000	3,660
Artiles et al. (data) (2010)	57,357	300
Wikilinks Corpus	40,323,863	2,933,659

What might you do with this data? Well, we’ve already written one ACL paper on cross-document co-reference (and received lots of requests for the underlying data, which partly motivates this release). And really, we look forward to seeing what you are going to do with it! But here are a few ideas:

• Look into coreference -- when different mentions mention the same entity -- or entity resolution -- matching a mention to the underlying entity
• Work on the bigger problem of cross-document coreference, which is how to find out if different web pages are talking about the same person or other entity
• Learn things about entities by aggregating information across all the documents they’re mentioned in
• Type tagging tries to assign types (they could be broad, like person, location, or specific, like amusement park ride) to entities. To the extent that the Wikipedia pages contain the type information you’re interested in, it would be easy to construct a training set that annotates the Wikilinks entities with types from Wikipedia.
• Work on any of the above, or more, on subsets of the data. With existing datasets, it wasn’t possible to work on just musicians or chefs or train stations, because the sample sizes would be too small. But with 10 million Web pages, you can find a decent sampling of almost anything.

Gory Details

How do you actually get the data? It’s right here: Google’s Wikilinks Corpus. Tools and data with extra context can be found on our partners’ page: UMass Wiki-links. Understanding the corpus, however, is a little bit involved.

For copyright reasons, we cannot distribute actual annotated web pages. Instead, we’re providing an index of URLs, and the tools to create the dataset, or whichever slice of it you care about, yourself. Specifically, we’re providing:

• The URLs of all the pages that contain labeled mentions, which are links to English Wikipedia
• The anchor text of the link (the mention string), the Wikipedia link target, and the byte offset of the link for every page in the set
• The byte offset of the 10 least frequent words on the page, to act as a signature to ensure that the underlying text hasn’t changed -- think of this as a version, or fingerprint, of the page
• Software tools (on the UMass site) to: download the web pages; extract the mentions, with ways to recover if the byte offsets don’t match; select the text around the mentions as local context; and compute evaluation metrics over predicted entities.

The format looks like this:

URL http://1967mercurycougar.blogspot.com/2009_10_01_archive.html

MENTION Lincoln Continental Mark IV 40110 http://en.wikipedia.org/wiki/Lincoln_Continental_Mark_IV

MENTION 1975 MGB roadster 41481 http://en.wikipedia.org/wiki/MG_MGB

MENTION Buick Riviera 43316 http://en.wikipedia.org/wiki/Buick_Riviera

MENTION Oldsmobile Toronado 43397 http://en.wikipedia.org/wiki/Oldsmobile_Toronado

TOKEN seen 58190

TOKEN crush 63118

TOKEN owners 69290

TOKEN desk 59772

TOKEN relocate 70683

TOKEN promote 35016

TOKEN between 70846

TOKEN re 52821

TOKEN getting 68968

TOKEN felt 41508

We’d love to hear what you’re working on, and look forward to what you can do with 40 million mentions across over 10 million web pages!

Thanks to our collaborators at UMass Amherst: Sameer Singh and Andrew McCallum.