Finding and applying threads of meaning in documents

In earlier research, Jane Morris and Graeme Hirst described a Roget's Thesaurus--based technique for finding lexical chains---`chains' of semantically related words within a text---and showed that these chains, or threads of meaning, were indicative of the structure of a text. David St-Onge and Graeme Hirst subsequently redesigned the technique for implementation with WordNet, and showed that the chains could also be used as an easily computable representation of context for the detection and correction of real-word spelling errors. Stephen Green used the technique to automatically create hypertext links within and between documents by looking at their patterns of lexical chains and where and how the chains converge. In interactive information retrieval, for documents without abstracts (such as newspaper and magazine articles), this creates a characterization of the document that is far better for the browsing user than simply a list of keywords or the introductory paragraph.

This work is continued in our project on semantic distance.

References:

• Green, Stephen J. ``Automated link generation: Can we do better than term repetition?'' Proceedings of the Seventh International World Wide Web Conference, April, 1998, Brisbane, Australia, 75--84.
• Hirst, Graeme and St-Onge, David. ``Lexical chains as representations of context for the detection and correction of malapropisms''. In: Christiane Fellbaum (editor), WordNet: An electronic lexical database, Cambridge, MA: The MIT Press, 1998, 305--332.
• Morris, Jane and Hirst, Graeme. ``Lexical cohesion, the thesaurus, and the structure of text.'' Computational linguistics, 17(1), March 1991, 21--48.

Also: Green thesis, St-Onge thesis.

Return to Research by Graeme Hirst and students