NLP Components

NLP components are used for processing text. CLAMP offers multiple NLP components that are the building blocks for performing any of the NLP tasks. The individual components are provided as pre-built modules that are to be used in building pipelines for automatic text processing, as well as training customized machine learning models. The following picture displays the CLAMP NLP components, as well as its associated tools, algorithms and resources. In this section, we will provide details about each NLP component including their function as well as ways to customize them using your own dictionary/model. In "Build a Pipeline" section, we have tutorials that touch on use cases wherein the components are utilized in various applications.

Sentence Detector

A sentence is defined as the longest whitespace trimmed character sequence between two punctuation marks. A Sentence Detector utilizes different methods to detect a sentence. As shown in picture below, CLAMP provides three different models to detect a sentence:

1. DF_CLAMP_Sentence_Detector
2. DF_CLAMP_Sentence_by_newline
3. DF_CLAMP_OpenNLP_sentence_detector

Each model is described in details in the following sections.

DF_CLAMP_Sentence_Detector

DF_CLAMP_Sentence_Detector is the default sentence detector in CLAMP. It is designed specifically for clinical notes and takes into account the distinctive characteristics observed in sentences found in clinical texts. To configure the DF_CLAMP_Sentence_Detector, please click on the config file. A pop-up window opens where you can modify two parameters: Medical Abbreviation, and Max Sentence Length.

Medical Abbreviation:

There are some medical abbreviations that have punctuation marks at their beginning (".NO2) while some of them have it at the end (spec.). Providing a list of such abbreviations would help the detector to identify sentences more accurately. By default, CLAMP has provided a comprehensive list of medical abbreviation which can be found in this file: defaultAbbrs.txt

1. To replace the abbreviation file:
   1. Double click on config.conf file to open it
   2. Click on the button with three dots to browse for your own file
   3. Click on the open button

      

2. To edit the current file:

1. Double click on the defaultAbbrs.txt file to open it
2. Add the terms that you want to include in the abbreviation file
3. Click on the Save button on the toolbar

Max Sentence Length

Checking the checkbox for "Break long sentences or not?" allows users to break long sentences into the number of words that they have specified in the input textbox. Please refer to the following picture for more information.

Tokenizer

A Tokenizer segments the text into a sequence of tokens. As shown in Figure 8.4, CLAMP provides three different models of tokenizer:

1. DF_CLAMP_Tokenizer
2. DF_ OpenNLP_Tokenizer
3. DF_Tokenize_by_spaces

Each model will be described in more details.

Pos Tagger

A Pos tagger allows users to assign parts of speech to each token. As shown in Figure 8.5, CLAMP currently provides only one pos tagger, DF_OpenNLP_pos_tagger, designed specifically for clinical text. This tagger is built from re-training the OpenNLP pos tagger on a dataset of clinical notes, namely, the MiPACQ corpus. (http://clear.colorado.edu/compsem/index.php?page=endendsystems&sub=mipacq) . Advanced users can use the config.conf file to change the default pos tagger modelmipacq_pos.bin.

Chunker

A chunker does a shallow parsing of a sentence and identifies the syntactic constituents such as noun phrases, verb phrases, and etc. As shown in Figure 8.6, CLAMP currently provides only one single chunker, DF_OpenNLP_chunker, which is a wrapper of the chunker in OpenNLP. Advanced users can use the config.conf file to change the default chunker model- en-chunker.bin.

Named Entity Recognizer

A named entity recognizer identifies named entities and their semantic types in text. Typically, named entities refer to clinical concepts in CLAMP. As shown in Figure below, CLAMP provides two different models for named entity recognition:

1. DF_CRF_based_named_entity_recognizer ,and
2. DF_Dictionary_lookup
3. DF_Regular_expression_NER

Each model will be described in more details.

Assertion Identifier

An Assertion identifier checks whether there is a negation related to a specific clinical concepts in the text. A negation means the absence or opposite of something positive. CLAMP Assertion Identifier provides a mechanism to examine the real-world implications of annotations in a given clinical text. The defaultNegexDict.txt file which contains common negation patterns is used by CLAMP to check for negation in a clinical text. You can either replace or edit this file by following the steps below. (See the picture below)

Ruta_ Rule_Engine

UIMA Ruta rules can be used to create or modify annotations as well as create features for annotations. Ruta rules in general can consist of a sequence of rule elements. A simple rule elements consist of four parts: A matching condition, an optional quantifier, an optional list of conditions and an optional list of actions. For more information please visit: https://uima.apache.org (See the picture below)

Section Identifier

The section header identifier component identifies the section headers in a clinical note based on a predefined dictionary and categorizes them into general categories (Figure below). E.g. the section header "ICD 10 code" will be assigned to the "icd_code" category.

You can replace or edit the default dictionary, section_map.txt, following the steps below:

UMLS Encoder

A UMLS Encoder matches the terms of clinical concepts to its corresponding CUIs in UMLS. For example, the term "breast cancer" will be encoded into the CUI of "C6006142" in UMLS. Currently CLAMP provides a default dictionary based on the UMLS encoder as shown in figure below

User Defined Components

DF_Drug_Attribute_Connector: This is a context free grammar parser which is extracted from Medex. It is used to connect medication to its possible attributes such as dose.
DF_Relation_connector_after_ruta: While connecting two named entities using Ruta is relatively easy, it can not be used to provide a name for that relationship. Advanced users can generate their own file and replace it with the system’s default file or edit the default file.

It is a type of word representation feature generated on the unlabeled data which is provided by the SemEval 2014 Challenge. Advanced users can eplace their own Brwon clustering file with the system’s default file.

For more information on how to create your own Brown Clustring file visit:
https://github.com/percyliang/brown-cluster

This extractor uses a dictionary consisting of terms and their semantic types from UMLS to extract potential features. Advanced users can replace or edit the default file following the steps below:
Note:The format of the content should be as the same as the default file: (phrase then tab then semantic type)

This module uses the words along with their part-of-speech (pos) tagging as NER features.

This function extracts the prefix and suffix of words that may be a representative of a specific type of named entities.

Similar to the brown clustering, it is a type of word representation feature generated on unlabeled data using a 3 rd party package. For more information visit: https://jcheminf.springeropen.com

This function extracts the section in which a candidate named entity presents.

This function distinguishes the pattern of a sentence by CLAMP built in rules.

Similar to the brown clustering and random indexing, it is a type of distributed word representation feature generated on the unlabeled data (MIMIC II) provided by the SemEval 2014 Challenge using a neural network.Advanced users can replace the default file with their own file.

This function extracts the type of a word; it identifies whether or not it begins with an english letter, number, and etc.

This function extracts the regular expression patterns of words that may indicate a specific type of named entity. Advanced users can create their own regular expressions or edit the default file

1. You need to create a project
2. You need to configure the pipeline
3. You need to import the files that you want to be analyzed
4. You need to process the imported files by running them through the pipeline.

1. Create a new project:
   1. Click on the plus (+) sign at the top left corner of the screen as shown in the Figure below.

      

   2. On the pop-up window (Figure below), enter a name for your project, for example: "Clinical_concept_recognition".

      

   3. Select NLP Pipeline as the project type.
   4. Click the Finish button.

      

Configure the pipeline

To configure a pipeline double click on the .pipeline file from the newly created pipeline project to open it in the middle window on the screen. Here you can drag and drop the NLP components from the Component panel. Since we want to recognize clinical concepts using NLP components, we drag the DF_CRF_based_name_entity_recognizer from the NLP_components to the pipeline.

Component dependency & Auto fix

As shown in the figure below, there is a red X sign in front of the newly added component, "CRF based named entity recognizer". This sign indicates that the named entity recognizer component is dependant on other NLP components that are missing from the current pipeline. In our example, the clinical notes first need to be processed by the sentence detector, tokenizer, section identifier, and POS tagger components before processing by the named entity recognizer. To fix this issue, simply click on the Auto fix button at the top of the panel. This automatically adds the required components to the pipeline. The sequence of the individual components from top to bottom reflect the order in which they will run to process your input data. After the required components are added (figure below), the red X sign changes to the green circle sign indicating the accuracy of the order of the components. Once you see the green sign for each of the displayed NLP components, click on the Save button at the top of the screen to save your changes.

Import input files

Once the pipeline is configured, you will need to import your desired files to the Input folder using the the following steps:

Now, you can double click on the Input folder to see the imported files

Similarly, you can also double click on each file to view its content (figure below).

1. Create a new project:
   1. Click on the plus (+) sign at the top left corner of the screen as shown figure below.

      

   2. On the pop-up window, enter a name for your project, e.g., Drug_name_annotation.

      

   3. Select Corpus Annotation as the project type.
   4. Click the Finish button.

A new project with the name that you have specified is created and placed in the Corpus panel.

Double click the project name to view its content. The created project contains two main folders:
Corpus: Contains the files that will be annotated
Models: Contains the machine learning models generated from the annotated files.In addition, the prediction results generated from the n-fold cross-validation process and gold standard annotations are included in this folder.

Import annotation files

After you have created a project, follow the steps defined below to import the files that you want to annotate: (Please note that you can import the files to either train or test folders.)

1. To import the files that you want to annotate:
   1. Right click on the train folder under the corpus folder in the CorpusView panel
   2. Select the import function from the context menu. A pop-up window will appear.

      

   3. On the pop-up window, select the import source. Here, you need to select "File System" which is already selected by default.
   4. Click on the Next button

      

   5. Click on the Browse button on the top of the window to choose the folder of your choice. The selected folder will be displayed on the left side of the window, and the content of the folder will be displayed on the right side. To import you desired files, check the checkboxes next to the files of your choice.

      You also have three options to choose from:
      Filter Types/ Select All/ Deselect All
      Filter Types: Allows you to define the type of files that will be imported. The only extensions that you will work with in CLAMP are txt, and .xmi. For example, you may only want to import files with the .txt extension Select All: Allows you to choose all displayed files Deselect All: Allow you to deselect the files that have already been selected

   6. Click on the Browse button next to the "Into folder" field to choose the folder that you want to import your files to. Here, we keep the default directory
   7. Click on the Finish button.

      

Now that the selected files have been imported to your desired folder, you can start annotating them. Double click on the files to open theim in the middle window and annotate them. Upon double clicking each file, you will notice that another file with the same name but a different extension (.xmi) has been added to your folder and displayed on the screen. This is the file type used by CLAMP for display and interaction purposes.

Define entity & relation types

Before starting annotation, you need to define the semantic types that you will use for this purpose. Semantic types in CLAMP refer to entities(e.g, ‘problem/treatment/test’) and the relations between them.

1. To define a new entity type:
   1. Double click on the typedef.xmi file under the models folder to open it. Using this file, you will be able to define a schema for entities and the relation types among them.
   2. Right click on the Entities node
   3. Go to "Add Child"
   4. Click on New Element

      

   5. In the pop up window, enter a name for the element
   6. Click the OK button

      

      The created element will be added to the Entity node (See the picture below)

      

2. To define a new relation type:
   1. Right click on the Relations node
   2. Go to "Add Child"
   3. Click on New Element

      

   4. In the pop up window, enter a name for the relation
   5. Click the OK button

      

      Then, you need to decide which entities are involved in this relation. There are two roles of arguments an entity can hold in a relation: From, and To. "From" refers to an independent entity while "To" indicates the dependent entity.

3. To select the entities that are involved in a relation:
   1. Right click on the newly created relation
   2. Go to "Add Attribute"
   3. Click on "New Attribute"

      

   4. In the pop up window, enter a name for the new attribute (you will use "from" for the independent entity, and "to" for the dependent entity)
   5. Enter the actual name of the entities for the Value field

      

   6. Click the OK button
   7. Click on Save at the top of the window

Start Annotation

Now that you have set your desired schema, you are ready to start annotating your corpus. First, open your desired .xmi file, then:
To assign entity: Place your mouse over a word/phrase to highlight it and assign an appropriate entity to the selected text.

Now that you have set your desired schema, you are ready to start annotating your corpus. First, open your desired .xmi file, then:
To assign relation: By dragging your mouse from an independent entity and dropping it to a dependent entity, the names of possible relations will occur. Choose the appropriate relation name by clicking on one of the displayed names.

Please remember that you can only assign a relation to the entities that have already been defined in that relation.
Once you have completed annotating the corpus, save your changes by clicking on the save button at the top of the screen.

Visualization of entity & relation

Although different colors are automatically assigned to the different items in the "Display Option", you are able to change them at any time.

1. To change the default colors for semantic types:
   1. Double click on the default color for the entity of your choice
   2. Pick a new color from the color picker window
   3. Press the OK button

      

Pre-Annotation of entity and relation

As annotating a corpus from scratch may be a time-consuming and costly process, CLAMP offers an advanced feature called "pre-annotation" function which facilitates this process. The "pre-annotation" function relies on the existing models in CLAMP and is highly customizable.

1. Using pre-annotation function
   1. Choose your desired pipeline to annotate your files in a corpus project. For more information on how to run a pipeline, go to "Run the pipeline" section.
   2. Select the .xmi files which contains the predicted named entities from the output folder.
   3. Copy them into the train/test folder of your desired corpus annotation project. To copy, right click on the selected files and choose copy.

      

      Double click on the files to view their contents in a new window. As you can see in the figure below, the identified named entities in the file are already highlighted. Now you can start your own annotation.

      

Building machine learning models
(NER model)

1. Select your desired train folder on the Corpus panel
2. Click on the "Train Model" button at the top of the window as shown in the figure below.

   

3. On the pop up window as shown in the figure below, enter a name for the model that you are building.

   

4. Click the checkbox for the features that you want to include in your model
5. In the Evaluation box, choose if you want to test the built model against a test dataset and/or if you want to do a n-fold cross-validation during the training process.

   If you choose to test the model against a test set, make sure that you have your desired annotated xmi files in the folder of your choice. You can browse for the folder by clicking on the three dot button next to the checkboxes. With the n-fold cross validation, you are not required to do so as the training data will be used to test the model performance.

6. Click on the Finish button to start building the model.

   Once the building process starts, you can check the progress in the Console window, as well as the progress bar at the bottom of the screen. You can also stop the building process at anytime by clicking the red stop button in the Progress window.
   Note:During the model building process, the training files can not be annotated. Clicking on the text of the training files pops up an alert window indicating that the user operation is waiting for a function to complete.

   

Check output models & logs

By default the built models, their associated logs, and the named entities predicted by each model (in the output sub-folder) are stored in the models folder. As shown in the figures below, the model built during n-fold cross validation and the model trained on the whole training set are also stored in this directory. The content of the log files includes the output information of the training process and the evaluation performance of each specific folder for cross validation.

Use your own model in pipeline

The steps below show how you should use your own model to recognize named entities:

1. Make sure that you have selected your desired project folder on the Corpus panel
2. Go to models> model_xxx> output, then, right click on the file labled model.jar and select copy (See the figure below)
3. Go to the pipeline panel and select the pipeline of your choice
4. Click on the small arrow next to it to expand it
5. Go to Components -> Named Entity Recognizer ->DF_CRF_based_named_entity_recognize

   

6. Paste the copied file into "CRF based name entity recognizer" folder by righ clicking on the folder and choosing past
7. Double click on the Config.conf file in the "CRF based named entity recognizer" folder to open it
8. Click on the three dots button to replace the default model for "CRF based named entity recognizer" with your own model.

   

9. Click on the Open button
10. Click the save button at the top of the page to save the changes.

    

Visualization for error analysis

You will be able to evaluate the performance of the NER model only if you have already checked the checkbox(es) for “Use test set” or/and “CV, fold” when creating the model. For more information on creating a new NER model, go to Building machine learning models section. Once the model is built, you can conduct an error analysis to compare the goldstandard annotations with the predicted ones (the annotations that are built based on the model that you have specified).
To perform error analysis: Double click on one of the .xmi files listed in the output folder of your choice on the corpus panel. This will open a new window where you can see the original text along with both gold-standard and predicted annotations.

Please note that all named entities in both gold-standard and predicted annotations are listed on the "Display Options" panel. You can choose which named entities to be highlighted in the text file and assign different colors to them as described in "Visualization of entity and relation types" section.