Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Info

This exercise is intended to teach you how to:

  • Install Elasticsearch and Snowstorm on a blank Debian / Ubuntu server
  • Load the SNOMED CT International Edition



Last Updated:  

On this page

Table of Contents
absoluteUrltrue

1. Accessing the server command line

Connect to the command line of the Ubuntu server, either by accessing the server machine physically or connecting remotely using SSH or similar.

The server should have around 16GB of memory and most of that should be available.

2. Install Java 11

Java 11 is required to run Snowstorm. Although this is an older version of Java it is a stable Long Term Service version. Snowstorm will switch to the newer Java 21 LTS version in the next couple of years.

Install Java 11 using the following command:

Code Block
languagebash
themeConfluence
sudo apt install -y openjdk-11-jre-headless

Check that Java 11 will be used:

Code Block
languagebash
themeConfluence
java -version

The first line of output should be something like:

openjdk version "11.0.18" 2023-01-17
3

If you are connecting to one of the AWS servers provided by SNOMED International please use the SSH private key file below:

View file
nametraining.pem
height250

Command to connect to the AWS server

chmod 700 training.pem
ssh -i training.pem ubuntu@IP-ADDRESS

Where IP-ADDRESS is the server IP address from the email you received. If the chmod command does not work on your machine try this windows solution to set the file permissions: https://superuser.com/a/1329702

2. Install Elasticsearch

The data store of Snowstorm version 8.x is Elasticsearch version 7.10.2, or a later 7.x version. Install Elasticsearch using the following commands:

Step

Command

Add Elastic.co package signing key

bash <(curl -
fsSL
s https://
artifacts
raw.
elastic
githubusercontent.
co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elastic.gpg

Add Elastic.co package repository

echo "deb [signed-by=/usr/share/keyrings/elastic.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" \
| sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

Update package manager

sudo apt update

Install Elasticsearch

sudo apt install -y elasticsearch=7.10.2

Set config options in temp file

echo -e "network.host: localhost\nxpack.security.enabled: false\n" > tmp.txt

Copy in remaining config

sudo cat /etc/elasticsearch/elasticsearch.yml >> tmp.txt

Replace config file without changing file permissions

cat tmp.txt | sudo tee /etc/elasticsearch/elasticsearch.yml

Set JVM options in temp file

echo -e "-Xms4g\n-Xmx4g\n" > tmp.txt

Strip existing memory settings from default options

sudo cat /etc/elasticsearch/jvm.options | grep -v '\-Xm' >> tmp.txt

Replace JVM options file without changing file permissions

cat tmp.txt | sudo tee /etc/elasticsearch/jvm.options

Update service manager

sudo systemctl daemon-reload

Enable automatic start in service manager

sudo systemctl enable elasticsearch.service

Start Elasticsearch

sudo systemctl start elasticsearch
Check that Elasticsearch is running using
com/IHTSDO/snowstorm-deploy/main/ubuntu_20.04_LTS/install_elasticsearch.sh)
Check that Elasticsearch is running using curl:
Code Block
languagebash
themeConfluence
curl localhost:9200

The response should be JSON, including the lines:

{
  "version": {
    "number": "7.10.2"
},
"tagline": "You Know, for Search" }

4. Install Snowstorm

Install Snowstorm version 8.1.0 using the following commands:installation script.

The script also installs Java 11, that is required to run Snowstorm. Although this is an older version of Java it is a stable Long Term Service version. Snowstorm will switch to the newer Java 21 LTS version in the next couple of years.

Add repoecho "deb [trusted=yes]maven3ihtsdotoolsorgrepositorydebianreleases bionic " | sudo tee /etc/apt/sources.list.d/maven3_ihtsdotools_org_repository_debian_releases.listsudo supervisorctl reload

Step

Command

Install

bash <(curl -s https://
raw.
githubusercontent.
com/
IHTSDO/
snowstorm-
deploy/
main

Update package manager

sudo apt-get update

Install Snowstorm

sudo apt-get install -y snowstorm=8.1.0

Update memory settings in supervisor config

sed -i 's/command.*/command = \/usr\/bin\/java -Xms4g -Xmx4g -Djava.security.egd=file:\/dev\/urandom -jar %(program_name)s.jar/g' /etc/supervisor/conf.d/snowstorm.conf

Reload supervisor with new config

/ubuntu_20.04_LTS/install_snowstorm.sh)

Tail Snowstorm log until startup complete (1 mins)

sudo supervisorctl tail -f snowstorm

If possible leave an SSH window open with this tail command running so you can easily access the log output. 

5. Loading SNOMED CT International Edition

For this German training session we will use the Austrian SNOMED CT Edition because some concepts have been translated to the German language.

The latest SNOMED CT Belgian Extension release Austrian Edition is from November 2022. This February 2023 and is dependant on the September December 2022 International Edition so  so we will load this first. 

  • Put Check that the International Edition release file "SnomedCT_InternationalRF2_PRODUCTION_20220930T120000Z20221231T120000Z.zip" into is in the /opt/snowstorm snomed-releases directory on the server.

  • Apply the snowstorm file permissions with the command: 

    Code Block
    themeConfluence
    sudo chown -R snowstorm:snowstorm /opt/snowstormsnomed-releases


The Snowstorm application has two APIs:

  1. The Snowstorm FHIR API - this is the HL7 standards based API that is recommended for use in healthcare systems. This can be used to access any loaded code system including SNOMED CT, ICD, LOINC and others.

  2. The Snowstorm Native API - this for the SNOMED CT Browser and Authoring Platform. It is the only way to load SNOMED CT editions and extensions.


We will use the Snowstorm Native API to run the imports. Access the native API swagger interface using the server ip-address and port 8080, for example - http://<ip-address>:8080/

First we need to start the import process by creating a new import job. Expand the "Import" section and then create a new import using the start-local-file-import endpoint with the following request body:

Code Block
languagexml
themeConfluence
{ "branchPath": "MAIN", "createCodeSystemVersion": true, "filePath": "/opt/snowstormsnomed-releases/SnomedCT_InternationalRF2_PRODUCTION_20220930T120000Z20221231T120000Z.zip", "type": "SNAPSHOT" }

... and click on 'Execute'.

The Response Headers will contain the "Location" of the newly created job, the format is http://<ip-address>:8080/imports/<import_id>. Copy this URL into a new tab to monitor the status of the job. The Snowstorm log also gives information about imports.

Warning

Make sure that you wait for the import to complete before starting the next step!

  • The import job status will say status "COMPLETED"

  • The log will say ..."Completed RF2 SNAPSHOT import on branch MAIN"...


There are 357359,473 active 658 active concepts in the September December 2022 International Edition (including more than 1 million descriptions, 1 million relationships and 4 million reference set members).

The import should take around between 40-60 minutes, depending mostly on disk speed.

(See Snowstorm documentation loading-snomed).

4. Check Loaded Content

As previously mentioned Snowstorm has two APIs: FHIR and Native. Let's use each of them to check that the content is loaded. Once the import has a status of "COMPLETED" we know that the content is there, this the following exercise is just a for this training exercisesession.

Snowstorm FHIR API - CodeSystem Resources

The FHIR API can list all loaded Code Systems including SNOMED CT code systems and others. When a new version of a SNOMED CT release is loaded it shows as a separate code system in FHIR.

  • Once the import is complete the FHIR CodeSystem listing will show the SNOMED CT code system - http://<ip-address>:8080/fhir/CodeSystem

  • It is also possible to check the number of concepts loaded by expanding the value set containing all active concepts in the International Edition. We use the module identifier "900000000000207008" to reference this - http://<ip-address>:8080/fhir/ValueSet/$expand?url=http://snomed.info/sct/900000000000207008?fhir_vs

Snowstorm Native API - Code System Registry

The Snowstorm Native API has a Code System Registry to manage SNOMED CT code systems. In the Native API code systems are listed as soon as they are created. Imported versions are listed separately.

  • Code systems that exist are listed here - http://<ip-address>:8080/codesystems

  • Once the import is complete the version will be listed here - http://<ip-address>:8080/codesystems/SNOMEDCT/versions

Check the registry throughout this exercise to see your changes reflected.

5. Loading the SNOMED CT

Belgian

Austrian Edition


  • Put Check that the Belgian Austrian SNOMED CT release package "SnomedCT_ManagedServiceBEManagedServiceAT_PRODUCTION_BE1000172AT1000234_20221115T120000Z20230215T120000Z.zip" into is in the /opt/snowstorm snomed-releases directory on the server.

  • Apply the snowstorm file permissions with the command: 

    Code Block
    themeConfluence
    sudo chown -R snowstorm:snowstorm /opt/snowstormsnomed-releases


When loading a second SNOMED CT Edition for the first time a new Code System must be created:


  • On the swagger interface in the ‘Code Systems’ section look for the ‘Create a code system’ endpoint. 

  • Use the following in the request to create the CodeSystem. Note the dependantVersionEffectiveTime allows you to create a CodeSystem dependent on a specific International Edition version.

Code Block
languagexml
themeConfluence
{ "name": "BelgianAustrian Edition", "shortName": "SNOMEDCT-BEAT", "countryCode": "beat", "branchPath": "MAIN/SNOMEDCT-BEAT", "dependantVersionEffectiveTime": 2022093020221231, "defaultLanguageCode": "en" }

click 'Try it now'.


Now the extension package can be imported. Like last time, we need to start the import process by creating a new import job. Look for the Import endpoints and then create a new import using the start-local-file-import endpoint with the following importRequest parameter

Code Block
languagexml
themeConfluence
{ "branchPath": "MAIN/SNOMEDCT-BEAT", "createCodeSystemVersion": true, "filePath": "/opt/snowstormsnomed-releases/SnomedCT_ManagedServiceBEManagedServiceAT_PRODUCTION_BE1000172AT1000234_20221115T120000Z20230215T120000Z.zip", "type": "SNAPSHOT" }

click 'Try it now'.

Copy the "location" from the Response Headers and open this in a new browser tab.

Refresh the import job in your browser to see the status change from WAITING_FOR_FILE to RUNNING and then COMPLETED. You can also see how this is progressing in the Snowstorm log.

This should take about 5 minutes.

Warning

Make sure that you wait for the import status to change to COMPLETED before going any further.


6. Check Loaded Content

Check that the Belgian Austrian Edition of SNOMED CT now appears in the FHIR API. The module identifier for the Belgian this edition is 11000172109 11000234105.