Page History

...

The data files for GMDN get updated monthly and should be loaded in a timely fashion.  To get the data files onto the prod mapping server the following process is used:

1. A .zip file of GMDN data is put on to the mapping.backup.ihtsdo Amazon S3 bucket

The "pullGmdn.csh" script on the prod server (in /home/ihtsdo/bin, and also in the repository as part of the prod config package) a cron job every 20 min to check for the availablility fo a GMDN package.  NOTE: this requires the cron user to be able to run "sudo aws s3 ls mapping.backup.ihtsdo"

Code Block
* 20 * * 0,1,2,3,4,5,6 csh /home/ihtsdo/config/bin/pullGmdn.csh > /home/ihtsdo/logs/pullGmdn.log 2>&1

2. Any files containing "gmdn" in the filename are downloaded
3. For each
   1. A file name with the pattern "gmdn.*(\d\d_\d\d).zip" is expected and a directory of the digits is created (e.g. 17_1) in the /home/ihtsdo/data/GMDN folder.
   2. If the filename does not follow this pattern, it gets unzipped into a directory matching the filename without .zip
   3. It is expected that this top-level directory contains the actual XML GMDN files

4. For example,

   Code Block
   % ls -lt /home/ihtsdo/data/GMDN/17_1 total 93700 -rw-rw-r-- 1 ihtsdo ihtsdo 58681302 Jan 3 12:02 termcollectiveterm17_1.xml -rw-rw-r-- 1 ihtsdo ihtsdo 34357440 Jan 3 11:57 term17_1.xml -rw-rw-r-- 1 ihtsdo ihtsdo 1889072 Jan 3 11:56 collectiveterm17_1.xml -rw-rw-r-- 1 ihtsdo ihtsdo 1003824 Jan 3 11:56 cttreenode17_1.xml

   As a separate (and currently manual process), the GMDN version should be loaded (the server must be taken down for this process). NOTE: if there are any problems, download the GMDN version (e.g. zip and copy to the "doc" folder and the file should be accessible, e.g. https://mapping.ihtsdotools.org/doc/17_1.zip) and try loading it into a local dev environment to track down the errors.  The loader assumes the structure of the .xml files is constant - if it changes that may cause problems.

   Code Block
   # This process takes about 15 min service tomcat7 stop cd ~/code/admin/loader mvn install -PGMDN -Dinput.dir=/home/ihtsdo/data/GMDN/17_1 \ -Dterminology=GMDN -Dversion=17_1 -Drun.config=/home/ihtsdo/config/config.properties

   Determine the older version(s) of GMDN

   Code Block
   mysql> select distinct destinationTerminologyVersion from map_projects where destinationTerminology='GMDN'; +-------------------------------+ | destinationTerminologyVersion | +-------------------------------+ | 16_10 | +-------------------------------+

   Remove the older version, then restart the server

   Code Block# This process takes under 5 min cd ~/code/admin/remover mvn install -PTerminology -Dterminology=GMDN -Dversion=16_10 -Drun.config=/home/ihtsdo/config/config.properties service tomcat7 startINSERT NEW PROCESS HERE
5. Navigate to the mapping tool (e.g. https://mapping.ihtsdotools.org).
   1. Log in as "mappingadmin"
   2. Change to the GMDN project
   3. Find the GMDN project in the "map projects" accordion
   4. Update the destination terminology version the latest version (e.g. 17_1)
   5. Save the map project.
       
6. Concepts from the updated GMDN version should now be available to mappers.

...