SNOMED International Mapping Tool

Space shortcuts

Child pages
  • Mapping Tool Release Process

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 31 Next »

Overview

Documents the release process details.  There is an admin page already that indicates how to run the mojo.  See Release Processing Tool for more information.

Assumptions

  • Source terminology is SNOMED CT
  • Release is in RF2 delta format
  • QA of overall release is done by an external system (though project-specific validation rules are processed here)
  • That editing was done in conjunction with a "drip feed" system that brought emerging content into the system as it was being developed, so it could be mapped.
  • That the mapping files are part of the source terminology release

Details

The mapping tool supports a release process that takes into account end-of-editing cycle activities, some quality assurance, production of data files, and upgrading terminology versions when the time comes.  The release processing is specifically tailored to the needs of IHTSDO and maintenance of the RF2 complex and extended map refsets that are part of the SNOMED CT distribution.

The release process requires the ability for mappers to resume editing before the release cycle fully completes.  Thus, this release strategy calls for completing the actual release processing in UAT and feeding back the final state of the map.   The overall steps for this process are:

  1. PROD: Mappers stop editing.
  2. PROD: Disable drip feed (and other cron jobs)
    1. crontab -e (on the prod system)
  3. PROD: Begin release (creates the "Release QA" report)
    1. Done via the "Begin" button in the "Release Handling" accordion of the "Project Details" page
    2. This produces a "Release QA" report which can be reviewed and fed into the QA workflow.
    3. Ensure all in-scope concepts are mapped and there are no validation errors.
    4. Double-check mapped, but out-of-scope concepts to ensure they should be out of scope.
    5. Mappers fix any blocker errors
  4. PROD: Obtain the alpha SNOMED release - /home/ihtsdo/data/SNOMEDCT/$version/...snapshot Refset/Terminology folders...
    1. The Snapshot Refset/Terminology directories should be downloaded to ~/data/SNOMEDCT/$version directory (replace any prior version data files)
    2. Remove the old SNOMED version from the system (using the Terminology Loader and Remover)
    3. Add new SNOMED version to the system (using the Terminology Loader and Remover)
  5. PROD: Run Begin release (creates the "Release QA report)
    1. (see #2 for details)
  6. PROD: Nightly backup following "ready for release" state
  7. UAT: Cloning PROD Database to UAT
  8. PROD: Run "Begin New Editing Cycle" 
    1. Done via the "Start New Editing Cycle" button in the "Release Handling" accordion of the "Project Details" page
    2. This merely sets the editing cycle start date in the map project
  9. PROD: Re-enable drip feed (and other cron jobs)
    1. crontab -e (on the prod system)
  10. UAT: For each: alpha, beta, member, final SNOMED Release
    1. UAT: Obtain and load the (alpha, beta, member, final) SNOMED release
      1. e.g. use ~/data/SNOMED/$version, replace any older version data files, or archive them)
    2. UAT: Begin release (creates the "Release QA" report) 
      1. (see #2)
    3. Perform release (into /home/ihtsdo/data/doc/release/$release/$destinationTerminology)
      1. Done via the "Process" button in the "Release Handling" accordion of the "Project Details" page
      2. This generates the data files in ~/data/doc/release/$version/$destinationTerminology
  11. PROD: After final UAT run on production SNOMED release, copy the ActiveSnapshot file back (into /home/ihtsdo/data/doc/release/$release/$destinationTerminology)
  12. PROD: Run "Preview Finish" → produces "Release Finalization QA" report
    1. Done via the "Preview Finish" button in the "Release Handling" accordion of the "Project Details" page
    2. This generates a "Release Finalization QA"  report that summarizes changes between the map records in the release on PROD and what was generated on UAT
    3. This process expects the active snapshot RF2 file in ~/data/doc/release/$version/$destinationTerminology
  13. PROD: mappers/admins agree on planned changes
  14. PROD: Run "Finish" → replaces the map refset entries, fixes map records to match final state from UAT and marks them as PUBLISHED, and removes out of scope records.
    1. Done via the "Finish" button in the "Release Handling" accordion of the "Project Details" page
    2. This does 3 things
      1. applies any changes made in UAT to the PROD copies of the map records (to avoid duplicate editing) 
        1. Any records that will be changed are reported in the "Release Finalization QA" report.
      2. removes out of scope concepts/map records
      3. Marks "ready for publication" records as "published"
    3. This process expects the active snapshot RF2 file in ~/data/doc/release/$version/$destinationTerminology
  15. PROD: Remove old reports

    • When starting a new editing cycle, it may be useful to remove reports more than a year old to keep the overall size of accumulated reports reasonable.

      service tomcat7 stop
# set date one year ago, e.g. 20141215
set refsetId=447562003
set date = 20141215
cd ~/code/admin/remover
mvn install -PReports -Drun.config=/home/ihtsdo/config/config.properties -Drefset.id=$refsetId -Dend.date=$date >&! mvn.log
service tomcat7 start

For each of the currently supported IHTSDO mapping projects, there is a project-specific page for enacting the specific steps needed for that project.

 

NOTE: paths shown above start ~/data/doc/...., however in principle everything is relative to the directory specified in "map.principle.source.document.dir" in config.properties (in practice, this is set to "/home/ihtsdo/data/doc" in the PROD deployment.  We already had a property pointing to a local file system directory so it was reused.  TODO: in the future, this property should probably be renamed.

 

Creating Custom Release Processing

If the process above is not needed or desired, alternative release processing can be built as a custom development task.  Things to consider are:

  • The release format
  • Whether the release is part of the source/target terminologies of the map
  • Whether the mapping versioning itself is coordinated with the source/target terminology versioning
  • Whether map records were built completely from scratch or were generated from terminology data
  • Whether a "delta" release is needed - e.g. a release that compares to the previous release and publishes only the changes.

These things are very project specific and are not the same for all projects.

Custom release processing can be implemented via an outside package, or in the mapping-custom project (see Custom Release Process).

  • n/a

 

  • No labels