| Subject | Owner | Notes & Actions |
---|
1 | Welcome! | All | Thanks to our members for all of their help. Welcome to our observers! INTRODUCTIONS... We've got several topics that we've resolved and closed down As always, we won't waste time going through them again in detail, but if you'd like to read through them they're listed below... I'll also run through them very quickly from a high level, and if you have any further questions/news on any of the discussions please let me know now and we can decide whether or not to re-open them... |
2 | Conclusion of previous Discussions topics |
3 | Annotations - Release Dates | ALL | The release date depends on two factors:
Firstly, the authoring platform being ready for annotations (which will be November 2023)Secondly, content authored for annotations. Language tags only involve about 8 concepts under 900000000000506000 |Language type reference set (foundation metadata concept)|. The attribution could be done by technical batch changes.
Because of the change to release date (to the 1st of the month), it is therefore unlikely to be possible to prepare everything in time for the December 2023 release. We are therefore currently aiming for the January 2024 releases - as this will also provide the largest audience for these major new changes (as opposed to Feb/March which most users still do no consume).
Is everyone ready for these changes, and comfortable with the proposed release date of January 2024 onwards?If so, we will publish the EMPTY files in the December 2023 release as a trail run, and then populate them with the first content in the January 2024 International Edition.This will likely be closely followed by more content being introduced in the various Extensions over the next few months:Anyone in the room intending to include any data in them anytime soon?If so, have we covered off all necessary bases?Any other considerations?NO - Everyone happy with the proposed timelines
|
4 | Active Discussions for October 2023
|
5 | Welcome and thank you! |
| Welcome to new members! |
6 | Member Nominations |
| Please let us know if anyone is interested (and who has the requisite domain knowledge and expertise) in applying for a seat on the TRAG - thanks! We're looking for new members to take the place of some outgoing chairs - if you have any Nominations please let me know either this week or by email after. Thanks! |
7 | Naming Conventions for Refsets | All | Proposal was sent to TRAG Members at the start of May 2024 for urgent feedback. This is because the intention would be to trial these changes in the NZ (and possibly also EE + SE) Extensions in Q4 2024, and we would therefore need to provide several months' notice to end users.... Full discussion required in October 2024 meetings: |
8 | Exception listing features | All | In our next meeting we should walk through how we would expect our end to end Exception listing functionality to work, from all points of view (ie) - International Edition visibility for everyone
- Managed Service process and visibility (also across extensions, not just within each one)
- Derivative products
- How we want to see the end result of Known Issues?
- Just discoverable in Release Notes when you need to find them?
- Or pushed out in pro-active comms?
- Or just searchable in Confluence somewhere if and when needed?
- Any other concerns?
|
9 | Validation feedback | All | Now that we've been running Frequent Delivery for some time, both in International and some MS products, we would like to ask for feedback on any and all Validation gaps that need plugging. - So anything major that anyone has found in any Production release packages?
- Anything minor that anyone is repeatedly finding in any Production release packages?
|
10 | Proposal to remove the Release Notes PDF file from Releases | All | Start discussions on the utility of the Release Notes PDF file that we currently publish as one of the deliverables with each of our Release packages. |
11 | New style Release Notes review | All | EITHER: - Heads up that the new style will be coming down soon, or
- DEMO of the new style (from 1st October Parallel run) so that you can provide initial feedback to us for use in future improvements
|
12 | Association Relationship file | All | In the MAG the requirement for the new Association Relationships (previously called "Additional Relationships") has been identified. Although they will take the same form as the Inferred Relationships, and therefore be stored in the same format as the Inferred Rel's in the RF2 package, there was still a discussion on whether or not they should be held in the Inferred Rel's file or a completely separate file. The arguments for retaining them in the Inferred File are: - The format is the same, and therefore it could be confusing to users for them to be in a separate file
- Slightly smaller overhead to maintain and upload
The arguments for putting them into a separate file (with the same format as the Inferred file) are: - The precedent for implementing new types of content is to hold them in a separate file, in order to:
- ...allow for backward compatibility (to allow users to continue to use the Inferred file standalone)
- ...make it clear for users that this is a new type of content (even though it happens to be in the same format)
- ...prevent confusion for users who might not see them as different from the existing Inferred Relationships
- The source of these Association Relationships (which are almost stated relationships) are different from the Inferred Relationships (which come from the classifier), and there is a potential for confusing systems by adding in a new source to the same file.
- The maintenance of the ID's could therefore potentially be impacted by having them in the same file.
The arguments for the latter, therefore, appear to be more compelling, and this is the MAG's recommendation. However, if anyone has any strong opinions either way then please raise them now? |
13 | RF2 Refset file naming convention | All | We have identified a requirement for Refset filenames to be a) more machine readable, and b) more maintainable c) more consistent (as we've already seen many filenames change over the years, due to mistakes in the descriptions, or valid changes to the purpose of the refsets)- especially for Extensions who have hundreds or more refsets. The human readable description in the file name is all very well, but: - It's not actually readable in countries with non english languages, and
- It's often concatenated due to machine limits of filenames (especially in Windows)
Therefore we have (**** CONFIRM ALREADY IMPLEMENTED BY THE TIME THE NEXT TRAG MEETING HAS OCCURRED ****) trialled a refined naming convention for ne of the MS extensions: - FROM:
- der2_Refset_AjccHaematologicMalignancyPathologicalMCategorySimpleRefsetSnapshot_NZ1000210_20231001.txt
- TO:
- der2_Refset_253731000210103SimpleRefsetSnapshot_NZ1000210_20231001.txt
This stays within the RF2 spec, which states that this description is "An optional short camel case summary of the usage of the file. The value of this sub-element may be determined on a case-by-case basis but, in conjunction with the ContentType element, should be adequate to identify the content and purpose of the file." Therefore: - The spec states that this should be "adequate to identify the content and purpose of the file", and the RefsetID is adequate because you can easily plug it into the browser to find the descriptions, etc
- The spec states that the description itself is actually completely optional and therefore we could actually remove it completely!
It also proves useful in both directions, as if you want to: a) search for the refset in the package then searching by refsetID in the name is much faster, or b) allow systems to create/maintain the names of the files is also much faster by ID, as the descriptions are all manually curated c) if you want continuity in the refset file names, using the ID is far more consistent, as the descriptions can change (as we have already proven!). Therefore using the ID's allows complete continuity, whereas using the descriptions can mean the refset filenames change from one Release to another! Thoughts? - Long term....
- ...do we consider moving the INT derivatives to the same approach??
- ...do we even then potentially talk about other files (maps, lang refsets, etc) moving to the same convention, as otherwise they would then be slightly conflicting with the others in the same package? (though changing core filenames might be a leap too far for end users - perhaps we draw the line in the sand between core files and optional files?)
- .
- ALTERNATIVE OPTIONS TO MOVING STRAIGHT TO MANDATING THE USE OF ID'S IN FILENAMES
- We could potentially move to a hybrid state, whereby we allow EITHER ID or name to be used in the fielnames.
- This would then continue to allow people to either host one or multiple refsets in the same file
- This would also give users some time to transfer across
- BUT it would potentially create inconsistency and confusion for end users...
- We could migrate to using ONLY refsetID's in filenames (eg)
- RefsetID would be used when only ONE refset is included in the file
- RefsetID OF PARENT (eg. "Simple Refset Type Concept") would be used where the file hosts multiple refsets
- DUE TO THE SHEER VOLUME OF REFSETS IN THE NZ EXTENSION, WE'VE TRIALLED THE TRANSITION IN THE OCTOBER 2024 NZ RELEASE :
- ....
- Any feedback????
- ....
- IF NOT, LET'S JUST PUSH FORWARD AND MAKE THE CHANGES ACROSS ALL PRODUCTS, IN ORDER TO ENSURE CONSISTENCY AND INTEROPERABILITY!!
|
14 | Association Relationship file naming conventions | All | Once the above decision on the implementation of the Association Relationship records has been made, we then need to consider the naming and packaging conventions for any new files that are required. The proposal would be to follow previous conventions, and therefore be something like this:
- sct2_AssociationRelationship_Snapshot_INT_[date].txt
- sct2_AssociationRelationshipConcreteValues_Snapshot_INT_[date].txt
- However, these do make for long names, so any alternatives are welcome!
- Also, are there any objections to hosting them in the standard Terminology folder (with the package structure)?
- Finally, just to check that we want to standardise these files (even if they remain empty) across ALL SNOMED International products, rather than just retaining them in the International Edition? The assumption is that we would want to follow similar precedents (for OWL, Annotations files, etc) and replicate them across all products, but we wanted to make sure there were no contradictions to this before going ahead?
-
- This has been requested, but not yet been prioritised in the Development team, so currently on hold...
|
15 | Redesign of the Map Reference Set formats | All | We have now successfully migrated MedDRA over to the new Map formats: We now, therefore, need to start work on standardising this new format across our Product portfolio: - International Edition:
- File naming conventions for Simple + Complex maps may need to change (eg)
- SIMPLE MAPS:
- der2_sRefset_SimpleMapSnapshot_INT_20240701.txt
- NOW WOULD NEED TO BECOME 2x FILES:
- der2_sRefset_SimpleMapToSNOMEDSnapshot_INT_20240701.txt +
- der2_sRefset_SNOMEDtoSimpleMapSnapshot_INT_20240701.txt
- Map content would then need to be split into the relevant files (eg)
- Content in module 446608001 |SNOMED CT to ICD-O simple map reference set| would be held in:
- der2_sRefset_SNOMEDtoSimpleMapSnapshot_INT_20240701.txt
- Content in module 900000000000497000 |CTV3 to SNOMED CT simple map reference set| would be held in:
- der2_sRefset_SimpleMapToSNOMEDSnapshot_INT_20240701.txt
- Would the refsetID's also need to change??
- (eg) with MedDRA we had to create a new refsetID for MedDRA to SNOMED (1193497006) as the old refsetID (816209002) was not compatible with the new format
- IF SO, the refsetDescriptor records will also need to be changed in order to describe the new format.
- EXTENDED MAPS:
- DO WE NEED TO CHANGE THE RELEASE FILE SPEC IN A SIMILAR WAY THAT WE DID TO THE SIMPLE MAPS (in order to add in the directionality to the format - see above)??
- OR DO WE SEE A DIMINISHING RETURN ON EXTENDED MAPS LIKE ICD-10 THAT RENDERS THIS REDUNDANT?
- .
- Internally, SNOMED International can't see many (if any) strong use cases for reverse directionality Extended Maps, as our intention is usually to map from SNOMED to external classifications when it comes to Complex maps.
- However if you can foresee use cases then please let us know now!
- Even if the use case is a local one (because you would like to develop your own complex/extended maps in the other direction), we can still update the overall spec in order to allow support for your local use cases in the global SNOMED standard...
- .
- Derivatives:
- ORPHANET - Need to align this and plan transition for next year
- GMDN - already aligned? (is the refset under the correct new refsetID?)
- EDQM - already aligned? (is the refset under the correct new refsetID?)
- .
- FUTURE:
QUESTIONS: - How much notice should we be providing?
- Comms scope?
|
16 | Derivative product Release package formats | All | The SI Standards for Derivative packaging formats (and therefore the precedents set for all existing Releases such as MedDRA, GMDN, etc), are that we create packages that are inherently dependent on the relevant International Edition package (as per the MDRS). These Derivatives are solely single refsets/maps, and so don’t mean anything to the end users without the supporting terms and other components from the International content. This is why we have always created the metadata components (refset/module concepts, descriptions, relationships, etc) in the International content, and then made the Derivatives dependent on the relevant International Edition. The main benefit so far identified has been to avoid the situation experienced in the EDQM Alpha release: - the metadata for the EDQM product was published in the December 2022 International Edition,
- and so the EDQM map content was upgraded in line with the December 2022 International content.
- the maps themselves however were not updated, and were still based on July 2022,
- but the release itself was dependent on the December 2022 International Edition.
- this created a slight contradiction in terms of the map release which appeared to be based on December 2022, but actually the metadata itself was the only thing baselined on this release. The content was only brought up to date with the July 2022 release.
- Having said all this, the flip side is that if we just push the collaborators to request the metadata to be added to the International Edition at an earlier time, then this situation can also be avoided!
HOWEVER, is it possible that the issue below in the RT2 release process could also be mitigated by moving the metadata into the Derivative packages themselves? If we did this, could we potentially then retain the Derivative content in the Refsets' child branches and release from there? (eg) - For GPFP we would maintain the content via the RT2 FE which would read/write to MAIN/Refsets/GPFP
- Given that we would then also include the Module + other metadata in the same branch (instead of in MAIN),
- ...Could we then version say into MAIN/Refsets/GPFP/2023-04-30, and Release straight out of that branch??
Changing the way we do things currently would take some work, and so would require a strong business case in order to a) Change our standards b) Incur the cost of making the technical + release changes Thoughts on the original decision to package them in this way? - Everyone comfortable that this was the , but not anymore.
Thoughts on the benefits/risks of refining this standard? - Consensus is that moving the Metadata components into the Derivative packages brings benefits to both the maintainers (SI + NRC's) + also to the end users, as it's easier than having to pull the metadata down from the dependent International Edition.
- It doesn't have a huge impact however, as the end users still need to download and consume the relevant International Edition when using the derivatives.
- The big benefit however, comes from combining this change with the new way of managing refsets in RT2 (see below for full details). The inclusion of this metadata in the Derivative branches in Snowstorm (instead of in the INT branch)allows us to move the Derivatives into their own codesystems, and thereby allow us to retain the different dependencies of the Derivative products on older version of the INT Edition than it would do otherwise in the new world of RT2
- (as things currently stand the Derivatives would have to be rebased against the LATEST INT Edition content whenever we need to promote them up to MAIN to publish them. This would force us to bring the Derivatives up to date with the very latest INT Edition content just before publishing the Production Derivative releases, which would be April/October. All users have confirmed this would be a huge problem for them as they're not yet ready to take multiple monthly releases of the INT Edition, and so desperately want us to retain the dependencies on the Jan/July INT Releases)
- Therefore we can continue to Publish the Derivatives in April/October (which is the earliest possible date for most of them because of the delays in collaborating with the external entities), based on the Jan/July INT Edition releases, exactly as the users want.
-
- If we agree on moving the module concepts to the Derivative package(s), do we also need to remove them from the International Edition, in order to avoid duplicates when users implement them in conjunction with the International content?
-
- EVERYONE IN AGREEMENT -
-
- SO ON RELEASE OF EACH NEW DERIVATIVE IN 2024 we need to:
- a) DEMOTE the various Derivative metadata components down from the INT Edition in the July 2023 Release
- (this would simply involve inactivating them all in the International Edition)
b) PROMOTE them in the various Derivative packages in the Sept/October 2023 Derivative releases. (this would simply involve activating them in the relevant Derivative packages)
-
- UPON FURTHER CONSIDERATION, WE REALISED THAT INACTIVATING THE INTERNATIONAL CONTENT IS UNNECESSARY, AND POTENTIALLY CONFUSING FOR END USERS. This also removes the challenges involved with the timing of the demotions, which was going to be problematic either way - either creating active content in Derivatives first, or inactivating in INT first (which causes problems in the Browser)
- THEREFORE THE NEW PLAN IS TO, ON RELEASE OF EACH NEW DERIVATIVE IN 2024:
- a) LEAVE the various Derivative metadata components Active in the International Edition
- b) ADD the Active Derivative metadata components into the various Derivative packages in the 2024 Derivative releases
- c) This results in covering off all use cases:
- i) Users who just want to use the Derivative packages STANDALONE, have the relevant necessary Metadata in the Derivative packages
- ii) Users who just want to query the International Edition can still see the Derivative metadata
- iii) Users who want to continue combining the International + Derivative content before use, the Derivative Metadata records should naturally take precedence over the International records, due to the change of module + later effectiveTimes.
- EVERYONE happy with the new plan - only possible risk is that one set of the metadata (in either INT or the Derivative package) could potentially be updated without updating the other set and keeping them aligned. This was agreed to be a very small risk however, as there is no reason to ever update this metadata without inactivating and recreating descriptions, etc
-
- Andrew Atkinson TO IMPLEMENT ASAP AT THE SAME TIME AS MIGRATION TO RT2
- TRANSITION BEGUN - PLEASE CAN EVERYONE REVIEW THE FIRST FEW PACKAGES CONTAINING THE NEW METADATA, IN ORDER TO ENSURE THAT IT'S IMPLEMENTED AS EXPECTED?
- GMDN - January 2024
- NCPT - January 2024
- MEDDRA - April 2024 (to be published at end of April)
- HPO - May 2024 (to be published at end of May)
- EDQM - May 2024 (to be published at end of May)
- ICNP - May 2024 (to be published at end of May)
- GP/FP - May 2024 (to be published at end of May)
-
- ****** WAS NOT POSSIBLE TO IMPLEMENT FOR THESE PRODUCTS AS THEY'RE STILL IN THE OLD FREESET FORMAT:
- IHE freeset - Jan 2024
- ERA Freeset - Jan 2024
-
- SPANISH EDITION - Already contains
-
- ANY FEEDBACK ALREADY FROM ANYONE ON IMPLEMENTATION OF THESE DERIVATIVES?????
-
-
- DEMO EXACT CHANGES MADE IN GMDN IN MARCH:
- Records taken from International Edition:
- Concept
- Description (x2)
- OwlExpression
- Records NOT taken from Internaitonal Edition:
- Inferred Relationship
- Simple Map (as this is just CTV3 - no need to take that???)
- ****** SHOW EVERYONE ACTUAL RECORDS IN FILES, ENSURE THEY'RE HAPPY WITH THEM (inc. Inferred output in GMDN after classifying - should we actually take the INT Inferred record into the GMDN package?) *******
- +++++++ NCPT IN APRIL:
- Records NOT TAKEN from International Edition AS THIS IS A FIRST TIME RELEASE, AND THEREFORE THESE ARE ALL BRAND NEW COMPONENTS:
- Concept
- Description (x2)
- OwlExpression
- Inferred Relationship
- Language Refset
- +++++++ MEDDRA IN APRIL:
- Records taken from International Edition:
- Concept
- Description (x2)
- OwlExpression
- Records NOT taken from Internaitonal Edition:
-
- QUESTIONS:
- Any issues with the new approach?? (eg) VERIFY THAT THE FOLLOWING IS EXPECTED:
- 1. Reference of moduleID concept to itself (eg) MedDRA module concept (similar to how the Model Component Concept module is defined in the INT Edition) is part of it's own module:
- id effectiveTime active moduleId definitionStatusId
816211006 20240101 1 816211006 900000000000074008
- 2. Change of effectiveTime to 20240101 (or later) for inclusion in the Derivative package (standard operating procedure for Extensions (which these Derivatives are now effectively treated as), as although content is the same, module is changing)
- 3. Change of UUID's for components such as OWL records (eg) MedDRA OWL record changes from:
- id effectiveTime active moduleId refsetId referencedComponentId owlExpression
- 864e67df-6999-4aa8-819c-c4807b54ce6c 20200131 1 900000000000012004 733073007 816211006 SubClassOf(:816211006 :900000000000445007)
- ...to
- id effectiveTime active moduleId refsetId referencedComponentId owlExpression
c0d051d1-c830-4d0c-add8-772c8b4d42a8 20240101 1 816211006 733073007 816211006 SubClassOf(:816211006 :900000000000445007)
-
- NO - UUID'S MUST BE RETAINED IN ORDER TO ENSURE CONTINUITY OF HISTORY!!!!
- Are we happy with the approach for the NEW Derivative products (eg) NCPT ??
- The current implementation is to ONLY add the metadata into the Derivative packages, and NOT add them into the International Edition
- Whilst this is potentially slightly inconsistent, it also provides less confusion for users
-
|
17 | DescriptionType spec improvements | ALL | - Back in October 2023, the MAG agreed that increasing the DescriptionType spec to allow longer terms in Descriptions would be a good idea - see item 4 here: 2023-10-24/25 Full MAG Atlanta Hybrid Meeting
- FYI - In the meantime, NZ require a new limit for a handful of terms, and so we extended their DescriptionType in the April 2024 NZ release, in order to allow for slightly longer terms locally until the International spec has been updated.
-
- Because the spec already allows us to increase the length of the field, the concern is predominantly for impact to end implementers.
- It’s mostly, therefore, a question of warning implementers that the status quo is going to change
- We have therefore started a full community consultation in 2024, in order to garner feedback from anyone who may be impacted by the changes.
- We will also ask if anyone has any issues with the length that we change it to, as we can foresee terms breaching 512 characters (with vaccine product with 10 ingredients for example) - so 1024 would be the next obvious target. However, at that point it's not clear why we don’t just jump to 4096, given that we already have that configured for textDefinitions… otherwise we could end up having to extend it again within the next few years.
- The potential problems are for end implementations who have local hard coding, or worse technical restrictions on imports, etc - but it feels like that’s going to be the same problem at 1024 as at 4096?
- Therefore it would appear that the best plan would be to increase it from 255 to 4096?
- Can anyone foresee any implementation issues? (other than providing a reasonable lead time to allow implementers to potentially change hard coded limits, etc?)
- At the moment the only implementation issues we're seeing are that several countries are finding terms that contravene the 255 spec (both in the INT Edition + the MS terms) and so only positive reinforcement for increasing the limits (eg) https://ihtsdo.freshdesk.com/helpdesk/tickets/49593
- But could we foresee, perhaps, any issues with implementers who might have
- a) hardcoded the 255 limit and could take a long time to get it changed (especially given how long the limit has remained the same), or
- b) still be running systems that can't actually cope with >255 characters even if the implementers want to increase the limit?
- The reason we ask this is that if terms are concatenated (especially Drugs, etc) by out-dated systems, then this could cause Clinical Risks, which we want to avoid at all costs.
- So perhaps we need a lengthy Community Consultation period (say 6-9 months) to socialise the change before we implement?
- This would be okay for the current known offenders, as:
- a) The International terms have successfully been reduced down to comply with the 255 limit for now
- b) The MS customer who had longer terms specialised their DescriptionType format to 4096 in their own extension - the only issue here of course is global interoperability, but this is a lower risk for now than contravening our own specs.
-
- WE ALSO NEEED TO INCLUDE THE SPANISH EDITION + ALL OTHER SI PRODUCTS IN THIS CONSULTATION!! (as ALL our products would move to 4096)
-
-
- SNOMED International Content team was asked to confirm how many FSN's (especially in the Drug content) in the International edition are pushing up and above the existing 255 limit -
The project lead advised that the current content is usually truncated with a comma if it exceeds the 255 limit. A report for content that may use a comma to reduce the character count was run and it looks like a very small number (approx. 17) - however there is no accurate way of identifying the exact numbers as this truncation is historical and the issue that arose with https://ihtsdo.freshdesk.com/a/tickets/49593 only occurred because the descriptions were being modified for a different reason. The DROOLS The DROOLS warning (which had been switched off for awhile on the international platform in response to a request from a member country) will be reinstated for 255 characters in the international release until the increase in character numbers is agreed https://jira.ihtsdotools.org/browse/MAINT-2532
- This confirms that there is no urgency for this change from an International perspective, and Extensions can continue to specialise the config in their own DescriptionType refset where required.
- We can therefore discuss and analyse this change properly, and apply a suitably lengthy Consultation Period for the Community, given the high potential for impact to end users.
|
18 | Proposal to deprecate the Concept Non-Current (CNC) Indicators | ALL | SOME REFINEMENTS TO THE proposal, for us to review and agree: The Case For Removing Description Concept Non-Current Indicators - The key points of the proposal that are salient to our group are:
- TODAY = Discuss whether or not there are any known users still using the CNC indicators?
- = Discuss whether there are any valid use cases still in existence to retain them? (whether in use or not)
- = Are there any dissenting arguments against the assertion that CNC indicators are completely
- obsolete, and that it's more efficient and reliable to determine the relevant Concept's state from the
- Concept record, rather than from the AttributeValue record?
- = Are there any dissenting arguments against the assertion that CNC indicators cause additional work
- for all creators of SNOMED CT content, in terms of maintenance, packaging and validation?
- = Are there any dissenting arguments against the assertion that the removal of CNC indicators
- will serve to simplify the understanding of SNOMED CT, + help to lower barriers to adoption?
- = Discuss any impacts to the terminology, or to any users, when removing the CNC indicators?
- (beyond our internal impact, which is restricted solely to removing/simplifying existing code)
- = Discuss who, if anyone, we should specifically target for feedback on the proposal?
- = Agreement in principle of the deprecation of CNC indicators by the TRAG.
-
- PROPOSED TIMELINE (updated)
- .
- Any other Feedback?
-
|
19 | DEPRECATION METHODS: | ALL | - Multiple different use cases:
- 1. Transparency in allowing users to see what's being Validly removed from history due to mistakes
- 2. Obsolete content that wasn't a mistake, is just no longer relevant and we want to tidy up history (especially where we're talking about thousands of records (like CNC indicators)
- 3. Legal issues (such as licensing issues) - where you CAN'T retain hisotry ANYWHERE (even in a static folder) and so history has to be completely deleted and never seen again
- 4. UNSAFE data (ie) clinical risks, technical issues that cause dangerous content issues (duplicate ID's, etc) - that should again not be kept in a separate package that's available to users, in order to prevent them being loaded accidentally!
-
- Method for 1 + 2:
- Inactivate it all first for a certain period...
- Remove it completely on agreed date
- Put all removed data into its own separate package, in a "static package" in MLDS
- This allows users who absolutely CANNOT live without the deleted content to reload it into their extensiovs
- We would need a spearate place in MLDS ??per product?? and also split between each use case above?? (so a repo for "removed content due to obsolescence" + a repo for "removed content
- Do we also then need to change status to something other than 0 or 1?
- Or perhaps even move it to a new "trash" module or something?
- OR do we retain each row in its exact state from the last Published state, as this gives users the opportunity to know exacty what needs to be deleted from their own extensions, etc (as the Triple index still exists as it would in Prod)
- the Problem with this approach is whether or not the risk of someone then misinterpreting this "removed" package (and uploading it into their systems instead of using it to remove data!) is too high to keep the records in the same status?
- OR perhaps we retain the info of which records to delete in the Annotations refset? So we could add an Annotations record for each deleted record, which would contain the Triple (ComponentID + ModuleID + effectiveTime) so that users could know exactly which records need to be deleted...
- ALSO NEED TO CONSIDER HOW TO PROVIDE THE USERS WITH A STANDARDISED METHOD
- Release Notes would also need link to MLDS static pacakge
- Naming convention for the Static Pacakge needs to be OBVIOSU + content in the JSON and Readme files made clear to mitigate the risk
Discussions to continue in October 2024 meeting... |
20 | Spanish Edition release date | Spanish Edition users | Now that we have proven that we can refine the timelines required to build, test and validate the Spanish Edition release packages, we now need to start the discussion regarding the Release Dates. Recently we changed them to bring them forward to 31st March/30th September, in order to reduce the time between the dependent International Edition + the Publication of the Spanish Edition package. Now that this has been trialled and confirmed, we should discuss what dates would be most helpful to the Spanish Edition users. The end users can most likely benefit from moving our release closer to theirs (rather than moving them earlier and continuing using the "July" and "January" releases). For example, Argentina publishes November 30, Spain December 1st, Uruguay December 15th. So we are considering moving the Spanish release to 1) October 30th, or mid-November 2024, or to a fixed date late in the month (e.g., November 25 to 27). (see email trail with Guillermo 27/02/2024 18:45) This would be with a view to socialising the plan for several months (assuming that we can agree on the best approach), and giving the end users time to adjust to the new schedule. This would therefore likely be targeted for actual transition in 2025. - WE NOW HAVE A NEW REQUIREMENT AS THE SPANISH COMMUNITY AREN'T ABLE TO CHANGE THEIR RELEASE DATES TO BRING THEM FORWARD, in order to take advantage of the additional time gained by us releasing the Spanish Edition a month earlier!! (they Release Nov/May)
- So instead of continuing the refine the process down in order to release the Spanish Edition even earlier each year, should we instead:
- Revert the Spanish Edition releases back to April/October
- Just cut off the translations much later in the cycle, in order to allow us to use a much later monthly INT Edition as the baseline? (eg) Sept/March?
- This would allow many more translations to get into each Spanish Edition release, but still allow the users to retain their own Release dates?
- HOWEVER - IS THIS JUST A COUPLE OF CUSTOMERS, OR ALL OF THEM?????
- NEED TO UNDERSTAND THE FULL USE CASES INVOLVED HERE, SO NEED TO HEAR FROM:
- Guillermo
- Spanish Extension NRC?
- WHAT ARE OUR THOUGHTS? ANY IMPACT ON ANYONE, OR SHOULD WE SOCIALISE WITH SPANISH COMMUNITY?
AAT to socialise a proposal to Change to: - May 10th + November 10th (based on INT Edition April 1st + October 1st)
- The first release on the new cycle would be targeted for 10th May 2025
-
|
21 | Spanish Edition frequency of delivery | Spanish Edition users | - Guillermo initially confirmed that many Spanish Users are requesting more frequent delivery of the Spanish content.
- Some ideas they have for addressing this are:
- 1. Allow Spanish users to build + publish their extensions based on the UNOFFICIAL preview release that we currently share with termMed - BUT this is a risk, and not great for interoperability
- 2. Allow Spanish users to version the content "locally" at ANY TIME, in order to baseline for their extensin - again not great for interoperability
- 3. Move Spanish Edition to a monthly release
- BUT this is a significant overhead for termMed + SI, and so we asked for use cases to support this requirement...
- Perhaps best would be a move to a Frequent Delivery of a Spanish Drugs extension, as this is the predominant use case Guillermo's users have for needing more freuqent delivery of Spanish contnet - and therefore trying to move the entire Spanish Edition (with it's rigid requirements for FULL translation of the INT Edition, etc) would be a large (and potentially unnecessary) overhead.
- To be discussed in October 2024 meeeting...
|
22 | Differences between the format of Extensions and the SI Spec | Matt/Dion to present | Australia have identified multiple differences between their Extension (which was valid from a Spec perspective), but different to other Managed Service extensions - would be great to discuss them through in APRIL 2024 and decide: a) Should the spec be tightened? b) Should the Validation (RVF, etc) be tightened to ENSURE alignment of extensions with the spec?
|
23 | MD5 Hash update | | As we all know, our current MD5 uses the standard 128 bit encryption, giving a 32 digit hash. However there are newer and more effective methods out there which we could in theory upgrade to, for example sha256Hash (which has been tested and proven already elsewhere in the community) - Do we think that this would be worth the effort to migrate over to?
- Does any have any feedback on use (or non-use) of the MD5 files?
|
24 | Frequent Delivery for Managed Service |
| Whilst the overall MS move to Frequent Delivery won't be made available to MS customers until after the International Edition transition, we also don't want to diverge the code bases. Therefore, we need to consider and include configuration items within the code to allows the MS Projects to move through the new Frequent Delivery workflow WITHOUT moving to Frequent delivery (for example, we could just enable the basic mandatory automated SAC and nothing else?) - We have already had to introduce a small amount of change into the MS authoring processes, in order to ensure that the MS code base remains in line with the International code.
- Comments and feedback welcome...
-
- **** SI have now made the decision to standardise ALL of our Products in terms of the format of the packages
- This means that the MS packages are now being migrated over to Delta-less packages
- Any feedback on this?
- Same goes for the Derivative products - so far:
- GMDN
- MedDRA
- Have been migrated over - any feedback?
- APRIL 2022 - only feedback was from Guillermo, who confirmed they are still creating extensions with Delta files - we assured him that we're not at the point of enforcing the new standards across ALL SNOMED Releases, just across all products published by SNOMED INTERNATIONAL - so he can continue to include/exclude the Delta files as required in his own extensions.
-
- WE ARE IN THE PROCESS OF MIGRATING NOW - MORE FEEDBACK FROM USERS NOW??? NEW REQUIREMENTS???
- Australia provided great example in October 2023 meetings, concluding that:
- the key is in bringing as much validation up front to authoring point as possible +
- also managing scope of projects properly so promotion is efficient +
- making the Release packaging and validation processes as thin as possible (Release Management now happens at time of Promotion to MAIN)
- HEAR FROM NORWAY IN 2024 TRAG MEETING WITH A FULL REPORT ON MIGRATING TO FRI IN THE MS?????????
|
25 | Annotations - Language Code | ALL | Hi Matt Cordell Dion McMurtrie Michael Lawley Alejandro Lopez Osornio Mounir Bouzanih Patrick McLaughlin Mikael Nyström Stuart Abbott Gábor Nagy Reuben Daniels There are several options for managing the Language code - - see the options here: Annotations review
- Plus also see the "Representation of annotation data type" section of the MAG proposal: SNOMED CT Annotations (this is because the other option was to include "@language" in the "annotationValue" field, which would be a problematic idea for implementations as the entire field would have to be parsed each time in order to extract the language code)
- Please provide feedback asap before the next TRAG meeting in October, so that we can try to unblock development. thanks!
- Decision made to use the "@[language]" (eg. "@en") in the "annotationValue" field
- ***** BUT THEN THE MAG JUST MADE A NEW (unanimous) DECISION - TO REMOVE ALL LANGUAGE CODES FROM THIS IMPLEMENTATION (as they're extraneous at present, and no valid use case is apparent at present)
- Any other major concerns with the decision made to remove the language code completely?
- YES - Mikael has strong objections so we took a vote and adding a new Column (which would be empty (NOT null) when not required) won 8 votes to 5
- Changes upcoming (within next few months):
- Dialect added to Language column
- Changes to refsetDescriptor
- Upcoming content in future INT Releases
|
26 | Annotations - Documentation for Refset File types | ALL | The Primary Use Cases for Annotations have been provided as follows: - Attribution - (eg) AJCC vs. UICC tumor staging concepts
- Editor notes - (eg) Reference to section of editorial guide when "nonconformance" is used as an inactivation reason
- Authoritative reference - (eg) Point to sources of truth such as UpToDate or Clinical key pages
- Regulatory data for drugs - (eg) approved uses, off-label uses
- All of the above examples appear to fall nicely within the new "METADATA" definition for Annotations...
- Thoughts??
The new Annotations Refsets do not conform to any of the existing Refset types/patterns: We likely therefore need to agree on a new type/format - this will be discussed first in the MAG in the morning and then in the TRAG in the afternoon, with the aim to agree new refset types in these meetings so that they can be used from the December 2023 International Edition Release onwards, and also to create the necessary documentation for the new Refset types in Confluence (as we did for the last new Refset type - the OWL Expression refsets: - New types / formats agreed?
- Do we need to DEPRECATE the earlier versions of the Annotations refset here 5.2.1.6 DEPRECATED: Annotation Reference Set ?
-
- Refset Type formats:
- .
- Additional fields for the Member Annotation Refset (created to support annotations on members of any refsets):
- refsetId - Identifies the reference set to which this reference set member belongs. In this case, a subtype descendant of |Member annotation type reference set|.
- referencedComponentId - A referred the referencedComponentId in the referencedMember entry in a refset.
- referencedMemberId - A reference to the UUID of a member in a reference set. The entity to which the annotation is being applied.
- Annotation - Any descendant of 900000000000459000 |Attribute type (foundation metadata concept)| in the metadata hierarchy.
- .
- Additional fields for the Component Annotation Refset (created to allow annotations to be assigned to any SNOMED CT component):
- Documentation complete?
- NO - Andrew Atkinson to complete proposed Specs and send out internally for review, before sending to TRAG for final review
|
27 | Annotations - Additional Relationship file | ALL | - Question - do we need to rush the Additional Relationship file into the December/January release urgently (in order to get Language tag in)?
- .
- Or can we put this in a future release once we've got the refsets in now?
- .
- AAT to take this offline and put a proposal together to get the Additional Relationships file (same format as ConcreteValues file) + some technical content (for initial use cases like language tags etc - see Australia) in as soon as possible
- TOPIC FOR THE MAG ON WEDNESDAY
|
28 | Annotations - Validation | All | - What validation, if any do we need?
- MUST BE UTF8 compliant (and we need to lock down what we mean by this as means different things to different people - see Peter)
- Exsiting refset validation for COMPONENT ID's (or UUID's) - (doesn't have to be Active, just EXISTING component)
- non -empty Annotation validation
- Andrew Atkinson to write up in tickets and request Dev work ASAP
- Anyone already have assertions they'd like to donate?
|
29 | SNOMED Release Package causing file path length issues in Windows environments.
| | WEDNESDAY (MEETING 2) WHEN US NRC IS IN ATTENDANCE: - The base file name is 63 characters: SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z.zip
- When the zip package is decompressed on a windows based computer using 7zip, the base folder name is the same length as the zip package file name:
- Drilling down into the second level directory, we see that the base folder is duplicated:
- The user must click through the duplicate folder before actually getting to the Full and Snapshot folders:
- The duplicated folder is easily viewable when looking at the file path.
- C:\Users\snyderjw\Desktop\SNOMED\
- ...SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z\...
- ...SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z
- To access files in the release package, the SNOMED path and file name can reach up to 218 characters.
- ..\SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z\...
- ...SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20230301T120000Z\
- ...Snapshot\Refset\meta\
- ...der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20230301.txt
- The maximum file path length allowed in Microsoft windows is 260 characters. This leaves distributed file systems only 42 characters, which can easily be exceeded by simply placing the zip package in the following directory
- “C:\Users\username\Documents\SNOMED\Downloads\USEdition\March2023\”
- When attempting to copy / paste or work with the files, cause the windows operating system to display an error message to the user as follows:
- This error causes the user to move and unzip the package in a shorter directory path to work with the files, which is not always feasible. While the duplicated folder structure issue has always existed, it has been accentuated by the recent change in the zip package and upper level folder renaming which increased the length.
Solution Request: While some windows server and client systems have been configured to allow file lengths greater than 260 characters, not all systems or local computers have been so configured. The US NRC is requesting that the duplicate folder structure be removed from the zip package during the zip package generation process to minimize file path length issue in windows environments. ACTIONS - If I download the US Edition from March 2023, I don't get the duplicate base folder structure - albeit using a Mac!
- PROVEN THIS IS A WINDOWS-ONLY ISSUE...
- Has anyone else experienced this issue on Windows or anywhere else?
- YES!
- HOWEVER Mikael confirmed this is NO LONGER an issue with the native winzip program in the latest version of Windows! So upgrading Windows resolves this issue.
- Stuart confirmed that it worked for him too, and also if we change the format we would have to make EVERY package different for every user in the world - not feasible!
- Gabor had checked and it is a configuration item that you can change in the Windows Registry, even on older versions
- THEREFORE ONLY ACTION IS FOR SNOMED International to add some guidance for Windows users - perhaps to the Readme files?
- ALSO AAT to check Release Packaging Conventions doc, as Gabor said that our MS PAckages contravene OUR OWN CONVENTIONS for the 4th Element (eg) "_US1000124_", etc ????????
- NOTE: Everyone agreed that the TRAG should occasionally review ALL of these RELEASE DOCS and confirm that they're still up to date, as things change quite frequeently so we should have a formal ACTION in place to review and update where needed at least once per year!!
- .
|
30 | IPS Terminology Product | All | Quick run through of the changes that we're proposing to make in the final Production release in Q4 2022, as compared to the BETA release (ie) discussion of the feedback that we accepted and have implemented in the Production release: - INCLUSION OF THE “EML” (new Drugs refset) IN THE FEEDER FOR THIS PRODUCT FROM 2022 ONWARDS
- IPS Terminology URI:
*** PLEASE SEE SECTION E here for final solution: ACTIONS: - Reminder that this is a SNOMED International product, but NOT a SNOMED CT product, which means it's non conformant to many of our normal standards
- "RF2-like", purely from a structural point of view
- HOWEVER, it will NOT be a full, formal RF2 package
- It will contain nothing but SNAPSHOT data, because it's going to be re-generated every year based on the latest data, rather than authored from release to release
- This means NO HISTORICAL mechanism will be provided - users will need to create their own if required
- Another reminder that this product is NOT for members, it's only useful for non-members (mostly those new to SNOMED)
- Questions on any changes planned?
- OCTOBER 2022: Any final feedback before finalise first Production release?
- YES!!! Following feedback received:
- a) FHIR and others have problems with the format - they have to add extra functionality for their non-member countries to use a new format - in addition, the use base is very diverse, across members and non members - this is because entities like HL7 are trying to support different users across these domains. This means that whilst the intention was only ever to target non-members with this product, this hasn't been the practical reality.... THEIR REQUEST IS TO THEREFORE:
- b) Various users therefore require the MDRS file to be included, in order for this product to be more usable across both types of users.
- c) It has also been requested that we consider turning this product into an EDITION! This would mean including all of the historical information and potentially other content, in order to turn it into a "mini SNOMED", however this was never the intention of the original product, and so is not likely to be accepted.
- d) The problem with using the IPS FREESET is that the Freeset contains only 8000 concepts, whereas the IPS SUB-ONTOLOGY expands everything to about 16,000 concepts!!
- It was therefore suggested that perhaps in order to address this requirement instead, we could scale up the IPS RF2 Refset product, to include all concepts in the IPS Terminology product? (currently there are nearly double the amount in the IPS Terminology product due to the expansion of the sub-ontology). This would then allow the users to get the historical info from the IPS RF2 Refset product instead...
- e) Michael Lawley raised the following query:
I know that IPS is "NOT SNOMED" and thus maybe doesn't need an update to the SNOMED URI spec?!?, but the following is not documented in that spec and again creates cost for venders to do custom support for IPS rather than just re-using the existing http://snomed.info/xsct/... approach – it's not really clear what the value of using /ips/... is?
- AAT to discuss with the business and come back to everyone with potential solutions on Wednesday...
- We can include the MDRS, just with a circular reference to itself (as it's supposed to be a standalone product, not dependent on INT or any other package!)
- With respect to the history requests, there is no appetite to include these in the IPS Terminology release format.
However, we have one potential compromise - how about adding the additional content from the IPS Terminology scope into the IPS RF2 Refset release? That way you naturally get both the MDRS + Histroy mechanism included?Only drawback we can see is that removal of parents etc in the calculation of the sub-ontology, would not then be perfectly represented in the RF2 inactivations, and so we'd have to all be happy that there may be concepts that are removed each cycle because of the unusual circular mechanism involved in a) firstly calculating the sub-ontology based on the original scope of the IPS Freeset, then b) using that wider scope to feedback into a new Refset in the Refset tool, and finallyc) basing the IPS RF2 Refset Release on this new refset in the tool(otherwise if we just feed it straight back into the original IPS RF2 Refset in the tool, it will grow exponentially, because the next cycle of sub-ontology calculation will start from the 16,000+ scope and then expand it again from there!
So this won't really work!
- So we agreed to trial a new version of the IPS Terminology format:
- MDRS file to be added to the package
- IPS RF2 Freeset file to be added to the package
- Change URL spec to "xsct" as per Michael Lawleys' recommendation in the TRAG meeting...
- Extra step in calculating the subontology Snapshot FROM 2023 ONWARDS (as no history required for this first 2022 Prod Release):
- Add in any concepts that had "previously" been in the IPS Terminology package, BUT check they are no longer, either because:
- They're now inactive, or
- The modelling has changed and these concepts are no longer in the scope of the sub-ontology
- NOVEMBER 2022:
- We published the Production release package, with the agreed improvements:
- MDRS file added to the package (with circular self-reference!!)
- IPS RF2 Freeset file added to the package
- THOUGHTS????
- APRIL 2023:
- ANY FEEDBACK FROM USING IT IN PRODUCTION SYSTEMS???
- YES!
- Previous changes to the file format addressed the issues that they had - so that's good
- People were however unhappy that this is being published separately,
- ...and via a different mechanism to the usual MLDS distribution method
- This creates more work for implementers and NRC's to consume
- MLDS is already full of historical non-SNOMED CT content (Resources, etc)
- The use-case for Members using the IPS Terminology product (that was originally designed specifically for non-members to use as an intro to SNOMED before getting full SNOMED licence), is that they want to be able to create queries (FHIR value sets, etc) that work for BOTH Members and non Members, allowing Members to transfer data to and from non-Members. Therefore in order to make this happen, and to be able to test the end to end, they need to be able to test them against not only the FULL SNOMED (that Members are using) but ALSO against the IPS Terminology scope (that non-Members are using).
- HAVING DISCUSSED THIS INTERNALLY WE WOULD BE HAPPY TO PUBLISH IPS TERMINOLOGY VIA MLDS (as well as the IPS part of the SNOMED website) - WOULD THIS RESOLVE THIS ISSUE??
-
- IN ADDITION, some people are unhappy with the separate IPS URI (eg) "http://snomed.info/ips/999991001000101"
- HAVING DISCUSSED THIS INTERNALLY WE WOULD NEED A REALLY STRONG USE CASE TO CHANGE THIS AT THIS POINT - CAN ANYONE PROVIDE ONE, OTHER THAN THAT IT'S A BIT IRRITATING?
- .
-
-
- 2023 - ROB TO PROPOSE ADDITIONS TO THE IPS TERMINOLOGY SCOPE...to include 14 structural concepts
-
- AAT TO TALK TO DEV TEAM TO GET THESE ADDED (Kai + Rory) IN TIME FOR THE NOVEMBER 30th 2023 RELEASE - COMPLETE
-
- AAT to start publishing IPS Terminology on MLDS AS WELL as on the IPST download site, to allow easier access for Members.
- Request was also made for the URI to change from http://snomed.info/ips to something more standard
- This was initially rejected internally, as the entire point of this was to distinguish IPST from other SI products
- However, Australia then confirmed that Ontoserver CANNOT consume this type of API.
- Peter Williams also suggested that maybe Snowstorm and/or the Browser might not consume it either...
- If this is the ALSO the case then we would have a stronger business case to change the URI
- Peter will therefore confirm shortly and we will decide from there...
- ONCE ALL DECISIONS MADE WE NEED TO
- a) Inform the community if any changes to be made, and
- b) Update the SI URI Spec (again if any changes are to be made, or even if we're keeping it as .../IPS/... as this isn't in the spec??
-
|
31 | Documentation | | Question: Do we need to update the release file specification to reflect the new practice of not including the delta release file type in the International Edition release package? - As we know, the delta format is still "valid", but it is no longer provided as part of the International release package.
- Also we now provide a tool to generate delta files.
- We've updated the Delta file specifications (3.2 Release Types) to specifically mention the removal of Delta files from the International Edition:
Please note : - Delta files have been removed from the SNOMED International release package, but a Delta Generation Tool is available for those who need it. The Delta Generation Tool allows users to create their own Delta between two fixed release dates - you can find it here: https://github.com/IHTSDO/delta-generator-tool/releases.
- BUT this doesn't include ALL Extensions + Derivative products - does it need to??
- It also states at the top that :
- ALSO the RELEASE PACKAGE CONTENTS page is definitely out of date:
- 3.4 Release Package Contents
- UNLESS we consider this to still be useful info for historical packages that still contain DELTA files? (or other extensions outside of SI control that still include them??)
-
- If we need to do this, let's walk through it in the Release File Spec docs...
- Andrew Atkinson to implement changes in the Documentation??????
|
32 | Refset processes in the post-RT2 world | | Options to be discussed (see local Notes "RT2 New Process") MAIN POINTS: OPTIONS: - BEST option appears to be to allow the Derivative refsets to be dependent on the month prior to that in which they're Published (eg) March/September for most derivatives.
- However, is it possible that if we were to make a different decision on the item above, then this issue in the RT2 release process could also be mitigated by moving the metadata into the Derivative packages themselves? If we did this, could we potentially then retain the Derivative content in the Refsets' child branches and release from there?
(eg) - For GPFP we would maintain the content via the RT2 FE which would read/write to MAIN/Refsets/GPFP
- Given that we would then also include the Module + other metadata in the same branch (instead of in MAIN),
- ...Could we then version say into MAIN/Refsets/GPFP/2023-04-30, and Release straight out of that branch??
- The final option is moving to Monthly releases for Derivative products as well - however:
- This is problematic from a capacity/practical point of view, and
- This would require significant discussions with all of our collaborating entities to sign off on new agreements (for those who have tangible inputs into the process), including monthly deadlines, etc - This could not be achieved quickly so would need to be discussed with 2024 in mind at the earliest.
- The alternative to this would be to move just SOME of the derivatives (the straightforward refsets for example which don't require much/any external collaboration) to monthly schedules, and keep the others on annual/6 monthly schedules.
- However, this would be a significant operational issue, as the processes are already complex enough without introducing further complexity in terms of different derivatives runing on completely different processes.
We should discuss options and agree the best way forward for retaining quality within the Release process vs impact to the users. - Option 1
- No-one is in agreement with this option!
- This would cause real problems for NRC's (let alone end users) who would struggle to keep up with downloading and consuming multiple monthly releases. In addition, they wouldn't be able to then publish multiple releases of their own to the end users, containing the usual Jan/July changes + then extra releases for each month that is a dependency for the Derivative releases.
- Finally, many of the entities doing Translations are struggling enough to keep up with the pace, without adding additional stress and complexity. This option would therefore completely prevent them from consuming any of the Derivative products.
- This Option is a non-starter.
-
- Option 2
- The Pro's are far greater here - as all of the users who can't keep up can continue to download just the Jan/July INT Edition releases, and then consume whichever Derivatives they need.
- Andrew Atkinson to put forward this proposal, which would be to:
- a) MOVE all of the Derivative content in Snowstorm from the INT Edition branch into their own Codesystems (checked with Rory and Terance and should not be a problem)
- b) CHANGE RT2 to use the new individual codesystem branches for reading + writing each Derivative content to and from RT2 (checked with Brian and Rick and should not be a problem)
- c) DEMOTE the various Derivative metadata components down from the INT Edition in the July 2023 Release
- (this would simply involve inactivating them all in the International Edition)
- d) PROMOTE them in the various Derivative packages in the Sept/October 2023 Derivative releases.
- (this would simply involve activating them in the relevant Derivative packages)
- e) THEN EACH CYCLE WE WOULD:
- i) Upgrade each Derivative Codesystem to the relevant Jan/July INT content
- ii) FREEZE the content in each of those "in flight" codesystems, to prevent any more re-basing until the Release cycle is complete
- iii) VERSION in each relevant Derivative Codesystem
- iv) RELEASE from each relevant Derivative Codesystem
-
|
33 | Update to the .JSON file metadata - addition of "Package Composition" data
| | Now we've used this file for several months, do we have any suggestions for improvements? - Andrew Atkinson to put together proposal to include following New fields:
- URI that identifies the package being published
- PackageType (ie) Edition or Extension or Derivative
- List of modules used
- Default Namespace
- Default module to start authoring
- Country tag
- Custom tag
- ALL INFORMATION FROM THE README FILE (which is neither human nor computer readable)
- MANIFEST INFO
- Licencing info
- JSON SCHEMA for this JSON file
- (to allow computer readable version of how to read the JSON file)
- .
- ANY OTHERS????
- .
- JUST TO CHECK WE STILL NEED THIS GIVEN ANNOTATIONS IMPLEMENTATION??
-
- TRAG TO REVIEW THIS PROPOSAL IN OCTOBER 2024 + IF RATIFIED WE'LL IMPLEMENT SHORTLY AFTER
- Removed fields:
|
34 | Computer readable metadata
* MAG crossover | Andrew Atkinson | Suzy introduced the topic for discussion...
Suzy would like to raise the question of creating computer readable metadata, and raise questions such as whether or not to include known namespace & modules? Or just the current metadata for the files in a machine readable format?
CAN WE PLEASE REQUEST THAT PEOPLE SHOUT NOW IF THEY HAVE ANY FURTHER REQUIREMENTS FOR THE METADATA PROJECT?? WE NEED TO FORMALISE THE SCOPE IF WE'RE GOING TO BE ABLE TO ADD THIS INTO THE WORKPLAN FOR 2024 - PLEASE SUBMIT REQUIREMENTS TO ME BY 30th September OTHERWISE THEY MAY NOT MAKE IT INTO THE SCOPE OF THE FIRST PHASE OF THIS PROJECT... Suzy Roy to provide an update on progress: - All agreed that whilst this is a large topic, we should start somewhere, and get at least some of the quick wins in (then request the change to content via the CMAG):
- Check where the progress with the namespace metadata has got to - can we progress this?
- Code systems (and versions) of the map baselines
- Common strings such as boiler plate licence text etc
- Description of use cases for the various refsets (using the text definition of the Refset concetps themselves) - either json or markdown representation of multiple pieces of info within the same field.
- Michael Lawley to provide an update from the related MAG topic...
- TRAG agreed that this should be incorporated into the discussions with the continuous delivery, in order that we can plan the changes here in line with the transition to more frequent releases. To be continued over the next few months...
- Michael Lawley to kindly provide an update on his work with David to help design and implement the solution - this will now be in the second TRAG meeting of the April 2019 conference, after they have met together....
- Ideas:
- Some human readable metadata could potentially live as descriptions (which can then be translated)? David to discuss further...
- David will mock up something in Json...
- Michael + David + Harold agreed to create a straw man to put up in the next meeting and take this further...
- This should now be combined with the Reference set metadata topic, to address all updated metadata use cases - Human readable, Machine readable, etc
- We need to setup a JOINT Working group to deal with this!
- Dion kindly volunteered for this group
- We've added a new machine-readable file to the International Edition this cycle, which can be refined for future usage:
- Standard July 2020 International Edition metadata file:
- NON-Standard July 2020 International Edition Rollup Delta metadata file:
-
- Suzy Roy kindly volunteered to run a Project Group later in 2021 to refine and improve this data as needed going forward:
- Volunteers confirmed in October 2021 TRAG meeting:
- Dion, Mikael + Alejandro
- + Andrew + Peter from tech team
- + need 2 volunteers from MAG (as SMT decided we need wide range of views)
- Suzy to setup meetings once we have MAG volunteers confirmed...
- This will be rolled into the holistic discussions on Metadata in the new Metadata Working Group... Working Group: Refined Metadata
-
- Plus new requirements from other discussions:
- HOWEVER, WE'RE STILL MISSING THE IDENTIFICATION OF THE ACTUAL MAP PRODUCT ITSELF, AND THE VERSION OF THAT ENTITY
- (eg) "ICNP version Jan 2019" should exist as metadata somewhere within the ICNP map product package...
- + possibly even the direct URI?
- SUGGESTION IS TO USE THE JSON FILE FOR THIS - Andrew Atkinson to take this forward in the Metadata working group...
-
- ANOTHER DISCUSSION POINT FOR THE JSON FILE:
- Are the "DeltaToDate" and "DeltaFromDate" fields in the JSON file now misleading in the new world of Frequent Delivery where we have no Delta files in the INT package itself?!
- "deltaFromDate" : "20210930",
"deltaToDate" : "20211031",
-
- FINAL DECISIONS:
- Agreed that these fields should ONLY be available in packages with Delta files
- Monthly International Releases going forward, should instead just have:
- EffectiveTime
- PreviousPublishedPackage (that the current release is based upon)
- Any retracted releases + their replacements
-
-
- Examples of extending this metadata:
- .json format 5 ?? (Please see Michael Lawley's comments on 16/04/2021 here: Re: Working Group: Refined Metadata)
- Namespace data
- Individual external Refset data
- ranges of permitted values
- mutability, etc?
- Package Name? (Please see Michael Lawley's comments on 20/04/2021 here: Re: Working Group: Refined Metadata: Yes, regarding the "Name" entry, it would be ideal if it could be used to populate the "Product Name" field in a list of available packages (and other required and relevant fields for MLDS). Then the zip contents would be sufficient to automatically populate MLDS (or an ATOM-based Syndication feed))
- WE'RE STILL MISSING THE IDENTIFICATION OF THE ACTUAL MAP PRODUCT ITSELF, AND THE VERSION OF THAT ENTITY
- (eg) "ICNP version Jan 2019" should exist as metadata somewhere within the ICNP map product package...
- + possibly even the direct URI?
- SUGGESTION IS TO USE THE JSON FILE FOR THIS - group to provide examples of how this would look to the TRAG for review...
-
- ANYTHING TO SUPPORT FREQUENT DELIVERY USEFULLY????
- Also create 2 new pages -
- one to capture the requirements for all different use cases
- one to discuss and agree on the proposed solutions....
- ALL AGREED NO OTHER REQUIREMENTS CURRENTLY, BEYOND THOSE NOW COVERED OFF BY EITHER:
- Extension of JSON file metadata, or
- Annotations, or
- Additional Relationships file
- .
- THEREFORE WE'LL HOLD THIS TOPIC OPEN ANOTHER MEETING OR TWO, AND CLOSE DOWN IF NO NEW REQUIREMENTS COME IN
- JUST TO CHECK WE STILL NEED THIS TOPIC GIVEN ANNOTATIONS IMPLEMENTATION??
|
35 | Run other (non Managed Service Extensions) through our RVF etc to see what differences are out there between their Extensions and the spec |
| This should help prove what barriers to adoption some of our implementors might face Andrew Atkinson to Discuss with Alejandro and Kai to see if they can help - as this might be a good implementation team task? |
36 | Appetite for front end for RVF configuration and building! |
| Not enough outside of the Managed Service - however a Service where people could feed a package into RVF and get results out would be useful Discuss further in 2024? Maybe spec out the new service? |
37 | Potential move towards Editions | ALL | MAIN POINTS: - Over the past couple of years we've noticed a trend towards requests for Edition packages containing multiple combined modules, rather than individual extensions/derivatives that are dependent upon other packages.
- In addition, some of our Managed Service work (in particular wrt the Common French + Common German content) is driving us in this direction as a natural part of the products' evolution.
- We'd therefore like to open the discussion on whether or not we should start moving towards producing more Editions, with an eventual aim of publishing most of our products in Edition format.
- Would this make things easier for yourselves as consumers?
- Would it make things easier/more difficult for smaller users such as affiliates and licensees?
- What are the benefits / drawbacks of such a move?
- Spain is another example:
- The Articles of Association appear to specify nothing, but one of our country contracts state that SNOMED International need to "Provide at least twice-yearly updated versions to the international release of SNOMED CT, including the English and Spanish editions" -
- However it does NOT specify format or structure.... So perhaps this allows us to change the Spanish Edition to a true Edition??
FEEDBACK SO FAR - National Extensions should move to Editions in order to remove blockers for end users:
- Knowing which International Edition to download in conjunction with the Extension
- Knowing how to combine Extension + INT packages to make something useable
- ESPECIALLY as this all needs to be done consistently across all countries and users, in order to make the final results useable and compatible for interoperability purposes.
- Edition packages also remove the ambiguity around modules that have been included in a package that aren't part of the MDRS - composition versus dependency.
- International Derivatives should remain as dependent Extensions
- Surely the same benefits for end users apply as with the National Extensions? (ie)
- Knowing which International Edition to download in conjunction with the Extension
- Knowing how to combine Extension + INT packages to make something useable
- HOWEVER:
- As they are mostly Refsets, having an enormous Edition (500mb+) for one tiny refset (10kb!!!) seems redundant and a massive overkill
- Many users want multiple derivatives, and so combining Editions then becomes extra-complicated!
- Derivatives (maps and reference sets, even language translations) don't typically incur the same level of risk as Extensions (wrt module dependencies etc) as they don't modify components declared in upstream modules.
- ANOTHER OPTION:
- Would be for SI to create a "Super-Edition" containing INT content + ALL derivatives, but we moved away from this several years ago for many reasons:
- Most Users don't want all derivatives, just one or two
- All derivatives are on different maintenance schedules, so no matter when we published this we'd be out of date with several of them.
- If something goes wrong with just one derivative, it holds up the release of ALL derivatives.
- What are the thoughts on the current proposal?
- Any further feedback?
- QUESTION - IS IT WORTH PUSHING EVERYTHING (and all Extension owners, both MS and external) TO EDITIONS, given that we're also driving towards Service-based delivery in the long run? What's the benefit of having 2x major transitions instead of just 1???
- Any comments on the feedback received so far?
- Mikael would also like International Derivatives to be left as Extensions and NOT Editions (too fat for such small releases, too complex to combine Editions (and they often want to combine Derivative releases into one package), etc)
- Stuart confirmed the UK have clinical Edition + their Drugs Extension (monthly because their users don't want so much data, and the Drugs extension is over 1GB for the Snapshot on its own!)
- So they want to choose what components to download
- + they want to releases some components annually, and others monthly, so this is another vote for EXTENSIONS (certainly for EXISTING Extensions - maybe NEW Extensions could be moved to Editions and leave existing as they are?)
- General suggestion from the TRAG is that "going forward" we should always promote Editions, BUT existing Extensions are too invested in Extensions to want to move, unless we can provide a better business case!
- However this doesn't increase efficiency in MS management as we would still have to spec-up our internal systems to build and validate Editions better than is currently the case, so this approach doesn't improve anything.
- Maybe instead we should just
- a) improve human training, and
- b) improve our upload tools to work better with Extensions? (allow automatic combinations, etc)
-
- If we approve the proposal, should:
- a) All Managed Service Extensions be transitioned to Editions?
- b) Does that even include Translation-only Extensions?
- c) Should NRC's not using the MS be required to create Editions instead of Extensions?
- d).Without unity and consistency across all NRC's, does it help to move to Editions, or create a larger interoperability blocker?
|
38 | Redesign of the Map Reference Set formats | All | Please find below a proposal for redesigning the map reference sets to support maps in either direction: https://docs.google.com/document/d/14bmRaVQYI7-Kz2EPgv00muGqdO6wRrMycCPCJqp5W2s - This proposal was signed off, ready to take to the MAG on 20/10/2021...
-
- APRIL 2022 - Review and sign off of final formats:
An opportunity has been identified to improve the format of the SNOMED International Map Reference Set products. This will apply to all types of simple and complex Map Reference Sets going forward, including (but not limited to) the SNOMED CT MedDRA Simple Map package, first released back in April 2021. The existing SNOMED CT map reference sets were originally designed for maps in the direction from SNOMED CT to another code system, manifested by the use of a ‘mapTarget’ string attribute used to represent the code in the other code system. The new and improved map reference set patterns will be introduced with a ‘mapSource’ attribute, in order to more accurately represent maps from other code systems to SNOMED CT. The refined format provides users with more clarity when using maps of either direction, as well as additional map metadata representing the new refset patterns and correlation values. Users will also benefit from clearer and more predictable naming of the map refsets, as the map reference set concepts have been reviewed and updated to follow the refined description patterns. Please see the links below for the updated technical details including the improvements:
The first product to be improved using the new designs will be the SNOMED CT MedDRA Simple Map package. After in-depth discussions with the communities’ expert advisory bodies, the users confirmed that their preference was to retain the historical data from April 2021.
It was therefore agreed that the 2022 SNOMED CT MedDRA Map package will be published as follows: - …with all new 2022 content in the improved format
- …with all relevant historical MedDRA data (from April 2021) also in the new format
- …with the historical April 2021 map records that were in the original format inactivated (in order to retire the relevant UUID’s) - the inactivations would likely be published a) in the new package in the new format, and b) in a separate file/package in the original format. However this is still to be confirmed.
- All of this means that the 2022 file will appear as if the original April 2021 MedDRA release was actually published in the new format. Therefore, the 2022 MedDRA Release will be published as a complete, consolidated package, with all original data from 2021 plus all new inactivations/changes from the latest cycle presented in the new and improved format.
-
- To be taken forward in metadata working group:
- HOWEVER, WE'RE STILL MISSING THE IDENTIFICATION OF THE ACTUAL MAP PRODUCT ITSELF, AND THE VERSION OF THAT ENTITY
- (eg) "ICNP version Jan 2019" should exist as metadata somewhere within the ICNP map product package...
- + possibly even the direct URI?
- EXAMPLES FOR MEDDRA??
- SUGGESTION IS TO USE THE JSON FILE FOR THIS - Unless we need to discuss now, we will take this forward in the Metadata working group...
- CONFIRMED THAT WE WANT THE FOLLOWING FIELDS ADDED TO THE .JSON FILE FOR RELEVANT DERIVATIVES :
- External MapSource (or MapTarget) - (ie) If we're publishing a map from SNOMED TO GMDN then we should state that this is from
- SNOMED CT version Jan 2022 to
- GMDN Version 2019
- If we're publishing a map from MedDRA to SNOMED CT we should state:
- from MedDRA version 2023 to
- SNOMED CT Version July 2023
- Directionality of the map - Some people would like the Directionality to be explicitly stated so that it's machine-readbable, instead of just implied in the Map Package naming convention
- (ie) If we're publishing a map from SNOMED TO MedDRA then we should state that this is
- Direction: FROM SnomedCT TO MedDRA
- (ie) If we're publishing a map from MedDRA to SNOMED CT then we should state that this is
- Direction: FROM MedDRA to SNOMED CT
-
- Simple Map transition complete and Documentation Published:
- Additional Metadata now included in following 2023 Releases:
- ANY ISSUES WITH THIS, OR IMPROVEMENT SUGGESTIONS???
- YES:
- The JSON Metadata file can be further refined to be more useful:
- Should be machine readable
- Can use SNOMED URI's to denote SNOMED components, and
- Perhaps use FHIR URI's for the non-SNOMED parts?
- The scope of this file was discussed in great detail, and it was agreed that it was still useful to target having package-specific metadata ONLY in this file. Everything else can and should be put into Annotations instead, so that it's all machine readable in the content itself.
- We may need to increase the breadth of the package-specific metadata - to cover Composition elements as well as the current nominal scope - see the ECRS discussions for further use cases...
- We could make the directionality of map products machine-readable by including an array of metadata (eg)
- <conceptID (of the Map)>
- <Parent (of the Map Concept)> - i.e. Simple Map from SNOMED type, or Simple Map to SNOMED...
- etc...
- Discuss with Peter W as he has good ideas on this...
- Then run the final Proposed formats past the Australian NRC, as they have good use cases and can not only feedback but also test thoroughly...
- AAT to discuss further and come back with new proposals for refined format...
-
|
39 | IMPROVEMENTS TO THE RELEASE FORMAT |
|
|
40 | a) | Proposed deprecation of the CTV3 Identifier simple map | Due to it coming to the end of its useful life, SNOMED International would like to propose planning for the deprecation of the CTV3 Identifier simple map (that currently resides in the RF2 International Edition package) as of the January 2020 International Edition.
Some Member countries have already identified the reduction of the effectiveness of the product, and have already put plans in place to withdraw support for the CTV3 Identifiers from 2020 onwards.
The TRAG therefore need to discuss whether or not there are any apparent problems with the proposed deprecation, and if so how they can be mitigated.
We must also discuss the most effective method to pro-actively communicate out announcements to the community to warn them of the upcoming changes, in order to ensure that everyone who may still be using the Identifiers has plenty of notice in order to be able to make the necessary arrangements well in advance.
Finally, we will need to decide on the best method for extricating it from the package, in order to ensure the smoothest transition for all parties, whilst remaining in line with the RF2 standards and best practices. - AAT CHECKED THE PREVIOUS IMPLEMENTATIONS OF DEPRECATION OF BOTH ICD-9-CM and RT Identifiers, AND AS THOUGHT BOTH WERE IN THE CORE MODULE, AND REMAINED IN THE CORE MODULE IN THE STATIC PACKAGES - SO ANY ISSUES WITH DOING THIS AGAIN?
- So the plan would be to follow the same deprecation process as we did with ICD-9-CM (ie)
- move all of the content to a Static Package in July 2020, and inactivate all of the content
- publish the reasons for inactivation in the historical associations
- Release Notes similar to ICD-9 = SNOMED CT ICD-9-CM Resource Package - IHTSDO Release notes
- CREATE A STATIC PACKAGE FOR CTV3 BASED ON THE JULY 2019 MAP FILES AND PUBLISH ON MLDS (and link through from Confluence link as well). ALSO LIFT THE CTV3 SPECIFIC DOCS FROM THE Jan 2020 RELEASE NOTES TO INCLUDE IN THE PACKAGE.
- Date of the files should be before the July 2020 edition (so say 1st June), in order to prevent inference of dependency on the July 2020 International edition
- So we set the effectiveTime of the Static package to be inbetween the relevant international edition releases (eg) 1st June
- This is to ensure that it's clear that the dependency of the Static package will always be the previous International Edition (here Jan 2020), and not continually updated to future releases
- It cannot therefore have an effectiveTime of July 2020 (as we would normally expect because we're removing the records from the July 2020 Int Edition) as this would suggest a dependency on the July 2020 content which doesn't exist
- It also can't have an effectiveTime of Jan 2020 as we need to distinguish between the the final published content which was Active in Jan 2020, and the new static package content where everything is Inactive.
- Inside the files should be all International edition file structures, all empty except for:
- Delta ComplexMap file needs to be cleared down (headers only), as no change in the content since the Jan 2020 files, so no Delta
- Full and Snapshot ComplexMap files exactly as they were in Jan 2020 release (including the effectiveTimes)
- ModuleDependency file needs to be blank, as CTV3 was in the core module (not in its own like ICD-10 is), and therefore the dependency of the core (and therefore the CTV3 content) module on the Jan 2020 edition is already called out in the Jan 2020 ModuelDependency file, and therefore persists for the static package too.
- Date of all of the files inside the package should be the new date (1st June)
- But all effectiveTimes remain as they were in Jan 2020
- Leave refsetDescriptor records as they are in the International edition
- RELEASE Notes Should describe all of the thinking we went through when creating this package, why the moduleDependency file remains blank, and why we’ve wiped the Delta, etc (see above)
- AND ALSO COMMS SAME AS WE DID WITH THE RT IDENTIFIER REFSET DEPRECATION:
RT Identifier Refset deprecation: We need additional comms around the July 2017 release, in addition to the usual Release Notes wording, in order to confirm what is happening and the rationale behind it. To re-iterate what was discussed on the previous call, Legal counsel confirmed that from a legal perspective, he doesn’t consider that it’s either necessary (or even advisable) for us to send CAP any further communications on this matter. Legal counsel is confident that the informal discussions that we’ve already had with them (in order to remind them about what they need to do), are sufficient to cover our legal obligations, given that the licence is theirs and not SNOMED International's. Therefore we no longer need to send a formal letter to CAP.
Has anyone identified any issues with the proposed deprecation? Is everyone still in favour of the refined process to use to deprecate??
If all good then Andrew Atkinson to begin formal deprecation process
|
41 | b) | Proposal to remove the Stated Relationship file completely | Link to proposal Inactivated all records in July 2019 - long enough now for everyone to be on Axioms? |
42 | Continual Improvement to the Frequent Delivery process - Presentation on Frequent Delivery | | In our last TRAG meetings several people confirmed that many people within the community were asking for more information on our transition to Frequent Delivery - therefore it was requested that we create a presentation on how the migration went, benefits realised, lessons learned, etc - Maria and Andrew to Present findings on our transition to Frequent Delivery:
- Processes transitioned
- This is the benefits of the transition so far
- [ ] 1. These are the issues we faced - [ ] a) This is how we resolved them - [ ] 2. This is how we see the future going - improvements + benefits
- Lessons learned
- Questions?
- We could setup a blog post with Kelly (similar to Jim's on collaborative authoring - https://www.snomed.org/news-and-events/articles/embracing-collaborative-authoring) that would be based on this presentation...
- Would this be helpful to people?
- FEEDBACK ON PRESENTATION:
- Would be useful to add Example monthly cycle for Authoring team in terms of Dates for cut off's, delivery, etc
- Would be great to clarify how often the authors have to "unpromote" concepts from MAIN because of conflicts, etc?
- Mapping - no impact moving to keeping up with monthly cycles - useful to specify this so people aren't guessing?
- USER DOCUMENTATION
- Several end users have contacted Guillermo to request more information on the transition to more frequent delivery (and/or more frequent updates to the dependent content of both extensions and derivatives).
- It would be great if we could provide people with a white paper or presentation (both from a Content and Technical perspective) on
- How the transition went
- Benefits realised
- Lessons learned
- Risks
- etc
- This should be targeted at both the Supplier level + the end users level for max effect
- Does the presentation myself and Maria gave cover this? Or could we add an area on the website??
-
- RELEASE NOTES
- Can translators please have more detailed release notes - automated to the extent of having EACH component change listed out?
- We need to be able to automate the generation and publishing of the Release Notes -
- this work is still underway, but will take quite awhile for the Dev team to complete....
- NEW REQUIREMENTS FOR SONJA:
- Option for detailed release notes as well as the standard level for all Releases (as per Guillermo's requirement above) - either in the RNMS (preferable) or in the Delta Generation Tool?
- It would be great to also link each Component change to the relevant high level change note in the official Release Notes as well (eg)
- Release Notes March 2023:
- Drugs Changes:
- Component 1 changed
- Anatomy changes:
- Component 1 added
- Component 2 removed
- Component 3 changed
- Quality Initiative:
- Component 1 changed
- Component 2 inactivated
-
INFRA-8739
-
Getting issue details...
STATUS
RAISED
- RNMS-65 created to track development
-
- Explicit annotations in the Release Notes to that each component change in the Delta period is linked to a specific Template
-
INFRA-8740
-
Getting issue details...
STATUS
RAISED
- MAINT-1958 created to track development
- DELTA GENERATION TOOL
- Has anyone trialled the Delta File Generation Tool as yet? Any feedback?
- YES GABOR HAS TRIED IT AND FED BACK ISSUES TO PETER - PETER IS STILL WORKING ON IMPROVEMENTS SUCH AS THE Snowstorm issue whereby if there are multiple different states for the same component withint the Delta period, snowstorm loses the history of all those changes and just spits out a (random) one line state! So if a concept has started active, been inactivated and then reactivated in the Delta timeframe, snowstorm might decide to spit out active or inactive as the latest state! It will also lose those 3 changes and only export one change!
- UPDATE ON THIS WORK:
"...the problem here is more with the Terminology Servers not being able to deal with deltas that contain more than one state for the same component than it with with the DGT itself. However we added a flag to the tool (see https://github.com/IHTSDO/delta-generator-tool/releases/tag/1.2.0 ) so that you can generate output that will contain multiple effective times, but only the most recent effective time for each component. This is a workaround that means your TS does not end up with the correct Full file representation. So really the only additional work would in theory be in Snowstorm so that it can treat deltas as successive updates to the Full file and generate release branches as it needs to. However so far this has not been raised as a requirement by the community, and so is not planned work...
- DO WE NEED FURTHER ENHANCEMENTS, OR NO APPETITE FOR THIS??
- a) Delta's to be generated from any point in time to any other point in time
- b) Metadata to be included somehow (to be discussed further in the Metadata Working Group) to record critical information, such as which Dates the Delta is from + to, which Modules are incorporated, etc
- c) Compound Delta's (including ALL changes since the relevant date, including ALL changes in the dependent release package(s), rather than just the latest state - so these are "Full file to Full file" Delta's, as we are used to) are favoured so far, however we should continue to assess any potential use cases for Atomic Delta's (effectively "Snapshot file to Snapshot file" Delta's) as we go along, in case it becomes apparent that there is a valid Business Case to ensure that the new Delta generation tool can provide either or both of these Delta file types...
- d) It needs to support the future requirements for Service Based delivery, once we transition over
-
- VALIDATION advances:
OWL testing - anyone worked on this as yet? Template validation - thoughts? Implementation testing feasible? (see Implementation Load Test topic below) - Need to identify Modelling areas that need improving - for example where concepts have 2x parents, this is usually an indication of areas that need re-modelling
- Need automation of the QA system itself - so some quick way to validate RVF + DROOLS Assertions, both old + especially new!
- Whitelisting - API required?
- Specific extensions to the automation Validation scope (eg)
- New idea for an RVF assertion regarding the ordering of OWL records (based on first concept) with disjoints:
-
- Critical Incident Policy update
- We need to refine the Critical Incident Policy:
- Need to ensure categorisation is solid (as otherwise requests may be made for minor issues to be fixed immediately as "Critical Incidents" just because it impacts one institution (but not Internationally))
- Currently Content Team critical incident policy states:
- If it's a Clinical Risk then it has to be fixed
- OR if it's not a Clinical Risk but impacts certain number of members etc then still Critical, etc
- April 2023
- What other criteria do we need to use?
- How strict should we be with the criteria?
- We need to balance the risk of NOT fixing an issue vs the risk of impact to a stable Release candidate from the fix...
- As discussed yesterday, we should keep the policy flexible on the solution that will need to be implemented in order to resolve any critical incidents:
- The best solution is simply to mark the Release in question as "Invalid" and advise all users to download the next stable Release
- However, can we reliably contact all users who've consumed a Release, given all the possible end users who've downloaded it via NRC's as well as direct from MLDS?
- We should only use Negative Delta files and other potentially confusing and destructive techniques if there is no other option (eg) Critical Legal Incidents.
- Any useful lessons learned from anyone else's Critical Incident Policies?
|
43 | Continual Improvement to the Frequent Delivery process | All | Potential Improvements: - USER DOCUMENTATION
- RELEASE NOTES
- DELTA GENERATION TOOL
VALIDATION advances: - Critical Incident Policy update
|
44 | NNF Generation | | Essentially it is about what is considered redundant in the NNF generation, and the implications that has for ECL. At the moment things that are more specific than statements inherited from further up the ancestry replace the more general statements in the NNF calculation, which makes sense.However I introduced equivalence axioms which created necessary conditions that were equivalent - not more or less specific. This resulted in the NNF calculation removing (seemingly arbitrarily) one of the two sets of conditions - this is kind of right because they are "redundant" in the sense that you don't need both, however they also aren't redundant in the sense that they are both necessarily true and neither is "more specific" than the other. Which is picked will affect how ECL works and is evaluated.I'm pushing the boundaries here having equivalence axioms with expressions on both sides, but that should be theoretically possible and I suppose what we need to determine is if that's to be supported what should the NNF look like. Presumably a deterministic selection of one of the axioms or a merged set of all the necessarily true conditions may be more useful for ECL.There's a related point with property chains which I can demo with ECL too to explain and provoke discussion. |
45 | RVF improvement discussions | | CSIRO have been working on improvements to the RVF, and would like to report on and discuss some of the results with us... - Dion to Present current status + plan...
- Comments and feedback welcomed...
- Plenty of feedback and so further discussions required as we move through the project...
- The main feedback for the past few months has been the RVF failures for the new MDRS assertions, which appear at first glance to be false positives. However, they have been proven to be valid failures, as long as you consider that the MDRS format itself is (and has always been) inherently flawed.
- The closure of this topic is therefore dependent on the outcome of the discussions on the Proposal for a complimentary file to the MDRS - the "ECRS" ("Edition Composition Reference Set")
- If this concludes that we need to change the MDRS, then this RVF topic can be closed down.
- If, however, we decide to retain the MDRS format, then we need to revisit these RVF assertions...
- We need to use the new planned changes to .JSON metadata file: Update to the .JSON file metadata - addition of "Package Composition" data in order to fix the RVF assertions and remove the false positives...
-
FRI-169
-
Getting issue details...
STATUS
-
- QUESTION FOR DION/MICHAEL - CAN WE JUST REFINE THE .JSON FILE (as per the proposals here: Update to the .JSON file metadata - addition of "Package Composition" data) IN ORDER TO ALLOW THE MDRS ASSERTIONS TO WORK PROPERLY FOR NOW????
- YES, but the question is how best to do this?
- We need to work together to create examples of how we might extend the JSON file to include more useful data, both for the a) users b) syndication + c) to allow the MDRS assertions to work as expected for SI Products.
- Examples to be added here, and assessed by the TRAG at a later date:
- NEED TO ADD EXAMPLES OF EACH USE CASE + SAMPLE MANIFESTS - This will not only help with MDRS assertions, but also Syndication info...
|
46 | Refset containing the semantic tags? | | This topic was closed down by the TRAG a few years ago due to the lack of requirements vs the complexity of finding a robust solution. However, new requirements and a potential solution from Ed Cheetham have now been submitted for our review and discussion - please see here for details: Refset containing the semantic tags? We will discuss in detail in the next TRAG meeting in April, however please feel free to contribute to the online discussion in the above link in the meantime. - Slide deck here for advanced review:
- Feedback from the group:
- Excellent identification of issues that need addressing
- The first target should be to discuss the application of the Validation that Ed has kindly brought to us, both in the AP + Release validation stages.
- The second sim is to bring the discussions on the potential Formalisation of the Semantic tags to the relevant AG's for furhter consideration
- Yong has kindly agreed to add this to the agenda for the next MAG meeting, to be discussed further.....
- UPDATES FROM THE MAG???
|
47 | Extension Management in the new world of Frequent Delivery | | - With the change in release cycle to monthly, extension management has become intractable and merits consideration for tooling enhancements or procedural change on the part of SNOMED International.
- Since extension modules have dependencies on one or more other modules, periodic reconciliation with their parents is a requirement if they are to support interoperation of their content.
- Multiple dependencies for an extension creates opportunity for parent modules with dyssynchronous release cycles, introducing further complexity.
- Since it is seemingly unrealistic to require alignment of versioning and release schedules between independent institutions, the situation calls for tooling support that would compare modules for reconciliation and prepare a systematic step-by-step workplan for the content manager to follow to achieve expedited systematic reconciliation that will validate and classify. The tooling would ideally execute the workplan to guide the manager in the process.
- Potential extended requirement for the Delta Generation Tool??
- Any new thoughts on this since October?
- .
|
48 | Dependent Releases for Derivative Products | All | To be discussed in April 2022, in time to make a final decision before the 2022 Derivative cycle begins shortly afterwards... - The original intention of more frequent delivery was to continue using the January + July Releases as the dependent INT Editions for all derivatives. This was to ease users through the transition, by allowing them to continue using the Jan + July releases indefinitely, rather than moving to individual monthly releases.
- However, this causes a potential conflict with the July Derivative cycle, as if we don't start the (many) feeder derivatives for the September GPS product until 1st August (instead of starting on 1st July as we did last year because we had the luxury of cutting off the July 21 editing cycle in May 21), not only do we reduce the amount of time that everyone has to migrate the refsets + get external reviews completed, but more importantly we clash with the European holiday season in terms of getting reviews signed off by the key external stakeholders, who are often away during August.
- We are in the process of discussing this with the relevant stakeholders, to see if they will be available in August 2022, but if not we are wondering if it would be acceptable for the 2022 Derivatives (including GPS) to be based on the May or June 22 release (instead of July 22)? Whilst this may initially seem inconvenient, it would have the benefit of increasing the quality of these derivative products by allowing thorough internal + external reviews before publishing.
- Feedback?
-
- As Matt mentioned, another option is to try to feed the derivative authoring process with monthly updates, thus reducing the necessary workload in the final Release cycle.
- However, in order to have the desired effect, this would also require us to not only author new changes more frequently, but also to migrate each derivative multiple times per year, in line with each monthly release.
- Whilst this could resolve the time crunch in August, it would necessarily introduce an additional overhead to the workload of the authors throughout the year,
- ...as even though they'd technically migrate the same number of concepts over 6 monthly migrations as they would in one large migration per 6 months, the process is cumbersome enough to have an impact on capacity
- ...this could (in theory) have a slightly positive effect however, as it would mean that authors get to know the migration process more intimately, if having to do it every month instead of every 6 or 12 months!
- We need to discuss with WCI to ensure that the tool would support this however...
- ...for example, we'd almost certainly need a new Delta generation process in the Refset tool, in order to enable it to provide roll-up Delta files for the past 6 or 12 months' worth of migrations in one file...
- The vast majority of the group are in favour of retaining the Jan + July releases as the dependent releases for all derivatives, mostly because of the comms that we sent out confirming that most users won't be impacted by the Monthly Releases if they don't want to be, as they can continue to use only the Jan/July releases for the foreseeable future.
- This is especially true for NRC's like Sweden, who Mikael says are using quite a few derivatives to package up different products for their users, and so having a conflict between the dependent releases of their extensions and thos of the derivatives will be very unhelpful for them
- We need, therefore, to explore different options, such as
- a) Updating the refsets monthly (though this is confirmed as an overhead for the team by Maria)
- b) Removing the review stage for all derivatives (except those which are brand new), s most feedback on BAU derivatives finds nothing of use nowadays...
- c) Postponing the final delivery of the refsets impacted by lack of people to review in July/August to November say, so the reviews can take place in September and work can continue after that. This si probably the most popular option in the group, but then not many people in the group are dependent on the derivatives releases...
-
- REVIEW AGAIN IN 2023 TO SEE IF THE APPETITE TO REFINE THIS IS NOW THERE, BASED ON USERS' EXPERIENCES OF BASING EXTENSIONS ON DIFFERENT MONTHS, ETC??
|
49 | NEW DEPRECATION PROCESS! |
| Link here (if JMI completed in time - if not push this to 2023) - 2022
- We will shortly be refining the deprecation process for SNOMED CT Products, especially derivatives such as Nursing Activities + Nursing Health Issues.
- If you have any pre-emptive ideas of how we can improve this process, please let us know now, as this is the time when we can easily impact the final solution?
- For example:
- Communication improvements?
- Comm out as far and wide as possible...
- Changes to the way we leave (or don't leave) the deprecated packages in MLDS?
- Some suggested leaving on MLDS for a short period (1 year?) then removing to keep MLDS clean
- Most others prefer to ALWAYS leave the latest (in this case Final Deprecated) version on MLDS permanently, so people know
- HOWEVER, this should be accompanied by clear labellling on MLDS to state
- a) That the product is deprecated
- b) the reason for deprecation (no longer used vs INVALID vs Dangerous, etc!)
- c) And keep the packages in a separate folder in MLDS marked "Deprecated" to make it very clear to only use them if you know what you're doing
-
- Changes to the way in which we deprecate the RF2 records? (inactivation, just leave them active but static, etc)
- This should be optional depending on the Reason for Deprecation (ie)
- a) If it is just no longer being maintained, then everything should remain Active, with a note clearly stating that there is no longer ACTIVE MAINTENANCE being done on this Product, and so should be used with caution as it's definitely out of date
- b) If the content is "WRONG" or "UNSAFE" then it should be inactivated and flagged as Unsafe for use
- c) etc
- Changes to the way we deal with the metadata? (inactivating refsetDescriptor records, module records, etc?)
- Metadata can never be "unsafe" in and of itself, and so refsetDescriptor and Module records ashould always remain active in all cases
-
-
- 2023
- Confirm if everyone is happy with the new process?
- Confirm if they are then also happy with applying the new process to all following planned deprecations:
- 2x Nursing Refsets
- Old FORMAT MedDRA Maps (but not entire Product)
-
|
50 | Proposal for a complimentary file to the MDRS - the "ECRS" ("Edition Composition Reference Set") | | The TRAG had discussions a couple of years ago to clarify the best application of the Module Dependency Reference Set (MDRS) - some background reading is here: - Re: 4.2.1.0 Using SNOMED CT with FHIR
- Proposal for a complimentary file to the MDRS - the "ECRS" ("Edition Composition Reference Set")
- Miscellaneous Documents
Michael and Dion then walked through the proposal and answered questions, but Michael and Linda both confirmed that the use case was not a critical priority at the time, and therefore didn't need to be actively discussed until new cases were proposed... WE THEREFORE CLOSED THE DISCUSSION DOWN AT THE TIME DUE TO A LACK OF MULTIPLE USE CASES, AND SO THIS WAS DE-PRIORITISED UNTIL SUCH TIME AS MORE USE CASES CAME TO LIGHT.
We have now identified more use cases for this proposal, as the new automated MDRS validation picks up what appear at first to be false positives, but which are actually valid failures due to the historical shortcomings of the MDRS format. - We therefore need to discuss and agree an approach that allows us to both express the correct moduleDependencies + the new module composition (to express which modules comprise the Edition package, for URI + validation purposes).
- This should then be used to properly validate the MDRS and moduleDependencies within the Edition and Extension packages.
- There was a lot of feedback on the original proposal - however in this meeting we should:
- a) Ask Dion/Michael to walk through the proposal in person to ensure that everyone's on the same page (and remembers the original discussions)
- b) Answer the feedback (plus any new feedback in light of new situations and/or use cases)
- c) Agree what the final proposal should be, and what are the next steps we need to take in order to get it signed off (MAG, design authority, etc?)
- Michael, Dion and Reuben were going to create the Australian version as an example, in order to include that in Michael's updated version of the proposal document - did this happen?
- New proposal for representing the ECRS information in the .JSON Metadata file will be kindly brought to the table by Dion + Michael tomorrow, for further review
- This will include an example of how the INT Edition might look...
-
- As part of the discussions on this topic, we need to decide what to do about the transitivity of dependencies in the MDRS - Linda will kindly present the background and options to discuss...
- Initial discussion were had on 18th October 2021, leading to a provisional decision that the best course of action might be to:
- State that transitivity is the primary method, but that
- Explicit statement of all moduleDependencies (even though that could be inferred through their transitive inclusion) would remain an option in all cases, to be used whenever the transitive dependencies would lead to potential confusion or conflict, for example in the case where two different components (eg. ICD-10 map + IPS refset) of an Edition (eg. Pangea Edition) were themselves dependent on two different versions of the same product (eg. the July 2021 INT Edition + the October 2021 INT Edition respectively). In this case the MDRS in the Edition which incorporates the modules would explicitly state the dependencies of all it's constituent modules, and therefore resolve the conflict that would otherwise have arisen -
- so in this example, the Pangea Edition would explicitly state that both ICD-10 + IPS modules were dependent on the October 2021 INT Edition
- NB the curator of the Pangea Edition would first be responsible for testing and confirming that the ICD-10 maps (which were implicitly dependent on the July 2021 INT Edition rather than October) worked cleanly with the October 2021 release as well, before publishing the Pangea Edition.
- However, the one drawback raised in response to this option was that we need a strong use case to warrant changing the RF2 spec. So we need to decide if we're happy that the use cases in the proposal are strong enough for that (ie)
- Resolving issues with pre-existing Editions that did not originally spec out the URI with this in the mind
- Enabling more comprehensive targeted automated validation of the MDRS files
- This is currently not possible without resolving the transitivity question, and
- The imminent transition to Frequent Delivery brings this to the forefront of our current considerations, as without the necessary breadth in the automated validation, we cannot guarantee the quality of the monthly releases.
- FINAL DECISIONS:
- a) We will use the new JSON data on Package Composition to resolve the issues with the false positive results in the current MDRS RVF assertions, by having the assertions check the new JSON data to confirm whether or not the modules that are not explicitly called out in the packages' MDRS file (as its an extension or similar), or that have conflicting versions.
- b) We will use the new .JSON data to allow correct resolution of URI's
- c) We will NOT change the RF2 spec to move to transitive dependencies in the MDRS.
- New planned changes to .JSON metadata file: Update to the .JSON file metadata - addition of "Package Composition" data
- FEED INTO THE METADATA WORKING GROUP DISCUSSIONS...
|
51 | Reference set metadata * MAG crossover | | Replacement of the Refset Descriptor file with a machine readable Release Package metadata file See David's proposal here: Reference set metadata (plus sub page here: Potential New Approach to Refset Descriptors) - Everyone confirmed no issues with the proposal in principle, in April 2018
- However, do we consider this to just be relevant to refsets in the International Edition release package?
- Or to all derivative products as well?
- Both refsets and maps?
- Also, are we talking about only human readable descriptive information, or also machine readable metadata such as
- ranges of permitted values
- mutability, etc?
- Michael Lawley to kindly provide an update on his work with David to help design and implement the solution - this will now be in the second TRAG meeting of the April 2019 conference, after they have met together....
- Michael + David + Harold agreed to create a straw man to put up in the next meeting and take this further...
- Michael Lawley - where are the discussion on this currently?
- Michael confirmed (20210420) that this straw man was never created, and so we should use the published .json file as the straw man for future discussions...
- Can we link this in to the .JSON file above? (Computer readable metadata) - yes, done!
-
- IN FACT, are there any requirements for machine-readable or human-readable metadata that can't be addressed with extensions to the new .JSON file in the release packages?
- No, not that people can foresee!
-
- This will be therefore be rolled into the holistic discussions on Metadata in the new Metadata Working Group...
-
- ANY UPDATES FROM THE MAG???
|
52 | Refset Descriptor Inactivation | Matt Cordell | Question here is whether or not RefsetDescriptor records themselves should remain active for retired reference sets? TRAG to decide on correct policy and feedback to Matt... - The consensus so far is that we should keep the RefsetDescriptor records themselves active, which has been the precedent for all cases in RF2 history so far, with the exception of the Non-human refset which was physically removed from the International Edition package.
- The UKTC and others have previously requested these RefsetDescriptor records to be inactivated (
ISRS-112
-
Getting issue details...
STATUS
, etc) - for consistency purposes, but the corollary of this is that the refset structure itself (which the refsetDescriptor describes) remains valid and active, despite the refset itself having been inactivated.
- TRAG TO DISCUSS AND AGREE BEST SOLUTION...
- Then propose an addition to the TIG to provide clear guidance on this for all users...
- AGREED:
- Happy to leave the RefsetDescriptor Active for all normal circumstances
- If we're removing the Refset entirely from the Extension/Edition, we should
- a) if it's just for space or something, then leave refsetDescriptor record in place
- b) if it's for CRITICAL INCIDENTS ONLY (and even then only certain subsets of this - most likely only legal issues), we'll remove RefsetDescriptor completely
-
- Matt Cordell to write up and send to all of us for review.... confirmed on 20/04/2021 that Matt will write this up and present to the TRAG in future meetings
- FINAL DECISIONS:
- Matt Presented - no contentious points, so Matt is ready to take this proposal further...
- UPDATES FROM MATT???
|
53 | Implementation Load Test | All | RVF has now been open sourced to allow people to contribute towards it more easily, so that Implementation issues can be reverse engineered into the assertions. All of the NRC validation systems should remain separate, in order to ensure as great a coverage across the board as possible. However, it makes sense to ensure the critical tests are included in all systems, in order to ensure that if, say, one NRC doesn't have the capacity to run Alpha/Beta testing for a certain release, we don't miss critical checks out. We are working on this in the Working Group, and also in the RVF Improvement program, where we are including the DROOLS rules, etc. These are also being incorporated into the front end input validation for the SCA. TRAG to therefore discuss taking the Implementation Load test forward, including the potential to incorporate key rules from NRC validation systems into the RVF. So we should discuss the tests that are specific to the Implementation of vendor and affiliate systems, in order that we can facilitate the best baseline for the RVF when agreeing the generic testing functionality in the Working Group. - Matt Cordell will promote some useful new ADHA specific rules to the RVF so we can improve the scope... report back in October 2019
- Chris Morris to do the same - get the RVF up and running and then promote any missing rules that they run locally.... report back in October 2019
-
- THIS NEEDS TO BE CONSIDERED AS PART OF THE OVERARCHING Shared Validation Service PROJECT GROUP
- Anything we can add to the Shared Validation Service going forward?
-
|
54 | Versioning Templates * MAG crossover * EAG crossover
|
| The EAG have proposed the need to version templates in some way, and potentially even make them "Publishable" components (with all of the reletive metadata that goes along with that). Also the potential to make them language sensitive. They would then also need to be automatically validated themselves, as well as then being used in the automated validation of the International Edition! - Keep an eye on EAG + MAG discussions on this topic an
- Ensure that the decisions are fed into our Continuous Delivery proposal
-
- October 2021 TRAG meeting:
- Peter confirmed no longer being discussed in EAG or MAG
- Instead, Linda confirmed that the templates are still being developed internally, and once the final proposal is ready they will share it with the TRAG and MAG for review+ for decisions such as how best to publish them, in what format, etc.
- So one to revisit in future conferences...
- UPDATES FROM THE MAG???
|
55 | Release packaging conventions and File Naming Conventions | All | TRAG to review and provide final feedback. Reuben to provide feedback on progress of the URI specs + FHIR specs updates... - Document updated by Andrew Atkinson in line with the recommendations from the last meeting, and then migrated to a Confluence page here: SNOMED CT Release Configuration and Packaging Conventions
- To be reviewed in detail by everyone, and all feedback to be discussed in the meetings. AS OF OCTOBER 2017 MOST PEOPLE STILL NEEDED TIME TO REVIEW THE DOC - Andrew Atkinson INFORMED EVERYONE THAT THIS DOCUMENT WILL BE ENFORCED AS OF THE JAN 2018 RELEASE AND THEREFORE WE NEED REVIEWS COMPLETED ASAP... so now need to check if reviews still outstanding, or if all complete and signed off??
- AAT to add in to the Release Versioning spec that the time stamp is UTC
- AAT to add the trailing "Z" into the Release packaging conventions to bring us in line with ISO8601
- AAT to add new discussion point in order to completely review the actual file naming conventions. An example, would be to add into the Delta/Full/Snapshot element the dependent release that the Delta is from (eg) "_Delta-20170131_" etc. AAT to discuss with Linda/David. Or we hold a zero byte file in the Delta folder containing this info as this is less intrusive to existing users. Then publish the proposal, and everyone would then take this to their relevant stakeholders for feedback before the next meeting in October. If this is ratified, we would then update the TIG accordingly.
- AAT to add in a statement to the section 4 (Release package configuration) to state that multiple Delta's are not advised within the same package.
- AAT to add in appendix with human readable version of the folder structure. Done - see section 7
- IN ADDITION, we should discuss both the File Naming convention recommendations in the Requirements section (at the top of the page), PLUS Dion's suggestions further below (with the diagram included).
- Dion McMurtrie to discuss syndication options for MLDS in October 2018 - see hwat they've done (using Atom) and discuss with Rory as to what we can do. Suzy would be interested is this as well from an MS persepctive. UK also interested. This shouldn't hold up the publishing of the document. Discussions to continue in parallel with the creation of this document...
- Reuben Daniels to raise a ticket to update the fhir specs accordingly
- Reuben Daniels to talk to Linda to get URI specs updated accordingly.
- URI Specs to be updated and aligned accordingly - Reuben Daniels to assist
- EVERYONE TO REVIEW TONIGHT AND SIGN OFF TOMRORROW
- ONLY outstanding point from earlier discussing was Dion's point from the joint AG where he talked about nailing down the rules for derivative modules... -
- Dion McMurtrie to discuss/agree in the October 2018 meetings - REPORT FROM DION??
- Everyone is now happy with the current version, therefore Andrew Atkinson to publish - we can then start refining it as we use it.
- Andrew Atkinson to therefore agree all of the relevant changes that will be required as a result of this document internally in SNOMED International, and publish the document accordingly.
- FIRST POINT WAS THEREFORE TO have it reviewed internally by all relevant stakeholders...
- This has been completed and signed off
- Do we consider anything in here needs to be incorporated into the TIG?
- or perhaps just linked through?
- or not relevant and just separate? YES - NOT RELEVANT!!
- the litmus test should be whether or not implementers still use the TIG, or whether people now use separate documentation instead?
- We also need to make a decision on the final Freeset distribution format(s), as I want to ensure we only have a MAXIMUM of 2 distribution formats - RF2 + the agreed new Freeset format (whatever that may be)
- YES everyone is happy with this!
- Add this into the Release Packaging Conventions and publish
- APRIL 2021 - DO WE NEED TO MAKE ANY REFINEMENTS IN ORDER TO PREPARE FOR CONTINUOUS DELIVERY? Did ADHA need any formatting changes when moving to monthly?
- No, nothing beyond the new .json file and refinements to that
- We really need to tackle the Delta from and to release version in the Delta file naming, and possibly package file naming. At the moment it is impossible to know what a Delta is relative to making it hard to safely process it. Perhaps beyond the scope of this document, but quite important
- THIS IS NOW ADDRESSED IN THE NEW Metadata file:
- Standard July 2020 International Edition metadata file:
- NON-Standard July 2020 International Edition Rollup Delta metadata file:
- January 2021 International Edition metadata file:
- March 2021 Belgium Extension metadata file:
- DOES IT NOW COVER EVERYTHING NEEDED???
- No! We'll continue to discuss and agree ideas for new fields in this file as we progress towards Frequent Delivery....
- This will therefore be rolled into the holistic discussions on Metadata in the new Metadata Working Group...
- THIS NEEDS TO BE CONTINUALLY REFINED OVER THE NEXT YEAR WHEN WORKING TOWARDS MORE FREQUENT DELIVERY:
- Once all happy, the document will be published and opened up to anyone to view.
-
- Everyone was invited to either join the Working Group, or contribute ideas towards it - we will therefore continue to report back on how this is going...
-
|
56 | Community Content | All | COMMUNITY EDITION(s) - What should the criteria be that differentiates between what goes in each Edition:
- SNOMED CT Core
- SNOMED CT International Edition
- SNOMED CT Community Edition
- What level of quality do we allow into the Community Edition?
- Any quality (quick and sharable) vs validated (slower but better)
- One suggestion is that instead of certifying the content, we could certify the authors themselves - so we could differentiate between projects which are authored by newbies, vs those who have say passed our SNOMED CT authoring certification level 1, etc
- Another suggestion is that whoever delivers content to the Community content would have to provide the MRCM to support it, + conform to editorial guidelines, etc
- So a list of “quality indicators” could be automated against each project (eg):
- MRCM compliant
- Automated validation clean
- Authors have SNOMED CT certification
- Peer reviewed
- Release Notes
- Etc
- And then people can make their own minds up about which projects to use based on comparing the quality indicators between projects
- SOME AGREEMENT TO SUPPORT AND MAINTAIN BY @SOMEONE@ AT LEAST…
- For example, what happens if we change something in the core which breaks someone way down deep in the Community Edition? (Which we can’t possibly test when we make the change in the core)
- The idea here would be that whoever creates the branch in the Community Edition then manages and maintains it - so everyone maintains their own branch, and is therefore responsible for resolving the conflicts coming down from the core, etc
- Versioning also becomes important, as whoever creates it needs to specify which Versions of each dependency their work is based on - (eg) they would state that their work is based on the 20190131 International Edition, and therefore any impact we have on the downstream community content would only happen when the owners of that content decided to upgrade their dependency(s) to the new version
- Promotion criteria important - thoughts?
- Do we remove the need for local extensions, as they can then simply become part of the Community Edition, with any local content just existing in a “country specific” edition within the Community Edition
- This also provides some level of assurance of the quality of the content in the Community Edition - as these would be assured by the NRC’s (and SI in some cases) and therefore provide a good baseline of high quality content for people to then start modelling against
- ModuleDependency is going to be important -
- perhaps we answer this by making the entire Community Edition part of the same module - therefore it will all classify as one entity?
- However a lot of people will ONLY want to cherry pick the things that they want to take - so we need a method for taking certain modules (or realms or whatever we call them) and allowing people to create a snapshot based on just that content instead of the entire community edition
- Dependencies need to be properly identified:
- Could the CORE be standalone and published separately?
- Or would the CORE need to have dedpendencies on the wider International Edition, etc?
- HOWEVER, how do we classify the entire Community Edition when there could be different projects dependent on different versions of the dependencies (such as the international Edition)?
|
57 | IHTSDO Release Management Communication Plan review | All | This document was reviewed in detail and all feedback was discussed and agreed upon - new version (v0.3) is available for review, attached to the IHTSDO Release Management Communication Plan review page. AAT has added in details to state that we'll prefix the comms with "Change" or "Release" in order to distinguish between the type of communications. See version 0.4 now - IHTSDO Release Management Communication plan v0.4.docx Once we've collated the feedback from the revised comms processes that we've implemented over the past year (in the items above), we'll incorporate that into the final version and discuss with the SNOMED International Executive Lead for Communications (Kelly Kuru), to ensure that it is aligned with the new overall Communication strategy. Once complete, the Release Management comms plan will be transferred to Confluence and opened up for everyone to view. We have publicised the Release Management confluence portal to both NRC's and the end users to get people to sign up as and when they require the information. Do we know of anyone still not getting the information they need? We also agreed last time that the community needs more visibility of significant, unusual changes (such as bulk plural change, or case significance change). These changes should be communicated out not just when they're assigned to a release, but actually well in advance (ie) as soon as the content team start authoring it, regardless of which future release it will actually make it in. I have therefore created a new Confluence page here: January 2020 Early Visibility Release Notices - Planned changes to upcoming SNOMED International Release packages I've left the previous items up (from the July 2017 International Edition) because there are no examples yet from the Jan 2018 editing cycle - so please take a look and provide feedback on whether or not this is useful, and how it can be improved. - ACTION POINT FOR EVERYONE BEFORE OCTOBER 2018: (Dion McMurtrie, Matt Cordell, Orsolya Bali, Suzy Roy, Corey Smith, Harold Solbrig, Mikael Nyström, Chris Morris)
The final version of the communication plan needs to be reviewed by everyone and any comments included before we agree the document internally and incorporate it into our communication strategy - Suzy Roy will discuss the end use cases of her users with them and come back to use with feedback on the practical uses of SNOMED CT and any improvements that we can make, etc
- We may now also need to add a new section in here wrt the comms for the TRAG, so that this is standardised and agreed with the community? Or is it outside of the scope for the Release Communication Plan? This was felt to be out of scope, and should this be restricted only to communication related to actual releases of products.
- Everyone is now happy with the current version, therefore Andrew Atkinson to publish - we can then start refining it as we use it.
- Andrew Atkinson to therefore agree all of the relevant changes that will be required as a result of this document internally in SNOMED International, and publish the document accordingly.
- AAT MIGRATED THE DOCUMENT FROM WORD TO CONFLUENCE, AND THEN SENT IT TO THE EPS Team for first review.....
- The feedback has been incorporated and the document refined accordingly.
- https://confluence.ihtsdotools.org/display/RMT/
- SNOMED+CT+Release+Management+Communication+plan
- Andrew Atkinson has now sent to the relevant members of the SMT for final sign off....
- This has now been signed off and is ready for publication
- Do we consider anything in here needs to be incorporated into the TIG?
- or perhaps just linked through?
- or not relevant and just separate?
- the litmus test should be whether or not implementers still use the TIG, or whether people now use separate documentation instead?
-
- THIS NEEDS TO BE CONTINUALLY REFINED OVER THE NEXT YEAR WHEN WORKING TOWARDS FREQUENT DELIVERY:
- Do we need more Communications over the first few months to ensure that everyone knows what's going on?
- Or do we actually need LESS now that we have regular, monthly releases?
- Once all happy, the document will be published and opened up to anyone to view
-
|
58 | What constitutes a true RF2 release? | | Harold would like to introduce this topic for discussion... - Language refset conflicts are not yet resolved - Linda has been discussing this in terms of how to merge Language refsets or dictate whether or not one should override the other in cases of multiple language refsets - in the UK they combine them all into one but this is not ideal either. In translation situations they use the EN-US preferred term as the default where there is no translated term in the local language. Perhaps we need to do a survey on the members and who's using what how.
- Suzy Roy (or Harold Solbrig) to get Olivier's initial analysis and come back to us on what worked and what didn't, and we can take it from there.
- Suzy would like to ask Matt Cordell if he can share his ppt from his CMAG extensions comparison project.
- Matt Cordell will distribute this to everyone for review before the April 2019 meeting.....
- Harold to continue analysis and report back with the results of reviewing the specific examples that Olivier identified in the next meeting....
- Can you please present the revisited presentation Matt Cordell ?
-
|
59 | Modularisation of SNOMED CT
* MAG crossover | All | Dion McMurtrie completed the Alpha release - did anyone have chance to review it? (I haven't had any requests for access to the remainder of the package) The subject of Modularisation needs to be discussed between the various AG's who are considering the topic, before we can proceed with the Release-specific sections.
We need to discuss any red flags expected for the major areas of the strategy: - Modularisation
- Members who want to abstain from monthly releases, and therefore need to use delta's with mulitple effective times contained within.
- Also need to consider if we continue to hold the date against the root concept - works perhaps still for 12 monthly releases, but not necessarily for continuous delivery daily!
- THIS NOW BECOMES CRITICAL TO THE STRATEGIC DIRECTION WE DISCUSSED IN TERMS OF MODULARISING OUR CONTENT, AND IMPROVING THE WAY THAT THE MDRS WORKS, IN ORDER TO ALLOW RANGES OF DEPENDENCIES. THIS WILL ALLOW THE "UNIT" OF RELEASE TO BE REFINED ACCORDING TO THE RELEVANT USE CASES.
- Understand the Use cases thoroughly, and refine the proposal doc to provide people with more real information - Dion McMurtrie TO PROVIDE THESE USE CASES FOR Andrew Atkinson TO DOCUMENT
- Does the POC allow for concepts to be contained within multiple modules? NO - BUT DION CAN'T THINK OF ANY CONCRETE EXAMPLES WHERE THIS WOULD BE NECESSARY
- What about cross module dependencies? Michael Lawley's idea on having a separate Module purely for managing module dependencies
- IN THE FINAL PROPOSAL, WE NEED TO CREATE A NESTED MDRS TO MANAGE THE INTER-MODULE DEPENDENCIES (as per Michael's comments)
- NEED TO PROVIDE GOOD EXAMPLES AND WHITE PAPERS OF THE USE CASES FOR MODULARISATION IN ORDER TO ENGAGE OTHERS...
- AFTER SIGNIFICANT DISCUSSION AND CONSIDERATION, THERE ARE NO VALID USE CASES LEFT FOR MODULARISATION. IT CAUSES A LOT OF WORK AND POTENTIAL CONFUSION, WITHOUT ANY TANGIBLE BENEFIT.
- THE PERCEIVED BENEFIT OF HAVING A WAY TO REDUCE THE SIZE/SCOPE FO RELEASE PACKAGES IS BOTH a) invalid (due to everyone's experience of being unable to successfully do anything useful with any small part of SNOMED!), and b) easily answered by tooling that using the ECL to identify sub-sections of SNOMED to pull out for research purposes, etc.
- THEREFORE AS OF APRIL 2018 THE FEEDBACK FOR RORY AND THE STRATEGY TEAM WAS THAT MODULARISATION SHOULD NOT BE IMPLEMENTED UNLESS A VALID USE CASE CAN BE IDENTIFIED.
- HOWEVER, KNOWING THE HISTORY OF THIS ISSUE, THIS WASN'T NECESSARILY GOING TO BE THE FINAL WORD ON THE MATTER, SO IS EVERYONE STILL SURE THAT THERE ARE NO KNOWN USE CASES FOR MODULARISATION?? (eg) linking modules to use cases, as Keith was talking about with Suicide risk assessment in Saturday's meeting,etc??
- This topic came up several times again during other discussions in the April 2019 meetings, and it was clear that people had not yet given up on the idea of Modularisation - we therefore need to discuss further in October 2019....
- Agreed to see where the linked discussions in the MAG etc end up going, and then discussing the proposals rather than just in abstract....
- UPDATES FROM THE MAG???
|
60 | "Negative Delta" file approach | All | This approach was successfully implemented in order to resolve the issues found in the September 2017 US Edition - is everyone comfortable with using this approach for all future similar situations? If so we can document it as the accepted practice in these circumstances... - NO! Everyone is decidedly uncomfortable with this solution! In particular Keith Campbell, Michael Lawley and Guillermo are all vehemently opposed to changing history.
- The consensus is that in the particular example of the US problem, we should have instead granted permission for the US to publish an update record in the International module, thus fixing the problem (though leaving the incorrect history in place). This would have been far preferable to changing history.
- ACTION POINT FOR EVERYONE FOR OCTOBER 2018: (Dion McMurtrie, Matt Cordell, Orsolya Bali, Suzy Roy, Corey Smith, Harold Solbrig, Mikael Nyström, Chris Morris
We therefore all need to come up with potential scenarios where going forward we may need to implement a similar solution to the Negative Delta, and send them to AAT. Once I've documented them all, we can then discuss again and agree on the correct approach in each place, then AAT will document all of these as standard, proportionate responses to each situation, and we will use these as guidelines in future. If we have issues come up that fall outside of these situations, we'll then come back to the group to discuss each one subjectively, and then add them back into the list of agreed solutions.
- Preference now is to retain EVERYTHING in the Full file, regardless of errors - this is because the Full File should show the state at that point in time, even if it was an error! This is because there is not an error in the Full file, the Full file is accurately representing the error in the content/data at that time.
- The problem here is that the tools are unable to cope with historical errors - so we perhaps need to update the tools to allow for these errors.
- So we need the tools to be able to whitelist the errors, and honestly document the KNOWN ISSUES (preferably in a machine readable manner), so that everyone knows what the historical errors were.
- The manner of this documentation is up for debate - perhaps we add it to a new refset? Then we could use something very similar in format to the Negative delta, but instead of it actually changing history retrospectively, we simply document them as known issues, and allowing people to deal with the information in their own extensions and systems in whatever way they feel is appropriate.
- Only situation we can think of where we couldn't apply the above gentle response, would be copyright infringement - whereby if we discovered (several releases after the fact) that we had released content that was in direct infringement of copyright, then we would potentially have to revoke all releases since the issues occurred. However, this would raise a very interesting situation where patient safety might be compromised - as if we remove all historical content that contravened the copyright, then we run the risk of patient data being impacted, thus potentially adversely affecting decision support. This is simple to resolve when the problem is in the latest release (simply recall the release), but if found in a 5 year old release for example, it could be very problematic to recall 5 years' worth of content and change it!
- October 2018 - Guillermo proposed a separate possibility, which is to introduce a new Status (eg) -1 whereby if you find this status in the latest snpashot you would just ignore it - this doesn't however address the use case where there is a legal contravention and you need to physically remove the content from the package - the use case where you would have something that contravenes RF2 paradigm, you can't use the RF2 format to correct something that is RF2 invalid! So this is unlikely to work...
- Nobody is on board with this idea, as it's too fragile and introduces unnecessary complexity such as we had with RF1...
- April 2019
- If we're still all in agreement with this, then steps 1-5 above should all be documented and disseminated to get confirmation of approval from everyone??
- Did everyone read through everything? Has anyone got any further scenarios that we can include in the documentation?
- The EAG raised this issue again on 08/04/2109 - Peter to try to make it to the next TRAG to explain the use case that was raised today and elaborate on the new proposal...
- The TRAG discussed this issue at length, and came to the conclusion that we cannot address ALL potential use cases with a standard, generic, solution (certainly not any of those offered above).
- Instead the solution in each case should be agreed on given each specific use case that comes up each time
- So INSTEAD we should update the Critical Incident Policy to very clearly define the process to be followed each time we need to remove something from the Published release(s):
- Which group of people should make the decision on the solution
- Perhaps we also provide examples of how each use case might be resolved:
- For Legal/IP contraventions, we should either remove content from history entirely, or redact it (leave the records in place, but remove all content from fields except for UUID, effectiveTime, moduleID, etc - thus allowing traceability of the history of the components, without including the offending content itself)
- For Clinical risk issues, we can remove it from the Snapshot, but leave the Full file intact to leave a historical audit trail whilst ensuring that the dangerous content shouldn't get used again (as most people use the snapshot) - see Full file steps 1-5 above, etc
- How to communicate it out to the users, etc
- OCTOBER 2019 - DISCUSSION RE-OPENED AS PART OF THE MAG:
- ONCE FEEDBACK OBTAINED FROM MAG:
- Andrew Atkinson to update the Critical Incident Policy with
- the various use cases that we've identified so far
- the governing bodies who should be the deciding entities
- the process for making the decision in each case
- including the critical entities that need to be collaborated with in each case (all NRC's, plus 3rd party suppliers (termMed etc) who represent some of them), to ensure the final solution does not break outlying extensions or anything
- the process for communicating out those decisions to ALL relevant users
-
- UPDATES FROM THE MAG???
|
61 | Potential for adding a "withdrawn" reason for inactivated content
* MAG crossover | All | Discussions around the future strategy for SNOMED CT have included the potential for adding new statuses for content.
In particular, many people have suggested that problems are created for those either mapping or translating from content that's still "in development". If (as is often the case) they use Daily Builds etc as input data, they can often get tripped up by content which is created but then withdrawn before it's versioned and officially released. It would be extremely useful to those users to have access to traceability data describing the reasons behind why they were removed, in order to support accurate mapping/translation.
In another use case, there's the possibility that content needs to be formally withdrawn from the International Edition AFTER it's been officially released. This would be the case if, for example, content has unintentionally been published that breaks the RF2 paradigm, or contravenes licensing laws, etc. In this case mere inactivation is not sufficient, the content instead needs to be completely withdrawn from the releases and sometimes even from history.
The TRAG needs to discuss all of this and be ready with recommendations if these proposals are taken forward. - ONE OF THE POTENTIAL SOLUTIONS TO THE ISSUE ABOVE: "Negative Delta" file approach
- Use cases:
- undo a historical issue (that break RF2 paradigm, etc) but don't want to pretend it never happened - in this case we should use the Negative Delta approach - but only used in EXTREME circumstances
- Legal contraventions - in this case we should use the Negative Delta approach - but only used in EXTREME circumstances
- Dead on arrival components - it should be okay to have these, and have them openly dead on arrival and therefore inactive to not map to them etc. However it's useful to be able to see these (even though they'd been activated + inactivated within the same release cycle) - so for those people who need to map/translate etc DURING the release cycle, they have to rely on the Daily Build and use live data still in development. Therefore if those concepts disappear by the time of the International Edition it causes problems for those maps/translations already including those concepts.
- Therefore the best answer is for us to move to having 2x Daily Builds - the existing one + a separate true Daily Builds - where each Daily Build is built relative to the previous Day, and NOT to the previous Published release. This new Daily Build could then be properly relied upon by mapping and translation projects.
- Can we align this with the transition to the more Frequent Releases?
- HAS ANYONE HAD ANY MORE THOUGHTS ON THIS SINCE OUR LAST DISCUSSIONS??
- MAG to discuss tomorrow (30/10/2019)
-
- UPDATES FROM THE MAG???
-
|
62 | Clean modularization | All | There are 22 module concepts, that are on the 900000000000012004|SNOMED CT model component| module.
I don't think it's documented anywhere, but we (AU) have made the assumption that the concept for a module, should be on itself. I suspect we've started to discuss this before, but can't recall how accepted this position was. The 22 concepts below (including the core module) aren't part of the core release, but clutter up the heirarchy. We also get enquiries about this content, some of which is non-existant/available. - Thoughts please from everyone on whether or not this proposal would have any impact (negative or positive) on the International Edition?
- Ready to close down?
|
63 | Proposal to increase the level of metadata available for authors to log decisions made during content authoring | | This is a subject that would be helpful to include Jim in the discussions, as he has some definite opinions on how to improve the metadata in this area.
Some suggestions would be to make more detailed information available for authors to describe their reasons for inactivation (especially in those areas where currently they are forced to use inactivation reason codes that aren't completely representative of the reasons in that instance). Adding Jim Case - ... |
64 | Discussion on the conflict between Extension content and International content | All Jim Case | The answer to this may be quite simple: - If extensions promote content via RF2 delta, we just need to retain all ID's, and only change the ModuleID and effectiveTime, and therefore it is all managed by effectiveTime.
- If IHTSDO reject content this is also managed
- The only issue then comes if IHTSDO want to change the FSN, then we need a way to manage the change of the meaning of the concept without creating 2 FSN's - as then we need a feedback loop to ensure that it's also corrected at source in the extension as well as in the International edition.
TRAG to continue the discussion and come to a conclusion that will work for all. - Has this been answered in its entirety by Jim's new agreed approach? (link here to his final position)
- Most people consider that Jim's approach covers this under most circumstances. We also need to ensure that we follow the approach listed to the left - so we should confirm all of this has been working in practice since April 2018, and if so close down?
-
- OR, do we have any new requirements here in order to ensure that Promotion/Demotion works efficiently once we move to more frequent delivery?
-
- In addition to this, we have had several issues with Promotions recently, with concepts being promoted without the related components (descriptions, relationships, etc) - so perhaps it's worth writing a full process document on exactly how, when and why content should be promoted + all related tasks that must take place at the same time in order to ensure a smooth and accurate promotion?
|
65 | AG Declarations of Interest | All | Could each of you please go in and update your information? If there has been no change, then you can simply update the last column with the date. |
66 | Any other questions / issues? | All |
|