Date & Time
20:00 to 21:00 UTC Wednesday 20th March 2020
Location
Zoom meeting: https://snomed.zoom.us/j/471420169
Goals
- To discuss syntax and advice for collation and folding
- To develop examples to illustrate new term searching functionality
Attendees
- Chair: Linda Bird
- Project Group: Daniel Karlsson, Ed Cheetham, Peter Jordan, Anne Randorff Højen, Michael Lawley, Guillermo Reynoso, Rob Hausam
Apologies
Agenda and Meeting Notes
Welcome and agenda ON HOLD: SCG, ECL, STS, ETL - Ready for publication, but on hold until after MAG meeting in April confirming requirement for Boolean datatype. QUESTION FROM SNOMED ON FHIR - Can/should we register ECL as a MIME type? ADDED TO DRAFT SYNTAX - Child or self (<<!) and Parent or self (>>!) Following up on our homework: UCA/CLDR/Case/accent folding + Unicode collation - What advice should we be giving in the specification? I have personally found trying to answer this torture! Ideally we want to be try and get predictable (per locale) search behaviour. This could then be neatly summed up in a sentence in the guidance something like this: “The search specification assumes that descriptions are indexed for search using the default UCA, or UCA tailored for a specific language or locale according to CLDR. The selected locale can be specified using the ‘language=[ISO 639-1 code]’ filter. Descriptions indexed this way are compared with unmodified search tokens.” However, it looks as though ‘default UCA’ doesn’t ignore case (but bafflingly how case is handled is predominantly specified using a parameter called ‘strength’!). The UCA specification states that “…Language-sensitive searching and matching are closely related to collation…”, but this also indicates that they are not the same. The required collation strength for case insensitive searching is ‘secondary’, whilst the default for collation is ‘tertiary’. This may be explained here and/or here , and is probably buried somewhere deep in here, but to me is actually most clearly described by the kind people who maintain the mongoDB documentation. If we therefore need to add something about case insensitivity to the assumption statement above (and possibly even make case sensitivity configurable in our filters), could we just say ‘“The search specification assumes that descriptions are indexed for search using case insensitive default UCA…”? From a practical point of view this is tempting (commercial product configurations seem to use the “_CI” notation when setting collation (e.g. “>>mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci"). However if we are going to reference UCA then it’s worth noting that the Unicode materials don't seem to use the phrase ‘case insensitive’. Instead they talk in terms of secondary or tertiary ‘strength’ (as does the configuration page of mongoDB). On balance I suspect that if we make case sensitivity configurable then we should name the filter ‘case=’ with values of ‘case sensitive’ and ‘case insensitive’ (implicit default). The alternative is to name the filter ‘strength’ with values of ‘secondary’ and ‘tertiary’ and so on. Whilst the latter looks more principled I suspect it’s just confusing. I’ll stop there, but will just add for info that the W3C reference we looked at last time was coming at this from a different direction. Their concern relates to string matching as it applies to the syntactic content of web pages etc. Consequently their recommendation is for a normalization step that changes nothing - to avoid changes in element names/markup. Other content (what that paper calls natural language content) may well benefit from extensive normalisation - closer to case insensitive UCA transformation. Proposed syntax to support querying and return of alternative refset attributes (To be included in the SNOMED Query Language) Proposal (by Michael) for discussion For example, I can write: << 404684003|Clinical finding| : 363698007|Finding site| = <<66019005|Limb structure| << 404684003|Clinical finding| . 363698007|Finding site| But I can't get all the attribute names that are used by << 404684003|Clinical finding| Proposal for discussion What refsets is a given concept (e.g. 421235005 |Structure of femur|) a member of? Expression Templates Examples: [[+id]]: [[1..*] @my_group sameValue(morphology)] { |Finding site| = [[ +id (<<123037004 |Body structure (body structure)| MINUS << $site[! SELF ] ) @site ]] , |Associated morphology| = [[ +id @my_morphology ]] } Note that QI Project is coming from a radically different use case. Instead of filling template slots, we're looking at existing content and asking "exactly how does this concept fail to comply to this template?" For discussion: Is it correct to say either one of the cardinality blocks is redundant? What are the implications of 1..1 on either side? This is less obvious for the self grouped case. Additional note: QI project is no longer working in subhierarchies. Every 'set' of concepts is selected via ECL. In fact most reports should now move to this way of working since a subhierarchy is the trivial case. For a given template, we additionally specify the "domain" to which it should be applied via ECL. This is much more specific than using the focus concept which is usually the PPP eg Disease. FYI Michael Chu FUTURE WORK Examples: version and dialect << 64572001 |Disease| {{ term = "*heart*" }} VERSION http://snomed.info/sct/900000000000207008/version/20180131 << 64572001 |Disease| {{ synonym = "*heart*" }} VERSION http://snomed.info/sct/900000000000207008/version/20180131 << 64572001 |Disease| {{ FSN = "*heart*" }} VERSION http://snomed.info/sct/900000000000207008/version/20180131 << 64572001 |Disease| {{ preferredTerm = “*heart*”}} VERSION http://snomed.info/sct/900000000000207008/version/20180131, DIALECT Y << 64572001 |Disease| {{ acceptableTerm = “*heart*”}} VERSION http://snomed.info/sct/900000000000207008/version/20180131, DIALECT Y (* {{ term = "*heart*" }} VERSION http://snomed.info/sct/900000000000207008/version/20180131, DIALECT Z) MINUS Notes Next meeting is scheduled for Wednesday 22nd April 2020 at 20:00 UTC.Concrete values Linda Bird Expression Constraint Language Linda Bird
999000691000001104 |National Health Service realm language reference set (pharmacy part)| ) }}
2. Constructing a Language Refset from other Language RefsetQuerying Refset Attributes Linda Bird
FROM 799 |Anatomy structure and part association refset|
WHERE 123 |referenced component| = (< 888 |Upper abdomen structure| {{ term = "*heart*" }} )
FROM concept
WHERE id IN (< |Clinical finding|)
AND definitionStatus = |primitive|
FROM concept, ECL("< |Clinical finding") CF
WHERE concept.id = CF.sctid
AND definitionStatus = |primitive|
FROM concept ( < |Clinical finding| {{ term = "*heart*" }} {{ definitionStatus = |primitive| }} )
(|Anatomy structure and part association refset| {{ |referencedComponent| = << |Upper abdomen structure}} )? |targetComponentId|
734139008 |Anatomy structure and part association refset|
734139008 |Anatomy structure and part association refset|
734139008 |Anatomy structure and part association refset| :
449608002 |ReferencedComponent| = << |Upper abdomen structure|
734139008 |Anatomy structure and part association refset|
{{ 449608002 |referencedComponent| = << |Upper abdomen structure| }}
734139008 |Anatomy structure and part association refset| :
449608002 |ReferencedComponent| = (<< |Upper abdomen structure|) : |Finding site| = *)Returning Attributes Michael Lawley Reverse Member Of Michael Lawley Road Forward for SI
Description Templates Kai Kewley Query Language
- Summary from previous meetings
(* {{ term = "*heart*" }} VERSION http://snomed.info/sct/900000000000207008/version/20170731, DIALECT W)Confirm next meeting date/time
1 Comment
Daniel Karlsson
Here is the table for Swedish (latest version as of today):
https://unicode-org.github.io/cldr-staging/charts/37/collation/sv.html
The search part of the table is where the interesting stuff is rendered, from this:
https://github.com/unicode-org/cldr/blob/master/common/collation/sv.xml
It imports some information which I cannot find right now, and it's late...
/Daniel