Page tree
Skip to end of metadata
Go to start of metadata

Team testing of the Content Tracker Prioritisation Framework. 

Tracker detailsCathy - PriorityCathy - EffortDaniel - PriorityDaniel - EffortLinda - PriorityLinda - EffortElaine - PriorityElaine - EffortComments

IHTSDO-162 - Getting issue details... STATUS

CI: 1

UI: 2

TI: 3

Result: 2

Effort: 3

Time: 3

Expert: 3

Tooling: 1

Result: 2. 5

1/1/1=12/3/2/1=2  1/2/3=2 3/3/1/1=21/2/2=1.672/3/2/1=2

Priority: ratings range from 1-2.

  • Broader range of results here.

Effort: ratings 3 at 2, 1 at 2.5.

IHTSDO-376 - Getting issue details... STATUS

CI: 1

UI: 1

TI: 2

Result: 1. 3

Effort: 2

Time: 3

Expert: 2

Tooling: 1

Result: 2

2/1/1=1.332/3/3/1=2.25 1/2/2/=1.67 3/2/3/1=2.251/1/2/=1.332/3/3/1=2.25

Priority: 1.33- 1.67

Effort: 2 - 2.25

IHTSDO-712 - Getting issue details... STATUS

CI: 2

UI: 1

TI: 1

Result: 1.3

Effort: 1

Time: 1

Expert: 2

Tooling: 0

Result: 1

2/1/2=1.672/2/3/1/=2  2/2/2=22/2/3/2=2.25

Priority: 1.33 - 2.

  • Slightly broader range of results here, thought CRI for TI- could run with a 2 bringing up to 1.67.

Effort: 1- 2.5

  • Broader range of results here.

Intermediate primitive concept.


IHTSDO-240 - Getting issue details... STATUS

  2/1/1=1.332/2/2/1=1.75  2/1/1=1.332/2/2/1=1.75

Priority: 1.33

Effort: 1.75

Priority and Effort ratings match.

 

IHTSDO-981 - Getting issue details... STATUS

CI: 1

UI: 1

TI: 1

Result: 1

Effort: 2

Time: 1

Expert: 1

Tooling: 1

Result: 1.25

       

IHTSDO-953 - Getting issue details... STATUS

CI: 1

UI: 1

TI: 1

Result: 1

Effort: 2

Time: 1

Expert: 2

Tooling: 1

Result: 1.5


       

IHTSDO-867 - Getting issue details... STATUS

CI: 2

UI: 3

TI: 3

Result: 2. 67

Effort: 3

Time: 3

Expert: 3

Tooling: 2

Result: 2.75

    3/3/3=33/3/3/2=2.75

Priority: 2.67- 3

Effort: 2.75.

IHTSDO-703 - Getting issue details... STATUS

CI: 3

UI: 2

TI: 2

Result: 2.33

Effort: 3

Time: 3

Expert: 3

Tooling: 3

Result: 3


2/1/3=2

3/3/3/3=3

  2/2/3/=2.333/3/3/3=3

Priority: 2 - 2.33

Effort: 3.

DKA: Borderline impossible?

CRI: on EAG agenda for October

IHTSDO-351 - Getting issue details... STATUS

CI: 2

UI: 2

TI: 1

Result: 1.67

Effort: 2

Time: 3

Expert: 2

Tooling: 1

Result: 2

2/1/1=1.332/2/2/1=1.75  3/1/1=1.672/2/2/1=1.75

Priority: 1.33 - 1.67

Effort: 1.75 (Daniel and Elaine)

CRI: Given range in priority rating I tried this one also.

EWO: Is also related to IHTSDO-756 Rework hierarchy: Dermatitis. Allergies one of priorities in work plan so should this then be an automatic 3 though this particular one is small in scale?

IHTSDO-633 - Getting issue details... STATUS

      1/1/1=11/1/1/1=1EWO: Should be Fast track and simple one for authors to go ahead with.

IHTSDO-347 - Getting issue details... STATUS

CI: 2

UI: 3

TI: 3

Result: 2.67

Effort: 3

Time: 3

Expert: 3

Tooling: 3

Result: 3

    2/3/3=2.673/3/3/3=3

Priority: 2.67

Effort: 3

CRI: A key piece of work

IHTSDO-761 - Getting issue details... STATUS

CI: 2

UI: 3

TI: 3

Result: 2.67

Effort: 3

Time: 2

Expert: 1

Tooling: 1

Result: 1.75

    2/3/3=2.673/3/3/3=3

Priority: 2.67

Effort: 1.75 - 3.

EWO: Situations work has stalled (item on content development roadmap)

This item is blocking a number of other tracker items and this associated work is large scale.

IHTSDO-789 - Getting issue details... STATUS

         

IHTSDO-969 - Getting issue details... STATUS

         

IHTSDO-512 - Getting issue details... STATUS

CI: 1

UI: 3

TI: 3

Result: 2.33

Effort: 2

Time: 1

Expert: 2

Tooling: 2

Result: 1.75

       

IHTSDO-927 - Getting issue details... STATUS

CI: 2

UI: 1

TI: 1

Result: 4

Effort: 2

Time: 1

Expert: 1

Tooling: 1

Result: 1.25

       

IHTSDO-865 - Getting issue details... STATUS

CI: 1

UI: 1

TI: 1

Result: 1

Effort: 2

Time: 1

Expert: 1

Tooling: 0

Result: 1

    1/2/1=1.332/2/3/1=2

Priority: 1 - 1.3

Effort: 1 - 2

IHTSDO-504 - Getting issue details... STATUS

      2/3/2=2.332//3/2=2.5 

IHTSDO-178 - Getting issue details... STATUS

CI: 2

UI: 2

TI: 3

Result: 2.33

Effort: 2

Time: 2

Expert: 2

Tooling: 1

Result: 1.75

       

IHTSDO-741 - Getting issue details... STATUS

CI: 1

UI: 1

TI: 3

Result: 1.66

Effort: 2

Time: 1

Expert: 1

Tooling: 2

Result: 1.5

       

Analysis of testing work undertaken

Performed by DKA.

  • No labels

3 Comments

  1. My apologies to all. The additional tracker issues will be posted on Wednesday. Regards, Cathy

  2. Daniel Karlsson Elaine Wooler Linda Parisien

    Hi all, I haven't at this point had any updates to the revised framework, except for a couple of minor ones I made.

    Priorisation testing is ready to run. I have made each of us a subpage to this one so we can work on the testing a bit more independently.  We can bring our results across prior to the meeting. We each have 14 to review- the blue cells. Post here if there are questions.

    Happy testing! Cathy

  3. The inter-rater agreement for our study (so far, data set not complete) showed that we we're doing reasonably good. The ones with a lower score we might want to have a look at, or possible it has been addressed by the changes done after the initial work.

    The details for the agreement measure calculation (Krippendorff's Alpha, kind-of-similar to Cohen's Kappa):

     Agreement, Alpha (95 % CI)
    Clinical impact0.64 (0.35 , 0.93)
    User impact0.24 (-0.18, 0.67)
    Terminology impact0.71 (0.32, 1)
    Clinical+User+Terminology0.71 (0.48, 0.94) 
    Effort (Complexity)0.65 (0.29, 1) 
    Time0.48 (0.17, 0.80) 
    External input-0.05 (-0.43, 0.33) 
    Tooling changes0.49 (0, 0.99)
    Effort+Time+Externals+Tooling0.41 (0.02, 0.80)