Pages

Thursday 12 October 2023

Multisource Feedback and Systematic Bias

 

Multisource Feedback is currently an established method to obtain information on individuals at workplace. It is an essential component to be able to continue careers for doctors in the UK as it is a requirement for revalidation to remain on the medical register.



On colleague MSF a GMC report states:



‘There was evidence of systematic bias amongst some groups of colleague respondents in respect of assessments provided’



‘Certain doctor characteristics appeared to predict systematic variation in colleague assessments.’



‘In interpreting data arising from such surveys of colleagues, consideration should be given to the possibility of systematic bias in a colleague’s report based on non-clinical aspects of care, as well as the extent of colleagues’ familiarity with the doctor.’

 

On Patient MSF a GMC report states: 


'There was evidence of systematic bias in reporting amongst some groups of patient
respondents. '


'In interpreting data arising from such surveys of patients, consideration should
be given to the possibility of systematic bias in a patient’s report based on nonclinical aspects of care, as well as the socio-demographic profile of the patient
sample'



Software systems that are used for MSF have begun to use benchmarking – it is not clear if these benchmarks are statistically adjusted or standardized to consider and recalibrate for these biases. It is also not known whether the software systems identify these biases and inform the doctor that the responses could be impeded by biases.



It is not clear if doctors undergoing revalidation for GMC licence to practice are explicitly informed of these biases and offered guidance and support on how to choose MSF respondents to negate the existing biases or counselling on how to recognise, challenge and deal with any biases that becomes apparent.



Appraiser training does have an EDI component, but it is unsure if this training includes methods on how to consider these biases including any undue positive biases?



There are publications that suggest that the MSF could be one of the triggers that may be used when considering performance. This makes any issues regarding bias paramount to the topic.



It is very important to prevent issues – unfortunately it may be the case that the MSF methodology as currently implemented for doctors (and perhaps other health professionals) has a built-in flaw in the structure when it does not consider and adjust for biases.



Doctors should be aware of this and be prepared to address this issue should it affect them. Doctors’ organisations should resolve this at the earliest opportunity.

 

 

 

Ref:

  https://pubmed.ncbi.nlm.nih.gov/23095930/ 

 https://www.gmc-uk.org/-/media/gmc-site/about/rtguidanceresearchandpilotingofgmcquestionnairesdc8593pdf64915124.pdf

Friday 26 March 2021

A surgeon's and QI enthusiast's views on COVID in UK


 

My views in the video covers following topics

Indian Origin Doctors - Backbone Of NHS (National Health Service) In The UK Approx. 65,000 Indian Origin Doctors In The NHS, 

UK Role Of Surgeons In Treating COVID-19 Patients Surgeries Have Reduced Adaptability Is Key Business Of Surgeons Is Virtual Impact Of COVID-19 On Business Of Surgeons High Stress On Surgeons 

Helping BAME (Black, Asian and Minority Ethnic) Community Fight , COVID-19 BAME Healthcare Staff Affected The Most 

Slow Uptake Of Technology In Healthcare, Pre-COVID Implication Of Using AI And Technology In Treating 

BAME Community Preparedness To Fight A Pandemic Like COVID-19 

Focus on Localism And Temper Ambitions On Globalism 

Can A Pandemic Worse Than COVID-19 Hit The World? 

New Forms Of Social Behaviour 

Robotic Operations Already Available 

The New Normal Post-COVID Struggle With New Normal Healthcare Going To Change Drastically Technology To Play A Vital Role

Wednesday 25 November 2020

Design is the key for human factors. Behaviour is a small bonus.

 Design is the key for effective human factors


In a recent famous case, one surgeon and two anaesthetists were said to be anaesthetising two patients at the same time; this would be risky and unnecessary for patients to be anaesthetised for longer than absolutely essential. It is unethical.


https://theworldnews.net/gb-news/derek-mcminn-patients-put-in-danger-so-scandal-hit-surgeon-could-perform-two-operations-at-same-time


The question is: how did this even happen? There must be policy in place to prevent this. There must have been people who could and should have questioned this and prevented this. Sure. Let’s assume we had policies and people in place – do they prevent for sure two patients being anaesthetised for the same one surgeon at the same time? No.


It happened because there was the structure, infrastructure and facilities to do it.


In UK hospitals there is something called the anaesthetic room which is separate from the operation theatre. This means for one surgeon, there could be a patient anaesthetised inside the operation theatre and another patient anaesthetised in the anaesthetic room.


This is a fairly unique UK NHS practice. Historically, the subsequent patient was brought into the anaesthetic room and the process of anaesthesia began or anaesthesia given when the patient on the table in the operation theatre was nearly done. This was thought to be efficient. It worked when the so called ‘registrars’ both anaesthetic and surgical were experienced. This anaesthetic room concept was then followed by UK private hospitals.


The presence of the anaesthetic room means that it would be physically possible for two patients to be under anaesthetic simultaneously for a single surgeon.


Recently, when designing the theatres of a private hospital, we argued for not having an anaesthetic room and prevailed.


This means that in that private hospital which does not have an anaesthetic room, there is no possibility of two patients being under an anaesthetic at the same time for a single surgeon because there is no physical infrastructure/facility that enables/allows it. No policy or person(s) would have been able to achieve this.


We cannot design a problem to be built into a system and then expect policies and people to overcome it consistently.


Design is the fundamental for human factors – people and behaviours are simply an add on bonus.



© Hemadri



Saturday 1 December 2018

UK postgraduate medical examination pass rates – what are your chances if you an IMG/BME?

The GMC annual survey report for 2018 is now published (https://www.gmc-uk.org/about/what-we-do-and-why/data-and-research/national-training-surveys-reports#). While the GMC’s focus seems to be on the training environment, it is important to delve into the online reporting tool (https://webcache.gmc-uk.org/analyticsrep/saw.dll?Dashboard ) where the detail resides.

I have a long interest in poor outcomes for discriminated groups and looked at the pass rates for postgraduate examinations. The results are still not that great or even for IMGs and BMEs. Overall, if you are a UK graduate with a white ethnicity – then the pass rate is 75.1% and if you are an IMG-BME the pass rate is 42.2% and if you are an EEA-BME then it falls to 37.2%. Basically if you are white UK graduate you have a 100% higher pass rate than some BME doctors. We can argue about the unfairness of this and have done so in the past.


However, let us focus on UK graduates alone for now. Overall there a white UK graduate has a 12 percentage point higher pass rate than a BME UK graduate. That is pretty gross. When you point this out, the classic false arguments based on sub-classification starts (schools in UK, candidates from abroad joining UK medical schools, etc) which essentially devalues the many years of UK medical school teaching and training.

Then if we start looking at how individual specialties are faring in their faculty and royal college pass rates, we find that most have an approximately 12 point difference biased towards white UK graduate doctors and biased against BME UK graduate doctors.

There are one postgraduate examination where UK white doctors and UK BME doctors have more or less even pass rates (with only a 3 percentage point lower for UK BME doctors) – that would be the Faculty of Public Health examination.


Then…

There is one postgraduate examination where UK BME graduates fare better than the UK white graduates by 10 percentage points. That would be the Faculty of Occupational Health Examination.




The biggest difference between UK white doctors and UK BME doctors is the RCGP examination where there is a 14.5 percentage point difference in favour of UK white doctors.




It is a sad fact of UK healthcare that such differences exist.

Overall if you are an IMG-BME you have a 57.8% failure rate in UK postgraduate examinations according to the GMC’s latest report.

Due to the medical staff vacancy crisis in the UK, the powers that be have increased the ‘MTI’ duration from two to three years with a view to attracting more IMGs to UK. The IMGs would hope to obtain some UK qualifications – though the reasons they are recruited is because they are already qualified specialists in the first place.

Make wise decisions.

Would you put your money in a scheme where you have a 57.8% chance of losing it? That is the downside. On the upside, you do have a 45.2% chance of not losing your money and in the bargain getting your qualification – what happens then? You need a more detailed analysis of what your chances of getting into a training post, chances of returning to the country of origin without a CCT/CESR, getting back to your country of origin with a CESR, etc.

As a doctor with a scientific qualification data based decision making would be recommended. Think deeply.

Also think deeply of the impact of the process, – knowing that you have a high chance of failing examinations, knowing about the realities of career progression – even if you were eventually successful in your examination or career progression.

Think deeply, very deeply. Make wise decisions. It is not easy but it does affect your life.



©M HEMADRI


Follow me on Twitter @HemadriTweets 


If you are a doctor wanting interview skills training for job interviews you may want to check out my website:

https://successatmedicalinterviews.co.uk/

We also offer highly bespoke  Leadership and Management training and unique QI skills