Conflicts of interest in Meteoriks juries, and how we're dealing with them
category: general [glöplog]
Howdy all!
There's a difficult topic that the Meteoriks team has had to deal with time and time again, and in the interest of transparency I'd like to take some time to explain the problem, how we're approaching it, and why we're taking that approach.
We'd like the jurors for a given category to be as experienced in that field as possible, so that their decisions will be based on strong knowledge of the field, and the results will be backed by their reputation. Unfortunately those folks who are best suited for the job are also usually very active and productive in the scene, and so there is the occasional situation where a juror for a given category has also made a prod in the previous year that may get considered for nomination, and we would also like to maintain our integrity as an award and make sure we can't be rightfully accused of monkey business :)
This year we're in a similar situation again: We have a few jurors who volunteered, who are super motivated and who are a perfect match for some of our categories, but who have also released something in 2020 that may end up being nominated. We have taken that opportunity to discuss conflicts of interest internally and how to approach them, and the end result is that we are going to go with the same approach we've taken in similar situations in the past: We allow those jurors to be in the jury, but they do not take part in discussions or polls regarding prods they had a part in - those are left to the remaining jurors in the jury.
Our rationale for picking this approach over other, more "pure" approaches is as follows:
Removing the juror in question from the jury would cause more harm than it might prevent - unless we have other equally knowledgable, capable and motivated jurors standing by to replace them, removing them would lead to lower-quality results due to the remaining jury being fewer people, having less diverse opinions and less total skill. Assuming the juror can be trusted to behave in a way that preserves the jury's integrity (we're all mostly adults after all) the added risk of keeping them, with the rule above in place, is small in comparison.
Excluding the prod in question from the category would significantly skew the results - we would no longer be nominating the best prods of the year.
Any jury's results go through a week of internal review by all jurors across categories before we announce them, so we have a second layer of scrutiny on top of the "no selfvoting" rule.
All that said, of course there's no black or white and there's a continuum of greyscale. In each case we're weighing the pros and cons of keeping a juror in a category against those of removing them, and if we had an overabundance of jurors who would be a good fit for a given category we would be more inclined to remove someone from a jury and avoid the problem entirely - but skilled and motivated jurors don't grow on trees unfortunately.
It is not an easy topic, and honestly we would prefer not to have to think about stuff like this in the first place, but given how it's a truth we have to deal with, we think we picked the best approach all things considered. In the interest of being fully transparent we're letting you know here ahead of time, and when we make the list of jurors public after the show we will also clearly mark those jurors who were involved with any nominees in their own category, and again explain how we counteracted those apparent conflicts of interest.
Cheers,
dojoe for the Meteoriks team
There's a difficult topic that the Meteoriks team has had to deal with time and time again, and in the interest of transparency I'd like to take some time to explain the problem, how we're approaching it, and why we're taking that approach.
We'd like the jurors for a given category to be as experienced in that field as possible, so that their decisions will be based on strong knowledge of the field, and the results will be backed by their reputation. Unfortunately those folks who are best suited for the job are also usually very active and productive in the scene, and so there is the occasional situation where a juror for a given category has also made a prod in the previous year that may get considered for nomination, and we would also like to maintain our integrity as an award and make sure we can't be rightfully accused of monkey business :)
This year we're in a similar situation again: We have a few jurors who volunteered, who are super motivated and who are a perfect match for some of our categories, but who have also released something in 2020 that may end up being nominated. We have taken that opportunity to discuss conflicts of interest internally and how to approach them, and the end result is that we are going to go with the same approach we've taken in similar situations in the past: We allow those jurors to be in the jury, but they do not take part in discussions or polls regarding prods they had a part in - those are left to the remaining jurors in the jury.
Our rationale for picking this approach over other, more "pure" approaches is as follows:
All that said, of course there's no black or white and there's a continuum of greyscale. In each case we're weighing the pros and cons of keeping a juror in a category against those of removing them, and if we had an overabundance of jurors who would be a good fit for a given category we would be more inclined to remove someone from a jury and avoid the problem entirely - but skilled and motivated jurors don't grow on trees unfortunately.
It is not an easy topic, and honestly we would prefer not to have to think about stuff like this in the first place, but given how it's a truth we have to deal with, we think we picked the best approach all things considered. In the interest of being fully transparent we're letting you know here ahead of time, and when we make the list of jurors public after the show we will also clearly mark those jurors who were involved with any nominees in their own category, and again explain how we counteracted those apparent conflicts of interest.
Cheers,
dojoe for the Meteoriks team
This is exactly how it is done in some academic grant review committees as well.
I don’t know what the best answer is, but it means a lot that so much care is taken with this. Kudos to Dojoe and the organisers. I hope this is resolved fairly easily.
Quote:
This is exactly how it is done in some academic grant review committees as well.
Good to know there's precedent - thanks :)
Quote:
I hope this is resolved fairly easily.
Well it is, we did pick our answer - see third paragraph starting in "This year...". But we figured it important to let y'all know about the options that were on the table, which one we chose and why.
Actually, there's also precedent in academic circles for trying to exclude all grant reviewers, whose *institution* had submitted a proposal to the call i.e. they tried to be super paranoid about conflict of interest. It didn't work - rather clueless people ended up judging the top applications, for example giving poor marks for being "unfeasable" (maybe to them, but not necessarily to top applicant).
So, what you're doing is just fine.
So, what you're doing is just fine.
happened in the scene.org awards jury many times as well, but didn't matter as fairlight would win the category anyway :P
In academia, grant review panel members are typically required to declare all conflicts of interest. One's own work is the most obvious but it can and usually does exclude the review of grants written by people you've recently collaborated or published (produced) with.
It can go on and on though.. it just depends on how deep your judging panels are. That'll define what options you have and how far you can go.
It can go on and on though.. it just depends on how deep your judging panels are. That'll define what options you have and how far you can go.
Thanks for your insights and support! Seeing the positive feedback here strengthens our belief that we're going with the right approach.
We'll continue to be vigilant and transparent about these things.
We'll continue to be vigilant and transparent about these things.
What ruairi said. Thanks!
dudes <3
