It is currently Thu Nov 27, 2014 5:25 am

All times are UTC - 5 hours [ DST ]




Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 7 posts ] 
Author Message
 Post subject: sandbox Contest Discussion
PostPosted: Wed Oct 01, 2008 4:12 pm 
Offline

Joined: Sun Apr 30, 2006 6:22 am
Posts: 7
Location: the Netherlands
Congratulations to all the winners and participants. The overall winning Bonsai by: Min Hsuan Lo, is simply amazing and deserved to win in this competition as like it deservedly won the CERTRE AWARD as well! Beautiful Bonsai Art, BRAVO!
Now I do now, that you cant debate about taste, but there is one thing that immediately caught my eye looking over the jury score sheets. I would love to hear the explanation for the big differences between the scores. I think that this would also be helpful to the novice bonsai enthusiast, who now is faced with some substantional mixed messages about what is considered no bonsai and good bonsai.
Regards,
Hans van Meer.




Entry Category Score
left: W. Valavanis
Middle: M. Invernizzi
Right: L. Bebb

aob_054 Conifer___ 6 0 1
aob_086 Conifer___ 9 2 2 winner of price.
aob_087 Conifer___ 7 not bonsai 4
aob_098 Shohin____ 8 2 4
aob_107 Display____ 7 0 1
aob_124 Landscape_ 8 3 3
aob_142 Display____ 7 0 1
aob_143 Display____ 6 0 2
aob_146 Landscape_ 6 1 1
aob_154 Display____ 8 2 3
aob_178 Broadleaf__ 6 2 1
aob_204 Shohin____ 8 2 4
aob_230 Shohin____ 7 1 2
aob_231 Broadleaf__ 7 2 3


Top
 Profile  
 
 Post subject: Re: Contest Discussion
PostPosted: Wed Oct 01, 2008 6:35 pm 
Offline

Joined: Wed Oct 01, 2008 6:22 pm
Posts: 1
Location: Brazil
Congratulations to all winners and participants.

Just like the Hans van meer, I would very much like to understand the reason for this difference so pronounced. Without debunk the judges, of course, had technical criteria for trials or just personal taste?

I signed up two plants:

AoB_079:

William Valavanis Scored: 6
Marco Invernizzi Scored: 2
Lindsay Bebb Scored: 7
otal Points: 15
Average Score of all Judges: 5.00

aob_095:

Judge William Valavanis Scored: 3
Judge Marco Invernizzi Scored: 2
Judge Lindsay Bebb Scored: 3
Total Points: 8
Average Score of all Judges: 2.67

All notes granted to the second plant are compatible and can not fit any comment. In the first plant, the note issued by the second judge, totally different from the others. I'm having a terrible question: Who is right?

Regards,

Sergivaldo


Top
 Profile  
 
 Post subject: Re: Contest Discussion
PostPosted: Thu Oct 02, 2008 7:01 am 
Offline

Joined: Thu Oct 02, 2008 6:55 am
Posts: 1
Location: Bucharest, Romania
First of all, I congratulate the winners... indeed, wonderful trees and displays (I hope to get there someday, as we always hope for the best).

Many thanks to all those who made this contest possible. And many thanks to the judges, who gave me the opportunity to see my work evaluated by the professionals. It is a great way to learn something (about yourself, and about your trees).

By the other hand, indeed there were some big differences between the judges scores on some entries. I don’t wanna comment anything, but it’ll be interesting to know the scoring technic.

Thanks again, AoB.


Top
 Profile  
 
 Post subject: Scoring Discussion
PostPosted: Fri Oct 03, 2008 11:15 pm 
Offline

Joined: Sat Jan 29, 2005 2:11 am
Posts: 6469
Location: Michigan USA
To address the comments concerning the scoring of this contest and what some have seen as an unusual discrepancy between the individual judges and the scores given as compared to the scores given by other judges:


When we first decided to host contests here at AoB we knew that we would have to address the flaws that were common in all other Internet bonsai contests in order to raise the level, integrity, and professionalism of internet bonsai contests to fit with the philosophy of our project. We knew that in order to create contests that would draw the best artists in the world and set the standard for all future contests, we would have to address these flaws. After much discussion, the two major flaws we wanted to address first were as follows:

  • Popular (popularity) Voting.

    Most every forum that hosted a contest in the past used the members of the forum as the judges, allowing each member to vote on their favorite entry. While this may be fun, the results were never representative of actual quality.

    Besides the fact that the majority of the voters were of the beginner to intermediate level, certainly not qualified to judge such a contest, it was quite common for voters to give high scores, not to the entry, but to the person they liked and low scores to those they did not. Some forums tried to address this obvious biased “judging” method by making the contest blind, meaning that the entrants name was hidden until after the judging. This only made matters worse, as friends would tell friends which entries were theirs and much guessing was done using backgrounds, photo quality, and other such clues to identify the other entries.

    In some other rare cases, the moderators themselves conspired behind the scenes to assure that favorites or even their own entries received favorable scores.

    To address this we decided to use only world-renowned bonsai artists to judge our bonsai contests, leaving even the editors here unable to influence the outcome. We selected judges that not only had the experience to judge, but that also who had the integrity, honesty, and reputation to judge fairly and without bias.

    We also considered the possibility that someday, in spite of our best precautions, we might get a judge that was indeed biased. To counter this remote possibility we decided never to have less than three judges of the same caliber and to add all judges’ scores together, divide them by the number of judges, and use the resulting average score of all judges to determine the winners.

    Thinking ahead to the possibility of ties, we decided that ties would be broken whenever possible by bringing in yet another world-renowned artist to judge the tied entries individually.

    There is no other internet contest anywhere who uses the quality of judges, the number of such judges, or the other methods we do to assure a fair, impartial, and unbiased judging process.

  • Lack of Transparency

    We also noticed that except for votes that were posted, which in many cases could be altered by management or moderators, the inner working of the contest judging, entry selection process, and other pertinent information related to the contest, were kept hidden from the public. We felt that this lack of transparency, even if innocent, led many to question results, thereby casting a shadow of doubt on any results.

    To address this we created judging sheets that the judges used to score and comment on entries. Once the results of the contest were released, we published each judging sheet for all to see. This allowed not only the public to see that all scores remained unchanged, but it also allowed the judges themselves to verify that their actual scores were used.

    We created a scoring sheet in which all the judges’ scores were listed, as well as the math combining them and dividing them for the total average score used to determine the winners. Again, once the results were announced, this scoring sheet was published for all to see.

    Besides these measures, every submission email, every email sent to us by the judges or entrants, and every communication the sponsors of the contest and the editors here are stored on our ftp server for future reference. Every discussion in the editors’ forum here at AoB are stored and were made available for review by the managing judge for this contest and they are open for review by our founders, advisors, and editors at any time.

    We knew that in order to meet the high standards expected of us, to retain our high level of integrity, and to exceed the expectations bonsaists around the world have for these record setting contests, every step of the contest would have to be documented and available for review at any time. This is what AoB is all about.

Using these new ideas, as well as many others, we wrote a set of rules that has been praised by some of the most experience artists in the world. They have been used as a foundation for other contests by other forums, and the base was used to create the rules for our record setting article contest recently.

Yet, we still dissect the rules each year, tweak and adjust them, and add to them when needed. We are determined to continue setting the standards, content, discussion, and contests here at AoB and input from our readers guide us in these endeavors.


The issue at hand, bought up by some of these very readers, concerns what some see as a huge discrepancy in the scores of the judges. A few readers have expressed a concern that on some of the entries the judges’ scores vary greatly.

In this years’ contest, our main judges are from Italy, Australia, and America, while our tie breaking judges are from Germany and Japan. Our managing judge is from America as well. All of the judges are respected in the bonsai community and their names read like a “Who’s Who” of bonsai masters. Each have judged more shows and events than most of us have ever attended. (A complete list of the judges for this contest can be seen at http://artofbonsai.org/galleries/aobawards2008.php )

It should be understood that each of these judges has individual styles and tastes, some lean more toward the traditional and others more toward the modern. Each has undergone different methods of training and development and each have been influenced in different ways throughout their career. Each comes from different cultures. In short, each judge is an individual with different expectations, tastes, and preferences.

This is exactly what we wanted for our contests, a wide range of knowledge from different cultures and backgrounds used to judge a wide range of entries created by artists from different cultures and backgrounds.

Considering the above, I would fully expect that some entries would receive scores from the judges that were widely varied. To confirm this, I looked at some of our previous contests such as the North American vs Europe Photo Contest , the AoB's First Annual Display Contest , the Bonsai Today / Art of Bonsai - Photo Contest , and even the article and editorial contests. I was not surprised to find some widely varied scores in these contests as well.

Thinking about this issue, I realized that I would be much more concerned if the judges scores were exactly the same or just off by a point on every single entry. Even the mainstream art world can not come to that tight of a consensus on art.

While I can’t pretend to understand why one judge would score a tree high, while another would score it low, I do know that these judges are all experienced, respected professionals and I trust that they scored according to their best evaluation of the tree.

I, like others, would be greatly interested in hearing comments on the trees, as I am sure such insight would be educational and inspirational. However, I ask for knowledge, not justification, I ask for education, not explanation.


Top
 Profile  
 
 Post subject: Re: Scoring Discussion
PostPosted: Mon Oct 13, 2008 5:17 pm 
Offline
Editor

Joined: Mon Oct 24, 2005 8:51 pm
Posts: 1442
Location: WI, USA
Sorry Will, but I did not read a thing you wrote.

All I will say is why is everyone expecting all bonsai judges to think the exact same way the world over? Every person the we would think of making a judge for any of our contests, knows what they are talking about when it comes to bonsai. They all have their ore conceived ideas of what they like and do not like, no matter what.

No matter who the judges are they are not going to all agree all of the time.

Best of luck to everyine next time.


Top
 Profile  
 
 Post subject: Re: Scoring Discussion
PostPosted: Tue Oct 14, 2008 12:07 am 
Offline

Joined: Sat Jan 29, 2005 2:11 am
Posts: 6469
Location: Michigan USA
Paul Stokes wrote:
Sorry Will, but I did not read a thing you wrote.


Too bad, you could have saved yourself some time.


Top
 Profile  
 
 Post subject: Re: Scoring Discussion
PostPosted: Sun Oct 19, 2008 3:15 pm 
Offline
Editor

Joined: Sat Jan 29, 2005 2:13 am
Posts: 1190
Location: Los Angeles, California
I love the fact that there is so much difference in the scores. This makes bonsai exciting, unpredictable. And considering the choices of the judges, bonsai is undoubtedly mysterious.

The scores should not be a surprise: how many times we've heard one world class painter calling another world class painter "talentless", "clumsy", and other unflattering adjectives? It happens all the time. Why shouldn't be the same in our art form?

If you want to hear why exactly Marco, Bill, or Lindsey voted the way they did, I guess you would have to take private lessons from them. It would be priceless if they would explain in detail their thinking and the reasons behind, but I think we would be asking too much. Considering the prices of high quality education today, free education is not a right, but rather a privilege.

It certainly kills me to find out why Marco voted "not bonsai", or "1" (or "2") while the other two judges gave decent marks to some of the trees, but I would just have to live with the knowledge that I will never have the answer. Instead, I should just focus on what I consider to be great bonsai.

Great show though, thanks for all the effort to organize it.


Top
 Profile  
 
Display posts from previous:  Sort by  
Forum locked This topic is locked, you cannot edit posts or make further replies.  [ 7 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Copyright 2006-2008 The Art of Bonsai Project.
All rights reserved.
Original MSSimplicity Theme created by Matt Sims © 2004
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group