| Stranica u temi: < [1 2 3 4] > | HowZ your K/Q? Postavljač teme: mediamatrix (X)
|
|---|
Hilde Granlund wrote:
The point that gets people so worked up is something I did not even realize until now: your KudoZ activity determines your position in the directory, and therefore probably your likelihood of being offered jobs.
It seems indeed unfair that it is based on points total, since in many cases it will be difficult for newcomers to measure up. How about changing it to basing the listing on activity in the last year (or some other suitable period)?
Hi Hilde, this is a great idea!
I agree that positioning in the directory should take into account only the points accumulated during the last year, thus leveling the field to old-timers and new arrivals.
Thanks a lot,
Enrique
| | | | | Nothing wrong with the system | Aug 8, 2007 |
“Unfair”?! What’s unfair about this? Some people who contribute to Kudoz work hard to provide proper answers so why not reward them for their work? If you answer hundreds of questions over a period of several years spending time and effort I don’t see why it’s unfair that these people are being rewarded for it? A person who has contributed to Kudoz for five years is more likely to have a better ranking in the directory. So what? They worked for it. There are two people who gave brillia... See more “Unfair”?! What’s unfair about this? Some people who contribute to Kudoz work hard to provide proper answers so why not reward them for their work? If you answer hundreds of questions over a period of several years spending time and effort I don’t see why it’s unfair that these people are being rewarded for it? A person who has contributed to Kudoz for five years is more likely to have a better ranking in the directory. So what? They worked for it. There are two people who gave brilliant answers in the legal field (English/German) but who stopped participating a long time ago. I still benefit from answers they gave years ago much more than from many answers given now.
Also, it is not difficult to get exposure in the directory even for newcomers and they should accept that others who contributed to the community much longer are perhaps in a better position. There is nothing wrong with this system IMO. However, the quality of Kudoz answers is of some concern and needs to be addressed.
The thread is actually about showing a “reliability ratio” or whatever you want to call it and I agree this would be a good idea and could lead to better answers although when you see the number of questions asked and the points earned you can figure it out yourself.
Hilde wrote:
“And the person topping the list could be someone who was extremely active three years ago, but has since disappeared from the scene - theoretically?
Would it not be more useful to outsourcers as well to see who is currently very active - in other words: keen to get a job”
Er – no, it would not. Why would it? Outsourcers don’t care about Kudoz. Have a look at your visitors feature and you will see that outsourcers type in their key words and then look at your profile, not at your Kudoz activity.
Again, it is the quality of the answers that makes them valuable, not the time when they were given.
And as Steffen pointed out, all these features where you can see the activity in the past 12 months etc. have already been implemented.
[Edited at 2007-08-08 12:11] ▲ Collapse
| | | | | Kudoz imperfections are fact, not fiction (a bit long, sorry...) | Aug 8, 2007 |
Hi,
Sorry, MV LegalTrans, but obviously there are a number of things that are wrong with the Kudoz system, otherwise there would not be so many threads or complaints about it (you can make a search in the forums to check this).
After being a registered user of Proz for more than 6 years and a more active online member for almost 2 years, I feel (probably like quite a number of other prozians) that, sooner or later, once you get the hang of it, whether you are a newbie o... See more Hi,
Sorry, MV LegalTrans, but obviously there are a number of things that are wrong with the Kudoz system, otherwise there would not be so many threads or complaints about it (you can make a search in the forums to check this).
After being a registered user of Proz for more than 6 years and a more active online member for almost 2 years, I feel (probably like quite a number of other prozians) that, sooner or later, once you get the hang of it, whether you are a newbie or an old-timer member/user, if you decide to participate in it, Kudoz may become clearly one of the most frustrating features of Proz.com.
Anyone who has some minimum experience as a translator will realize soon enough that the way the Kudoz system has been handled/is still being handled (with that almost "timed race" to get the points) has generated/keeps generating rather unbalanced/extreme results, where you can find very useful entries, but also a lot of trash (for example, repetitions and clearly incorrect translations).
This is fact, not fiction, and it has been "denounced" over and over again. So far the arguments I have read from the Proz.com staff in this respect basically say that Kudoz is not supposed to be an "authoritative" terminology base, but simply a means of "help". OK, if "help" were the sole argument, then why the point/"meritocracy" system?
The point system has obviously "corrupted" the idea of help in Kudoz and has often turned it into a "battlefield" where sometimes, because of this point race, you may find yourself (unconsciously/consciously) either "defending" yourself/your answers or even "attacking" someone else's whom you may consider (rightly or not) is not contributing with the idea of "helping" another colleague (I made that mistake not too long ago). Not to mention the clique mentality and the "popularity contests" that you often witness as the behavior to assign points.
This false sense of "meritocracy" that makes Kudoz points move you up to the top of the translator directory is hardly a realistic or "fair" (yes!) system. Being a Kudoz leader may simply mean that you have a lot of free time in your hands. And being free so much is not necessarily a good sign when you work as a freelancer. Obviously the busier professionals are the more sought after and, therefore, I doubt that they would have so much time to devote to something like Kudoz (or the Proz.com forums). Of course, there may be exceptions, but I am referring to the obvious general cases.
But then you could say, if I think so apparently "negatively" about Kudoz, why bother about it at all? Why not just let others continue with this point game, forget about Kudoz and just "live and let live"?
Well, because I think it is such a waste of precious resources that a potentially much more useful tool like Kudoz keeps losing (out of frustration) so many clearly valuable intellectual/professional contributions, both from past and current members/users and, instead, is often being monopolized by those who usually care more about the point circus and not necessarily about the *quality* of the database itself, which is what would really be helfpul to *everyone* in the long run (like, for example, the database of the European Union, IATE).
So, in spite of that resounding "Let's Improve Proz" thread that was opened about a year ago (where numerous suggestions were made to improve Kudoz), I can think of yet a couple more suggestions (though the first one may sound a bit radical, but, hey, my 2 cents, right?) to improve the quality of the contents of the Kudoz terminology database:
1) Eliminate the Kudoz relation with the "meritocracy" point system that improves your position in the translator directory. Make the directory listing have a random order that is changed, for example, every 15 minutes, just like the "you are featured" box.
People could still keep their Kudoz points and, furthermore, to please those who have been or become Kudoz leaders and have thousands of points, you could add to their profiles a special icon that showed their active participation in Kudoz, like the icon of those who have won a Proz translation contest, so that when a potential outsourcer saw their profile in the translator listing, maybe they would consider as an added asset the time this person has devoted to Kudoz (though I seriously doubt that any potential outsourcer would hire someone with more Kudoz points over someone with obviously better academic preparation and more experience).
2) Why not finally concentrate on something more like Wikipedia, where a team of clearly dedicated people would be seriously in charge of editing the entries made, in order to maintain a minimum overall quality?
The way I see it, Kudoz moderators seem like the least effective solution, because in some pairs there are practically hundreds (if not thousands) of questions per day, and you cannot expect that just one or even two moderators can handle this, especially not being paid. You need a team of really dedicated people for Kudoz (obviously the number of people in the team will depend on the volume of questions of the pair). Whether they are paid or not is another matter.
I understand that Wikiwords was supposed to become something like Wikipedia. What happened to this project? I have not heard much about it for a long time. I remember I sent once an email to the person in charge of this (I believe it was Colin at the time) and I never got a reply, which made me lose all interest in following-up.
Finally (did you actually read everything else I had written so far?), I also want to make clear that I am not the "complaining type", but rather try to keep a constructive attitude about Proz.com, which I find a useful idea in general. This, of course, does not exclude that I can be upfront critical with those aspects of Proz.com that I consider improvable.
Best,
Ivette ▲ Collapse
| | | | Kirill Semenov 
Ukrajina
Local time: 17:39
Član (2004)
engleski na ruski
+ ...
| Just an example from my real life | Aug 8, 2007 |
A week ago I was contacted by a very good outsourcer (great feedback at the Blue Board) with a project for a very good company (the end client was one of the biggest mobile phone producers) and the PM mentioned that she found me due to my high position at the ProZ Directory. Great. So it makes a difference sometimes, right? It's a great chance to find a regular client, that's what, and I say `Thank you, proZ'. Any ceremonial altar around to bring my sacrifice?
| | |
|
|
|
| (Sorry, off-topic, just a short comment to Kirill) | Aug 8, 2007 |
Kirill Semenov wrote:
Any ceremonial altar around to bring my sacrifice?
I'll play that "Mataculebra" song when you are sacrificed, he he he!
P.S.: sorry, I couldn't resist...
| | | | Kirill Semenov
Ukrajina
Local time: 17:39
Član (2004)
engleski na ruski
+ ...
| Fits the best :) | Aug 8, 2007 |
ICL wrote:
Kirill Semenov wrote:
Any ceremonial altar around to bring my sacrifice?
I'll play that "Mataculebra" song when you are sacrificed, he he he!
Sure, Ivette, I will work perfectly when I burn yet another witch onto this fire. A snake-killer is just the right idea.
| | | | Konstantin Kisin
Ujedinjeno Kraljevstvo
Local time: 15:39
ruski na engleski
+ ...
| ratio > recent kudoz | Aug 8, 2007 |
Hi Hilde, this is a great idea!
I agree that positioning in the directory should take into account only the points accumulated during the last year, thus leveling the field to old-timers and new arrivals.
Thanks a lot,
Enrique
Given a choice, I would definitely choose a reliability ratio over this idea. By doing this you completely disregard any previous contributions made by Proz members.
[Edited at 2007-08-08 17:02]
| | | | Nikki Graham
Ujedinjeno Kraljevstvo
Local time: 15:39
španjolski na engleski
| Agree with Konstantin and some | Aug 8, 2007 |
Konstantin Kisin wrote:
Hi Hilde, this is a great idea!
I agree that positioning in the directory should take into account only the points accumulated during the last year, thus leveling the field to old-timers and new arrivals.
Thanks a lot,
Enrique
Given a choice, I would definitely choose a reliability ratio over this idea. By doing this you completely disregard any previous contributions made by Proz members.
Personally, I think this is a terrible idea. It will encourage all the things people complain about in Kudoz, such as: point chasing on a large scale, disagrees with a nasty edge to them, fast answers with no evidence to support them, etc., etc. etc. (see Ivette's post for info on the state of play at the moment. It would all just get worse).
And it is, again, totally unfair. Why should people who spent some time in the past answering questions, but who no longer participate (or as much) have to be penalised in this way? I can think of mothers who take maternity leave, or who simply don't have the time to look after children, work AND help colleagues all day every day (my position). And I bet lots of fathers are in the same boat. And what about moderators? Many of them have proved themselves in the Kudoz arena, but just don't have the time to moderate and answer questions AND have a life.
Please, get real.
I would just like to add, as an edit, that if some sort of reliability ratio isn't introduced, then I agree with Ivette's suggestion, as follows, for the directory, which I have often thought of as a possibility myself:
Make the directory listing have a random order that is changed, for example, every 15 minutes, just like the "you are featured" box.
[Edited at 2007-08-08 23:02]
| | |
|
|
|
Andy Watkinson
Španjolska
Local time: 16:39
Član
katalonski na engleski
+ ...
Nikki Graham wrote:
Please, get real.
Andy.
| | | | | Regarding the Listing | Aug 10, 2007 |
I also recently learned that our place in the directory was based on KudoZ points. I have to admit that I became active again when I learned this fact. After all, this is a business, and outsourcers DO use the directory to find freelancers (they've told me on several occasions). If I'm 20 places down the list, it will certainly impact my bottom line.
However, it is a little frustrating to see how people climb up that listing. I provide a very specialized service. I only trans... See more I also recently learned that our place in the directory was based on KudoZ points. I have to admit that I became active again when I learned this fact. After all, this is a business, and outsourcers DO use the directory to find freelancers (they've told me on several occasions). If I'm 20 places down the list, it will certainly impact my bottom line.
However, it is a little frustrating to see how people climb up that listing. I provide a very specialized service. I only translate Spanish to English, and ONLY medical. As such, I only answer KudoZ questions that meet that criteria. I consider my expertise in this area to be an asset to myself, my clients and to the KudoZ community. I have a K/Q ratio of 2.61 in my specialty field (and seriously, sometimes you guys choose answers that are WAY off). I've managed to climb to number 2 in the "3 month" and "12 month" columns for my specialty/pair.
However, if you do a freelancer search for my specialty/pair, I come in 6th on the list. Number 1 on the list has a K/Q ratio of 1.27. That shows some persistence (and free time), but it hardly demonstrates any expertise. If you are an outsourcer looking for a specialist translator, wouldn't you expect the freelancer listing to reflect the level of expertise rather than total number of questions answered? Many freelancers list many "specialty" fields (which really calls into question the definition of "specialty"), but there should be a simple metric that outsourcers can use to choose who really is a specialist. The K/Q ratio might be the answer, though a more complex algorithm may be needed to account for years of experience/activity.
Granted, I'm only speaking from my own little corner of cyberspace and my opinion is skewed towards my own best interests, but if one had to consider the accuracy of their responses as opposed to the volume, it certainly would improve the quality of the responses as well as the usefulness of the directory order.
It might even result in a KOG that I can use, but that's another topic altogether.
SaludoZ! ▲ Collapse
| | | | | Quality improvement | Aug 10, 2007 |
Hi Jason
You are making some interesting points. I agree this could be a way of better reflecting people’s expertise. As you said, the points total is not an accurate tool to assess a translator’s expertise. Linking the reliability ratio to the listing could make people think more before they give an answer and at the same time would take into account any efforts long standing members have made so far.
You are also right in saying that a more complex algorithm ma... See more Hi Jason
You are making some interesting points. I agree this could be a way of better reflecting people’s expertise. As you said, the points total is not an accurate tool to assess a translator’s expertise. Linking the reliability ratio to the listing could make people think more before they give an answer and at the same time would take into account any efforts long standing members have made so far.
You are also right in saying that a more complex algorithm may be needed to account for years of experience/activity. If I answer four questions only and get the points for all of them then obviously it would not be ok to be the first one on the list when someone has answered 1000 questions and has a slightly lower reliability ratio. ▲ Collapse
| | | | tazdog (X)
Španjolska
Local time: 16:39
španjolski na engleski
+ ...
| acceptance rate | Aug 10, 2007 |
I agree with the last posts by Nikki, Andy and Jason, although I'm not too keen on the idea of the directory showing freelancers in random order.
Rather than implementing the "K/Q" reliability ratio proposed in this thread, why not use the info. that is already available in our profiles but not displayed to the public? (I'm not sure if this is a members-only feature or not, but it's in the "Kudoz Activity" section--click on the link--in the "Stats" tab.) There is a breakdown by lang... See more I agree with the last posts by Nikki, Andy and Jason, although I'm not too keen on the idea of the directory showing freelancers in random order.
Rather than implementing the "K/Q" reliability ratio proposed in this thread, why not use the info. that is already available in our profiles but not displayed to the public? (I'm not sure if this is a members-only feature or not, but it's in the "Kudoz Activity" section--click on the link--in the "Stats" tab.) There is a breakdown by language pair and also by specialty field, e.g.:
Spanish to English
Questions answered 1671
Answers accepted 1151
Acceptance rate 68.8
Percent non-PRO 2.9
Percent PRO 97
Average agrees per question 1.66
Average disagrees per question 0.01
This would eliminate the issue of answers not receiving the whole 4 points, and AFAIK also covers not-for-points questions (correct me if I'm wrong here).
I know I've proposed this before, but since we're on the subject (again!), I think it's worth another mention. ▲ Collapse
| | |
|
|
|
mediamatrix (X)
Local time: 10:39
španjolski na engleski
+ ...
POKRETAČ TEME | Further observations ... | Aug 10, 2007 |
Jason Faulkner wrote:
The K/Q ratio might be the answer, though a more complex algorithm may be needed to account for years of experience/activity.
I started this thread with the simple K/Q ratio largely because it can easily be calculated by any user for themselves or for any other member/user. And in that way I hoped to trigger a debate on how we might get a more meaningful indicator than 'total points'.
I agree that a more complex algorithm might be chosen with a view to taking account of some 'other factors'. What those other factors might usefully include (and what they certainly should not include, like sex, age, country of origin or work, etc.) would be a matter for further debate...
MV LegalTrans wrote:
If I answer four questions only and get the points for all of them then obviously it would not be ok to be the first one on the list when someone has answered 1000 questions and has a slightly lower reliability ratio.
Of course - and the solution to that problem is simply to not provide statistics until the newbie has answered a certain number of questions. I've just looked at my own results for the first few days of KudoZ activity. They show that I hit a K/Q of 4 with my first question - but by question number 5 it had (already) fallen to something very close to what it is today, more than 1600 questions further down the road. It would be a simple matter to run a few tests on the data to determine a suitable 'running-in' period before K/Q is displayed in public or used in anger.
Cindy Chadd wrote:
... why not use the info. that is already available in our profiles but not displayed to the public? I'm not sure if this is a members-only feature or not, but it's in the "Kudoz Activity" section--click on the link--in the "Stats" tab.
One reason I didn't suggest anything more complex than K/Q using publicly-available data is that as a mere 'user' I wasn't aware of these additional statistics you have mentioned.
That said, I'm not sure any of those additional data will be more representative of performance/reliability/ call-it-what-you-will than the simple K/Q.
Several contributors have suggested it might be interesting to provide a breakdown of K/Q by language pair or by specialization. Of course, its perfectly feasible - but I'm not at all sure that it's a good idea. We need to accept that in any system having as many uncontrolled (and uncontrollable) variables as KudoZ, there comes a point where more tightly focussed statistics cease to give better guidance either to the KudoZ asker seeking the best answer or to the client/agency seeking the best translator for a given job.
No statistical indicator can be more reliable than the worst data used to generate it. From a review of my own KudoZ data, I reckon that my K/Q in my main specializations would be about 50% higher if:
- askers correctly categorized their questions. For example I answer many questions about broadcast engineering and the media that are categorized as 'law/contracts' (which I don't claim as specializations) simply because the asker is translating a contract but has problems with a clause concerning broadcast system specifications. Not unnaturally, askers tend to give points to answerers flagged as specialists in the declared category, so we have lawyers getting points for technically incorrect answers about television studios, for example, when correct answers from qualified broadcast engineers are there on the screen. The lawyer's 'legal' and 'overall' K/Q both go up, without due cause, and the engineer's overall K/Q goes down, and his engineering K/Q remains unchanged, despite having provided the correct answer.
- askers knew enough about the subjet-matter of their source text to make a sensible choice among the answers to their questions (time and again I see askers pick 'stupid' answers when they have the correct answer there on the screen, staring them in the face).
The first problem suggests good reasons to avoid doing a breakdown of K/Q by specialization:
- You end up excluding data that should be included in the calculation for a given specialization.
- You can easily get the impression that someone is an 'expert' in a subject when they are not. My overall K/Q is only 1.512 but my K/Q for 'Medicine' (which I know nothing about, except in my capacity as an occasional patient) is 2.208 and for my three main professional specializations of Electronics/Telecoms/Mechanical Engineering it is only around 1.7 to 1.9, and for 'Genealogy', which is just a hobby interest, it is 3.250.
Until there's a system allowing answerers to grade askers I don't see how the second problem can be resolved...
So, for as long as any more-focussed stats will continue to be contaminated by these kinds of error, it's surely better to use the simpler overall K/Q, on the understanding that it doesn't give the full picture but it is at least more indicative of current performance than the league tables we have today based merely on total points.
MediaMatrix
| | | | | I don't see how reliability ratios can work... | Aug 24, 2007 |
and the thought that every time I gave an answer that wasn't selected my "reliability ratio" would decrease is worrying. What about those questions where people ask for help with a slogan, jingle or pun? - such questions are often considered "fun" and can attract 10 or 12 answers. A number of those may be very good, but only one can be selected. The thought that all the others would see their reliability ratio decrease, for no particularly valid reason, is most alarming.
| | | | Richard Jenkins
Brazil
Local time: 11:39
Član (2006)
portugalski na engleski
+ ...
| Some further notes.... | Sep 17, 2007 |
Hi Mediamatrix,
The Ratio process is good fun (statistically) but doesn't really hold up as a fair measure of reliability because of the following:
1)
If you take a look at the Kudoz history of some of the senior/long-term members (going back five years +) you'll notice that many Kudoz questions were originally posted as NON-Pro. It was much more common then. Many well-established translators have thousands of 'easy' / 'non-pro' Ku... See more Hi Mediamatrix,
The Ratio process is good fun (statistically) but doesn't really hold up as a fair measure of reliability because of the following:
1)
If you take a look at the Kudoz history of some of the senior/long-term members (going back five years +) you'll notice that many Kudoz questions were originally posted as NON-Pro. It was much more common then. Many well-established translators have thousands of 'easy' / 'non-pro' Kudoz points. This ratio does NOT show up on the visitor profile page of these site users; the ratio is calculated on PRO points only. Should these more established translators therefore be penalised because of this?
2)
If you don't like your ratio statistic or the particular answer that you have given then you can simply 'Remove' your answer by hiding it. Equals: one less question answered, slightly improved statistics. That's a serious flaw in your proposal.
However, ratios do reflect attention to detail and a statistical, precise mentality. That's good .
[Edited at 2007-09-17 14:14] ▲ Collapse
| | | | | Stranica u temi: < [1 2 3 4] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » HowZ your K/Q? | Protemos translation business management system |
|---|
Create your account in minutes, and start working! 3-month trial for agencies, and free for freelancers!
The system lets you keep client/vendor database, with contacts and rates, manage projects and assign jobs to vendors, issue invoices, track payments, store and manage project files, generate business reports on turnover profit per client/manager etc.
More info » |
| | Anycount & Translation Office 3000 |
|---|
Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
