I came across a quote the other day that completely explains IT:
There will always be people who are ahead of the curve, and people who are behind the curve. But knowledge moves the curve. – Bill James
You can be competent in a technology, but when that technology changes, when it evolves, when the knowledge about that technology changes, you may end up behind the curve. For instance, if you grabbed on to Visual Basic when it first came out (like me) you may have become competent or even proficient in that technology but unless you changed, unless you grew, you fell behind the curve.
This explains why someone who is praised by everyone as a “fantastic developer”, “fabulous programmer”. “awesome dude”, may end up being a dude in your organization. The level of competency, the level of “average” may be much higher than in your organization than it was in his previous. Whereas he used to be outstanding in his field (insert joke here) he is now just average, or even below average.
The curve moves, not just from a world-wide perspective, but from an organizational and even personal perspective. The more that is known about a topic the greater the basis of knowledge and the more the curve shifts. As you know your expectations shift. As the organization grows and its needs change its curve shifts. As the IT community at large gets more comfortable with a new technology and is able to better understand and utilize it their curve shifts.
The difficulty is trying to determine someone’s knowledge level, their competency, in an objective manner. Well, there isn’t one. Even an exam isn’t totally objective as it is based on a point in time and needs to be constantly evaluated. The scope of the exam also needs to be reviewed in order to make sure that it covers all of the appropirate areas. For something like a computer language, there are tests that can tell you if someone knows the syntax, but knowing the syntax and being able to program are two different things. (I once passed an exam for Shorthand Dictation without knowing a thing about shorthand. Good guessing.) And being able to program (competent) and being able to really develop (proficient) is another step.
So, what do you do?
Well, you need to put a stake in the sand/dirt/concrete at some point. If you want something which can be considered objective, find an organization that offers testing in that subject. If you feel that you can’t trust anyone else, create your own exam, but be careful of the biases that you bring to the exam. Start somewhere, anywhere, and the odds are that you will be better off than you were before you started. At least you’ll have some metrics.