How i judge software engineers

My kid’s teachers routinely provide rubrics for assignments. At first blush, rubrics are tools to make grading an assessment easier. They are effective in that role. They turn out to be at least as effective at communicating expectations. Readers of a rubric can quickly and easily determine what is important.

Recently i developed a rubric to help me judge the performance of engineers (it was review season yet again). The engineers on my team have appreciated the transparency and clarity this tool provides. Hopefully, it will be helpful to others.

The Rubric is divided into two sections. One about results and the other about behavior. Both of these are important. Good, pro-social behavior is just as important in engineers as strong results.

Results

3 2 1
Continuous improvement
  • always leaves code substantially better than they found it
  • manages scope of refactors to match time available
  • often improves code as part of normal work
  • sometimes improves code as part of normal work
  • often has to abandon refactors due to time constraints
  • rarely improves existing code
Communication
  • communicates intentions early and clearly
  • often provides material support to teammates (developers, QA, on-call person, etc)
  • regularly creates & maintains runbook (particularly when on call)
  • keeps stakeholder informed (particularly when on call)
  • communicates intentions after it is hard to change course
  • regularly supports teammates
  • occasionally maintains runbook
  • rarely communicates intentions
  • sometimes supports teammates
  • doesn’t maintain runbook
  • never communicates intentions
  • never supports teammates
  • never creates or improves runbook entries
Production support
  • prioritizes concurrent incidents correctly
  • use critical thinking and problem-solving skills to resolve issues quickly
  • handles lower priority issues (eg, jenkins nodes down) when there are no higher priority incidents
  • prioritizes concurrent incidents correctly
  • use critical thinking and problem solving skills to resolve issues
  • handles lower priority issues (eg, jenkins nodes down) when there are no higher priority incidents
  • prioritizes concurrent incidents correctly
  • ignores lower priority issues (eg, jenkins nodes down) even when there are no high priority incidents (works stories while on call)
  • relies too heavily on others (rather than using critical thinking and problem-solving skills)
  • bad attitude
  • incorrectly prioritizes concurrent incidents
  • always ignores lower priority issues (jenkins nodes down)
  • doesn’t communicate incident status to stakeholders
  • relies on others to resolve issues (throws it over the wall)
Code Quality
  • functional
  • well factored
  • documentation on classes/modules and public method/functions
  • PRs require trivial changes
  • functional
  • well factored
  • poorly documented
  • PRs require minor changes
  • functional
  • poorly factored
  • undocumented
  • PRs require some rework
  • buggy
  • poorly factored
  • undocumented
  • PRs require substantial rework
Productivity
  • usually delivers more stories per sprint than the average engineer
  • usually delivers more points per sprint than the average engineer
  • occasionally delivers more stories/points per sprint than the average engineer
  • usually delivers fewer stories/points per sprint than the average engineer
  • delivers substantially fewer stories/points per sprint than the average engineer
Testing
  • public contracts well tested
  • key scenarios have acceptance tests
  • tests are independent of current implementation
  • public contract partially tested
  • key scenarios have acceptance tests
  • tests are independent or current implementation
  • public contract partially tested
  • tests dependent on current implementation
  • acceptance tests check too many edge cases
  • no unit or functional tests
Feedback
  • often reviews PRs
  • feedback is substantive and useful
  • reviews show understanding of PRs intent and the code that interacts with it
  • regularly reviews PRs
  • advice would materially improve PRs
  • reviews show understanding of PRs intent
  • sometimes reviews PRs
  • reviews are superficial
  • reviews are hard to understand
  • never reviews PRs
Product & domain knowledge
  • understands the domain
  • understands most of the supported features of the product
  • understands some of the historical features of the product
  • understands the domain
  • understands many of the supported features of the product
  • some knowledge of the domains
  • limited knowledge of product features
  • no knowledge of utility and grid edge domain
  • no knowledge of product
Personal goals
  • goals are SMART
  • goals drive achievement of team and corporate goals in material ways
  • achieves goals on time
  • goals are SMART
  • goals weakly support team and corporate goals
  • achieves goals
  • goals have no relation to team and corporate goals
  • achieves goals
  • goals are vague or unattainable
  • goals work against team and corporate goals
  • doesn’t achieve goals

Behavior

Attitude
  • polite and engaging even when under stress (eg, when on call)
  • accepts setbacks and moves forward
  • normally polite but brusque when under stress
  • accepts setbacks and moves forward
  • normally polite but rude when under stress
  • rude
  • dismissive
Courage
  • courageous in all aspects of work every day
  • strives for greatness even when difficult
  • occasionally fails spectacularly
  • often courageous in most aspects of work
  • occasionally fails
  • sometimes courageous in some aspects of work
  • rarely fails
  • often timid
  • usually takes least risky (and rewarding) approach
Motivation
  • highly motivated to succeed
  • accepts challenges and new responsibilities
  • motivated to succeed
  • grudgingly accepts new challenges and responsibilities
  • resists new challenges and responsibilities
  • lacks motivation
  • rejects all new challenges and responsibilities
Strategic thinking plans for 6 month – 2 year horizon plans for 3 – 6 month horizon plans for 1 – 3 month horizon no planning
Trustworthiness
  • earns trust of others
  • reliably meets commitments
  • honest
  • occasionally fails to meet commitments
  • sometimes fails to gain the trust of others
  • fails to meet commitments
  • occasionally misconstrues the facts
  • often misconstrues the facts
  • widely distrusted
Learning
  • seeks out learning opportunities
  • applies lessons learn to enhance success
  • elicits relevant experiences from others
  • interested in learning
  • applies lessons learn to enhance success
  • learns when pushed
  • resists change
  • uninterested in learning
  • resists change
Pairing*
  • improves productivity and morale of pairs
  • pairs most of the time
  • pairs effectively
  • pairs most of the time
  • pairs ineffectively
  • pairs some of the time
  • reduces productivity and morale of pairs
  • rarely pairs
  • Optional. If your team pairs at a matter of course this is very important. If your team works as individuals then ignore this row.