· Peter Williams

How i judge software engineers

13 Mar 2018 • Software Development

My kid’s teachers routinely provide rubrics for assignments. At first blush, rubrics are tools to make grading an assessment easier. They are effective in that role. They turn out to be at least as effective at communicating expectations. Readers of a rubric can quickly and easily determine what is important.

Recently i developed a rubric to help me judge the performance of engineers (it was review season yet again). The engineers on my team have appreciated the transparency and clarity this tool provides. Hopefully, it will be helpful to others.

The Rubric is divided into two sections. One about results and the other about behavior. Both of these are important. Good, pro-social behavior is just as important in engineers as strong results.

Results

	3	2	1
Continuous improvement	always leaves code substantially better than they found it manages scope of refactors to match time available	often improves code as part of normal work	sometimes improves code as part of normal work often has to abandon refactors due to time constraints	rarely improves existing code
Communication	communicates intentions early and clearly often provides material support to teammates (developers, QA, on-call person, etc) regularly creates & maintains runbook (particularly when on call) keeps stakeholder informed (particularly when on call)	communicates intentions after it is hard to change course regularly supports teammates occasionally maintains runbook	rarely communicates intentions sometimes supports teammates doesn’t maintain runbook	never communicates intentions never supports teammates never creates or improves runbook entries
Production support	prioritizes concurrent incidents correctly use critical thinking and problem-solving skills to resolve issues quickly handles lower priority issues (eg, jenkins nodes down) when there are no higher priority incidents	prioritizes concurrent incidents correctly use critical thinking and problem solving skills to resolve issues handles lower priority issues (eg, jenkins nodes down) when there are no higher priority incidents	prioritizes concurrent incidents correctly ignores lower priority issues (eg, jenkins nodes down) even when there are no high priority incidents (works stories while on call) relies too heavily on others (rather than using critical thinking and problem-solving skills)	bad attitude incorrectly prioritizes concurrent incidents always ignores lower priority issues (jenkins nodes down) doesn’t communicate incident status to stakeholders relies on others to resolve issues (throws it over the wall)
Code Quality	functional well factored documentation on classes/modules and public method/functions PRs require trivial changes	functional well factored poorly documented PRs require minor changes	functional poorly factored undocumented PRs require some rework	buggy poorly factored undocumented PRs require substantial rework
Productivity	usually delivers more stories per sprint than the average engineer usually delivers more points per sprint than the average engineer	occasionally delivers more stories/points per sprint than the average engineer	usually delivers fewer stories/points per sprint than the average engineer	delivers substantially fewer stories/points per sprint than the average engineer
Testing	public contracts well tested key scenarios have acceptance tests tests are independent of current implementation	public contract partially tested key scenarios have acceptance tests tests are independent or current implementation	public contract partially tested tests dependent on current implementation acceptance tests check too many edge cases	no unit or functional tests
Feedback	often reviews PRs feedback is substantive and useful reviews show understanding of PRs intent and the code that interacts with it	regularly reviews PRs advice would materially improve PRs reviews show understanding of PRs intent	sometimes reviews PRs reviews are superficial reviews are hard to understand	never reviews PRs
Product & domain knowledge	understands the domain understands most of the supported features of the product understands some of the historical features of the product	understands the domain understands many of the supported features of the product	some knowledge of the domains limited knowledge of product features	no knowledge of utility and grid edge domain no knowledge of product
Personal goals	goals are SMART goals drive achievement of team and corporate goals in material ways achieves goals on time	goals are SMART goals weakly support team and corporate goals achieves goals	goals have no relation to team and corporate goals achieves goals	goals are vague or unattainable goals work against team and corporate goals doesn’t achieve goals

Behavior

Attitude	polite and engaging even when under stress (eg, when on call) accepts setbacks and moves forward	normally polite but brusque when under stress accepts setbacks and moves forward	normally polite but rude when under stress	rude dismissive
Courage	courageous in all aspects of work every day strives for greatness even when difficult occasionally fails spectacularly	often courageous in most aspects of work occasionally fails	sometimes courageous in some aspects of work rarely fails	often timid usually takes least risky (and rewarding) approach
Motivation	highly motivated to succeed accepts challenges and new responsibilities	motivated to succeed grudgingly accepts new challenges and responsibilities	resists new challenges and responsibilities	lacks motivation rejects all new challenges and responsibilities
Strategic thinking	plans for 6 month – 2 year horizon	plans for 3 – 6 month horizon	plans for 1 – 3 month horizon	no planning
Trustworthiness	earns trust of others reliably meets commitments	honest occasionally fails to meet commitments sometimes fails to gain the trust of others	fails to meet commitments occasionally misconstrues the facts	often misconstrues the facts widely distrusted
Learning	seeks out learning opportunities applies lessons learn to enhance success elicits relevant experiences from others	interested in learning applies lessons learn to enhance success	learns when pushed resists change	uninterested in learning resists change
Pairing*	improves productivity and morale of pairs pairs most of the time	pairs effectively pairs most of the time	pairs ineffectively pairs some of the time	reduces productivity and morale of pairs rarely pairs