Watts S. Humphrey

Evaluation Process

For each nomination, the following steps are completed:

1. The Humphrey Award Selection Subcommittee evaluates the degree to which the nominee's work satisfies the award criteria for significant, measured, sustained, and shared improvement.
   • To evaluate whether the work is significant, the subcommittee considers as factors not only the breadth and depth of the work's deployment throughout the target organizations (e.g., the number of projects that have benefited) but also the certainty of the connection between the software engineering quality improvement work and achievement of the target organizations' performance goals (e.g., the strength of the link between the work and the improvements in quality and productivity). If a heterogeneous set of organizations (or projects) have different business goals (or different improvement starting points), it may not be sensible to aggregate deployment or impact data across organizations. In this case, the nominator should select a small set of individual organizations (possibly three to five) and report specifically the work for these.
   • To evaluate whether the work is measured, the subcommittee considers factors such as the breadth to which the data demonstrate the improvement (e.g., the measurement of customer satisfaction factors as well as workforce capability-related factors), the depth of the data regarding the improvement (e.g., the consideration of defect containment as well as defect incidence data), and the use of well-defined measures to demonstrate the improvement (e.g., the definition of a concrete measure relating planned vs. actual project effort).
   • To evaluate whether the work is sustained, the subcommittee considers factors such as the work's level of documentation (i.e., whether the results of the work are described in informal memoranda, internal reports, or organizational standards), the likelihood that the results achieved to date are permanent (e.g., as indicated by the support of senior executives), and the likelihood that there will be additional improvements in the future (e.g., as indicated by establishing procedures for collecting, considering, and implementing improvement suggestions).
   • To evaluate whether the work is shared, the subcommittee considers factors such as the degree to which the work has directly affected other organizations (e.g., the number of other organizations using methods influenced by the nominee's work) and the extent to which the nominee's work has impacted (or will impact) the nominee's local and global software engineering communities (e.g., in terms of presentations in professional society meetings).
2. The Selection Subcommittee decides whether the nominee's work satisfies all of the criteria to at least a minimal degree. If this is not the case, a detailed list of deficiencies is sent to the nominator and nominee accompanied by advice regarding a resubmission in the future. If the work at least minimally satisfies all of the criteria, then a detailed list of questions and issues is sent to the nominee.
    
3. The Selection Subcommittee interacts with the nominee to gather information addressing the questions and issues. If the Selection Subcommittee feels it is necessary (and, recently, this has been the exception rather than the norm), they may conduct an on-site visit to interact with the nominee and personnel from the target organizations.
    
4. The Selection Subcommittee requirements are consistent with IEEE CS award standards:
   • Nominations should comply with IEEE Policies and restrictions on awards. Incidents of misconduct including, but not limited to, violations of IEEE's publication policies, will be strongly considered by the awards committee and may be grounds for denial of an award or leadership position.
   • Voting shall be conducted in accordance with Robert’s Rules of Order. The Subcommittee Chair shall have no vote except if the vote is by secret ballot or unless the Chair's vote can change the outcome of the vote.
   • Conflict of Interest concerns shall be disclosed and addressed in accordance with IEEE Policies 9.9 A, B. and C. Any real and perceived conflict of interest (COI) shall be avoided. Anyone with a COI shall recuse themselves from nominating, endorsing, discussing, and evaluating.
   • Individuals serving on any board or committee involved at any stage of the recipient selection or approval process for an award shall be ineligible to receive, or act as a nominator or reference for that award.