Nominations are evaluated by the Award Selection Subcommittee consisting of senior, knowledgeable, and experienced software engineering quality improvement professionals. The restricted length of the nomination package generally does not allow a fully adequate description of the nominee's activities and results, but it should provide enough information for the Award Selection Subcommittee to determine that the criteria are likely to be satisfied. Nominees that pass this initial screening review are then engaged in focused interactions with the Award Selection Subcommittee to elicit more detail about the nominee's achievements. A positive decision requires that the nominee satisfies the award criteria to an exceptional degree. In the case of a negative decision, the Chair of the Award Selection Subcommittee provides an explanation, a detailed list of deficiencies, and advice regarding a resubmission in the future.
For each nomination, the following steps are completed:
- The Humphrey Award Selection Subcommittee evaluates the degree to which the nominee's work satisfies the award criteria for significant, measured, sustained, and shared improvement.
- To evaluate whether the work is significant, the subcommittee considers as factors not only the breadth and depth of the work's deployment throughout the target organizations (e.g., the number of projects that have benefited) but also the certainty of the connection between the software engineering quality improvement work and achievement of the target organizations' performance goals (e.g., the strength of the link between the work and the improvements in quality and productivity). If a heterogeneous set of organizations (or projects) have different business goals (or different improvement starting points), it may not be sensible to aggregate deployment or impact data across organizations. In this case, the nominator should select a small set of individual organizations (possibly three to five) and report specifically the work for these.
- To evaluate whether the work is measured, the subcommittee considers factors such as the breadth to which the data demonstrate the improvement (e.g., the measurement of customer satisfaction factors as well as workforce capability-related factors), the depth of the data regarding the improvement (e.g., the consideration of defect containment as well as defect incidence data), and the use of well-defined measures to demonstrate the improvement (e.g., the definition of a concrete measure relating planned vs. actual project effort).
- To evaluate whether the work is sustained, the subcommittee considers factors such as the work's level of documentation (i.e., whether the results of the work are described in informal memoranda, internal reports, or organizational standards), the likelihood that the results achieved to date are permanent (e.g., as indicated by the support of senior executives), and the likelihood that there will be additional improvements in the future (e.g., as indicated by establishing procedures for collecting, considering, and implementing improvement suggestions).
- To evaluate whether the work is shared, the subcommittee considers factors such as the degree to which the work has directly affected other organizations (e.g., the number of other organizations using methods influenced by the nominee's work) and the extent to which the nominee's work has impacted (or will impact) the nominee's local and global software engineering communities (e.g., in terms of presentations in professional society meetings).
- The Selection Subcommittee decides whether the nominee's work satisfies all of the criteria to at least a minimal degree. If this is not the case, a detailed list of deficiencies is sent to the nominator and nominee accompanied by advice regarding a resubmission in the future. If the work at least minimally satisfies all of the criteria, then a detailed list of questions and issues is sent to the nominee.
- The Selection Subcommittee interacts with the nominee to gather information addressing the questions and issues. If the Selection Subcommittee feels it is necessary (and, recently, this has been the exception rather than the norm), they may conduct an on-site visit to interact with the nominee and personnel from the target organizations.
- The Selection Subcommittee requirements are consistent with IEEE CS award standards:
- Nominations should comply with IEEE Policies and restrictions on awards. Incidents of misconduct including, but not limited to, violations of IEEE's publication policies, will be strongly considered by the awards committee and may be grounds for denial of an award or leadership position.
- Voting shall be conducted in accordance with Robert’s Rules of Order. The Subcommittee Chair shall have no vote except if the vote is by secret ballot or unless the Chair's vote can change the outcome of the vote.
- Conflict of Interest concerns shall be disclosed and addressed in accordance with IEEE Policies 9.9 A, B. and C. Any real and perceived conflict of interest (COI) shall be avoided. Anyone with a COI shall recuse themselves from nominating, endorsing, discussing, and evaluating.
- Individuals serving on any board or committee involved at any stage of the recipient selection or approval process for an award shall be ineligible to receive, or act as a nominator or reference for that award.
Recipient Support Process
The following steps occur after a proposal is approved by the IEEE Computer Society and the SEI:
- The Humphrey Award Selection Subcommittee consults with the recipient to confirm recipient's availability to present at a webinar hosted by the SEI.
- The Selection Subcommittee identifies one of their members to mentor the recipient in preparing a high-quality webinar presentation and a high-quality SEI technical report regarding the recipient's "improvement journey" and the resulting insights, experiences, and proven practices.
- The IEEE Computer Society and SEI jointly announce the recipient(s) of the current year’s award. The award is bestowed at the IEEE Computer Society award ceremony.
- The Chair of the Selection Subcommittee facilitates publication of the SEI technical report to coincide with the webinar presentation and makes arrangement for delivery of an honorarium upon completion of the report.
Details of the nomination and award process, deadlines, and past recipients can also be found at https://www.computer.org/volunteering/awards/humphrey-software-quality.