Nominations are evaluated by an Award Committee consisting of senior, knowledgeable, and experienced software engineering process improvement professionals. The restricted length of the nomination package generally does not allow a fully adequate description of the nominee's activities and results, but it should provide enough information for the Award Committee to determine that the criteria are likely to be satisfied. Nominees that pass this initial screening review are then engaged in focused interactions with the Award Committee to elicit more detail about the nominee's achievements. A positive decision requires that the nominee satisfy the award criteria to an exceptional degree. In the case of a negative decision, the Chair of the Award Committee provides an explanation, a detailed list of deficiencies, and advice regarding a resubmission in the future.
For each nomination, the following steps are completed:
- The Humphrey Award Selection Subcommittee evaluates the degree to which the nominee's work satisfies the award criteria for significant, measured, sustained, and shared improvement.
- To evaluate whether the work is significant, the subcommittee considers as factors not only the breadth and depth of the work's deployment throughout the target organizations (e.g., the number of projects that have benefited) but also the certainty of the connection between the software engineering process improvement work and achievement of the target organizations' performance goals (e.g., the strength of the link between the work and the improvements in productivity and quality). If a heterogeneous set of organizations (or projects) have different business goals (or different improvement starting points), it may not be sensible to aggregate deployment or impact data across organizations. In this case, the nominator should select a small set of individual organizations (possibly three to five) and report specifically the work for these.
- To evaluate whether the work is measured, the subcommittee considers factors such as the breadth to which the data demonstrate the improvement (e.g., the measurement of customer satisfaction factors as well as workforce capability-related factors), the depth of the data regarding the improvement (e.g., the consideration of defect containment as well as defect incidence data), and the use of well-defined measures to demonstrate the improvement (e.g., the definition of a concrete measure relating planned vs. actual project effort).
- To evaluate whether the work is sustained, the subcommittee considers factors such as the work's level of documentation (i.e., whether the results of the work are described in informal memoranda, internal reports, or organizational standards), the likelihood that the results achieved to date are permanent (e.g., as indicated by the support of senior executives), and the likelihood that there will be additional improvements in the future (e.g., as indicated by establishing procedures for collecting, considering, and implementing process improvement suggestions).
- To evaluate whether the work is shared, the subcommittee considers factors such as the degree to which the work has directly affected other organizations (e.g., the number of other organizations using methods influenced by the nominee's work) and the extent to which the nominee's work has impacted (or will impact) the nominee's local and global software engineering process improvement communities (e.g., in terms of presentations in professional society meetings).
- The Selection Subcommittee decides whether the nominee's work satisfies all of the criteria to at least a minimal degree. If this is not the case, a detailed list of deficiencies is sent to the nominator and nominee accompanied by advice regarding a resubmission in the future. If the work at least minimally satisfies all of the criteria, then a detailed list of questions and issues is sent to the nominee.
- The Selection Subcommittee interacts with the nominee to gather information addressing the questions and issues. If the Selection Subcommittee feels it is necessary (and, recently, this has been the exception rather than the norm), they may conduct an on-site visit to interact with the nominee and personnel from the target organizations.
Recipient Support Process
The following steps occur after a proposal is approved by the IEEE Computer Society and the SEI:
- The Humphrey Award Selection Subcommittee consults with the recipient to confirm recipient's availability to present at a conference selected by the SEI and the IEEE Computer Society.
- The Selection Subcommittee identifies a member to mentor the recipient in preparing high-quality conference presentations and a high-quality SEI technical report regarding the recipient's "improvement journey" and the resulting insights, experiences, and proven practices.
- The SEI announces the recipient(s), presents the commemorative plaque, introduces the recipient'(s') presentation, and promotes the recipient'(s') work and its implications for the conference attendees' work.
- The Chair of the Selection Subcommittee facilitates publication of the SEI technical report to coincide with the second-occurring conference and makes arrangements for delivery of an honorarium upon completion of the report.
Details of the nomination and award process, deadlines, and past recipients can also be found at http://www.computer.org/web/awards/humphrey-spa.