24 Figure 5: Definitions and Example Metrics for KPIs and Risk in Impact Scoring Figure 6: Impact Scoring Variable Metrics and Scatterplots. levels of management to identify the desired level of KPIs that a Gen-AI integration analysis should benchmark against. Analysts must develop metrics that incorporate both leadership’s KPI priorities and those of individual teams, as KPIs at one level of the organization are not always aligned with those at supporting levels. Therefore, multi-level coordination is critical to identify the right KPIs, and consequently, the appropriate metrics, for Impact Scoring. Once the right KPIs are identified, analysts will create relative metrics for tasks against each KPI. Figure 5 below demonstrates the definitions and metrics for KPIs and Risk. Measuring the variable Risk, on the other hand, requires looking at historical cumulative outcomes— successes and failures—in any given step of a process. Analysts must define the range of outcomes based on the analysis of past documentation, as well as stakeholder interviews with team members, and quantify the outcomes on a positive to negative scale. The analysis will vary between organizations and teams, but measurements can include risks such as the potential impacts on finances, timelines, safety, and regulatory frameworks. As each step of the process contributes differently to the overall outcome, assigning relative scores can allow analysts to generate a relative Risk Ranking for each step. Once Risk Rankings and KPI Rankings are established, they can be combined to identify an Impact Score for each task. Building on our Cognitive Load Scoring for the Naval Aviation Community flight mission process, in Figure 6 below, we demonstrate Impact Scoring for pilots: Step 2: Score Impact variables and graph scores on quadrant scatter plot Variable Description Potential Input Measures KPI Alignment The relative magnitude of impact to the organization’s defined key performance indicators (KPIs). • Cost • Schedule • Performance • Workforce Development Risk The relative potential for damage, unfavorable or hazardous outcomes for a given mission or military force. • Financial Impact • Timeline Impact • Safety Impact • Regulatory Impact Generative AI Adoption in the US Military