The limits of automation and the role of the human factor in making emergency IT decisions

Abstract

In the context of emergency decision-making in IT companies, automation does not eliminate risks but transforms them, increasing the likelihood of rare but critical failures. Despite the effectiveness of artificial intelligence systems in performing routine operations, it is humans who remain the key element in stabilizing the system in conditions of uncertainty and non-standard situations. The study focused on the concepts of automation bias and human models in the adaptive control circuit. It was found that cognitive factors affect the quality of decision-making in critical situations when standard algorithms encounter uncertainty. This is of particular importance in incident management and cyber defense processes, where decisions must be made quickly and in conditions of information scarcity. The levels of automation and human involvement were analyzed, with the most common management risks in mind. The specifics of the application and the effectiveness of Human-SRE and AI-SRE were also analyzed. Effective practical approaches to minimizing the negative impact of automation and maximizing human potential in the IT sphere are proposed. It has been shown that excessive reliance on automation can lead to cognitive degradation among personnel, loss of critical-thinking skills, and reduced situational awareness. In this regard, IT companies should invest not only in digital technologies but also in developing employees' autonomous response skills. An important condition for security is ensuring the transparency of algorithms and personal responsibility for emergency decisions. Ukrainian experience of IT companies operating in wartime demonstrates the effectiveness of combining a high level of automation with the adaptability of human capital. It is the balance between technological support and human control that underpins the stability of emergency management systems in the modern digital environment. The domestic IT sector's experience during the war demonstrates the effectiveness of the human-centric automation model. The stability of emergency management systems is based on preserving a person's role as an active and autonomous subject. In this paradigm, digital tools do not replace human capital but expand human capital's adaptive capabilities, which is critical to system stability in a dynamic digital environment.