Overview

Machine learning components are becoming an intrinsic part of safety-critical systems, from driver-assisted vehicles to medical imaging devices. Since undesired behaviours in such systems can lead to fatal accidents, the dependability and trustworthiness of such systems, including qualities such as reliability and integrity, is paramount for their broad and safe adoption. However, contrary to traditional system development, we are ill equipped to ensure the dependability of systems with learned components. For example, robustness testing of ML components may apply targeted noise to inputs, but that type of noise may never be encountered in a deployment context. Meanwhile, environments with unexpected features may affect learned components in unsafe ways.

ML-components are intrinsically different from traditional software in many ways: (1) they lack precise requirements (relying instead on proxies), lack accuracy; (2) they depend on data with multiple sources of provenance; (3) they rely on architectural considerations that have to do with capacity to efficiently achieve accuracy and not on dependability; (4) they are implemented through an optimization process that is fraught with undeterminism and has many degrees of freedom with subtle dependencies; and (5) their performance does not come with a human-consumable explanation. These differences are creating development challenges that are crucial to the deployment of safety-critical systems with ML components that we are going to discuss through this workshop. The divergence of ML and SE development practices becomes an SE issue when ML components are incorporated into a system. ML components are inherently vulnerable, and features of safety-critical systems, such as fallback routines in the event of a failure, require an understanding of those vulnerabilities to be properly created and deployed. Because these features are often written during system development, the vulnerabilities of ML at that point become the system responsibility.

Goal

The primary goal of this workshop is to foster innovative ideas on the use of SE techniques for the dependability and trustworthiness of safety-critical systems with ML components and to promote cross-fertilization of research with the most relevant underlying areas. This workshop will provide a forum for discussing the challenges associated with dependability and trustworthiness of systems with ML components, identifying areas of responsibility for the SE community, strategizing solutions for SE-owned challenges, and incorporating the (potentially enhanced) SE practices into ML-based system development processes. We aim to bring together researchers from different SE communities, e.g., requirements, modeling, safety, testing and formal verification, together with ML experts to highlight ongoing work and present new ideas. Additionally, we aim to sketch out an ongoing research agenda for future work and promote cross-community collaborations. We plan to summarize the findings of SE4SafeML as a public-facing lessons learned document, including SE theory versus praxis when addressing these challenges, any consensus reached in discussion, and minutes from various panels. We hope to use this as a jumping-off point for future work and discussion on these challenges.

Topics of interest

Include but are not limited to:

• Safety requirements and specification of ML components
• Model-based safety analysis of ML components
• Architectures to manage scale, uncertainty, and safety of ML-components
• Dataset development for ML-based safety-critical components
• Verification and validation methods of ML components
• Trust and trustworthiness of ML-based safety-critical systems
• Privacy analysis of ML-based safety-critical systems
• Explainability of ML-based safety-critical components
• Safety and security guidelines, standards and certification of systems with ML components
• Hazard analysis of ML-based safety-critical systems
• Safety and security assurance cases of ML-based systems
• Risk assessment and reduction of ML-based safety-critical systems
• ML safety education and awareness
• Establishing a baseline of techniques for ensuring dependability of ML-enabled systems

Meet our keynote speakers!

Speaker 1

University

Speaker 2

University

Speaker 3

University

Speaker 4

University

Speaker 5

University