For a complete audio recording and transcript, see this link: https://www.buzzsprout.com/1769590/episodes/8434486
In this podcast episode, Paul Starrett and Jermand Hagan discuss how artificial intelligence (AI) and machine learning (ML) are audited. The episode covers various aspects of auditing AI, including the three lines of defense framework, the CRISP-DM (Cross-Industry Standard Process for Data Mining) model, and the importance of ethical considerations in AI development and application.
Jermand Hagan, the founder and principal of 2fifth consulting, introduces himself and his extensive background in IT audit, compliance, risk, and regulatory examination roles. He highlights his experience working with various companies, including large financial institutions, manufacturers, and others.
The conversation begins with an exploration of the three lines of defense framework commonly used in financial institutions. The first line represents the business operations, the second line focuses on risk management, and the third line involves external or internal audit. Jermand explains how this framework helps ensure that businesses execute operations with acceptable risk levels, while audits validate these processes.
The podcast then delves into the CRISP-DM model, a process used for developing data science projects. This six-stage model involves business understanding, data understanding, data preparation, model development, model evaluation, and model deployment. The hosts discuss how these stages align with the three lines of defense and the necessity of considering privacy, data protection, and security throughout the entire AI development lifecycle.
The conversation also touches on the importance of explainability in AI models, as understanding how a model makes decisions is crucial for assessing potential risks and biases. The hosts provide an example of an AI model that achieved high accuracy by focusing on irrelevant features, underscoring the importance of understanding model behavior.
Ethical considerations and data ethics are emphasized as key factors in AI development and application. The hosts stress that companies should not only ensure their AI tools are functional but also ethically applied. They highlight the need for explainability, fairness, and bias mitigation in AI systems, emphasizing the role of data ethics in maintaining public trust.
In conclusion, the podcast emphasizes the unique strengths of the partnership between 2fifth consulting and PrivacyLabs in providing comprehensive AI audits. Their combined expertise covers both the IT audit perspective and the data science and privacy perspective. The hosts encourage listeners to reach out to them for assistance in preparing for audits and ensuring ethical AI deployment.
The podcast touches on various aspects of AI auditing, including the three lines of defense framework, the CRISP-DM model, explainability, fairness, bias mitigation, and the importance of data ethics in AI development and application. Overall, it provides valuable insights into the complexities and considerations surrounding auditing AI and ensuring its responsible and ethical use.