Interview Outline
In this discussion, Dr Paul Bleicher, a physician scientist, discusses his experience with observational research and the problems with confounders that can lead to false conclusions. He cites the example of vitamin D, which was thought to have numerous benefits beyond bone health due to observational research, but those claims were not supported by clinical trials.
He then discusses how, despite these issues, observational research is critical as it uses real-world data, has a broader population scope, and can be conducted more quickly than clinical trials. He emphasizes that by designing and iterating on observational data in a rigorous way, it's possible to dissect confounders and come closer to causal inference. His team at Optim Lab initiated a project called the Operand with Brigham and Women's Multi-Regional Clinical Trial Center to replicate clinical trials using rigorous methods without actually knowing the results of the trials.
Dr Bleicher also highlights the importance of integrating expert knowledge in AI and deep learning. Without this, AI can become a 'black box' affected by the same confounders as observational research. If expert knowledge is applied upfront, it can help deal with bias, provide more transparency and explainability, and improve trust in the results.
Dr Bleicher also discusses the evolution of causal inference and the need for a deeper understanding of statistics and causal tools among data scientists and engineers working with AI. The speaker emphasizes the importance of involving subject matter experts in the process to ensure that AI isn't used to rediscover well-known correlations that offer no new insights.
He ends by stating that although we have these tools for causal inference and know they work, we are still in the early stages of using them properly and need to transition from the 'toy stage' to the 'engineering stage.' He sees signs of this transition happening and is hopeful for the future.