Questioning the Answers: LLMs enter the Boardroom

Deep dive into the Practical Application of LLMs & Generative AI tools for Quantitative Alpha Research utilising S&P’s Machine Readable Earnings Call Transcripts. In their Questioning the Answers Research the Quantitative Research & Solutions (QRS) team analyses the earnings call Q&A dialogue between executives and sell-side analysts. When analysts ask about topics not addressed in the prepared remarks, it may suggest that executives either overlooked these issues or chose to avoid them. The way executives respond in these moments is equally revealing. Transparent answers build trust. Evasive answers raise concerns. Large Language vector embeddings of 192,000 earnings calls were used to score executives on: (1) proactiveness, proxied by how well the prepared remarks addressed analysts' questions, and (2) on-topic alignment, proxied by whether executives’ responses remained on-topic to the question.

• Firms with on-topic executives outperformed off-topic peers by 390 bps/year in a long/short portfolio.
• Proactive executives outperformed reactive peers by 170 bps/year in a long/short portfolio.
• Combining both factors yielded 506 bps/year in a long/short portfolio.

Agenda:

• Opening Remarks and webinar outline – Daniel Sandberg
• Methodology outline for executive behavioural signal construction and alpha results – Liam Hynes
• LLM, Vector Embedding & Signal Construction Coding Tutorial using S&P’s Machine Readable Transcripts – Henry Chiang
• Topic Analysis and thematic trends utilising S&P’s Pronto NLP – Ronen Feldman