Dear Participant,

Earnings calls of publicly traded companies serve as key resources for analysts, investors, and other interested stakeholders to determine a company's financial performance during a given reporting period, such as a quarter or a fiscal year. The call transcripts, released by the firms after the earnings events, are typically in the form of long unstructured documents without any prescribed length limit or format. Automatically summarizing relevant facts from these transcripts is thus an important task.

We have trained two different systems to automatically generate short summaries (around 4-8 sentences long on an average) from earnings call transcripts (hereby refered to as ECTs). An example transcript (hosted on Motley Fool) and its corresponding summary are as follows:

ECT for Graham Corp. Q1 2021:
https://www.fool.com/earnings/call-transcripts/2020/07/31/graham-corp-ghm-q1-2021-earnings-call-transcript.aspx

Summary:
1. graham corp reports q1 loss per share $0.18.
2. q1 sales $16.7 million versus refinitiv ibes estimate of $14.2 million.
3. q1 loss per share $0.18.
4. fiscal 2021 revenue guidance of $90 million to $95 million from expected improved performance for remainder of fiscal 2021.
5. averaged about 50% of normal staffing capacity during quarter.
6. encouraged by pipeline activity that co is addressing in both defense industry and emerging markets, specifically india.


This survey expects you to evaluate the two system-generated summaries for a given ECT across three different metrics: factual correctness, relevance, and coverage. The survey is organized as follows:
1. In the next three sections, we define and illustrate the metrics.
2. In each section thereafter (total 5), first you will be given a link to the ECT of a certain company. You are expected to read the transcript. Then, you will be shown the two system-generated summaries that need to be evaluated on each of the three metrics.

You are requested to meticulously follow the instructions and answer all the questions individually and judiciously.

Clicking on the "Agree" button below indicates the following:
1. You have read the information presented above.
2. You voluntarily agree to participate in this survey.