Scribe

Scribe streamlines the analysis of large volumes of free-form text. With Scribe, you can:

• Identify Key Topics: Extract and highlight the main topics from a large volume of text responses like surveys and interviews.
• Quantify Topic Prevalence: Measure and display how frequently each topic appears, presenting unstructured data in a structured format.
• Visualize Thematic Clusters: Explore and interact with thematic clusters and individual data points in an intuitive 2D visual format.
• Filter Data Dynamically: Apply filters based on metadata attributes, relevant entities, and sentiment to refine the data view.
• Augment Insights with Research: Enhance findings by retrieving and incorporating the most relevant past research from your organization's archives.

[...]

Case Study: Shorter Working Week Trial

This demo showcases responses from a survey of several hundred participants in a shorter working week trial. Each respondent answered four questions. Large language models were used to extract individual statements from each response, providing a clearer visualization and analysis of the diversity in opinions.

Method

Inputs

Scribe requires the following input data:

• List of questions
• List of responses to each question
• Optional metadata for each respondent (e.g. job role, nationality)
• Optional list of relevant entity types (e.g. organizations, locations)
• Optional database of relevant PDF files (e.g. research publications, news reports)

This input data is then processed through a series of steps to generate the required data for Scribe's interactive visualisations.

Processing

1. Responses to Statements

Responses to interview questions often contain multiple points of discussion. For a clearer visualisation and analysis of the unique positions raised across responses, a large language model processed each response generating a series of statements each capturing a distinct viewpoint. As previously mentioned, this allows for clearer visualization and analysis of the diversity in opinions.

2. Semantic Space

An embedding model calculates a vector for each statement, mapping it in a semantic space where closer proximity indicates similar meanings. Dimensional reduction techniques were applied to identify clusters of statements similar in meaning, which are then summarized using a large language model to generate a topic title and description of the trends within the topic.

3. Entity Filters and Sentiment

A model trained for named entity recognition was instructed to detect instances of people and activities mentioned within each statement. All of the found entities (i.e. Activity and Person values) were then categorised into higher level 'types' by a large language model (i.e. Activity Type and Person Type values). Furthermore a model finetuned for sentiment analysis was used to classify each statement as positive, neutral or negative in sentiment.

4. Further Reading

To find existing research relevant to the topics identified within each cluster, Scribe integrates a vector database to search for publications by semantic similarity. For this case study, Autonomy's database of research was uploaded to Scribe's vector database to generate relevant citations for each topic cluster. A large language model was used to summarize the relevance of each report to its corresponding cluster.

Explore

To explore survey data with Scribe:

• Select a survey question from the dropdown menu in the Config box to load the corresponding statements from each respondent.
• Individual statements are mapped within the Semantic Space box, with common topics appearing in the Topics box.
• Hover over points in the Semantic Space box to see the statement and respondent metadata in the Data Point box.
• Hover over topics in the Topics box to see the corresponding cluster of statements highlighted in the Semantic Space box.
• To filter statements by respondent metadata or entities, open the Apply Filters drop-down menu in the Config box and select the metadata or entity value(s) to filter by.
• To view the sentiment of each statement (positive neutral negative ), open the Apply Filters drop-down menu and select the Sentiment checkbox.
• Expand cluster titles in the Topics box to find statistics, summaries and individual responses for each cluster. Each cluster includes a Further Reading section containing publications from Autonomy's archive identified as relevant to the cluster topic.

✉️ Please reach out if you would like to explore how Scribe can power your own quantitative research.