AI and LLMs¶
Introduction¶
We are investigating a variety of possible ways of leveraging LLMs, but some key points:
The underlying structures of Chimera are being leveraged (for instance, when you create a chart via Chat it is also a chart you can modify in the advanced charting or add it to a report)
We are building Chimera to be agnostic about what providers and models are used
We want to ensure we are using these technologies in responsible ways
We attempt to provide feedback on what you are asking (for instance, if you ask for a chart on data that does not exist in your analysis so far)
We are in very much an exploratory phase just like everyone else
Asking LLMs about Every Page (LLM Prompt Fields)¶
Chimera has a method to define sets of prompts so we can ask questions about many aspects of each piece of content in an efficient way – LLM Prompt Fields can be used to get values like topics, audience, and other elements that may require a deeper understanding of a page than more traditional approaches like scraping.
Chat Overview¶
We see chat as an exciting new way to explore and understand a site. Chat is in two modes: Chat Content Analysis and Chat Semantic Search.
Of course this Chat has a question-and-answer approach, but in addition it:
Injects system messages as well, that can include things like suggested prompts or places you can go for deeper exploration on the question you asked
At the bottom of the page you will see any system-generated suggested places for more analysis or suggested prompts
When charts are created they are full-blown Chimera charts (that you can click on, etc)
Dimension |
Content Analysis |
Semantic Search |
|---|---|---|
Content Considered |
All |
All |
Content Analyzed |
All |
Less than ten pages |
Data Source |
Existing fields in the analysis |
The text of the content itself |
Level of Automated Understanding |
Low |
High |
Chat Content Analysis¶
Content Analysis can respond to the following types of questions:
Chart - Questions asking for charts, graphs, plots, or “show the distribution”
Single Number - Questions that would be answered with one number, like “How many” or “what is the percentage”
Table - Questions that would be answered with a table with rows and columns
Single Category - Questions where the answer is a simple string, like “What is the most popular folder?”
Fields - Questions asking what data or fields are available
It’s important to check what fields are being used for the responses – that’s up for you as the human to confirm.
Note
Chat Content Analysis has the advantage of looking at detailed metadata for every piece of content, but that also means the data must already exist. Note that in Content Analysis mode you can ask “What fields are available for analysis?” to see the fields at your disposal. Chimera is built to add and transform the information it has, but that functionality cannot currently be initiated from chat.
Chat Semantic Search¶
This is a RAG (Retrieval-Assisted Generation) search that happens in two steps:
Look against all content in the site (using a vector database) to find the most relevant pages
Pass that information to an LLM to answer the question
This approach allows for more nuanced questions about the actual content of the site, leveraging the full text rather than just structured metadata fields.
Note
Semantic search considers all the pages as potentially useful to answer your question, but it only passes the most relevant content to an LLM to answer. This means it cannot answer questions across all the content like Content Analysis mode can. That said, you can use LLM Prompt Fields to use an LLM to glean information off of every page (or a section of pages), which are then fields that can be used in Content Analysis. Managing LLM Prompt Fields is not currently available directly in Chat.
