Knowledge
The Knowledge section of the Workspace is where you can manage your organization's knowledge. These are used in RAG (Retrieval Augmented Generation) pipelines. Knowledge is organized in Knowledge Bases, and each Knowledge Base can contain multiple documents.
Additional permissions are required to access the workspace. If you don't have access, but think you should, please contact your organization's administrator.
Knowledge Bases
Knowledge Bases are used to organize your documents.
Creating Knowledge Bases
You can create a new Knowledge Base by clicking the "+" button in the top left corner. When creating a new Knowledge Base, you are asked to provide a name and a description. Provide a clear name and description to help you and your colleagues find the right documents.
Uploading Documents
After creating a Knowledge Base, you can open it and start uploading documents by clicking the + icon on the topright. You can upload .docx (word) or .pdf documents. If word documents are available, they are recommended over pdf documents, because they are more accurate.
For pdf documents, there are two different loaders available: the PyPDF Parser directly extracts text from the pdf document, whereas the LLM Loader takes screenshots of the pdf pages and then uses the LLM's visual capabilities to extract text. The former is recommended for text-heavy documents; the latter is recommended for documents with lots of visualizations. The PDF loader type is a global setting configured in RAG Settings.
Cleaning Chunks
By clicking the Clean Chunks button, you can ask an LLM to go over all of your chunks and clean their texts, e.g. from formatting issues. The prompt used for cleaning is configured globally in RAG Settings under "Global Chunk Cleaning Prompt".
Transcription Instructions
By clicking the Transcription Instructions button, you can provide custom instructions for the vision model (VLM) when the LLM Loader is used for uploading .pdf files. These instructions are per Knowledge Base and are appended to the global transcription system prompt configured in RAG Settings. For example, you can instruct the model to transcribe in a specific language or to ignore decorative elements.
Viewing and Editing Documents
After a file has been processed, selecting it in the file explorer will open the file viewer. The file viewer will show the file content and allow you to edit the chunks.
Downloading Documents
You can download a document by first selecting the document in the file explorer and then clicking on the filename in the file viewer. A download should start automatically.
Deleting Knowledge Bases
You can delete a Knowledge Base by clicking the "..." button in the top right corner of the Knowledge Base and selecting "Delete". Knowledge Bases can only be deleted by the user that created the Knowledge Base or by a knowledge admin.
Deleting Documents
You can delete individual documents by hovering over the document in the file explorer and clicking the 'x' button that appears.