Connectors

Overview

The Connect to Source panel allows administrators to link cloud storage services — Dropbox, Microsoft OneDrive, and Microsoft SharePoint — to the application. Once connected, you can browse folders, select files, and kick off background processing that extracts text, generates embeddings, and indexes content for AI-powered search. Only vectors and file links are stored; file content is streamed when needed for citations.

---

Table of Contents

1. Accessing the Connect to Source Panel
2. Adding a Connector
   • Adding a Dropbox Connector
   • Adding a Microsoft Connector (OneDrive or SharePoint)
3. Opening the Browse Modal
4. Browsing and Selecting Files — Dropbox
5. Browsing and Selecting Files — OneDrive
6. Browsing and Selecting Files — SharePoint
7. Processing Selected Files
8. Background Jobs
   • What Are Background Jobs?
   • Viewing the Background Jobs Panel
   • Job Statuses Explained
   • Job Detail Panel
   • Cancelling a Job
   • Retrying Failed Items
9. Troubleshooting

---

Accessing the Connect to Source Panel

1. Log in with admin privileges
2. Open the Admin Dashboard from the main navigation
3. Select the “Connectors” tab

For a map of all Admin Dashboard tabs, see the Administration Introduction.

The panel displays:

• A Dropbox integration card
• A Microsoft integration card (for both OneDrive and SharePoint)
• A list of your existing connectors as source pills

---

Adding a Connector

Adding a Dropbox Connector

1. In the Dropbox integration card, type a display name for this connector (e.g. “Marketing Assets”) in the name input field
2. Click Connect Dropbox
3. Your browser will be redirected to Dropbox OAuth — log in and grant access when prompted
4. After authorisation you are returned to the panel automatically
5. The new connector appears as a source pill beneath the integration card

Note: You can add multiple Dropbox connectors, each scoped to a different Dropbox account or team member file.

---

Adding a Microsoft Connector (OneDrive or SharePoint)

1. In the Microsoft integration card, select the source type using the OneDrive / SharePoint toggle:
   • SharePoint — Team sites and document libraries
   • OneDrive — Personal drive files
2. Type a display name for this connector
3. Click Connect Microsoft
4. Your browser will be redirected to Microsoft OAuth — sign in with your Microsoft 365 account and consent to the requested permissions
5. After authorisation you are returned to the panel automatically
6. The new connector appears as a source pill labelled with the display name you chose

Tip: The source type (OneDrive vs SharePoint) is locked when the connector is created and determines which sections are available when browsing.

---

Opening the Browse Modal

Once one or more connectors exist, you select a source to browse its files.

Step 1: Select a Source Pill

1. Locate the row of source pills (coloured tags) beneath the integration card
2. Click the pill for the connector you want to browse
3. The pill highlights to show it is active

Step 2: Open the Connector

1. Click the Open button (or double-click the pill) to open the full Browse modal
2. The modal slides in from the right side of the admin panel at full height
3. A loading spinner appears while the root folder listing is fetched

Step 3: Modal Controls

At the top of the browse modal you will see:

Control	Purpose
Connector name	Shows the currently open connector
Shared link / Scope URL field	Optionally scope the browse tree to a folder link (Dropbox) or a specific URL (OneDrive/SharePoint)
EXIT button	Closes the browse modal and returns to the source list

---

Browsing and Selecting Files — Dropbox

Step 1: Optional — Set a Shared Link (Scoped Browse)

1. In the shared link input field at the top of the modal, paste a Dropbox folder or file shared link
2. Click Open URL to browse only that linked folder
3. Optionally click Save URL to persist the link to the connector so it loads automatically next time

Leave the field empty to browse the root of the connected Dropbox account.

Step 2: Browse the Folder Tree

1. The file and folder tree loads automatically
2. Click the arrow / chevron next to any folder to expand it
3. Child items load inline — the tree is lazy-loaded, so deeper levels fetch on demand
4. A loading spinner appears next to a folder while its children are being fetched

Step 3: Select Files and Folders

1. Check the checkbox next to any file or folder to select it
2. Selecting a folder selects all files inside it recursively at processing time (you do not need to expand it)
3. The header updates to show a green badge: “X files selected” or “X files + Y folders (processed recursively) selected”
4. Uncheck an item to deselect it

---

Browsing and Selecting Files — OneDrive

The OneDrive browse modal includes a left sidebar with navigation sections.

Step 1: Optional — Set a Scope URL

1. In the OneDrive URL input at the top, paste a OneDrive folder URL (e.g. from the OneDrive web app)
2. Click Open URL to navigate directly to that folder
3. Click Save URL to persist it — the connector will default to this folder on next open

Step 2: Choose a Browse Section (Left Sidebar)

The left sidebar lists four sections. Click any section to switch the file tree:

Section	What It Shows
Home	OneDrive home view — recommended starting point
My files	All files and folders in the signed-in user’s OneDrive
Shared	Files and folders shared with you or by you
Favorites	Files you have starred/favourited

Switching section resets the current selection and folder expansion state.

Step 3: Shared Section Filters

When the Shared section is active, two additional filter rows appear:

• People filter: “Shared with you” / “Shared by you”
• Type filter: All / Folder / Word / Excel / PowerPoint / PDF

Use these to narrow down shared items before selecting.

Step 4: Browse and Select

1. Expand folders using the chevron arrow
2. Check/uncheck files and folders as needed
3. The header badge shows the running count of selected items

---

Browsing and Selecting Files — SharePoint

The SharePoint browse modal also includes a left sidebar and additional site-level navigation.

Step 1: Optional — Set a Site URL

1. In the SharePoint site URL input at the top, paste a URL to a specific SharePoint site (e.g. https://contoso.sharepoint.com/sites/Marketing)
2. Click Open URL to scope the tree to that site
3. Click Save URL to persist it for future sessions

Removing the URL pill (click × next to the saved URL) resets the connector to browse all accessible sites.

Step 2: Choose a Browse Section (Left Sidebar)

Section	What It Shows
Start page	Sites and documents you follow, recently visited, and frequently used — the best starting point
My sites	All SharePoint sites you have access to
My files	Files in your personal OneDrive for Business within SharePoint
My list	SharePoint lists you have access to

Switching section resets the current selection and folder expansion state.

Step 3: Start Page — Following, Recent, Frequent

When Start page is active, three groups of cards appear:

• Following — Sites or documents you explicitly follow
• Recent — Recently visited sites and files
• Frequent — Sites and files you access most often

Click a site card on the Start page to navigate into that site’s document libraries. You can then expand library folders and select files.

Step 4: Browse and Select

1. Expand a site or library using the chevron arrow
2. Check/uncheck folders and files
3. Selected folders are processed recursively — you do not need to expand them first
4. The header badge updates to confirm the selection count

Important: If a SharePoint folder shows in the tree but was not fully loaded, you may see an error when processing. Open the folder in the tree first, then try again.

---

Processing Selected Files

After selecting files or folders across any connector type:

Step 1: Assign to a Project

1. Use the Project selector at the bottom of the modal to choose one or more projects this content should be indexed into
2. All selected files are assigned to every checked project

Step 2: Click Process

1. Click the Process button (primary action button in the modal footer)
2. The system validates your selection:
   • At least one file or folder must be selected
   • At least one project must be assigned
3. If validation passes, the request is sent to the backend

Step 3: Watch the Progress Bar

• A progress bar appears at the bottom of the modal
• The bar shows: files done / total files, estimated time remaining, and throughput (files/min)
• The bar turns green when all files complete successfully
• During processing the header shows: “Streaming, extracting text, and vectorising files. You can switch tabs or leave; check Background jobs to see progress.”

Tip: You can close the modal or navigate away immediately after clicking Process. The background workers continue running independently.

---

Background Jobs

What Are Background Jobs?

When you click Process, files are queued. Each worker:

1. Streams the file from the source (Dropbox/OneDrive/SharePoint) — the file itself is never permanently stored
2. Extracts text from the document (PDF, Word, Excel, etc.)
3. Generates vector embeddings using the configured AI model
4. Indexes the content into the database so it is searchable via AI chat

This is done asynchronously so the UI stays responsive. Large syncs with hundreds of files may take several minutes to complete. Jobs survive page navigation — they are tracked in both Redis (fast, 7-day TTL) and the database (persistent fallback).

---

Viewing the Background Jobs Panel

The background jobs panel can be accessed from the Connect to Source panel header.

Method 1: Background jobs button

1. Click the Background jobs button in the top-right corner of the Connect to Source panel
2. The inline jobs panel expands below the panel header
3. Click Hide background jobs to collapse it

Method 2: “View progress” badge

1. If a job is currently running, an In Progress badge and a green View progress button appear in the panel header
2. Click View progress to jump directly to the jobs panel

Panel Controls

Control	Purpose
Active tab	Shows jobs currently running or queued
Completed tab	Shows finished (success, failed, cancelled) jobs
All Jobs tab	Shows every job regardless of status
Auto-refreshing	The list refreshes every 4 seconds automatically
Jobs continue running in the background	Footer reminder — closing the panel does not stop jobs

---

Job Statuses Explained

Each job card displays a colour-coded status:

Status	Colour	Meaning
Processing	Blue (pulsing dot)	Worker is actively extracting and embedding files
Queued	Grey	Job has been accepted and is waiting for a worker to pick it up
Completed	Green	All files were processed successfully
Failed	Red	One or more files could not be processed; details available in the job panel
Cancelled	Amber	Job was cancelled by an admin before it completed

Each card also shows:

• Connector display name
• Started at timestamp
• Duration (elapsed or total)
• Progress bar — files done vs total, with skipped and failed counts highlighted

---

Job Detail Panel

Click any job card to expand the detail panel below it.

The detail panel has tabbed sections:

Overview tab

• Job ID (batch ID, useful for support)
• Connector name
• Duration and Started timestamp
• Error message (if status is Failed)

Files tab (when completed)

• Created — number of new document records indexed
• Files processed — total file count handled
• Skipped — files skipped (duplicates, unsupported types, etc.)
• Failed files — list of files that errored with individual error messages

Progress tab (when active)

• Live stage indicator: Preparing → Scanning → Processing → Done
• Current file being processed
• Files/min throughput and ETA

---

Cancelling a Job

1. On any active job card, click the Cancel button (shown when the job is in Processing or Queued state)
2. Confirm the cancellation if prompted
3. The job status changes to Cancelled and all related Celery tasks are force-stopped via Redis signals
4. Partial results are preserved — files that had already been indexed remain indexed

Cancel is irreversible. To re-process the same files, return to the browse modal, re-select the files, and click Process again.

---

Retrying Failed Items

1. Open a Failed job’s detail panel
2. Switch to the Files tab
3. A Retry Failed button appears listing the files that errored
4. Click Retry Failed — the connector browse modal re-opens with those files pre-selected
5. Click Process again to re-queue only the failed items

---

Troubleshooting

Issue: Connector button shows “Connecting…” for a long time

Solutions:

• The OAuth popup/redirect may have been blocked. Check your browser’s popup blocker
• Reload the page and try connecting again
• Ensure your Dropbox or Microsoft account has sufficient permissions

Issue: File tree shows “Failed to load” or stays empty

Solutions:

• Click the Refresh icon or reload the browse modal (click EXIT then re-open the connector)
• For Dropbox: verify the shared link is publicly accessible
• For SharePoint: ensure the saved site URL is correct and your account has read access
• Check your network connection

Issue: Process button is disabled

Solutions:

• You must select at least one file or folder
• You must assign at least one project
• If the loading spinner is still spinning in the header, wait for the file count to finish calculating before processing

Issue: Job stuck in “Processing” for a very long time

Solutions:

• Check the Background jobs panel for the current stage and file count
• Large folders with hundreds of files may take 10–20+ minutes
• Verify the Celery worker is running (check with your system administrator)
• If the job appears frozen with no progress for over 30 minutes, cancel it and re-process in smaller batches

Issue: Many files show as “Skipped”

Solutions:

• Skipped files are typically duplicates already indexed in the same project
• Files may also be skipped if they are an unsupported format (e.g. binary files, executables)
• If needed, remove the existing indexed version from the project first, then re-process

Issue: Background Jobs panel shows no jobs

Solutions:

• Jobs are stored with a 7-day TTL in Redis. Jobs older than seven days are no longer listed
• Ensure the Celery worker and Redis are running (contact your system administrator)
• If a job was running when the worker restarted, it will not reappear — re-process the files

Issue: SharePoint Start page shows no sites

Solutions:

• Your account may not be following any SharePoint sites yet — try the My sites section instead
• Sign in to SharePoint directly and follow some sites, then return and refresh
• If the issue persists, paste the specific SharePoint site URL into the scope URL field and use Open URL

---