By default, the Spotfire Copilot Orchestrator retains all conversation history indefinitely unless explicitly configured otherwise. If the system is deployed in a high-traffic enterprise environment without a data retention policy, the PostgreSQL container will eventually exhaust the host's available disk capacity, potentially leading to service degradation or database crashes.

All

Spotfire Copilot includes a built-in background daemon designed to purge stale data automatically, but it must be manually activated within the Orchestrator's environment configuration file.

To resolve this issue and prevent future database bloat, configure the automated retention policy by following these steps:

Connect to the host machine where the Spotfire Copilot Orchestrator is deployed.

1. Open the main Orchestrator environment configuration file i.e .env file used to deploy the Orchestrator using a text editor.

2. Locate the Conversation Retention section within the file.

3. Uncomment the CONVERSATION_RETENTION_DAYS variable (remove the leading #) and set it to your organization's preferred compliance window in days (e.g., 90).

Example:

# ============================================
# Conversation Retention (optional)
# ============================================

# Automatically delete conversations older than this many days.
# Set to 0 or leave unset to disable automatic cleanup (conversations kept indefinitely).
# When enabled, a background daemon runs daily at 3 AM UTC to remove old threads.
CONVERSATION_RETENTION_DAYS=90

4. Save your changes to the .env file.

5. Restart the Orchestrator Docker Compose stack to apply the new configuration:

docker compose down
docker compose up -d

Note: Once enabled, the background daemon runs automatically at 03:00 AM UTC daily to silently prune threads older than the specified threshold. This ensures the database footprint remains stable and prevents the host from running out of disk space.

After several months of operating in a production environment, the orchestrator-postgres Docker volume may consume an unexpectedly large amount of host disk space. Inspecting the PostgreSQL database tables reveals an accumulation of historical conversation threads, summaries, and user session metadata. This article talks about how to handle this scenario in Spotfire Copilot 2.3.1 and higher.

Community: Spotfire Copilot Installation Guide - Backend Infrastructure Setup

• https://community.spotfire.com/articles/spotfire/spotfire-copilot-installation-guide-backend-infrastructure-setup-r3575/