Upgrading to DefectDojo Version 3.5.300

Alert IDs are 64-bit

Every alert DefectDojo creates takes the next ID from a sequence. The alerts table is kept small (the cleanup_alerts task keeps the newest DD_MAX_ALERTS_PER_USER alerts per user, 999 by default), but the sequence counts every alert ever created. On a busy instance it could reach the 32-bit maximum, after which every new alert failed with nextval: reached maximum value of sequence "dojo_alerts_id_seq" (2147483647).

Migration 0312_alerts_bigint_id changes the alert ID column and its sequence to 64-bit (bigint). Columns in other tables that point at an alert are widened in the same step.

What you need to do

Usually nothing. PostgreSQL rewrites the alerts table, and any table that references it, while holding an exclusive lock on it, so creating or reading alerts waits until the migration finishes. The time depends on the number of alerts. Find yours before upgrading:

SELECT count(*) AS alerts,
       count(*) FILTER (WHERE user_id_id IS NULL) AS system_alerts,
       pg_size_pretty(pg_total_relation_size('dojo_alerts')) AS size
FROM dojo_alerts;

As a guide, one million alerts (about 1,000 users at the default limit) took about six seconds on a test machine.

The table can be much larger than the limit suggests when pruning is off (DD_MAX_ALERTS_PER_USER=-1), when Celery beat is not running, or when system alerts (alerts with no user) have built up, since cleanup_alerts does not remove those. To keep the upgrade short, shrink the table first: turn pruning back on and let cleanup_alerts run, and remove system alerts with python manage.py clear_alerts --system (--users <username> removes one user’s alerts, --all removes every alert). Otherwise, plan the upgrade for a maintenance window.

For more information, check the Release Notes.