Refactor add foreign key to ecu

[jesperhodge]

Internal issue: https://2u-internal.atlassian.net/browse/ENT-9961

Other PRs that are part of this:

• Refactor add foreign key to ecu part 2 #2339
• Refactor add foreign key to ecu part 3 #2341
• Refactor add foreign key to ecu part 5 #2345

Description

Add a Foreign Key, using Django’s from the EnterpriseCustomerUser model to the auth.User model. This will help greatly simplify the means by which we support filtering/searching/sorting on user fields (e.g. first/last name, email, etc.) in the context of Enterprise DRF viewsets.

This will require multiple PRs to avoid performance problems. This specific PR adds the new field without index and without foreign key reference.

Strategy

Zero-Downtime Step-by-Step Approach (Foreign Key Last)

Step 1: Add Nullable Column Without Constraint (Non-Blocking)

This operation is instant in Aurora when the column is nullable.

ALTER TABLE enterprise_enterprisecustomeruser 
ADD COLUMN user_fk INT NULL;

Non-blocking because the column is nullable and no default is set.

Step 2: Create Index on the New Column (Online, Non-Blocking)

Index creation in Aurora MySQL is online with these options:

ALTER TABLE enterprise_enterprisecustomeruser
ADD INDEX idx_user_fk (user_fk),
ALGORITHM=INPLACE,
LOCK=NONE;

Why this is critical:

MySQL uses the index for fast FK validation later, making the FK addition much faster and non-blocking.

Step 3: Backfill Data in Small Batches (Without FK Checks)

Because there’s no foreign key constraint yet, backfilling is fast and won’t lock the table.
Use small batches (size 1000) to avoid saturating the I/O or CPU.
The backfill should copy over the value from user_id to the new field.

This should use a management command. The management command can easily run in batches and be monitored, and it reduces the risk of downtime versus using a data migration.

At the same time, add something to the EnterpriseCustomerUser.save() method (or maybe a signal handler?) that will update user_fk with the value in user_id on save. That will help ensure that we don't end up with new ECU records created that don't copy over the data.

Step 4: Validate Data Integrity Before Adding FK

• Ensure all rows have valid references to auth_user
• Ensure that values in user_id and user_fk are identical and not null

Step 5: Add the Foreign Key Constraint (Online, Minimal Locking)

Finally, add the foreign key constraint using Aurora’s online DDL. For this, 'foreign_key_checks' must be temporarily disabled in the session to support the inplace algorithm:

SET FOREIGN_KEY_CHECKS = 0;

ALTER TABLE enterprise_enterprisecustomeruser
ADD CONSTRAINT fk_user_fk FOREIGN KEY (user_fk) REFERENCES auth_user(id),
ALGORITHM=INPLACE,
LOCK=NONE;

SET FOREIGN_KEY_CHECKS = 1;

Why this avoids downtime:
ALGORITHM=INPLACE: Adds the constraint without copying the entire table.
LOCK=NONE: Ensures DML operations (INSERT/UPDATE/DELETE) can continue during the operation.

Step 6: validate data integrity again after adding FK

Merging this

This should be a series of PRs that are merged and deployed in order.

Merge checklist:

• Any new requirements are in the right place (do not manually modify the requirements/*.txt files)
  • base.in if needed in production but edx-platform doesn't install it
  • test-master.in if edx-platform pins it, with a matching version
  • make upgrade && make requirements have been run to regenerate requirements
• make static has been run to update webpack bundling if any static content was updated
• ./manage.py makemigrations has been run
  • Checkout the Database Migration Confluence page for helpful tips on creating migrations.
  • Note: This must be run if you modified any models.
    • It may or may not make a migration depending on exactly what you modified, but it should still be run.
  • This should be run from either a venv with all the lms/edx-enterprise requirements installed or if you checked out edx-enterprise into the src directory used by lms, you can run this command through an lms shell.
    • It would be ./manage.py lms makemigrations in the shell.
• Version bumped
• Changelog record added
• Translations updated (see docs/internationalization.rst but also this isn't blocking for merge atm)

Post merge:

• Tag pushed and a new version released
  • Note: Assets will be added automatically. You just need to provide a tag (should match your version number) and title and description.
• After versioned build finishes in GitHub Actions, verify version has been pushed to PyPI
  • Each step in the release build has a condition flag that checks if the rest of the steps are done and if so will deploy to PyPi.
    (so basically once your build finishes, after maybe a minute you should see the new version in PyPi automatically (on refresh))
• PR created in edx-platform to upgrade dependencies (including edx-enterprise)
  • Trigger the 'Upgrade one Python dependency' action against master in edx-platform with new version number to generate version bump PR
  • This must be done after the version is visible in PyPi as make upgrade in edx-platform will look for the latest version in PyPi.
  • Note: the edx-enterprise constraint in edx-platform must also be bumped to the latest version in PyPi.