Migration 0019 fails with uniqueness violation

[dkliban]

Version
3.23.1

django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_artifact_sha384_pulp_domain_id_46f3e0f5_uniq"
DETAIL: Key (sha384, pulp_domain_id)=(aa14cc19a3bef3118208654d51af22dd1303c48a46eb9b30cbb4997c3505f1367c463dcff3253ba84103ed5ef2361693, 0199e33f-7907-7c1c-b026-99db98363d92) already exists.

The artifact never saves to the database. Seems like the transaction is rolled back. Seems like the batch has multiple Artifacts that have the same digests.

[dkliban]

The latest patch for this issue experiences a new error:

+ set -e
+ /usr/local/lib/pulp/bin/pulpcore-manager migrate --noinput
Operations to perform:
  Apply all migrations: admin, auth, certguard, container, contenttypes, core, file, gem, maven, npm, python, rpm, service, sessions
Running migrations:
  Applying python.0019_create_missing_metadata_artifacts...Traceback (most recent call last):
  File "/usr/local/lib/pulp/bin/pulpcore-manager", line 7, in <module>
    sys.exit(manage())
             ^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulpcore/app/manage.py", line 11, in manage
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/base.py", line 420, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/base.py", line 464, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/base.py", line 111, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/commands/migrate.py", line 353, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 255, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/operations/special.py", line 196, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulp_python/app/migrations/0019_create_missing_metadata_artifacts.py", line 207, in create_missing_metadata_artifacts
    batch_save_artifacts(domain_id)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulp_python/app/migrations/0019_create_missing_metadata_artifacts.py", line 155, in batch_save_artifacts
    for artifact in artifacts:
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/query.py", line 386, in __iter__
    self._fetch_all()
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/query.py", line 1954, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/query.py", line 93, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/sql/compiler.py", line 1623, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/sentry_sdk/utils.py", line 1841, in runner
    return sentry_patched_function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/sentry_sdk/integrations/django/__init__.py", line 645, in execute
    result = real_execute(self, sql, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/utils.py", line 99, in _execute
    self.db.validate_no_broken_transaction()
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/base/base.py", line 521, in validate_no_broken_transaction
    raise TransactionManagementError(
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

[dkliban]

+ set -e
+ /usr/local/lib/pulp/bin/pulpcore-manager migrate --noinput
Operations to perform:
  Apply all migrations: admin, auth, certguard, container, contenttypes, core, file, gem, maven, npm, python, rpm, service, sessions
Running migrations:
  Applying python.0019_create_missing_metadata_artifacts...Traceback (most recent call last):
  File "/usr/local/lib/pulp/bin/pulpcore-manager", line 7, in <module>
    sys.exit(manage())
             ^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulpcore/app/manage.py", line 11, in manage
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/base.py", line 420, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/base.py", line 464, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/base.py", line 111, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/management/commands/migrate.py", line 353, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 255, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/migrations/operations/special.py", line 196, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulp_python/app/migrations/0019_create_missing_metadata_artifacts.py", line 189, in create_missing_metadata_artifacts
    result = artifact_to_metadata_artifact(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulp_python/app/migrations/0019_create_missing_metadata_artifacts.py", line 96, in artifact_to_metadata_artifact
    artifact.file.seek(0)
    ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/core/files/utils.py", line 45, in <lambda>
    seek = property(lambda self: self.file.seek)
                                 ^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/fields/files.py", line 51, in _get_file
    self._file = self.storage.open(self.name, "rb")
                 ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulpcore/app/models/storage.py", line 132, in __getattr__
    storage = domain.get_storage()
              ^^^^^^^^^^^^^^^^^^
AttributeError: 'Domain' object has no attribute 'get_storage'
Sentry is attempting to send 2 pending events
Waiting up to 2 seconds
Press Ctrl-C to quit