eva.admin.alpha.api — com.workiva/eva 2.0.3

Namespace for collecting a suite of tools for administrative tasks on eva databases, such as migration and backup.

cas-node! clj

(cas-node! store cur-n new-n)

migrate-batch clj

(migrate-batch {:as migration-info-blob
                :keys [source-tx-log dest-store graph context
                       cur-migration-status status-key]}
               tx-nums)

migration-status clj

(migration-status key status tx-num)

now clj

(now)

stateful-migration! clj

(stateful-migration!
  source-storage-config
  destination-storage-config
  database-id
  &
  {:as options
   :keys [log-every migration-status-key-suffix active-parents]
   :or {log-every 25 migration-status-key-suffix "migration-status"}})

ALPHA

Performs a stateful migration of all data from the source store to the destination store. Will actively read from both the source and destination stores to provide both continuous and iterative migrations.

Periodically records the migration progress using a specific key-value pair in the destination store.

Options:

:log-every specifies that the status of the migration should be persisted to the destination store every <log-every> tx-log-entries that are migrated. Defaults to 25.

:migration-status-key-suffix provide a suffix for looking up and recording the state of a migration. The key will be of the form: <database-id>.<migration-status-key-suffix> Defaults to 'migration-status'.

:write-xform (NOT YET IMPLEMENTED) an optional xform on Traversable->Traversable that will be applied before nodes are written. Value store operations will be applied to the post-xform value, so create and replace in particular will have their restrictions applied to the post-transformation object. Great Caution MUST be exercised with xforms that alter the keys or values of traversable objects. Defaults to (map identity).

:read-xform (NOT YET IMPLEMENTED) an optional xform that is passed to the underlying traversals. Defaults to (traversal/unique-on traversal/k)

Properties:

1. Performs a lazy DFS traversal of the transaction log from either the start of the tx-log, or the last transaction log entry noted in the recorded migration state.
2. Stateful migrations are safe to run against an active source database.
3. Stateful migrations should not be run against an active destination database. They will likely fail in unexpected ways.
4. Stateful migrations can safely resume when unexpectedly halted part-way through, and will continue / resume migrations that have completed if the source database has advanced since the previous migration. If operating on a previously-completed stateful migration, will CAS on the transaction log head and database-info to ensure that no constraints are violated.
5. Will not overwrite extant nodes. To ensure this property is not violated, create-key with a try-catch pattern is used. If the create fails, we assert that the persisted node is equivalent to what we attempted to write.
6. Simple migrations are agnostic to storage location but are not agnostic to the database constructs. To provide iterative and continuous migrations, a working knowledge of the transaction log representation is required.
7. Writes occur in batches and follow a postordering traversal of stored data. This ordering ensures that all references under a node must exist before the node itself exists. This property is slightly violated when a completed migration is sourced against a database that has advanced: the tx-log root will exist and will not be updated until all new source tx-log entries have been migrated.