Importing Vault EDC Data

With Vault CDMS, you can easily move data from Vault EDC into Veeva CDB with the Workbench Export job. The Workbench Export job exports your study’s execution data from EDC to the CDB Workbench application. In CDB Workbench, CDB creates a Core Listing for each Form in your study. Each time you run this job, CDB updates the data within to reflect that in EDC.

If your vault was created after the 24R2 release, you have incremental import enabled in your vault. Skip to the Incremental Import section below. If your vault was created prior to the 24R2 release, you may not have this feature enabled yet.

Availability: Clinical DataBase (CDB) is only available to CDB license holders. Contact your Veeva Services representative for details.

Prerequisites

Before you can use the Workbench Export job to move data from EDC to Workbench, Veeva Services must perform some additional configuration tasks to connect your Study in EDC to your Study in Workbench. Contact your Veeva Services representative for details.


Users with the standard CDMS Lead Data Manager study role or the Vault Owner security profile can perform the actions described below by default. If your vault uses custom Study Roles, your role must have the following permissions:

Type Permission Label Controls
Standard Tab EDC Tools Tab

Ability to access the EDC Tools tab

Functional Permission Manage Jobs

Ability to create, edit, and delete scheduled jobs

Functional Permission View Import

Ability to access the Import page

Functional Permission Download Import Package

Ability to download import packages

If your Study contains restricted data, you must have the Restricted Data Access permission to view it.

Learn more about Study Roles.


Run the Workbench Export Job in EDC

You can choose to run the Workbench Export job once, or schedule it on a recurring basis.

This job sends a package of all study data for Forms in the Submitted and In Edit statuses.

To create a Workbench Export job:

  1. Navigate to Tools > EDC Tools > Jobs.
  2. Click + New Job. New Job Button

  3. For Type, select Workbench Export.
  4. To run the Workbench Export job once, click Run. To schedule a recurring export job, proceed to step 5.
  5. Select a Frequency other than Run Now.
  6. Enter a descriptive Name for your job. This displays in the Job Name column within the Job Schedule tab.
  7. Select a time to Run at. The New Job dialog to schedule a Workbench Export job, running daily at 12:00AM

  8. Click Save.

These job statuses only apply to the EDC job, Workbench Export. To check the status of your import into Workbench, go to Workbench > Import > Packages. See details below.

Job Statuses

Vault displays the status of your job in the job history, along with other details about the job. You can click the Information () icon for additional details about a job.

Status Meaning
Queued Another job of the same type is currently running. Vault will run this job when the previous job is completed.
In Progress Vault is currently performing the job.
Failed The job instance encountered one or more errors.
Completed The job instance completed without errors.

Job Logs & Output Files

Vault provides a CSV job log file for each job. This log includes details about which objects were affected by the job and whether or not the job was successful on each object. When applicable, Vault also provides a link to the output files from a job. For example, a link to download the ODM XML file for an ODM Export job.

To access these files, click the icon () in either the Log or File column.

You can view tables with potential log descriptions for each object below:

Casebook

Object Potential Audit Description
Casebook "Casebook Definition" changed from "[text]" to "[text]"
Casebook Casebook frozen
Casebook Casebook unfrozen
Casebook Casebook locked
Casebook Casebook unlocked
Casebook Casebook: [Screening ID #] created

Event

Object Potential Audit Description
Event "[Date]" was removed from the "Event Date" field
Event "[Date]" was removed from the "Overdue Date" field
Event "Change Reason" set to "[Reason]"
Event "Change Reason" changed from "[Reason]" to "[Reason]"
Event "Did Not Occur" set to "True"
Event "Did Not Occur" set to "False"
Event "Event Date" set to "[Date]"
Event "Event Date" changed from "[Date]" to "[Date]"
Event "Overdue Date" set to "[Date]"
Event "Overdue Date" changed from "[Date]" to "[Date]"
Event Event: [Event Name] created
Event Event Date DMR set to "False"
Event Event Date DMR set to "True"
Event Event Date SDV set to "False"
Event Event Date SDV set to "True"
Event Event Date signed. Signature meaning: [Signature Meaning]
Event Event Date unsigned
Event Event unfrozen
Event Event frozen
Event Event unlocked
Event Event locked
Event Event Date frozen
Event Event Date unfrozen
Event Event Date locked
Event Event Date unlocked
Event Query opened: [Query message]
Event Query answered: [Query message]
Event Query closed: [Query message]
Event Query reopened: [Query message]
Event "Overdue Date" set to "[Date]"
Event "Overdue Date" changed from "[Date]" to "[Date]"
Event "User Review Modified By" set to [User]
Event Event reset. Reason for reset: "[Reason]"

Event Group

Object Potential Audit Description
Event Group Event Group: [Event group name] created

Form

Object Potential Audit Description
Form "Change Reason" set to "[Reason]"
Form "Change Reason" changed from "[Reason]" to "[Reason]"
Form "Form Status" changed from "[Status]" to "[Status]"
Form "Intentionally Left Blank Reason" set to "[Reason]"
Form "[Reason]" was removed from the "Intentionally Left Blank" field
Form "Intentionally Left Blank" set to "True"
Form "Intentionally Left Blank" changed from "False" to "True"
Form "Intentionally Left Blank" changed from "True" to "False"
Form "Marked for Removal" set to "True"
Form "Marked for Removal" changed from "True" to "False"
Form "Number of Submits" changed from "[#]" to "[#]"
Form "Submission Date" set to "[Date]"
Form "Submission Date" changed from "[Date]" to "[Date]"
Form Form: [Form Name] created
Form Form signed. Signature Meaning: "[Signature meaning]"
Form Form unsigned
Form Form frozen
Form Form unfrozen
Form Form locked
Form Form unlocked

Item

Object Potential Audit Description
Item "DMR Mode" set to "[Review Mode]"
Item "SDV Mode" set to "[Review Mode]"
Item "User Review Modified By" set to [User]
Item Item: [Item Name] created
Item Item DMR set to "True"
Item Item DMR set to "False"
Item Item SDV set to "True"
Item Item SDV set to "False"
Item Item frozen
Item Item unfrozen
Item Item set as Intentionally Left Blank with reason "[Reason]". Reason for change: "[Reason]"
Item Item unset as Intentionally Left Blank. Reason for change: "[Reason]"
Item "Intentionally Left Blank Reason" changed from "[Reason]" to "[Reason]"
Item Query opened: [Query message]
Item Query answered: [Query message]
Item Query closed: [Query message]
Item Query reopened: [Query message]
Item Translated value set to "[Value]". Reason for change: "[Reason]"
Item Translated value changed from "[Value]" to "[Value]". Reason for change: "[Reason]"
Item Value entered "[Value]". Reason for change: "[Reason]"
Item Value changed from "[Value] to "[Value]". Reason for change: "[Reason]"
Item Value is "[Value]". Change Reason changed from "[Reason]" to "[Reason]"
Item "User Review Modified By" set to [User]

Item Group

Object Potential Audit Description
Item Group "Change Reason" set to "[Reason]"
Item Group "Change Reason" changed from "[Reason]" to "[Reason]"
Item Group "Intentionally Left Blank Reason" set to "[Reason]"
Item Group "Intentionally Left Blank" changed from "False" to "True"
Item Group "Intentionally Left Blank" changed from "True" to "False"
Item Group "Intentionally Left Blank" set to "True"
Item Group Item Group: [Item Group Name] created
Object Potential Audit Description
Link Form [Form Name](Seq #) link to [Form Name](Seq #) created

Protocol Deviation

Object Potential Audit Description
Protocol Deviation "Category" changed from "[PD Category]" to "[PD Category]"
Protocol Deviation "Date of Deviation" changed from "[Date]" to "[Date]"
Protocol Deviation "Description" set to "[description]"
Protocol Deviation "Description" changed from "[description]" to "[description]"
Protocol Deviation "Inactivated by System" set to "True"
Protocol Deviation "Last Inactivated Date" set to "[Date]"
Protocol Deviation "Protocol Deviation Status" changed from "[Status]" to "[Status]"
Protocol Deviation "Resolution" set to "[Resolution]"
Protocol Deviation "Resolution" changed from "[Resolution]" to "[Resolution]"
Protocol Deviation "Severity" changed from "[Severity]" to "[Severity]"
Protocol Deviation "Subcategory" changed from "[Subcategory]" to "[Subcategory]"
Protocol Deviation "Summary" changed from "[Summary]" to "[Summary]"
Protocol Deviation "User Modified By" changed from "[User]" to "[User]"
Protocol Deviation "User Modified By" set to "[User]"
Protocol Deviation "User Modified Date" changed from "[Datetime]" to "[Datetime]"
Protocol Deviation "User Modified Date" set to "[Datetime]"
Protocol Deviation Protocol Deviation: [PD Name] created

Subject

Object Potential Audit Description
Subject "Enrolled Date" set to "[Date]"
Subject "Enrolled Date" changed from "[Date]" to "[Date]"
Subject "Name" changed from "[Subject ID]" to "[Subject ID]"
Subject "Screened Date" set to "[Date]"
Subject "End of Treatment Date" set to "[Date]"
Subject "Subject Status" changed from "[Status]" to "[Status]"
Subject "Withdrawn Date" set to "[Date]"
Subject "Withdrawn Date" changed from "[Date]" to "[Date]"
Subject Subject: [Subject name] created
Subject "Randomized Date" set to "[Date]"

Vault automatically sends the output file to CDB Workbench. You can download and view the output from EDC Tools > Jobs or from Workbench > Import.

Importing the Data in Workbench

Workbench automatically imports the data exported from Vault EDC, without any additional action from you. Similarly to how 3rd party data is imported into Workbench, the data exported from Vault EDC is normalized at the Form level.

For the import to finish, a user within your study must open a listing. This moves the import into the Complete status. If no one opens a listing, the import remains in the In Progress status.

When import is finished, Workbench sends you, as well as any other users subscribed to the Source, an email notification. If the reprocessing of a package results in a change from the previous load, Workbench will also send a notification to you and users subscribed to that Source.

Workbench automatically generates a Core Listing for each unique Form in a Study. The default CQL query for these listings is:

SELECT @HDR, * from source.filename

For example, if a Study contains a Chemistry form and a Hematology form, CDB creates two Core Listings: Chemistry and Hematology, using these queries:

Chemistry
SELECT @HDR, * from labs.Chemistry
Hematology
SELECT @HDR, * from labs.Hematology

Viewing Import Status

You can check the status of your import package from Import > Packages. This page lists the status of every import package, from both Vault EDC and 3rd party tools. You can also download import packages and issue logs (errors and warnings) from this page.

Complete Status: For an import package to move into the Complete import status, a Workbench user in your Study must open a listing. Otherwise, the import stays in the In Progress status. If your Study has the auto swap feature enabled, this isn’t required.

Users without the Restricted Data Access permission can download the import package log, but they can’t download the data file. Users with restricted data access can download a package with blinded data.

Any time you import a package into Workbench, Workbench automatically reprocesses the most recent packages for all other sources. For example, when your nightly Workbench Export EDC job runs, and then imports into Workbench, Workbench reprocesses your most recent Lab Data and Imaging packages as well. For earlier packages from the same Source, Workbench marks them as Replaced by the newer package.

You can easily filter the list to show only complete or failed imports using the Import Status Filters. Click Error to show only failed imports, or click Complete to show completed imports.

Workbench Import Statuses

When an import package is able to import with only warnings, Workbench highlights the status in orange to indicate that there are warnings. When import finishes, you can download the issue log to review the warnings.

Status Description
Queued The package is in the processing queue. There is a package ahead of this one in line that also contains changes, and this package is waiting for a paused package to be approved or rejected.
Paused CDB detected a change in the manifest, and so import is paused until you approve or reject the package.
Approved The change in the manifest was approved. CDB will now import the package.
Rejected The change in the manifest was rejected.
Skipped The package was skipped and not imported. Another package was imported for the Source before this package was processed. This status can only apply to third party packages.
In Progress The import process has begun for this package, and Workbench has not identified any errors or warnings.
In Progress (with warnings) The import process is ongoing, but Workbench has identified a warning.
Error Import failed because there are one or more errors in the import package. Download the issue log and review the errors.
Complete Workbench imported the package successfully, with no errors or warnings.
Complete (with warnings) Workbench imported the package successfully, but there are one or more warnings. Download the issue log and review the warnings.
Not Imported Workbench skipped this package because a newer package for the same source was uploaded before processing began. When a package enters the Not Imported status, Workbench also replaces the processing date with “Replaced”.
Reprocess in Progress Workbench has begun reprocessing this package because a new package from another source was imported.
Reprocessed Complete Workbench finished reprocessing this package with no errors or warnings.
Reprocessed Complete (with warnings) Workbench finished reprocessing this package, but there are one or more warnings. Download the issue log and review the warnings.
Reprocessed Error Reprocessing failed because there are one or more errors in the import package. Download the issue log and review the errors.

Download the Import Package

To download the import package:

  1. Navigate to Import > Packages.
  2. Locate your import package in the list.
  3. Click the Package link. Click to download the import package

  4. Extract the files from the ZIP folder and view them in a tool of your choosing.

Download the Logs

You can download the Import Log (CSV) for any import and the Issue Log (CSV) for an import that fails. The Import Log lists details about the import job and the data ingestion into Workbench.

The import log lists the following:

  • Transformation Start Time
  • Transformation Completion Time
  • Transformation Duration
  • Import Start Time
  • Import Completion Time
  • Import Duration

To download the import log:

  1. Navigate to Import > Packages.
  2. Locate your import package in the list.
  3. From the Package () menu, select View Package Details.
  4. In the Package Details panel, click Issues.
  5. Optioanl: In the Issues tab, click Download () to download a CSV of the issue log.

Issue Log

The Issue Log lists all errors and warnings that Workbench encountered while importing the package.See a list of possible errors and warnings here.

To view the issue log:

  1. Navigate to Import > Packages.
  2. Locate your import package in the list.
  3. From the Package () menu, select View Package Details.
  4. In the Package Details panel, click Issues.
  5. Optional: In the Issue Log panel, click Download () to download a CSV of the log.

To download the issue log without first viewing it in the application:

  1. Navigate to Import > Packages.
  2. Locate your import package in the list.
  3. From the Package () menu, select Download Issue Log.

Restricted (Blinded) Forms

In Vault EDC, study designers can mark Forms as Restricted. This means that all Items on the Form are also Restricted.

For users who have access to restricted data (typically lead data managers), restricted data behaves the same way as unrestricted data. For blinded users (users without the Restricted Data Access permission), the following behavioral rules apply to any imported restricted data:

  • If an Item (column) is restricted:
    • The CQL projection doesn’t return the restricted item’s column.
    • The CQL projection doesn’t return any derived columns that reference the restricted item.
    • If a blinded user references the restricted Item in their CQL statement, CQL still doesn’t return the column.
    • SHOW and DESCRIBE do not return the restricted Item.
  • If a row is restricted:
    • The result set doesn’t return any rows from the Form or Item Group.
  • If a listing file (csv) is restricted:
    • The default @HDR columns are included in the listing, but no Item columns are included.
  • If a Source (package) is restricted:
    • CQL doesn’t return any Items or column results from the restricted Source in any listing.
    • CDB marks all Item Definitions, Item Group Definitions, and Form Definitions within the Source as restricted.
    • All data rows are marked as restricted.
    • The default @HDR columns will still appear in the core listings.

Incremental Import (24R2 & Later)

Incremental Import Enablement: CDB Incremental Imports is enabled by Veeva Support. Veeva is working with customers to enable this feature for customers between the 24R2 (August 2024) and 24R3 (December 2024) releases. Contact your Veeva Services representative for information about enablement.

With incremental import, CDB imports your study data from Vault EDC every fifteen (15) minutes for new data. This happens automatically, without any action required from the user. Data from Vault EDC is normalized at the form level.

Study design changes are loaded once a day at 12:00 GMT. If a design change is detected, all incremental refreshes are paused until 12:00 GMT when the latest study design is loaded.

These incremental import packages include the following data:

  • Study properties
  • Site properties
  • Subject properties
  • All casebook data
  • Queries
  • SDV Status
  • DMR Status

CDB creates a Core Listing for each Form in your Study. A Core Listing contains the following columns, but note that an administrator may have configured your Study to have different core listing columns:

  • Study.Name
  • Site.Name
  • Site.PI
  • Event.Name
  • Event.Date
  • Event.Status
  • Subject.Name
  • Subject.Status
  • A column for each Item on the Form

When import is finished, CDB sends a notification to any users subscribed to the Source, an email notification.