Importing Veeva EDC Data

With Veeva Clinical Data, data from Veeva EDC flows incrementally into Veeva CDB. In CDB Workbench, CDB creates a Core Listing for each Form in your study. Each time you run this job, CDB updates the data within to reflect that in EDC.

Importing the Data in Workbench

With incremental import, CDB imports your study data from Veeva EDC every fifteen (15) minutes for new data. This happens automatically, without any action required from the user. Data from Veeva EDC is normalized at the form level.

Study design changes are loaded once a day at 00:00 GMT. If a design change is detected, all incremental refreshes are paused until 00:00 GMT when the latest study design is loaded.

These incremental import packages include the following data:

• Study properties
• Site properties
• Subject properties
• All casebook data
• Queries
• SDV Status
• DMR Status

CDB creates a Core Listing for each Form in your Study. A Core Listing contains the following columns, but note that an administrator may have configured your Study to have different core listing columns:

• Study.Name
• Site.Name
• Site.PI
• Event.Name
• Event.Date
• Event.Status
• Subject.Name
• Subject.Status
• A column for each Item on the Form

When import is finished, Workbench sends you, as well as any other users subscribed to the Source, an email notification. If the reprocessing of a package results in a change from the previous load, Workbench will also send a notification to you and users subscribed to that Source.

Viewing Import Status

You can check the status of your import package from Import > Packages. This page lists the status of every import package, from both Vault EDC and 3rd party tools. You can also download import packages and issue logs (errors and warnings) from this page.

Users without the Restricted Data Access permission can download the import package log, but they can’t download the data file. Users with restricted data access can download a package with blinded data.

Any time you import a package into Workbench, Workbench automatically reprocesses the most recent packages for all other sources. For example, when your nightly Workbench Export EDC job runs, and then imports into Workbench, Workbench reprocesses your most recent Lab Data and Imaging packages as well. For earlier packages from the same Source, Workbench marks them as Replaced by the newer package.

You can easily filter the list to show only complete or failed imports using the Import Status Filters. Click Error to show only failed imports, or click Complete to show completed imports.

Workbench Import Statuses

When an import package is able to import with only warnings, Workbench highlights the status in orange to indicate that there are warnings. When import finishes, you can download the issue log to review the warnings.

Download the Import Package

To download the import package:

1. Navigate to Import > Packages.
2. Locate your import package in the list.

3. Click the Package link.

4. Extract the files from the ZIP folder and view them in a tool of your choosing.

Download the Logs

You can download the Import Log (CSV) for any import and the Issue Log (CSV) for an import that fails. The Import Log lists details about the import job and the data ingestion into Workbench.

The import log lists the following:

• Transformation Start Time
• Transformation Completion Time
• Transformation Duration
• Import Start Time
• Import Completion Time
• Import Duration

To download the import log:

1. Navigate to Import > Packages.
2. Locate your import package in the list.
3. From the Package () menu, select View Package Details.
4. In the Package Details panel, click Issues.
5. Optioanl: In the Issues tab, click Download () to download a CSV of the issue log.

Issue Log

The Issue Log lists all errors and warnings that Workbench encountered while importing the package.See a list of possible errors and warnings here.

To view the issue log:

1. Navigate to Import > Packages.
2. Locate your import package in the list.
3. From the Package () menu, select View Package Details.
4. In the Package Details panel, click Issues.
5. Optional: In the Issue Log panel, click Download () to download a CSV of the log.

To download the issue log without first viewing it in the application:

1. Navigate to Import > Packages.
2. Locate your import package in the list.
3. From the Package () menu, select Download Issue Log.

Restricted (Blinded) Forms

In Veeva EDC, study designers can mark Forms as Restricted. This means that all Items on the Form are also Restricted.

For users who have access to restricted data (typically lead data managers), restricted data behaves the same way as unrestricted data. For blinded users (users without the Restricted Data Access permission), the following behavioral rules apply to any imported restricted data:

• If an Item (column) is restricted:
  • The CQL projection doesn’t return the restricted item’s column.
  • The CQL projection doesn’t return any derived columns that reference the restricted item.
  • If a blinded user references the restricted Item in their CQL statement, CQL still doesn’t return the column.
  • SHOW and DESCRIBE do not return the restricted Item.
• If a row is restricted:
  • The result set doesn’t return any rows from the Form or Item Group.
• If a listing file (csv) is restricted:
  • The default @HDR columns are included in the listing, but no Item columns are included.
• If a Source (package) is restricted:
  • CQL doesn’t return any Items or column results from the restricted Source in any listing.
  • CDB marks all Item Definitions, Item Group Definitions, and Form Definitions within the Source as restricted.
  • All data rows are marked as restricted.
  • The default @HDR columns will still appear in the core listings.

Core Listings from EDC Import

Workbench automatically generates a Core Listing for each unique Form in a Study. The default CQL query for these listings is:

SELECT @HDR, * from source.filename

For example, if a Study contains a Chemistry form and a Hematology form, CDB creates two Core Listings: Chemistry and Hematology, using these queries:

SELECT @HDR, * from labs.Chemistry

SELECT @HDR, * from labs.Hematology