Data Import Configuration for Data Loader
MedTech companies that process lab samples, such as blood or DNA, for diagnostic purposes, or pharmaceutical companies that develop targeted therapies, use machines that receive samples in such large quantities that they produce a file containing the results of one or more tests. Vault EDC can import the data from those files and automatically enter it into configured Forms and their data collection Items, instead of users needing to manually enter it.
For data loaders to import this data into Vault EDC, a study designer must first create a Data Import Configuration for it. The Data Import Configuration identifies the destination Form and maps the columns in the CSV file to identifiers, such as Subject ID or Site, and data collection Items on the destination Form. When a data loader imports a CSV, they select the Data Import Configuration that Vault uses to process the data during import.
Configuration per Form Definition: You must create a Data Import Configuration for each Form Definition that you want to load data into. So, if you are loading machine or device data into more than one form definition, you must create a separate Data Import Configuration in Studio for each Form. This doesn’t apply to instances of a form in a single Form Definition.
Prerequisites
Contact Veeva Support to enable this feature in your vault.
Once this feature is enabled in your vault, you can enable it in your Studies.
To enable data loader for your Study:
- Navigate to your Development environment.
- Navigate to your Study in Studio > Study Settings.
- Click Edit.
- Select Yes for Enable Data Loader.
- Click Save.
Data loader is now enabled in your Development environment. As you deploy to your other environments, the feature will also be enabled there.
Users with the standard CDMS Study Designer study role can perform the actions described below. If your organization uses custom roles, your role must grant the following permissions:
Type | Permission Label | Controls |
---|---|---|
Standard Tab | Studio Tab | Ability to access the Studio tab |
Functional Permission | Design Study | Ability to create study design definitions and a study schedule from Studio |
Create a New Data Import Configuration
- Navigate to Studio > Studies > Data Loader for your Study.
- Click + New.
- Enter a Name for your Data Import Configuration. This Name can’t contain any spaces or special characters, except for dashes (-) and underscores (_).
- Enter a Label for your Data Import Configuration.
- Optional: Enter a Description.
- Select the Destination Form. This is the Form that your imported data is loaded into.
- Select your Column header creation method. See more details below.
- Upload file: Upload a CSV file. Vault will take each of the column headers in your CSV and create rows in the mapping table for them. Then, you can map each of those columns to a form item.
- Create manually: Create CSV column header mappings in the mapping table. There must be a mapping for each column in your CSV.
- If you chose Upload file, drag and drop your file into the Drag and Drop area, otherwise click into that area and upload your file.
- If you chose Create manually, enter your Column Header, then click + Add. Repeat this for each column.You can add up to 100 columns.
- Create your mappings.
- Click Save or click Save and Close to save and close the editor.
Column Headers
The primary purpose of the Data Import Configuration is to map CSV column values to an Item on the destination form or the metadata required to identify the destination form.
The following columns are required:
- Site Number: The Site Number (Name) of the subject’s Site.
- Subject ID: The Subject ID (Name) of the Subject.
- Event Group: The Name of the Event Group that contains the Event.
- Event: The Name of the Event (visit) that contains the destination Form.
- Event Date: (Optional) The date of the Event (visit).
- Data Items: Data items that you want to enter into the destination form.
There are two methods for column header creation.
You can choose to Upload file, where you can upload an existing CSV file, and Vault will add all of its column headers to the mapping grid.
You can also choose to Create manually. For this method, you manually enter each of your column headers. Enter the header value, then click + Add. Repeat this for each header. Click to remove a header.
Column Name Rules
The following rules apply to column names:
- Only use alphanumeric characters (a-z, A-Z, 0-9), dashes (-), or underscores (_)
- Do not include double underscores (“__”)
Fill CSV File Rows
The rows in your CSV files are where you identify the source, value, and format for imported values of the destination form’s Items. Vault adds the following items to the mappings table. You can map these to CSV columns (import file), collect the values as a prompt during import, or define a static value to use as the item value.
Import Source
The Import Source defines where the values come from. There are three options:
- Import File: There is a column containing this value in the import file. You can select the column in Import Value.
- Prompt: There is no column containing this value in the import file. Vault will show the data loader user a prompt to enter this value when they import the data.
- Static Value: There is no column containing this value in the import file. You provide a value in the Import Value field, and Vault inserts that value into the Item during import.
Import Value
The Import Value defines what value Vault uses to fill the field during import.
If you chose Import File for Import Source, select the Column that contains the data in your CSV file.
If you chose Static Value for Import Source, specify the Value that you want to use in the Import Value field.
If you chose Prompt for Import Source, you can’t select anything in this field, as the data loader will provide a field to enter the value when initiating import.
Import Format
For Event Date and date, time, and datetime Items, where you selected Import File for Import Source, you will be required to select a format in the Import Format field. This is meant to be the format of the date, time, or datetime values in the specified column in the import file.
Translate Values
For codelist-type Items where you selected Import File for Import Source, you can map Codelist Values to Import Values. Values from the file can be mapped to selections in an EDC codelist for cases where the values differ between the import file and the codelist definition in Studio. For example, the codelist code in Studio may be “Y” and the value in the import file may be “Yes”.
To translate values:
- Click the Bars in the Translate Values field.
- Enter an Import Value for each Codelist Value that could have a different value in the import file.
- Click Save.
Available Rows
The table below lists the available rows.
Item Name | Mapping Columns | Required | Notes |
---|---|---|---|
Site Number |
|
Yes | The Site Number (Name) of the subject’s Site. |
Subject ID |
|
Yes | The Subject ID (Name) of the Subject. |
Event Group |
|
Yes | The Name of the Event Group that contains the Event. |
Event |
|
Yes | The Name of the Event (visit) that contains the destination Form. |
Event Date |
|
No | The date of the Event (visit). |
Form Sequence |
|
Yes (Only for repeating forms) | The Sequence Number of the Form. This is only avaialble for Forms that are repeating. |
Data Items |
|
No | Vault creates a row in the mapping grid for each data collection Item on the destination Form. |
Copy a Data Import Configuration from Another Study
You can copy a Data Import Configuration from another Study that you have access to. This is useful when you use a certain Form across multiple Studies and plan to import data into both.
To copy a Data Import Configuration from another Study:
- Navigate to Studio > Studies > Data Loader for your Study.
- Click Copy From Study.
- Select Create a copy or Use existing to handle duplicates. See details here.
- For Copy From Study, choose where you want to copy from (From Another Study or From Library.)
- Select a Study Environment to copy from.
- Search or scroll to locate the Data Import Configuration you want to copy.
- Optional: Select the Select all checkbox to select all Data Import Configurations for copying.
- Click Copy Configurations. Vault begins copying the Data Import Configurations into your Study.
- Optional: In the Copy from Study dialog:
- Click View Summary to see a list of the copied Data Loader Configurations and the status of the copy.
- Click View Log to see detailed status information for each object record copied. If you copied more than 20 Data Loader Configurations, the log isn’t available in the dialog. When finished with the copy, Vault will send you an email notification with a link to download the log file.
- When finished, click Close. Once you click Close, you can no longer access the summary or log file.
How to Edit a Data Import Configuration
To edit a Data Import Configuration:
- Navigate to Studio > Studies > Data Loader for your Study.
- Locate the Data Import Configuration you want to edit in the list.
- Hover over the definition’s Name to show the Actions menu.
- From the Actions menu, select Edit.
- Make your changes.
- Click Save and Close.
How to Delete a Data Import Configuration
To delete a Data Import Configuration:
- Navigate to Studio > Studies > Data Loader for your Study.
- Locate the Data Import Configuration you want to delete in the list.
- Hover over the definition’s Name to show the Actions menu.
- From the Actions menu, select Delete.
- In the confirmation dialog, click Delete.
Deploying to Other Environments
The Enable Data Loader setting is part of the study deployment package, so changes to this setting are included each time you deploy.
Your Data Import Configurations are also included in the package. However, these are considered versionless changes. So, changes to a Data Import Configuration apply automatically once deployed, without the need for an amendment. Consider this when making changes to your Data Import Configurations.