Review Listings

Once a listing is Review Enabled, it becomes a Review Listing. In Review Listings, users can mark each row as In Progress, Reviewed, or as having a known discrepancy. These listings facilitate the review of data from within Workbench. You can mark records as reviewed based on and continue reviewing new rows as they are added to the listing.

Review Statuses

Workbench shows the Review Status of each row in the Review Column. Rows can have one of the following four (4) statuses:

Enable Review for a Listing

To enable review for a listing:

1. Create a new listing or open an existing one.
2. In the Listing () menu, select Properties.
3. Click Edit.

4. Select Enabled for Review.

5. Click Save.

To disable review for a listing, toggle Review Enable off. Disabling review removes the review status from all rows and deletes any recorded Known Discrepancies.

How to Mark a Row as In Progress

You can indicate that review is in progress by setting a row’s status to In Progress. For example, a data manager may want to set the review status to In Progress after creating a query on that row.

To mark a single row as In Progress:

1. Navigate to Listings > playlist_add_check Review.
2. Open the Review Listing.
3. Locate the row that you want to mark as In Progress.
4. In the Review column, click to show the Review Actions.

5. Click Mark as In Progress (update).

6. Optional: Enter a Reason.
7. Click Save.

To mark multiple rows as In Progress:

1. Navigate to Listings > playlist_add_check Review.
2. Open the Review Listing.
3. Locate the rows that you want to mark as In Progress.

4. Select the Checkboxes for the rows you want to review.

5. Optional: To select all rows on the current page, select the Select All checkbox.
6. Click Set Status .

7. Select In Progress.
   

8. Optional: Enter a Reason.
9. Click Save.

How to Review Data

Once review activities are complete, you can mark a row as Reviewed. A data manager may mark a row as Reviewed once all queries within that row are closed and the record is considered cleaned based on the review listing’s objective.

To review data:

1. Navigate to Listings > playlist_add_check Review.
2. Open the Review Listing.
3. Locate the row that you want to review.

4. In the Review column, click to show the Review Actions.

5. Click Mark as Reviewed (how_to_reg) or Mark as Known Discrepancy (sync_problem).

6. Optional: Enter a Reason.

7. Click Save.

To review multiple rows:

1. Navigate to Listings > Review.
2. Open the Review Listing.

3. Select the Checkboxes for the rows you want to review.

4. Optional: To select all rows on the current page, select the Select All checkbox.

5. Click Set Status .

6. Select a Status:

   

   • Select the playlist_add_checkReviewed status to set these rows to Reviewed.
   • Select Known Discrepancy to indicate that these rows have a known discrepancy.
7. Optional: Enter a Reason.
8. Click Save.

How to Unreview Data

You can mark a row as Unreviewed to indicate that it requires new review, for example, if a known discrepancy has been addressed.

To unreview data:

1. Navigate to Listings > playlist_add_check Review.
2. Open the Review Listing.
3. Locate the row that you want to unreview.
4. In the Review column, click to show the Review Actions.

5. Click Mark as Unreviewed (not_listed_location).

6. Optional: Enter a Reason.
7. Click Save.

Workbench moves the row into the Unreviewed status. You or another user can now mark this row as Reviewed or as a Known Discrepancy.

To unreview multiple form rows:

1. Navigate to Listings > Review.
2. Open the Review Listing.
3. Select the Checkboxes for the rows you want to unreview.
4. Optional: To select all rows on the current page, select the Select All checkbox.
5. Click Set Status.
6. Select a Status.
7. Optional: Enter a Reason.
8. Click Save.

Workbench moves the rows into the Unreviewed status. You or another user can now mark this row as Reviewed or as a Known Discrepancy.

Filter by Review Status

You can filter your listing based on the Review Status.

To filter your listing by Review Status:

1. Hover over the Review column header to show the Sort & Filter menu (filter_list).
2. From the Sort & Filter menu, click Review Filters to expand that section.

3. Select the Review Statuses that you want to include in the listing in the Review Filters section.

4. Apply additional filters as needed.
5. Click OK.

Filter by Queries

You can filter your listing based on query statuses or the presence of queries.

To filter with query filters:

1. Hover over the Review column header to show the Sort & Filter menu (filter_list).
2. From the Sort & Filter menu, click Query Filters to expand that section.

3. Select the Review Statuses that you want to include in the listing in the Review Filters section.

4. Apply additional filters as needed.
5. Click OK.

Export Review Listing to CSV

You can export the Review Listing to a CSV file for offline review or use in an external system. When Workbench exports listings, it includes all data shown in the listing, according to the currently applied CQL statement, as well as the following columns:

• Reviewed By
• Reviewed On
• Review Status
• Review Reason

To generate a CSV file of your listing:

1. Open the listing.

2. From the Listing menu, select Generate CSV.

3. Workbench begins a job to create the CSV file. When finished, Workbench shows a notification with a link to download the CSV in the bottom-left corner of the screen. You can click this link to download the CSV file.

Once any user generates a CSV for a public listing, Workbench shows a download link for that CSV in the Latest Download column of the Listings page. Click the Download icon () to download the CSV file.

Review Tracking Warnings

When your CQL is valid, but there is a possibility that your review data can get out of sync, CDB displays a Review Tracking Warning in the top-right corner.

Dynamic Results

When a listing’s CQL uses a dynamic function, for example, Today(), each time the row changes, CDB returns the row’s review status back to No Review. So, CDB shows the following warning:

This listing may not have predicatable review tracking because results are dynamic.

A dynamic column is a column where the value is dynamically generated, such as a value determined by a CQL formula. In the example below, the CURDATE() function results in a dynamic column cd:

select @HDR, @HDR.Event.Date, @HDR.Site.PI, CURDATE() as `cd` from `EDC`.`adverse_event`
where @Form.Status = 'submitted__v' or @HDR.Event.Status IN ('did_not_occur__v')

Rows Aren’t Unique

CDB determines the uniqueness of a row by concatenating all of the values in that row. When there aren’t enough columns to uniquely identify a row, CDB shows the following warning:

This listing may not have predicatable review tracking the rows are not unique.

For example, if a Form includes two repeating Item Groups, with an Item that is used in both of those repeating Item Groups, if the listing doesn’t include other columns to uniquely identify each row, any review that you apply to one row will apply to the other row.

When there are duplicate rows of the same unique key, those rows will share the same review status. For example, if rows 1, 3, 5, and 7 have the same concatenated values, and a user marks row 3 as a having a known discrepancy. After the user refreshes the page, the Known Discrepancy is applied to all four rows, even though the total number of known discrepancies is 1.

Modifying Review Listings

You can modify review listings, but modifying a listing will reset the review status of all rows to No Review.

To modify a review listing:

1. Navigate to the Review Lisitng you want to modify.

2. Click Modfiy.

3. In the confirmation dialog, click Yes. This opens the Listing Builder.

4. Make your changes in the Listing Builder.
5. When finished, click Validate and Save.

Review Listing Refresh

CDB refreshes Review Listings to update the data and apply any decorations related to those updates. This generates a “review listing snapshot” of the data at the time of the refresh. This snapshot is what CDB uses to generate the review listing. CDB refreshes Review Listings on an hourly basis. In addition to the hourly refresh, CDB refreshes the snapshot for a Review Listing upon listing creation or when an existing listing has review enabled.

CDB displays the time the listing was last refreshed in the top right corner, above the last modified date.

CDB also shows an indicator icon for a listing that is disabled or has a refresh warning next to the last refreshed date.

Review Listings have a low threshold of 30 seconds and a high threshold of 90 seconds to measure how long it takes to generate the snapshot. The following 3 scenarios can occur with respect to these thresholds:

1. A Review Listing runs below the low threshold : This is the ideal scenario. CDB will take no action and the snapshot for this Review Listing will continue to get generated on an hourly basis.
2. A Review Listing runs above the high threshold : CDB will allow the snapshot to generate. CDB will allow snapshot generation above the high threshold for a total of 3 times. After the 3rd run above the high threshold, CDB will disable snapshot generation for future runs. After getting disabled, these review listings will not get refreshed on an hourly basis. However, these listings will always retain review comments and review status from the last successful run.
3. A Review Listing runs between the low and high threshold : CDB will allow the snapshot to generate. CDB will allow snapshot generation between the low and high thresholds for a total of 50 times. After the 50th run between low and high threshold, CDB will disable snapshot generation for future runs. After getting disabled, these review listings will not get refreshed on an hourly basis. However, these listings will always retain review comments and review status from the last successful run. If a Review Listing runs below the low threshold after running between the low and high thresholds, then the count will be reset to 0 for this Review Listing.