The third and last step of the import process provides you with an import validation report (QC report) an interactive and detailed report of events that are going to happen when you commit (finalize) the import.
Please note: Once the data is committed to the CDD Vault database, it has to be deleted manually (there is no "undo" button after "Commit" - this is why it is called "Commit"); so the report provides you with an opportunity to cancel (Reject) the entire import, or to selectively reject import of individual event types within an otherwise OK file.
The third import step and the validation report are very important to ensure that quality results are entered into your vault. We would also like to point out that choosing the correct import setting for your data in the second step, is important too. As the second step usually comes before the third step, at least in euclidean spaces, here's what you need to know about the second import step.
Import validation report summary:
- Import validation report is provided in the last step of the import process before you click "commit data import" or "reject data import"
- In the top left corner of the report, you see the file name with a link to review the mapping from the previous import step, and to save a mapping template.
- In the top right corner of the report, you see the project name where this file is imported, and the file owner's name.
- The yellow panel shows import progress and status of the file: after the initial import is complete, it shows the total number of records (rows of a csv file, and individual records of the SDF file) that will be imported and/or rejected when the import is finalized.
The report sections which follow the summary, contain a breakdown of import events by category. There are 3 main categories: noteworthy events, suspicious events, and errors. Within each category events are separated by type.
Selectively Accept/Reject events by type:
You can choose to accept all records of a particular event type. This means that all records in the given event type will be imported successfully when you "commit data import". In the example shown, we have accepted 2 new molecule records which will be registered in your vault as soon as you commit.
You can choose to reject all records of one event type. This means that all records of the given event type will be skipped on import while the rest of the records will be imported successfully, once you "commit data import". We have now rejected 2 new molecule records, and those two molecules will not be imported.
Details of the event type:
You can find out more about each event by clicking on it's name:
- Find an explanation of what will happen if you choose to accept and commit all the rows/records in this event type. This event hasn't happened yet, and it's up to you to decide if the event will be accepted and committed, or rejected. If the explanation is long, you will see a "read more" link.
- See a preview (10 rows) of the rows/records in your file that fall under this event type. Scroll to the right to see an event description on each row/record.
- Download a complete file that contains all rows/records of this event. This file will be handy when you need to troubleshoot the import- you can work with individual event types, instead of trying to find them in the original import file. The file will contain event descriptions for each record.
What to do in each case:
Understanding the meaning of each event category and event types within them will help you decide whether each event should be accepted or rejected. It will also help you figure out how to correctly import each of the rejected sections. Here's a quick summary of the three main categories. Follow the links to get into each section in lots more detail!
Noteworthy events are simply events that will happen when the import is committed. As far as CDD Vault is concerned there is nothing problematic about them, therefore each event is set to "Accept" by default. We are drawing your attention to the facts and counts of what will happen. For example, "2 New Molecules" event means that two new molecule records will be created on commit, and "2 New Batches" event means that two new batches of existing molecules will be created on commit. However, some of the noteworthy events may be incorrect, in which case you will want to manually change from the default "Accept" to "Reject". For example, excel made a copy/pasting mistake while you were filling in plate information, and the import validation report shown " 384 New Plates" event, where you should only see "1 New Plate" (384W plate).
Suspicious events are highlighted for your review by CDD Vault. From the vault perspective, the data may be imported successfully, however, we believe that you should pay special attention to these events. The idea is to make the user look at the details. Typical events include tautomer registration, or sharing of molecules between projects: you may want to accept these events, but by default they are marked to be rejected. Once you have reviewed the suspicious events carefully, you can choose to accept them.
Errors are events that cannot be imported by CDD Vault. There is no choice to accept these events, they will be automatically rejected on import. An error may be an invalid compound structure, or a mismatch between structure and unique molecule ID, which violate the uniqueness constrains enforced by your CDD Vault. To correct an error, you will need to download the complete error file, correct the mistake, and re-import those records separately.
Finalize the import:
When (and only when) you have reviewed all of the events, and made all of the accept/reject decisions, you should finalize the entire import. These are the two big buttons at the bottom of the page:
- When an import is committed, every event that was set to "Accept" is finally imported into the vault.
- When an import is rejected, all data in the file are rejected- the import is essentially cancelled summarily.
- It is not possible to undo changes after the import is committed, except to manually go through the vault, and delete every molecule, batch, user-defined field, run of a project, or what have you.
Please note: An import that has not been committed OR rejected is "in limbo" - data in the file are not visible to any other members of the vault, and the import may be preventing other users from importing data. CDD Vault will send you e-mail reminders if you forget to finalize an import for over a week.
Of course, mistakes happen but the QC report gives you the power to prevent many of them. If you encounter an import validation event that you do not understand, please contact firstname.lastname@example.org- we will be happy to help!