Configuration

The Illumina MiSeqDx Integration Package v1.11.0 supports the integration between Clarity LIMS and MiSeqDx instruments.

This documentation describes the integration between Clarity LIMS and the MiSeqDx system. It includes information about protocols and automations, configuration options, installed components, and rules and constraints.

Workflows, Protocols, and Steps

The following protocols are included in MiSeqDx Integration Package v1.11.0:

CF 139-Variant Assay Library Prep 1.2
CF Clinical Sequencing Assay Library Prep 1.2
Illumina SBS MiSeqDx (CF 139-Variant Assay) 1.2
Illumina SBS MiSeqDx (CF Clinical Sequencing Assay) 1.2

There are three validation protocols. Each protocol is included in a workflow with the same name. The protocols are as follows.

MiSeqDx Validation (CF 139-Variant Assay) 1.2
MiSeqDx Validation (CF Clinical Sequencing Assay) 1.2

Each validation protocol includes the following steps from the Library Prep and Illumina SBS MiSeqDx protocols:

Extension-Ligation of Bound Oligos (Library Prep step)
PCR Amplification (Library Prep step)
Library Pooling (MiSeqDx) (Illumina SBS MiSeqDx step)
Denature, Dilute and Load Sample (Illumina SBS MiSeqDx step)
MiSeqDx Run (MiSeqDx) (Illumina SBS MiSeqDx step)
Variant Calling (MiSeqDx) (Illumina SBS MiSeqDx step)

For instructions on user interaction for each step and using the MiSeqDx Validation (CF 139-Variant Assay) 1.2 protocol to validate automated sample sheet generation, refer to MiSeqDx Integration v1.11.0 User Interaction, Validation and Troubleshooting.

The following table lists the automations included in this integration, and the steps on which they are configured.

MiSeqDx Integration Default Automations

Library Prep Protocol - PCR Amplification Step

This section discusses the index placement and validation automations configured on the PCR Amplification 1.2 Library Prep steps.

The example workflow uses the CF 139-Variant Assay Library Prep 1.2 protocol.

By default, in the CF 139-Variant Assay Library Prep 1.2 protocol, the PCR Amplification 1.2 step includes two automations. Both automations invoke the place_indexes script with different options.

Auto Place Indexes — Automatically invoked on entry to the Add Reagents screen.
Validate Index Placement — Automatically invoked on exit from the Add Reagents screen.

A list of reagents installed with this configuration is provided in Installed Components.

Auto Place Indexes Automation

Automatically triggered on entry to Add Reagents screen, this automation completes the following actions:

Invokes place_indexes script to place reagent indexes into the container based on the pattern defined in a specified index placement pattern file.

Reagent indexes are assigned to each sample in the destination container according to the specified index placement pattern file.

bash -l -c "/opt/gls/clarity/bin/java -cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/PlacementHelper.jar place_indexes -i {stepURI:v2} -u {username} -p {password} -f /opt/gls/clarity/extensions/conf/miseqdx/placementpatterns/placement_pattern_index_cf139_01.tsv -ic 'CF 139-Variant Assay 8-Sample Indexes' -locked true"

The automatic index placement pattern for CF 139-Variant Assay should not be modified.
Upon completion of the reagent index placement, a success message displays. Select OK to close the message. Indexes with the pattern specified in the placement file are then assigned to the samples.

Validate Index Placement Automation

Automatically triggered on exit from Add Reagents screen, this automation completes the following actions:

Invokes place_indexes script to validate the index placement:

Index placement cannot be modified for CF 139-Variant Assay. Any changes made by the user are disregarded and the automatic index placement is restored.

bash -l -c "/opt/gls/clarity/bin/java -cp /opt/gls/clarity/extensions/ngs-common/v5/EPP/PlacementHelper.jar place_indexes -i {stepURI:v2} -u {username} -p {password} -f /opt/gls/clarity/extensions/conf/miseqdx/placementpatterns/placement_pattern_index_cf139_01.tsv -ic 'CF 139-Variant Assay 8-Sample Indexes' -locked true -fc true"

Illumina SBS MiSeqDx v1.2 Protocols

This section describes the features of the key steps in the Illumina SBS MiSeqDx v1.2 protocols.

Library normalization CSV file generation in Library Normalization (MiSeqDx) step.
Reagent cartridge name validation and sample sheet generation in Denature, Dilute and Load Sample 1.2 step.
Primary analysis (sequencing) results parsing, which includes generation of the run report, in MiSeqDx Run (MiSeqDx) step.
Secondary analysis results parsing in Variant Calling (MiSeqDx) 1.2 step.

Library Normalization (MiSeqDx) Step

In each of the Illumina SBS MiSeqDx protocols, the Library Normalization (MiSeqDx) v1.2 step includes automated calculation of normalization buffer volumes. The results are provided in an autogenerated comma-separated file that is attached to the step.

Create Normalization CSV Automation

Buffer volumes are calculated by the normalizationBufferVolumes script. The script is invoked by the Create Normalization CSV automation when you select a button on the Record Details screen.

For details on the normalizationBufferVolumes script, refer to Normalization Buffer Volumes.

Denature, Dilute and Load Sample 1.2 Step

In each of the Illumina SBS MiSeqDx and Validation protocols, the Denature, Dilute and Load Sample 1.2 step includes automations to:

Validate single input
Validate the MiSeqDx reagent cartridge name
Generate sample sheet

Validate Single Input

Automatically initiated on entry to the step, the Validate Single Input automation validates that there is only 1 pool added to this step.

bash -l -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar -i {stepURI:v2} -u {username} -p {password} script:validateSampleCount -min 1 -max 1"

Validate Container Name Automation

Automatically initiated on exit from the Placement screen of the step, the Validate Container Name automation validates the reagent cartridge name to ensure:

The reagent cartridge Container Name has been renamed and is not the same as its Clarity LIMS ID.

if (output.container.name == output.container.node.@limsid) { fail(::The Container Name cannot be the same as its LIMS ID. Please scan the Reagent Cartridge RFID.::) }

The Container Name contains only letters and numbers, does not have special characters, and is in the format AB#######-XYZ (e.g., RG1234567-ABC).

else if (!output.container.name.matches(::^[a-zA-Z0-9-]*$::)) { fail(::The Container Name can contain letters, numbers and dashes only. No other special characters are allowed. Please scan the Reagent Cartridge RFID.::) };

Sets the Reagent Cartridge Barcode field attached to the output that is from the sample input.
```
output.::Reagent Cartridge Barcode:: = output.container.name;
```

Pads the suffix (the characters after the dash) of the Container Name so that it is at least five characters long. The Container Name is used by the MiSeqDx Sequencing Service to match the run data to the Clarity LIMS step in the MiSeqDx Run (MiSeqDx) step.

def match = output.container.name =~ /([a-zA-Z0-9]+-)([a-zA-Z0-9]{0,5})$/;
if (match.matches()) {
    int barcodeSuffixLength = 5;
    int zerosToAdd = barcodeSuffixLength - match.group(2).size();
    output.container.::name:: = match.group(1).toString() + ::0::*zerosToAdd + match.group(2).toString()
}

Generate MiSeqDx Sample Sheet Automation

The sample sheet generation automation is configured on the Record Details screen of the Denature, Dilute and Load Sample 1.2 step.

This automation is triggered by a button on the Record Details screen.

The automation sets the value of the Progress field on the sample to 'Library ready for sequencing'.

/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar -u {username} -p {password} -i {processURI:v2} script:setUDF -f 'Progress' -t '//input/@uri->//sample/@uri' -v 'Library ready for sequencing'

The automation starts a script that generates the sample sheet.

&& /opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/miseqdx/miseqdx-extensions.jar -u {username} -p {password} -i {processURI:v2} script:generate_miseqdx_sample_sheet -c {compoundOutputFileLuid1} -e {compoundOutputFileLuid2} --useSampleLimsID true --isMiSeqDx true

The automation labels the derived samples that do not have reagent labels. The label value applied is NoIndex.

&& /opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar -u {username} -p {password} -i {processURI:v2} script: labelNonLabeledOutputs -l 'NoIndex'

For more information on sample sheet generation, including details on file format and contents, master step and global fields, script parameters, and configuration options, refer to the following documentation:

Illumina MiSeqDx Sample Sheet Generation (v1.11.0 and later) — For details on the Diagnostic mode sample sheet designed for use with MOS software.
Sample Sheet Generation — For details on the Research mode sample sheet designed for use with MCS software.

The integration allows for generation of a sample sheet designed to be used with the MOS (Diagnostic mode) instrument software.

By default, one MiSeqDx instrument sample sheet is created for the reagent cartridge loaded during the step. The placeholders and sample sheet files in the following table are generated by the Denature, Dilute and Load Sample 1.2 step.

Generated Placeholders and Sample Sheet Files

MiSeqDx Run (MiSeqDx) Step

The MiSeqDx Run (MiSeqDx) step records information for each lane of the flow cell and generates a report summarizing the results. In addition, run parameters, run info, and a link to the run folder are automatically captured.

Generated and Captured Files

The following table describes the run information files, reports, placeholders, and links that Clarity LIMS automatically generates or captures during a sequencing run.

Run Information Generated or Captured by MiSeqDx Run (MiSeqDx) v1.2 Step

Metadata

The following metadata are stored as custom fields tagged to the MiSeqDx Run (MiSeqDx step). These fields are all read-only.

Finish Date* – run completion date
Run Type – Diagnostic or Research mode
Status – current status of the sequencing run on the instrument
Flow Cell ID
Flow Cell Version
Experiment Name – entered in software
Read 1 Cycles
Index 1 Read Cycles – intended Index cycles
Index 2 Read Cycles – intended Index cycles
Read 2 Cycles
Output Folder – run folder root
Run ID – the unique run ID
Reagent Cartridge ID
Reagent Cartridge Part #
PR2 Bottle ID
Chemistry
Workflow

There are two additional master step fields in this step:

Comments — multiline text field used for any comments that are attached to the run
Report Status — hidden read-only field that tracks if the run report has been successfully uploaded

If the End Run event contains a date in the format YYYY-MM-DD, Finish Date will be set to the date in the event file.
If the End Run event does not contain a date or the date is in the wrong format, Finish Date will be set to the date when the event file is processed.

Primary Analysis Metrics

The following table lists the Real-Time Analysis (RTA) primary analysis metrics Clarity LIMS automatically captures and records, per read, for samples in each flow cell lane. These metrics are captured upon run completion.

RTA Primary Analysis Metrics Captured by MiSeqDx Run (MiSeqDx) Step

How it Works

The sequencing service may run on Clarity LIMS server in an on-premise environment or on a remote server in a SaaS environment. The service detects event files that the instrument software (RTA) is producing as the run progresses. These event files let the service know where to find the run data.

As the run data are written out and the events come in (Begin Run, Cycle Complete, End Run), the data are matched to the step based on the reagent cartridge ID. This value was entered as the Container Name on the Denature, Dilute and Load Sample 1.2 step. The read-only field values on the Record Details screen are populated accordingly.

AUTOMATED - Run Report Generation Automation

The automation is configured to be invoked by a button on the Record Details screen. However, it is automatically started by the sequencing service when it has finished processing the end run event.

The automatin generates the run report and sets the Report Status field to completed. Run report generation is asynchronous. The sequencing service does not wait for the report generation to complete before processing the next event file.

bash -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/miseqdx/miseqdx-sequencing-report.jar -i {processLuid} -r {compoundOutputFileLuid0}_RunReport.pdf -l {compoundOutputFileLuid1}_RunReportLogFile.html && /opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar -i {stepURI:v2} -u {username} -p {password} script:evaluateDynamicExpression -exp 'step.::Report Status:: = ::Completed::' -t 'false' -log tempLog.html"

An additional shared result-file placeholder is configured on the step. This placeholder is for a new log file — Illumina Run Report Log File — that is generated and attached to the step when the report runs.
The run report can be successfully generated in a hosted deployment of Clarity LIMS where the sequencing service is running on a remote server on the customer network.

Verify Report Status Automation

The Verify Report Status automation prevents movement to the Assign Next Steps screen unless the Report Status field is set to Completed (this is done by the previous automation after the run report is attached to the step).

bash -c "/opt/gls/clarity/bin/java -jar /opt/gls/clarity/extensions/ngs-common/v5/EPP/ngs-extensions.jar -i {stepURI:v2} -u {username} -p {password} script:evaluateDynamicExpression -exp 'if (!step.hasValue( ::Report Status:: ) || step.::Report Status:: != ::Completed::) { fail(::Cannot advance because expected files are missing, please wait until the RunReport, RunInfo and RunParameters files are generated and attached before completing the step.::) } ' -t 'false' -log tempLog.html"

Variant Calling (MiSeqDx) 1.2 Step

The Variant Calling (MiSeqDx) 1.2 step attaches secondary analysis output files for each of the samples.

In addition, a Combined Variant Call File, a text file, and a variant call file for each individual sample are automatically captured following the completion of secondary analysis.

How it Works

Following primary analysis, secondary analysis runs on the MiSeqDx instrument. Clarity LIMS detects the completion of secondary analysis and lets the service know where to find the result files. While the EndRun event is processed successfully for primary analysis, the sequence service generates a BeginSecondaryAnalysis event file in the same directory with the miseqdx.seqservice.eventFileDirectory property.

An example of how Clarity LIMS shows the location of the secondary analysis event file is:

netFolder = D:\Illumina\MiSeqOutput\161017_M99999_0047_FC1234567-ABCDErunNetworkLocation = /mnt/ussd-dev-mdx01/MiSeqOutput/161017_M99999_0047_FC1234567-ABCDEeventType = BeginSecondaryAnalysissoftwareType = MRScreationTime = Thu Oct 20 22:56:32 GMT 2016

This event file signals that secondary analysis has started. The sequence service periodically checks if the analysis is complete by looking for the CompletedJobInfo.xml file. This file is in the folder specified by the runNetworkLocation parameter of the event file.

The frequency of checking is defined by the miseqdx.seqservice.synchronizationPeriod property.

If the completion file appears, the sequence service processes the event. The event is archived in a similar way to how the EndRun event file is handled. If the event completion does not appear within the number days specified by the miseqdx.seqservice.ignoreUnmatchedContainerIdsWaitDays property, the service stops monitoring the event file and the file is archived.

Captured Files

The following table describes the Variant Call Format (VCF) files that Clarity LIMS captures and attaches to the step after the secondary analysis.

Variant Call Format Files Captured by the LIMS After Secondary Analysis

Installed Components

The Illumina MiSeqDx Integration RPM package installs the following components.

This integration requires installation of the associated NGS Extensions Package (refer to Release Notes).

Scripts and Files

Properties

Refer to Integration Properties Details for the properties installed with the Illumina MiSeqDx Integration Package.

Reagent Categories/Label Groups

The following reagent categories/label groups are included in the default configuration for the MiSeqDx Integration Package:

CF 139-Variant Assay Indexes
CF 139-Variant Assay 8-Sample Indexes
CF Clinical Sequencing Assay 8-Sample Indexes

Reagent Kits

The following reagent kits are included in the default configuration for the Illumina MiSeqDx Integration Package:

CF 139-Variant Assay-Oligo Pool
CF Clinical Sequencing Assay-Oligo Pool
Custom Oligo Pool
Elution Buffer
EtOH
Extension-Ligation Mix
Hybridization Buffer
Index Primers
Library Beads
Library Dilution Buffer
Library Normalization Diluent
Library Normalization Wash
Library Storage Buffer
MiSeqDx Flow Cell - CF 139-Variant Assay
MiSeqDx Flow Cell - CF Clinical Sequencing Assay
MiSeqDx SBS Solution (PR2) - CF 139-Variant Assay
MiSeqDx SBS Solution (PR2) - CF Clinical Sequencing Assay
NaOH
PCR Clean-Up Beads
PCR Master Mix
PCR Polymerase
Stringent Wash Buffer
Universal Wash Buffer

Controls

The following controls are included in the default configuration for the MiSeqDx Integration Package:

Negative Control for MiSeqDx
PhiX Internal Control
Positive Control for MiSeqDx

Container Types

All one-dimensional container types with both numeric rows and numeric columns are supported.

The following container types are included in the default configuration for the MiSeqDx Integration Package:

MiSeqDx Reagent Cartridge - CF 139-Variant Assay
MiSeqDx Reagent Cartridge - CF Clinical Sequencing Assay

Instrument Integration

The following information provides an overview of the steps performed by the Clarity LIMS support team when configuring the instrument for use with the MiSeqDx Integration to Clarity LIMS.

Configure the MiSeqDx as follows.

Create a directory on the local computer to hold the batch files. These batch files write event files to the network-attached storage (NAS) shares.
Create a directory on the NAS to hold the event files.
Modify the software configuration files to call the batch files that create the event files.
Update sequencing service default properties to match the specifics of the installation.

Configuring Instrument Names

We recommend configuring instrument names in Clarity LIMS to track the specific lab instrument on which a run was completed.

This information should be configured on the MiSeqDx Run (MiSeqDx) 1.2 step, and should be set to the same name as is configured in the Illumina sequencing software (ie, this will be value of the Instrument Run ID field described in the Metadata section).

When the integration reads the Instrument Run ID field in the run results, it will update the instrument on the step in Clarity LIMS. This update eliminates the need to enter this information manually.

For instructions on configuring instrument names, refer to the Clarity LIMS (Clarity & LabLink Reference Guide) documentation.

Configuring Sample Sheet Generation

Sample sheet generation occurs on the step prior to the sequencing run—Denature, Dilute and Load Sample 1.2—which is the step where samples are placed on the container that will be loaded in the instrument.

By default, an instrument sample sheet is created for the container loaded during the Denature, Dilute and Load Sample 1.2 step.

Rules and Constraints

This integration operates with the following constraints:

There must only be a single input on the Denature, Dilute and Load Sample 1.2 step.
The Reagent Cartridge ID must be unique. There should not be multiple containers in the system with the same name.
The reagent cartridge ID must be scanned as the Container Name on the Denature, Dilute and Load Sample 1.2 step.
For MOS sample sheet generation constraints, refer to Illumina MiSeqDx Sample Sheet Generation (v1.11.0 and later).

PreviousInstallation NextUser Interaction, Validation and Troubleshooting

Last updated 2 months ago