Clarity LIMS Software
Illumina Connected Software

Installation and User Configuration

The Illumina Preset Protocols (IPP) v2.8 support the integration of Clarity LIMS with established lab protocols.

The integration provides preconfigured workflows that map to lab protocols and steps, and support Illumina library prep kits, reagent kits, assays, and instruments.

This guide provides instructions and describes the components installed in the default configuration of IPP.

Installation Components

IPP Integration v2.8 is distributed as the BaseSpaceLIMS-Illumina-Preset-Protocols RPM package. The RPM package installs the following components:

  • The illumina-preset-protocols-installer.sh script for configuration slices.

  • Templates for the following uses:

    • Sample sheet generation

    • Sample placement pattern

    • Covid testing setup and reporting

    • Illumina Genomics Architecture (IGA)

  • Automation scripts, including scripts to support IGA and Clarity LIMS Product Analytics (CLPA) automation.

  • Configuration slices.

  • Workflows for applicable instrument integrations.

Sample Sheet Generation Templates

  • BCL2FASTQ_Reverse_Complement_Samplesheet.csv

  • BCL2FASTQ_Samplesheet.csv

  • genomeStudio_Genotyping_SampleSheet_driver_file.csv

  • genomeStudio_Methylation_SampleSheet_driver_file.csv

  • iSeq_Samplesheet.csv

  • iSeqManual_Samplesheet.csv

  • miniSeq_Samplesheet.csv

  • MiSeqSamplesheetv3.csv

  • NextSeq1K2K_Pool1.csv

  • NextSeq1K2K_Pool2.csv

  • NextSeq1K2K_Pool3.csv

  • NovaSeq_BCL2FASTQ_Samplesheet.csv

  • NovaSeq_BCL2FASTQ_Samplesheet_v2.csv

  • NovaSeq_BCL2FASTQ_Reverse_Complement_Samplesheet.csv

  • NovaSeq_Standard_Bulk_Pool1.csv

  • NovaSeq_Standard_Bulk_Pool2.csv

  • NovaSeq_Standard_Bulk_Pool3.csv

  • NovaSeq_Xp_Bulk_Pool.csv

  • NovaSeq_Xp_Bulk_Pool2.csv

  • NovaSeq_Xp_Working_Pool.csv

  • NovaSeq_Xp_Working_Pool2.csv

  • NovaSeqDx_BCL2FASTQ_Samplesheet_v2.csv

  • NovaSeqXSeries_Bulk_Pool1.csv

  • NovaSeqXSeries_Bulk_Pool2.csv

  • NovaSeqXSeries_Dilute_Denature_Calculate_Volumes.csv

  • NextSeq1K2K_OnPrem_Samplesheet.csv

Sample Placement Pattern Templates

  • Hybrid Pattern_96 well plate_8x1 HD BeadChip.tsv

  • Hybrid Pattern_96 well plate_8x1 LCG BeadChip.tsv

  • Hybrid Pattern_96 well plate_12x1 HD BeadChip.tsv

  • Hybrid Pattern_96 well plate_24x1 HD BeadChip.tsv

  • Hybrid Pattern_96 well plate_24x1 HTS BeadChip.tsv

  • Hybrid Pattern_96 well plate_24x1 HTS BeadChip Multichannel.tsv

  • Hybrid Pattern_96 well plate_24x4 XT BeadChip.tsv

Covid-Related Templates

  • 7900sds_container_setup_template.txt

  • covid_report_results.csv

Illumina Genomics Architecture (IGA) Templates

  • BatchingInputFile_template.csv

  • CaptureWashInputFile_template.csv

  • ExtractionInputFile_template.csv

  • LibInputFile_NFE_template_withoffsets.csv

  • LibInputFile_NFPF_template_withoffsets.csv

  • LoadToFlowcell_NovaSeqXp_Incomplete_template.csv

  • LoadToFlowcell_NovaSeqXp_ManifestForConsolidation_template.csv

  • LoadToFlowcell_NovaSeqXp_template.csv

  • LoadToLibTubeFile_NovaSeqStd_template.csv

  • MakeBPFile_NovaSeq_template.csv

  • PlatingInputFile_template.csv

Python Scripts for Automation Support

  • autoPlaceCovid7500.py

  • clarityapiutil.py

  • parseCovidSeqRunReport.py

  • verifyBeadChipIDs.py

Python Scripts for IGA Automation Support

  • glsapiutil:

    • glsapiutil.py

    • glsapiutil3.py

  • hamilton_fast_prep:

    • 01_A_CFP_Batching_defaultContainerCheck.py

    • 01_CFP_Batching_confirmPlateID.py

    • 01_CFP_Batching_confirmPlateIDAndStoreInstrumentID.py

    • 02_ReagentParsing.py

    • 03_checkHamiltonFileNameBarcode.py

    • 04_A_checkIndexPlateBarcodeWithCategory.py

    • 04_B_autoAssignIndexesFromIndexPlanning.py

    • 05_reusedIndexPlateWarning.py

    • countSamples.py

    • countSamplesInPools.py

    • validate_sample_placement.py

  • iga_helpers:

    • clarityUIFileHelper.py

    • pop_up_msg.py

    • printCurrentWorkflow.py

  • iga_novaseq:

    • 01_NovaSeqStd_confirmPlateIDAndStoreInstrumentID_OptionalFile.py

    • 02_NovaSeqStd_confirmPlateIDAndStoreInstrumentID_OptionalFile_DiluteDenature.py

    • 03_NovaSeqXp_confirmBPPPlateAndStoreInstrumentID.py

    • 03_NovaSeqXp_consolidateBulkPoolFiles.py

    • 04_NovaSeqXp_completeAndConsolidateWPPZipFile.py

    • NovaSeq_automated_warning.py

    • NovaSeq_pooling_warning.py

    • validate_floorcell_container.py

    • validate_flowcell_pools.py

    • validate_multiple_destination_plates.py

  • illumina-cobalt-samplesheet-converter:

    • ss_converter:

      • converter.py

      • errors.py

      • v1_builder_plugins.py

      • v1_v2_handler.py

      • v2_converter_plugins.py

  • index_correct

    • apply_correction_factors.py

  • make_bpp:

    • apply_multiple_bulk_pools.py

    • calculate_sample_volume.py

  • utils:

    • common.py

    • igaapiutil.py

Clarity LIMS Product Analytics Automation Script

  • unified-product-analytics-automation.jar

Configuration Slices

The following configuration slices are included in IPP Integration v2.8:

  • Illumina_Instruments, including:

    • iScan-v1.0

    • iSeq-100-v1.0

    • Library-Prep-Validation-v2.3

    • miniSeq-v1.0

    • miseq-v3.2

    • NextSeq1k2k-v2.4

    • nextseq550-v1.1

    • novaseq-v3.8

    • novaseqdx-v1.2

    • NovaSeqXSeries-v1.1

    • NextSeq1k2kOnPrem-v1.0

  • QC_Protocols

    • qc-protocols

  • Ampliseq_for_Illumina_Protocols, including:

    • AmpliSeq-Illumina-BRCA-Panel-v1.1

    • AmpliSeq-Illumina-Cancer-HotSpot-Panel-V2-v1.1

    • AmpliSeq-Illumina-Childhood-Cancer-Panel-v1.1

    • AmpliSeq-Illumina-Comprehensive-Cancer-Panel-v1.1

    • AmpliSeq-Illumina-Comprehensive-Panel-v3-v1.1

    • AmpliSeq-Illumina-Custom-DNA-Panel-v1.1

    • AmpliSeq-Illumina-Focus-Panel-v1.1

    • AmpliSeq-Illumina-Immune-Repertoire-Panel-v1.1

    • AmpliSeq-Illumina-Immune-Response-Panel-v1.1

    • AmpliSeq-Illumina-Myeloid-Panel-v1.1

    • AmpliSeq-Illumina-TCR-beta-SR-Panel-v1.1

    • AmpliSeq-Illumina-Transcriptome-Human-Gene-Expression-Panel-v1.1

  • Covid, including:

    • CDC-COVID-19-RT-PCR-v1.1

    • Illumina-COVIDSeq-Workflow-v1.6

    • Respiratory-Virus-Panel-v1.0

  • DNA_Protocols, including:

    • ILASS-TruSight-Whole-Genome-Assay-v1.2

    • Illumina-DNA-PCR-Free-Library-Prep-Manual-v1.0

    • Illumina-DNA-Prep-M-Tagmentation-v1.0

    • Nextera-Mate-Pair-v1.0

    • Nextera-XT-v2.0

    • TruSeq-DNA-PCR-Free-v2.0

    • TruSeq-Nano-DNA-v1.0

  • IGA, including:

    • IGA_Library_Prep_Automated_v2.1

    • IGA_NovaSeq_Sequencing_v2.2

  • Infinium_Array, including:

    • Genome-Studio-v1.0

    • ILASS-Infinium-Genotyping-Assay-v1.0

    • Infinium-HD-Methylation-Assay-Manual-v1.2

    • Infinium-HTS-Assay-Manual-v1.2

    • Infinium-LCG-Assay-Manual-v1.2

    • Infinium-XT-Assay-Manual-v1.2

  • Methylation_Protocols, including:

    • TruSeq-ChIP-Seq-v1.0

    • TruSeq-Methyl-Capture-EPIC-v2.0

  • RNA_Protocols, including:

    • Illumina-Stranded-mRNA-Prep-Ligation-v1.1

    • Illumina-Stranded-Total-RNA-Prep-Ligation-with-Ribo-Zero-Plus-v1.1

    • TruSeq-Small-RNA-v1.0

    • TruSeq-Stranded-mRNA-v2.1

  • Targeted_Amplicon_Protocols

    • TruSeq-Custom-Amplicon-v1.0

  • Targeted_Enrichment_Protocols, including:

    • Illumina-DNA-Prep-with-Enrichment-S-Tagmentation-v1.2

    • Illumina-RNA-Prep-with-Enrichment-L-Tagmentation-v1.1

    • Nextera-Rapid-Capture-Custom-Enrichment-v2.0

    • TruSeq-DNA-Exome-v2.0

    • TruSeq-RNA-Exome-v1.0

  • TruSight_Oncology_Protocols, including:

    • TruSight-Oncology-500-ctDNA-v1.1

    • TruSight-Oncology-500-HT-v1.1

    • TruSight-Oncology-500-v1.1

    • TruSight-Tumor-170-v2.0

Instruments and Workflows

IPP Integration v2.8 contains workflows that are used to integrate the following instruments:

  • iScan

  • iSeq

  • MiSeq

  • MiniSeq

  • NextSeq 500/550

  • NextSeq 1000/2000

  • NovaSeq 6000

  • NovaSeq 6000Dx

  • NovaSeq X Series

The following workflows are included in the integration:

  • iScan v1.0

  • iSeq 100 v1.0

  • Library Prep Validation v2.3.3

  • MiniSeq v1.0

  • MiSeq Sequencing v3.2

  • NextSeq 500/550 Sequencing v1.1

  • NovaSeq 6000 v3.8

  • NextSeq 1000/2000 Sequencing v2.4

  • NovaSeqDx v1.2

  • NovaSeq X Series Sequencing v1.1

  • NextSeq 1000/2000 On-Prem Sequencing v1.0

These workflows require ClarityLIMS-NGS-Package v5.25 or later so that the Template File Generator tool can be used.

Protocols and Configuration Slices

IPP Integration v2.8 provides protocols that contain configuration slices. These configuration slices are used to install other workflows and protocols.

QC_Protocols

QC_Protocols configuration component contains the qc-protocols base slice. This configuration does not contain any workflows, but it does provide the following protocols that are used by other workflows:

  • DNA Initial QC

  • Library Validation QC

  • RNA Initial QC

The following table shows the components that are installed with the QC_Protocols configuration.

Components Installed with QC_Protocols

AmpliSeq_for_Illumina_Protocols

The following table shows the AmpliSeq_for_Illumina_Protocols configuration slices and the workflows that are installed with IPP Integration v2.8.

AmpliSeq_for_Illumina_Protocols Configuration Slices and Workflows

Additional Configuration Slices

The following table shows additional configuration slices and workflows that are installed with IPP Integration v2.8.

Additional Configuration Slices and Workflows

IPP Integration v2.8 Package Installation

The BaseSpaceLIMS-Ilumina-Preset-Protocols RPM package provides instructions for locating and running the /opt/gls/clarity/config/illumina-preset-protocols-installer.sh bash script that launches the IPP configuration installer.

The RPM package automatically installs template files that are used with Template File Generator (TFG) and Sample Placement Helper. These template files are accessible after the applicable configuration is installed. The files are installed at the following locations:

  • /opt/gls/clarity/extensions/conf/driverfiletemplates

  • /opt/gls/clarity/extensions/conf/infinium/placementpatterns

  • /opt/gls/clarity/extensions/conf/covid

The RPM package automatically installs the CLPA automation that is used for the CLPA integration. This automation is installed at the following location:

/opt/gls/clarity/extensions/unified-product-analytics/automation

If this is a new Clarity LIMS installation, you must install the QC_Protocols before the preconfigured workflows. The base configuration is included in the standard installation process.

After you have selected a workflow, the installer validates the import of that workflow and provides Warning/Error details in STDOUT. This process allows you to proceed with the import or cancel it.

Prerequisites

Before installing the RPM package, make sure that the following software is installed:

  • Clarity LIMS v6.2 or later

  • NGS-Package v5.25 or later

  • SecretUtil v1.2. For more information, refer to Guide to Secret Management in the Clarity LIMS (Clarity & LabLink Reference Guide) documentation.

RPM Installation

  1. On the server used for the IPP RPM installation, log in as the root user.

  2. Use the following yum command to install the RPM: The --enablerepo in the command line must be included to enable the repo. The Illumina Support team provides the repo file and the appropriate name to use.

    yum install BaseSpaceLIMS-Illumina-Preset-Protocols --enablerepo=<< repo name info from support >>

  3. When prompted to proceed with the RPM package installation, enter y to confirm.

For information on running the installer and supported commands, parameters, and options, refer to Installer Parameters and Operations.

Installer Parameters and Operations

The IPP Integration v2.8 installer uses parameters and operations to assist with the configuration installation.

You must run the installer as the glsjboss user. The following table describes the parameters that are available for use with the installer.

IPP Installer Parameters

List Operation

Run the installer with the list operation for a list of all configuration installation options in a two-column format. The first column shows the IPP workflow identifier (or Id) that the installer uses when running the install operation. The second column shows the name of the configuration associated with the identifier. As the glsjboss user, run the following command to view the list of IPP workflows:

/opt/gls/clarity/config/illumina-preset-protocols-installer.sh -o list

Install Operation

Use the install operation to install the configuration slices. Specify both the name of the parent item and the name of the configuration slice, separated by a period, as follows:

/opt/gls/clarity/config/illumina-preset-protocols-installer.sh -o install Parent_Item_name.slice_name

For example, to install the qc-protocols slice, run the following command:

/opt/gls/clarity/config/illumina-preset-protocols-installer.sh -o install QC_Protocols.qc-protocols

All Operation

The all operation is a special option that is available at the top level and parent item level.

  • At the top level, use the operation to install every configuration package from IPP Integration v2.8. To use the all operation at this level, run the following command:

    bash /opt/gls/clarity/config/illumina-preset-protocols-installer.sh -o install all

  • At the parent item level, use the operation to install the workflows associated with a specific parent item. For example, run the following command to install the workflows associated with the AmpliSeq_for_Illumina_Protocols parent item:

    bash /opt/gls/clarity/config/illumina-preset-protocols-installer.sh -o install AmpliSeq_for_Illumina_Protocols.all

Headless Mode

Headless mode allows the install operation to complete without prompting for input. Use this mode if you want to automate the installation. When running the installation in headless mode, the process still runs through the validation phase before importing the configuration. However, if a conflict is found, the configuration causing the conflict is skipped automatically.

Installation Validation

The IPP installer validates the configuration import. If conflicts are found during the validation, a warning message displays. This message provides the following options:

  • f/Import anyway

  • s/Skip this workflow

  • a/Abort

The f/Import anyway option runs the import command of the config slicer tool. It also allows the tool to handle conflicts. This option does not run the config slicer tool in importAndOverwrite mode.

The s/Skip this workflow option does not import workflows where conflicts were found, but continues to import the other selected slices.

The a/Abort option aborts all import operations. The conflicts found are captured in the log files and can be reviewed.

Track and Log the Installation

The IPP installer tracks the installation process in the installhistory table and generates log files that can be reviewed after the installation is complete.

Viewing the installhistory Table

After installation is complete, a record is stored in the installhistory table in the database.

To view the installation record, use the following command:

java -jar /opt/gls/clarity/tools/propertytool/omxprops-ConfigTool.jar getInstallHistory

If a configuration slice is installed multiple times, there are multiple entries in the record. This scenario can happen during an upgrade or if there are attempts to resolve conflicts during installation. The log shows the following information for each slice:

  • Product — The name of the slice (for example, IPP - LIMS 5 - TruSight_Oncology_Protocols.TruSight-Oncology-500-HT-v1.0).

  • Version — The version of the configuration slice that was installed (for example, 2.0.0.25).

  • InstallDate — The time stamp for when the slice was installed. The date displays first (in YYYY-MM-DD format), followed by the time (in HH:MM:SS format)

Viewing the Log Files

After the configuration package is installed, the following log files are available:

  • ipp-installer.log

  • configslicer.log

  • omxprops.log

The ipp-installer.log file captures the output of the IPP installer tool. The name and location of this file can be changed with the command line options described previously. The default location is as follows:

/opt/gls/clarity/logs/illumina-preset-protocols/{IPP version}/{Time-based Digits}/

The configslicer.log and omxprops.log files are raw outputs from their respective tools. These files contain more detail than the ipp-installer.log file. For example, if a conflict is identified in the ipp-installer.log file, the configslicer.log file shows the specific entry causing the conflict. Some workflow selections, such as the MiSeq and NextSeq integration workflows, import database properties in addition to a configuration slice. By default, only the ipp-installer.log file is generated in the following directory:

/opt/gls/clarity/logs/illumina-preset-protocols/{IPP version}/{Time-based Digits}/

When the -l operation is specified during installation, all three log files are available in the current working directory.

IPP Integration v2.8 Workflow Upgrades

There are some expected conflicts when upgrading from IPP Integration v2.7.

In IPP v2.8, the on-premise NextSeq 1000/2000 is now supported and can cause a conflict with the Sequencing Instrument derived sample custom field. The following workflows can be affected:

  • Illumina DNA Prep (M) Tagmentation v1.0.7

  • Nextera Mate Pair v1.0.8

  • Nextera XT DNA v2.0.8

  • TruSeq DNA PCR-Free v2.0.8

  • TruSeq Nano DNA v1.0.8

  • Library Prep Validation v2.3.2

  • TruSeq ChIP-Seq v1.0.8

  • TruSeq Methyl Capture EPIC v2.0.9

  • TruSeq Small RNA v1.0.8

  • TruSeq Stranded mRNA v2.1.8

  • TruSeq Custom Amplicon (TSCA) v1.0.8

  • Nextera Rapid Capture Custom Enrichment v2.0.9

  • TruSeq DNA Exome v2.0.8

  • TruSeq RNA Exome v1.0.5

  • TruSeq RNA Access v2.0.8

  • TruSight Tumor 170 v2.0.8

To resolve these conflicts, perform the following actions:

  1. When the expected conflicts are encountered during the IPP workflow installation, enter f to import the workflow when prompted. An example of the prompt is as follows.

    Conflicts were detected between this slice and the existing configuration in the LIMS!
What would you like to do with this slice? [(f|Import anyway)|(s|Skip this workflow)|(a|Abort)]:

  2. After the workflow is installed, resolve the conflicts as follows.

    1. From Configuration, select the Custom Fields tab, and then select the Global Fields tab.

    2. Search for Sequencing Instrument.

    3. For Additional Options, add NextSeq 1000/2000 On-Prem to the Dropdown Items list.

PreviousIPP v2.8NextManual Upgrade

Last updated