Published: Aug 30, 2023

Last Updated: Aug 30, 2023

Latest Patch: 6.2.0

These Release Notes describe the key changes made to software components for Clarity LIMS v6.2.

Clarity LIMS v6.2 is deployable in both cloud hosted and on-premise environments.

New Features

• The operating system (OS) changes from CentOS7 to Oracle Linux v8.8.

• Lab account management that allows you to create, update, and delete accounts in Clarity LIMS.

• You can now set and update account custom field values. These configured fields are available in the new Account management screen when you view, create, or update an account.

• A red dot alert that displays when a required field without a default value is added to Clarity LIMS. This alert shows up LabLink next to the Configuration menu, the drop-down list in the Custom Fields menu, and next to Custom Fields in the Configuration tab.

• Configure client custom fields in LabLink through the Custom Fields Configuration page after they have been created through the Custom Fields Configuration Page in Clarity LIMS. The Users page in LabLink shows the configured custom fields when creating, editing, viewing, and reviewing a user.

Defect Repairs

• The Queue View page performance has been optimized.

• Reviewed and addressed potential vulnerabilities for cross site scripting (XSS).

Bug Fixes

• Fixed a bug where the upload of the sample submission spreadsheet with leading or trailing spaces in any of the header tags (eg, </TABLE HEADER>) resulted in import errors.

• Fixed a bug where the LIMS ID (Process) in the Record Details step disappeared when you select No preset.

• Fixed a bug where the custom term for Projects did not propagate to LabLink.

• Fixed a bug where samples were not placed correctly when you drag and drop from the plate GUI. This bug happened when a step was configured to have samples ordered by Column and placed by Row in the placement stage.

• Fixed a bug where false errors occured when the Config-slicer tool was used to import automation template files. In this update, the Config-slicer tool ignores any differences in the automation file XML tag, but logs a reminder in the configslicer-issues.log file. The file lists the different automation names that the users should check against so that the automation files for those names are uploaded properly. The file also shows warnings or errors logged.

• Fixed a bug where the Config-slicer command-line interface (CLI) tool showed false negative reports with control types and reagent kits when the values were empty during the import or validation of the slice.

• Fixed a bug that affected the UI for the File Attachment Method for the Demultiplexing step in Master Step and Step during the Configuration stage.

• Updated UI and API application logs so that they contain only useful log information.

Known Issues

• There is a bulk indexing error due to the memory space limitation for search-indexer when the indexing batch size is too large.

• There can be additional or missing search results when using Date Received of a sample as one of the search criteria in Advanced Search when the client and server are in different time zones.

• Multiple accounts that share the same email address only receive a reset password email for one of the accounts when a reset password request is made using Email information.

Additional Notes

Clarity LIMS v6.2 supports both Python2 and Python3 versions. However, there is no symbolic link created for /usr/bin/python to Python2 and there will not be a default link. Scripts have to be updated specifically to either Python2 or Python3.

This document provides the steps required to install a new Clarity LIMS v6.2 instance to Oracle Linux/RedHat Enterprise Linux v8.8 and v8.9.

The installation procedure includes adding the Clarity LIMS repository, installing the Clarity LIMS RPM through yum commands, and configuring the installation through a series of configuration scripts.

Prerequisites

• Your system meets the requirements listed in Technical Requirements.

• You have installed and configured the required components. For more information, see Pre-Installation Requirements.

• You have a database user and two empty schemas on your database server. The schemas are populated during configuration.

• You have received the appropriate repository files from the Clarity LIMS Support team.

• All standard OS security updates have been applied.

• All instances of Clarity LIMS must have a purchased SSL/TLS certificate installed. Purchase the certificate before installation or upgrade. For instructions on installing purchased SSL/TLS certificates, see Install a Purchased SSL/TLS Certificate.

With the Oracle Linux/RedHat Enterprise Linux server, the following error messages can display when you perform the yum commands used to install Clarity LIMS:

Failed loading plugin "product-id": No module named 'urllib3'

Failed loading plugin "subscription-manager": No module named 'urllib3'

Failed loading plugin "upload-profile": No module named 'urllib3'

These messages do not affect the installation of Clarity LIMS. You can resolve these error messages by running the following command:

pip3 install urllib3==1.24.2

Step 1: Add the Clarity LIMS Repository

1. Using scp/sftp, WinSCP, FileZilla, PSCP, or similar, copy the repository to the following location: /etc/yum.repos.d.

2. Test the repo file with this command:

   Copy

   yum --enablerepo=GLS_Clarity62 search ClarityLIMS-App

Step 2: Install Clarity LIMS

1. Run the install command:

   Copy

   yum --enablerepo=GLS_Clarity62 install ClarityLIMS-App

2. Type y to download and install the Clarity LIMS RPM core components.

   NOTE: The installation of Clarity LIMS creates 3 operating system users:

   • glsai - User created to run the Automation Worker node

   • glsftp - User to access the SFTP file store

   • glsjboss - Runs the application server

   These users are created by the RPM installation process, and should not be created before starting the installation steps. The user home directories are created in the directory /opt/gls/clarity/users

   The operating system passwords for each of the above users should be set by the root user.

Step 3: Run Configuration Scripts

1. As the glsjboss user, change directory to /opt/gls/clarity/config/pending with the following command:

   Copy

   cd /opt/gls/clarity/config/pending

2. Run the first script listed sequentially in the directory listing with the following bash command:

   Copy

   bash /opt/gls/clarity/config/pending/05_configure_claritylims_secretutil.sh

   This script configures the Secret Utility password management tool so that secrets and passwords are accessible. It is recommended that you store application secrets in vault. If that is not possible, the configuration script supports file-based storage. For more information about the prompts, see Guide to Secret Management.

3. Run the following script:

   Copy

   bash /opt/gls/clarity/config/pending/20_configure_claritylims_platform.sh

4. Run the next script to initialize the database and overwrite any existing data:

   Copy

   bash /opt/gls/clarity/config/pending/26_initialize_claritylims_tenant.sh

   NOTE: This script requires that you enter the password for the glsftp user. Entering the password does not set the password for this user.

5. If your database server is standalone or remote, update the /opt/gls/clarity/tomcat/current/lib/activity-management-ui-config.groovy file with the following code snippet.

   Copy

   hibernate {
       isis {
           template {
               url="jdbc:postgresql://<Replace me: Remote DB IP>:<Replace me: Remote DB Port>/{0}"
           }
       }
   }

   NOTE: Substitute the Remote DB IP and Remote DB Port placeholders with the information for your server.

   If you do not update the script, log and database connection errors can occur.

6. Change to the root user, and then run the following script in the sequence to configure RabbitMQ:

   Copy

   bash /opt/gls/clarity/config/pending/32_root_configure_rabbitmq.sh

7. As the root user, install the Apache proxy with the following script:

   Copy

   bash /opt/gls/clarity/config/pending/40_root_install_proxy.sh

Step 4: Install Lablink

LabLink v2.4 is compatible with Clarity LIMS v6.2.

Before installing LabLink v2.4, make sure that a database named LabLink is created with the same database user as the Clarity LIMS database.

1. Turn off all Clarity LIMS services using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh stop

2. Install the LabLink RPM with the following yum command. Make sure that you have the correct repo enabled:

   Copy

   yum --enablerepo=GLS_Clarity6 install ClarityLIMS-LabLink

3. As the glsjboss user, run the pending initialization script using the following command:

   Copy

   bash /opt/gls/clarity/config/pending/60_initialize_lablink.sh

4. Restart all Clarity LIMS services using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh start

5. Make sure that LabLink is accessible at https://<your-Clarity-FQDN>/lablink

Step 5: Start the System

Clarity LIMS includes the run_clarity.sh script. This script starts (or stops) all Clarity LIMS services (Elasticsearch, RabbitMQ, Search Indexing, Tomcat, httpd/Apache proxy, Automation Worker) in the required order, with one command.

Run the following script as the root user:

/opt/gls/clarity/bin/run_clarity.sh (start|stop|restart)

If an error occurs starting any service, subsequent services will not be started. Stop all services before trying to start them again.

Start the system as follows.

1. Switch to the root user.

2. Make sure that no Clarity LIMS services are running.

3. Run the script with the following start command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh start

4. After the script has completed, all Clarity LIMS services should be ready for use.

If any services are running, the script exits and provides a list of services to stop. In this scenario, complete the following steps:

1. Use the script with stop command to stop services.

2. Open a supported browser window and make sure that you can access the Clarity LIMS client at the following URL: https://<your-Clarity-FQDN>/

This section provides an overview of the components of Clarity LIMS. For technical requirements, refer to Technical Requirements.

Environment Overview

Clarity LIMS is built on a platform that is easy to customize and supports off-the-shelf hardware, common database systems, and industry-standard data formats.

We offer both on-premise and Illumina cloud hosted deployment models. Both offerings use the same underlying software and security model as explained in the Clarity LIMS Security and Privacy technical note, available for download from the Illumina website.

For on-premise deployments, we provide flexibility in selecting tools that suit your needs and existing IT infrastructure. The Clarity LIMS environment includes:

• Application server

• Database

• File server

• Web client

Depending on the details of your contract with Illumina, the environment can also include:

• Automation Workers

For cloud hosted deployments, the hardware is scaled upward by Illumina as required.

System Architecture

For on-premise deployments, you can separate application and file server functions from the database functions by installing them on discrete hardware platforms. Alternatively, you can combine them on a single server hardware platform. Base your decision on the size of your installation and how much data your laboratory processes.

Whichever solution you choose, Illumina requires that the Clarity LIMS server environment reside on dedicated hardware, free from other Illumina or third-party server products. The server environment must also be on a 1 Gbs or faster network. The network should not contain links with a capacity lower than 100 Mbps on any network-connected devices, such as routers, firewalls, and switches.

The Clarity LIMS installation installs Apache Tomcat 9.0, Apache Webserver v2 (used as a proxy), ElasticSearch and RabbitMQ to support search, and Open JDK 8.0. These software versions are the only versions that Clarity LIMS supports. The software packages are supplied by Illumina.

For cloud hosted deployments, Illumina fully manages the system deployment and maintenance.

Application Server

Clarity LIMS uses a web application served by the Apache Tomcat server to manage the creation, collection, and retrieval of data and results. This core server is built on a Java architecture and allows for rapid deployment and custom configuration for other on-premise and Illumina cloud hosted deployments.

For both on-premise and Illumina cloud hosted deployments, the application requires the following standard secure ports for web and file communications:

• Web communications: Port 80, 443

• File communications: Port 22

Depending on included instrument integrations, additional open ports may be required.

Additionally, for cloud hosted deployments, a site-to-site VPN connection, using IPSEC, might be required for instrument integrations. The most current list of required open ports is included in the preinstallation documentation that is provided during an implementation project.

Database & File Server

Clarity LIMS supports PostgreSQL database to record data generated by the client, and references to file locations on the Clarity LIMS file server.

If the system needs to export a file, it issues a call to the database to find the file location on the file server. We recommend that you store files on a server or file system separate from the Clarity LIMS application server, such as a Network Attached Storage (NAS) appliance.

When handling data, Clarity LIMS saves files in their original format. The advantages of saving files in their original format are as follows:

• The size of files is restricted only by the size of your file system.

• You benefit from the built-in error-correction and integrity-checking features included in the file system.

The amount of storage space required by Clarity LIMS depends on the following:

• The number of samples your laboratory processes each day.

• The instruments you use.

• The number of files you save to the Clarity LIMS system.

Illumina will work with you to recommend an appropriate amount of storage space.

NOTE: We do not recommend that you place the Clarity LIMS file server on a remote mount to a Windows server. We recommend you discuss other network storage devices, such as high availability NAS, with your hardware and supported operating system vendors.

Web Client

User-centric, goal-based design has become the new standard in software interfaces. Clarity LIMS has a clean, helpful, easy-to-use interface. It is a lightweight web application that provides:

• A simple, fast, and efficient way for lab scientists to identify work they need to complete.

• The tools necessary to complete and record that work quickly and efficiently.

Automation Workers

The Clarity LIMS Automation Worker allows specifically designed scripts to automate and extend the functionality of Clarity LIMS. You can integrate a wide variety of laboratory instruments and software.

The Automation Worker runs as a Windows Service or as a Linux daemon. You can install the Automation Worker on any computer with a supported OS on your Clarity LIMS network. When you install multiple copies on different machines on your network, Clarity LIMS automatically distributes work across the machines to improve system performance.

NOTE: Only one Automation Worker node can be installed on a Windows server.

Mixpanel

Mixpanel™ is a system that provides Illumina with information about how users interact with the Clarity LIMS web client. We do this by tracking which features are being used, and how often. Gathering this information allows us to determine which interactions are most common, and how our users proceed through protocol steps and tasks. We can then use this information to improve system performance and ultimately enhance the user experience.

All data are collected anonymously. We collect data on Clarity LIMS usage only. We do not collect specific sample names, projects, or values entered. We do track total usage (number of samples selected, how many protocol steps executed, and so on). Data are collected across all customers for analysis in one group. We do not directly track which site is doing which work. If you require more information about Mixpanel, contact the Illumina Support team.

Security

All client traffic is encrypted over secure HTTP (HTTPS). To ensure the security of the transactions between Clarity LIMS and clients, on-premise deployments require a purchased certificate. The certificate should be from a well-known vendor such as DigiCert, Entrust, or QuoVadis. For information on the policies, processes, and controls enacted for security and privacy of data in cloud hosted deployments, see the Clarity LIMS Security and Privacy Technical Note, available for download from the Illumina website.

Released versions:

Release Notes Clarity LIMS v6.2.1

Release Notes Clarity LIMS v6.2.0

Published: July 30, 2024

Last Updated: June 07, 2024

Latest Patch: 6.2.1

These Release Notes describe the key changes made to software components for Clarity LIMS v6.2.1. The Clarity LIMS v6.2.1 patch release supersedes v6.2.0. It is intended for customers on or migrating to Clarity LIMS v6.2.0.

This release resolves customer-reported issues related to the following functions:

• Search-indexer performance

• Basic search failure when searching with multiple custom fields in the system

• Necessary handling of controls for a configured step

New Features

There are no new features added with this release.

Bug Fixes

• Fixed a bug where the search-indexer did not quickly consume messages from the queue. This issue caused data in the system to be non-searchable.

• Fixed a bug where the bulk indexing for a project, container, step, and file failed due to a limitation in memory space for search-indexer. This bug occurred when the indexing batch size was too large. Indexing bulk size can now be modified with an attribute in /opt/gls/clarity/search-indexer/conf/search-index-config.properties.

• Fixed a bug where the basic search failed when searching with large numbers of custom fields in the system.

• Fixed a bug where details about the step failed to show when selecting the LIMS ID link for steps in the Advanced Search results.

• Fixed a bug where controls and samples must be added in a certain order for the configured step. This issue caused an error that prevented the configured protocol step from starting.

• Fixed a bug where a step cannot be started when controls and samples are in the standard step and the Enable QC flag is set to Yes.

Known Issues

• There can be additional or missing search results when using Date Received of a sample as one of the search criteria in Advanced Search when the client and server are in different time zones.

• Multiple accounts that share an email address only receive a reset password email for one of the accounts when a reset password request is made using Email information.

• Users with the Read-Only permission do not have access to Advanced Search.

• {ProjectName} in the {ProjectName} token appears as duplicated when there is more than one sample in a step.

• On the Place Samples screen, the sample well position in the source container takes precedence in the placement order of the destination plate when placing samples from the Sample View List table. This placement order change only happens when samples are transferred from more than one source container. If the Sample View List table is sorted by a specific column, the order of the listed samples is ignored when they are placed in a destination plate.

• The config slicer tool will takes longer time (~8 times) to import a config slice into instance with Clarity LIMS v6.2 when compared to earlier versions (v6.1 and below).​

Additional Notes

• Customers connecting remotely to PostgreSQL database need to make sure that their JDBC driver is compatible with the PostgreSQL version used.

• For on-premise deployments, Clarity LIMS v6.2.1 is qualified to run with the following server OS versions:

  • Oracle Linux v8.8 and v8.9 (64-bit)

  • RedHat Enterprise Linux v8.8 and v8.9 (64-bit)

• Security vulnerabilities in PostgreSQL 15.x have been addressed. PostgreSQL 15.2–15.6 is at a low risk of security vulnerabilities for customers on Clarity LIMS v6.2.1.

• PostgreSQL 15.2 and 15.6 are now qualified to run in Clarity LIMS v6.2.1.

Secret Utility (secretutil) is a password management tool used to store, manage, and retrieve passwords. Secret Utility returns the passwords in plain text.

The following sections describe the configuration of Secret Utility, which is installed as part of the Clarity LIMS-SecretUtil RPM.

Installing Integration Packages on Clarity LIMS

Configuration Script

If Secret Utility has not been configured, the 05_configure_claritylims_secretutil.sh script is created in the /opt/gls/clarity/config/pending folder.

To reconfigure Secret Utility:

1. Remove the hidden file /opt/gls/clarity/tools/secretutil/.configured

2. Run the Secret Utility configuration script as follows: /opt/gls/clarity/config/configure_claritylims_secretutil.sh

The following table describes the entries prompted by the configure_claritylims_secretutil.sh script.

Configuration Script Entries

Managing the passwords (Vault Mode)

If Secret Utility is configured as Vault Mode, the passwords are stored and retrieved from Vault Enterprise.

• To use Secret Utility and perform the following steps, you must first remote into the instance before performing any of the following steps.

• To use the Vault user interface (UI) and perform the following steps, you must have the appropriate role and access control list (ACL) policies.

Managing the Passwords (File Mode)

If Secret Utility is configured as File mode, the passwords are encrypted and stored in /opt/gls/clarity/tools/secretutil/conf/secrets.properties. Encryption is based on the CLARITYSECRET_ENCRYPTION_KEY environment variable.

To manage the passwords and perform the following steps, you must first remote into the instance.

The following information is a summary of the technical requirements for an on-premise or cloud based deployment of Clarity LIMS v6 and later. To install and use Clarity LIMS, the client and server systems must meet these requirements. Clarity LIMS is designed to run on standard commodity hardware. The requirements provide general guidelines for your hardware configuration. You can obtain specific configuration quotes from the hardware vendor of your choice.

Allow enough time for the procurement of your hardware and software. Make sure that all components are installed and configured before proceeding with the installation of Clarity LIMS.

Production Server Requirements

For on-premise deployments, Clarity LIMS has two levels of recommended hardware. For larger labs in full production, we strongly suggest the high-throughput requirements.

The production server must be configured in US locale.

Recommended

• Server class 64-bit CPU with at least eight cores at 2.9 GHz

• 20 MB shared cache (L3) memory

• 32 GB RAM

  • 6 GB allocated to Tomcat

  • 6 GB allocated to the database

  • 2 GB allocated to ElasticSearch

• 100 GB hard disk drive space for the operating system, application, and log storage

• 1 Gbps Ethernet network or faster

High-throughput:

• Server class 64-bit CPU with at least 16 cores at 2.9 GHz

• 20 MB shared cache (L3) memory

• 64 GB RAM

  • 8 GB allocated to Tomcat

  • 8 GB allocated to the database

  • 2 GB allocated to ElasticSearch

• 100 GB hard disk drive space for the operating system, application, and log storage

• 1 Gbps Ethernet network or faster

For cloud hosted deployments, Illumina sizes the system accordingly for system load. We reserve the right to archive auditing information to maintain system performance as the data set grows.

If your subscription is not renewed for your cloud hosted deployment, at your request, we will supply you with an export of all user data. In practice, we will provide a database dump and details on the database schema for you to pull out any data you need going forward.

Server Operating System Requirements

For on-premise deployments, Clarity LIMS has been qualified to run with the following server operating systems versions:

• RedHat Enterprise Linux v8.8 and v8.9 (64-bit)

• Oracle Linux v8.8 and 8.9 (64-bit)

For cloud hosted deployments, Illumina uses the latest qualified Oracle Linux version.

Database Requirements

For on-premise deployments, Clarity LIMS has been qualified to run with PostgreSQL 15.2 and 15.6.

For cloud hosted deployments, Illumina uses the latest qualified PostgreSQL version.

Web Client Requirements

The following client requirements apply to both on-premise and cloud hosted deployments.

Hardware

• 64-bit processor (dual-core 3.0 GHz)

• 8 GB RAM

Operating Systems

• Windows 10

• Linux (restricted to the server-supported versions listed previously)

• Macintosh OS X (12 Monterey or later)

• iOS 15 on iPad running Safari browser

Web Browsers

• Google Chrome (latest update)

• Mozilla Firefox (latest update)

• Apple Safari on iPad only (latest update)

Other Requirements

• 1280 x 800 or higher

• Cookies and JavaScript must be enabled

For both on-premise and cloud hosted deployments, a 20 Mbs network connection speed from client to server is required. If remote access via VPN is needed for LDAP or instrument integrations, we recommend 100 Mbs network connection speed between your site and the hosted instance.

Automation Worker Requirements

The following requirements apply to automation workers installed on premise, for both on premise deployments and Illumina cloud hosted deployments, to support instrument integrations.

Hardware

• 64-bit processor

• 1 Gb RAM

• Hard disk drive space equivalent to twice the size of the largest file you are planning to transfer

Operating Systems

• Windows 10

• RedHat Enterprise Linux v8.8 and v8.9

• Oracle Linux v8.8 and 8.9

Applications

• Linux – Illumina installs Java Open JDK 8.0 update 362 (1.8.0_362)

• Windows – Clients must install Java Open JDK 8.0 update 362 (1.8.0_292)

Before installing Clarity LIMS, you must purchase hardware and software that meet the minimum requirements (see Technical Overview). Following those purchases there are several components that you must organize, install, or configure.

The following sections discuss these components, and how to install and configure them. These sections apply to on-premise customers only. Before completing the steps described, make sure that the server has the minimum requirements. See Technical Requirements for details.

Purchase SSL / TLS Certificate(s)

All instances of Clarity LIMS must have a purchased SSL / TLS certificate installed.

Certificate Authorities will no longer issue SSL / TLS certificates for internal server names. As a result, to obtain a certificate you must have a valid, public DNS entry for your server.

Before installing or upgrading Clarity LIMS, do the following:

1. Purchase an SSL / TLS certificate.

2. Save the certificate files on the server on which the Clarity LIMS server is installed.

3. Provide the Clarity LIMS Support team with the private key and password for the SSL / TLS certificate.

For instructions on obtaining a certificate, see Install a Purchased SSL/TLS Certificate.

Check SELinux Mode

Security-Enhanced Linux (SELinux) is not supported for use with Clarity LIMS. Make sure that SELinux is set to either permissive or disabled mode.

For instructions, see the following sections of the Red Hat documentation:

• 5.4.1.2 Permissive Mode

• 5.4.2 Disabling SELinux

Set Up Root Access Server

Clarity LIMS is installed using industry standard RPM packaging. The Clarity LIMS support team requires root credentials to the server during the installation process.

The following sections discuss the system user accounts that the support team sets up during the installation process. It is important that you do not change these system users.

The production server must be configured in US locale.

Install and Configure the Database

After installing a supported database, Clarity LIMS requires certain changes to the default database configuration.

Additional tablespace names and user profiles may be needed, depending on the configuration of your system.

For more information or for assistance with your database configuration, contact the Clarity LIMS Support team.

Confirm Hostname Resolution

To access the Clarity LIMS server via DNS, make sure that the following apply:

• The server local host file /etc/hosts does not contain an entry for that hostname bound to its loopback address.

• Any hostname entries correspond to their entries in DNS.

• The command hostname -f must return the fully-qualified domain name of the server.

For client systems:

• Users should use the fully-qualified domain name (FQDN) when logging on to the system. Using the FQDN ensures persistence of the session ID.

Set the TimeZone (TZ) Environment Variable

Clarity LIMS requires the environment variable TZ be set on the Clarity LIMS server to your correct timezone. If the value is not configured, a default of GMT is configured by Clarity LIMS in the file /etc/profile.d/clarity.sh.

This file might update on upgrade. Any changes must be manually applied across upgrades.

Configure TCP/IP Settings

To allow proper system communication, the following ports on the Clarity LIMS server must be accessible by the LIMS clients:

• TCP/IP Port 22 (SFTP) for file transfers between the client and server

• TCP/IP Port 443 (HTTPS) for Apache proxy

• TCP/IP Port 80 (HTTP) used to forward any unknown unsecured requests over SSL / TLS and port 443

The following ports are required on the local Clarity LIMS server:

• TCP/IP Port 4369 for Epmd for RabbitMQ

• TCP/IP Port 5432 for PostgreSQL database communications *

• TCP/IP Port 9009 for Tomcat

• TCP/IP Port 9200 for Elastic Search

• TCP/IP Port 9300 for Elastic Search

• TCP/IP Port 5672 for RabbitMQ

• TCP/IP Port 15672 for RabbitMQ

Configure automation worker TCP/IP settings

Computers running an automation worker must be able to reach the Clarity LIMS server via the following ports:

• TCP/IP Port 22 (SFTP) for file transfers between the client and server

• TCP/IP Port 443 (HTTPS) for Apache proxy

• TCP/IP Port 80 (HTTPS) used to forward any unknown, unsecured requests over SSL / TLS and port 443

Configure VPN Access for Hosted Systems

To facilitate instrument integrations, a site-to-site IPSEC VPN connection can be set up between your facility and the hosted instance.

There are two ports that must be opened: 4500/udp and 500/udp.

If a VPN is required, you must provide more detailed setup information to the Clarity LIMS Support team. Upon request, the Clarity LIMS Support team will provide the additional form required to do this.

Save any Apache Proxy Configuration

Clarity LIMS uses an Apache proxy and the Clarity LIMS installation process installs and configures it automatically. If the server already has an Apache proxy installed and configured, the installation process overwrites the current configuration. If that configuration is important, you must back it up before running the Clarity LIMS installation process. Any settings that are important to your organization must be reconfigured manually after an install or upgrade of Clarity LIMS.

Install and Configure HashiCorp Vault

In Clarity LIMS v6.0.0 and later, you can choose to install and configure a HashiCorp Vault to store Clarity LIMS-related passwords and secrets safely.

For more information, refer to Configure Your HashiCorp Vault.

Two levels of user passwords are created in the Clarity LIMS system: one at the operating system level and one at the Clarity LIMS level.

Following are details on the user passwords, instructions for changing them, and instructions for updating Clarity LIMS with the new database connection details.

Update Operating System User Passwords

The user passwords created at the operating system level are for the glsai, glsjboss, and glsftp users.

• glsai and glsjboss users:

  • These users have no configuration associated with them.

  • You may change their passwords at any time.

• glsftp user:

  • After installation of Clarity LIMS, you can change this password. However, you must also update it in the file/vault secret store, using the Secret Management Util tool.

To change the glsftp password after installing Clarity LIMS, using SecretUtil:

1. Change the glsftp user's password on the server.

2. Log in to the server as the root user.

3. Stop Clarity LIMS using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh stop

4. Go to /opt/gls/clarity/tools/secretutil.

5. Update the password in the secret store.

   • For vault-based secret storage, use either the Vault command line interface (CLI) or Vault user interface (UI) to update the password.

   • For file-based secret storage, use Secret Management Util to update the password as follows:

     Copy

     $ java -jar /opt/gls/clarity/tools/secretutil/secretutil.jar -u="new-password" app.ftp.password

6. Start Clarity LIMS using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh start

Update Clarity LIMS User Passwords

The user passwords created at the Clarity LIMS level are for the admin, facility, and apiuser users.

• admin and facility users:

  • These users have no configuration associated with them.

  • You may change their passwords at any time.

• apiuser user:

  • After installation of Clarity LIMS, you can change this password. However, you must also update it in the file/vault secret store, using the Secret Management Util.

To change the apiuser password after installing Clarity LIMS**:**

1. Check for any remote Automation Workers, and take note of their locations in your network. You will need to restart these after changing the password.

2. Log in to the server as the root user.

3. Stop Clarity LIMS using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh stop

4. Go to /opt/gls/clarity/tools/secretutil.

5. Update the password in the secret store

   • For vault-based secret storage, use either the Vault command line interface (CLI) or Vault user interface (UI) to update the password.

   • For file-based secret storage, use Secret Management Util to update the password as follows:

     Copy

     $ java -jar /opt/gls/clarity/tools/secretutil/secretutil.jar -n=integration -u="new-password" "apiusers/apiuser"

6. Start Clarity LIMS using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh start

Update Database Connection Details

In some circumstances (such as security breaches/compromises), the database connection details (eg, database password) are updated, which prevents Clarity LIMS from connecting to the database. You can correct this issue by updating Clarity LIMS with the new database connection details as follows.

1. Check for any remote Automation Workers.

2. Update the existing tenant with the new details.

3. Restart any Automation Workers.

To update database connection details:

1. Check for any remote Automation Workers, and take note of their locations in your network.

2. Log in to the server as the root user.

3. Stop Clarity LIMS using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh stop

4. Go to /opt/gls/clarity/tools/secretutil.

5. Update existing tenant with new details.

   • For vault-based secret storage, use either the Vault CLI or Vault UI to update the password.

   • For file-based secret storage, use the Secret Management Util to update the database password as the root user.

   Copy

   $ java -jar /opt/gls/clarity/tools/secretutil/secretutil.jar -u="db-password" db.<db-name>.password

6. Start Clarity LIMS using the following command:

   Copy

   /opt/gls/clarity/bin/run_clarity.sh start

After initial user training, but before the lab starts to use Clarity LIMS in production, a database cleanup is recommended.

This process removes any projects, samples, workflows, protocols, steps, and other artifacts from the system that were created during training but are not needed for production.

Contact the Clarity LIMS Support team to schedule a time for this cleanup procedure.

Preparation

In preparation for the clean-up, complete the following steps:

• Delete all unwanted custom fields and master steps.

• Set the status of all unwanted workflows to Archived. If there is an Archived workflow that you would like to keep, temporarily set its status to Active. Note the following:

  • Protocols that are part of an Archived workflow, or set of Archived workflows will be deleted.

  • Protocols that belong to an Active or Pending workflow, in addition to an Archived workflow, will not be deleted.

After these steps are completed, a Clarity LIMS Technical Support Analyst (TSA) helps to perform the cleanup procedure. This procedure takes approximately 15–20 minutes to complete.

The following steps use the HashiCorp Vault user interface (UI) to guide you through the configuration of your HashiCorp Vault instance.

These configurations are mandatory for on premise Clarity LIMS deployments. For Illumina cloud hosted deployments, this configuration is completed by Illumina.

Pre-requisites

1. You are planning to install Clarity LIMS v6.0.0 and newer.

2. You have installed the latest version of either HashiCorp Vault Open Source or Enterprise.

3. You have read the Getting Started tutorials for Vault on the HashiCorp website and/or possess a basic knowledge of HashiCorp Vault.

4. You have system administrator permissions to perform the necessary operations to your HashiCorp Vault instance.

5. You have allowed the necessary port 443 from the Clarity LIMS instance to your HashiCorp Vault instance.

6. You have access to all the passwords required to be configured in your HashiCorp Vault instance.

Configuration

Enable a new KV Secret Engine

To enable a new KV Secret Engine, refer to the Versioned Key/Value Secrets Engine tutorial provided on the HashiCorp Vault website.

Configuring Secret Engine

The following table lists the secrets required for Clarity LIMS. To use the paths shown in the table, replace $host with your fully qualified domain name (FQDN).

When configuration is complete, these secrets are listed in the Vault user interface.

Authentication Methods

AppRole

AppRole is the recommended authentication method to use with the Clarity LIMS Secret Utility tool.

1. To enable the AppRole authentication method, refer to the AppRole Pull Authentication tutorial provided on the HashiCorp Vault website.

2. When AppRole is enabled, create an AppRole with the appropriate Access Control List (ACL) policy (see the following section).

3. Make a note of the Role ID and Secret ID. You need these IDs when configuring Secret Utility.

Token

Alternatively, the Clarity LIMS Secret Utility tool also works with the token authentication method.

To learn more about tokens, see the Tokens documentation on the HashiCorp Vault website.

Authentication ACL Policies

After enabling the AppRole authentication method, create ACL policies to access the Secret Engine.

IMPORTANT: Replace "claritylims" with your Secret Engine path.

To create the ACL policy, refer to the Vault Policies tutorial provided on the HashiCorp Vault website.

Verification

1. SSH into the Clarity LIMS instance.

2. After installing Clarity LIMS and configuring Secret Utility using the instructions provided in Guide to Secret Management, run the command to read an existing password/secret from HashiCorp Vault.

\

Clarity LIMS can work with Named or WildCard certificates.

Typically, the process to install the certificates into Clarity LIMS is as follows:

1. Request a certificate from your IT organization, or purchase a certificate from a third-party SSL/TLS vendor.

2. Install the certificate using the script installCertificates.sh provided with Clarity LIMS. This script prompts for the required inputs and helps you to configure Clarity LIMS to use your SSL/TLS certificate.

Obtaining the Certificate

You will need your organization or the third-party SSL/TLS vendor to provide you with the following:

• An Apache 2.4-compatible SSL/TLS Certificate

• The Certificate private key

• The corresponding certificate chain, properly prepared for Apache 2.4. This component may not be required, depending on the organization that signs your certificate.

Your IT organization might provide you with a WildCard certificate. Clarity LIMS can use WildCard certificates, as long as the Apache 2.4-compatible certificate, private key, and certificate chain files are provided.

If purchasing from a third-party vendor, make sure that the vendor provides you with an Apache 2.4-compatible bundle that includes the components listed above. To purchase from a vendor, refer to their documentation.

Private key password considerations

By default, a private key has a password associated with it. On startup, Apache requests a passphrase to access the private key. You can use either of the following methods to resolve this issue:

Method 1 — Place a passphrase file on the system and reference it in your clarity.conf file.

1. Create a passphrase file in a directory that has read, write, and execute permissions for only the root or apache user.

2. Edit the clarity.conf file. The clarity.conf file is in the /etc/httpd/conf.d directory.

   Add the following line to your clarity.conf file, before the section:

Method 2: Removing passphrase from an OpenSSL key

• Remove the password from an OpenSSL key using the following command:\

Install SSL/TLS Certificates for Use with Clarity LIMS

Assumptions and Prerequisites

• You have installed BaseSpace Clarity LIMS and run the 40_install_proxy.sh script.

• You have OpenSSL (installed by default on the Clarity LIMS Linux server when you install Clarity LIMS). OpenSSL is used by the installCertificates.sh script.

• You have the files listed in the following table (obtained from the process described previously) available on the Clarity LIMS server. In the example shown below, these files are located at /tmp/certs.

Install the signed SSL/TLS Certificates

On the Clarity LIMS server, as the root user, run the installCertificates.sh script:

As of Clarity LIMS v5.4, the method used for managing passwords (secrets) for Clarity LIMS integration modules has changed.

The following diagram shows the installation steps required for installing Secret Utility and integration packages with Clarity LIMS v5.4 and later.

1. Install Clarity LIMS v6.2.

   • Install Clarity LIMS-App 6.2 and complete pending script.

   • Secret Utility (secretutil) is installed as part of Clarity LIMS-App 6.2 dependency, and the Secret Utility configuration is part of Clarity LIMS pending scripts.

2. Install Secret Utility.

   • In the automated installation tooling for Clarity LIMS v6.2, the installation and configuration of Secret Utility is included. No further action is necessary.

   • For manual installation:

     1. Install Clarity LIMS-SecretUtil.

     2. Configure Secret Utility by running the following script: /opt/gls/clarity/config/pending/05_configure_claritylims_secretutil.sh.
3. Check usage of custom API username.

   • Check for any needs for custom API username, if any (eg novaseq_user). The documentation for the integration package provides the requirements for API username.

     NOTE: In a typical installation, a default API username (apiuser) is used. It is not necessary to add the default apiuser username, because it is configured as part of Clarity LIMS v6.2 installation. If no custom API username is required, skip step 4 and proceed to step 5.

4. Configure custom API username.

   • If a custom API username is not required, proceed to step 5.

   • If a custom API username is required, configure the user/password with Secret Utility as follows. Substitute the values enclosed in double quotes with your own values, keeping the double quotes.

     java -jar /opt/gls/clarity/tools/secretutil/secretutil.jar -n=INTEGRATION -u="password" "key"

     Example:

     java -jar /opt/gls/clarity/tools/secretutil/secretutil.jar -n=INTEGRATION -u="mypassword" "apiusers/novaseq_user"

   • For a custom api username, set the key to apiusers/{custom api username}

5. Install integration package.

   There is no change to the installation of the integration package. Follow existing installation instructions.

   NOTE:

   The new configuration script in the new integration package retrieves passwords directly from Secret Utility.

   For IAP access token and BSSH access token, follow the existing setup guide. There is no change in the configuration step.

From Versions 4.2 / 4.3 / 5.0 / 5.1 / 5.2 / 5.3 / 5.4 / 6.0 / 6.1 to 6.2

This document provides details on the steps required to upgrade an existing Clarity LIMS to RedHat Enterprise Linux/Oracle Linux compatible Clarity LIMS v6.2.

For installation requirements and Oracle Linux compatibility, see .

Migration Paths

The following table shows the applicable migration paths.

Prerequisites

Before Illumina can proceed with the upgrade, complete the following prerequisite steps.

Provision the New Instance

We recommend you provision an instance with similar or higher specifications to the current Clarity LIMS instance.

Note the following:

• All standard operating system (OS) security updates must have been applied.

• Upgrades are only supported from Clarity LIMS v4.2/4.3/5.0/5.1/5.2/5.3/5.4/6.0/6.1.

Configure the New Instance

• Custom configurations: If you have made any additional configurations that are not part of the Clarity LIMS pre-installation requirements, apply these configurations to the new instance.

• Passwords: Configure all passwords to be same as the existing instance. After you have verified the new instance, you can change passwords as needed.

Verify User Accounts Email Addresses

Make sure that all user accounts have email addresses associated with them. Users must reset their passwords after the upgrade is complete.

Install and Run the Pre-Validation Script on the Source Server

To assist with validating the system before an upgrade, install the UpgradePreValidation RPM on the source server.

• This RPM is installed temporarily, and provides tools to help check the system before an upgrade.

• If validation is successful, you can remove this RPM and proceed with the upgrade.

1. Install the UpgradePreValidation RPM. Make sure you have the correct repo enabled.

   • On the source server, as the root user, run the following command:
2. [Optional] Set up Secret Utility.

   • If ClarityLIMS-SecretUtil was installed previously, run the following command to set up Secret Utility as the glsjboss user:

     NOTE: Using a vault is the safer way of storing application secrets. If using a vault is not possible, the configuration script supports file-based storage.
3. Run the validation script as follows.

   1. Make sure that the Clarity LIMS server is running.

   2. As the root user, run the following command:
   3. Review the output of the script to determine if you can proceed with the upgrade. If the script outlines any issues with the potential upgrade, review the generated log files and contact the Clarity LIMS Support team for further assistance.

4. Remove the PreValidation RPM.

   Remove the PreValidation RPM only after you confirm that you can upgrade. If you are unsure, consult the Clarity LIMS LIMS Support team.

   • As the root user, run the following command:

Back Up the Current Database and Clarity LIMS

Archive the backup in case a rollback is required.

Before performing the backup, stop Clarity LIMS. The following command stops all Clarity LIMS components, including Automation Worker and integration services.

1. Stop Clarity LIMS:

   • On the command-line interface, run the following command as the root user:
2. Back up Postgresql database:

   On the PostgreSQL server, best practice recommends backing up the database using the pg_dump utility.

   The following example assumes the following:

   • The database server and the application server are on the same server.

   • The pg_dump utility is accessible to the glsjboss user.

     Example

     The Postgres DBA uses the following commands to create a database backup in the glsjboss home directory. Substitute the variables as appropriate for the specific environment.

     • As the glsjboss user:
3. Make sure that the following items, and any other files and configurations, are backed up safely:

   • crontab -l

   • custom scripts

   • OS configuration files

   • firewall rules

   • network configuration

   • etc.

   If there are custom changes to any application configurations (to increase performance, security, etc.), restore/configure these items manually later by referencing the backup.

   We recommend that you back up the items into a single zip file and transfer them to the new instance.

   Directories

   /opt/gls/clarity/users/glsftp or /home/glsftp (Clarity LIMS file store location)

   /opt/gls/clarity/customextensions

   /opt/gls/clarity/glscontents

   /etc/httpd/conf.d

   /etc/httpd/sslcertificate

   Files

   .pgpass

   /opt/gls/pgsql/9.x/pg_hba.conf

   /opt/gls/pgsql/9.x/postgresql.conf

   Additional configurations:

   rpm -qa | grep "BaseSpace\|Clarity" > clarityrpms.txt

Upgrade steps

Install Latest Clarity LIMS on New Instance

1. Make sure that the repository file exists in the following location:
2. As the root user, install the RPMs required on the new instance by referencing the content of clarityrpms.txt.

   • If you are upgrading from Clarity LIMS v4.x, some RPMs (eg, Server RPM) are now included under other RPMs.

   • Install any other required RPMs (eg, Python packages) which are not part of the Clarity LIMS setup.

   • Do not install NGS, Illumina Preset Protocols (IPP), and Sequencing RPMs during this step. You install these RPMs later in the installation process.

3. When prompted for user passwords, enter the passwords used in the previous instance.

4. Stop Clarity LIMS. To do so, on the command-line interface, run the following command as the root user:

   /opt/gls/clarity/bin/run_clarity.sh stop

Install and Configure LabLink 2.4

LabLink v2.4 is compatible with Clarity LIMS v6.2.

If upgrading from Clarity LIMS v4.x, Illumina migrates your LabLink-related data with the following exceptions:

• sample submission templates

• customized UI CSS

• lablink property table configurations

Before completing the following steps, make sure that a database named lablink is created with the same database user as Clarity LIMS database.

1. Install the LabLink RPM. Make sure that you have the correct repo enabled.

   • On the new instance, as the root user, run the following command:
2. Run the pending initialization script.

   • As the glsjboss user, run the following command:
3. The script prompts for a Google reCAPTCHA URL, site key, and secret key.

   • Google reCAPTCHA URL: https://www.google.com/recaptcha/

   • Google reCAPTCHA site key and secret key: View these keys from the Google reCAPTCHA Admin Console, under Settings.

     NOTE: If you prefer not to use reCAPTCHA, leave the site-key and secret-key fields blank when running the configuration script. LabLink does not display the reCAPTCHA when these fields are left blank. You can also use your own reCAPTCHA accounts when configuring LabLink.

   • To reconfigure LabLink (without initializing the database), run the following command as the glsjboss user:

     bash /opt/gls/clarity/config/configure_lablink.sh

Restore Backup Onto the New Instance

1. Extract the backup zip file into a suitable location, e.g. /tmp/restore.

   NOTE: If using Oracle database, skip the following steps 2 and 3. Contact the Clarity LIMS Support team for assistance in performing the migration from Oracle to PostgreSQL.

2. If using PostgreSQL database, the DBA imports the database dump into the new database instance. Dropping and recreating the database might be necessary. If you need to do this, use the following command:

   dropdb -U postgres <OL8DB> createdb -U postgres <OL8DB> psql -U postgres -d <OL8DB> -c 'ALTER DATABASE "<OL8DB>" OWNER TO "<OL8User>"'

3. Restore the database exported from the previous instance.

4. Extract and restore the following directories to the same directory on the new instance:

   • Clarity LIMS file store: /opt/gls/clarity/users/glsftp or /home/glsftp

   • /opt/gls/clarity/customextensions

   • /opt/gls/clarity/glscontents

   • /etc/httpd/conf.d

   NOTE: As of RedHat Enterprise Linux/Oracle Linux 8.8, Apache v2.4 is installed. There are several configuration changes in this version. You can use the new configuration, or merge your previous configuration file cautiously to the new configuration file. For details on the changes, refer to: httpd.apache.org.

5. Copy the SSL certificate files to the following location (create the directory if it does not exist):

   /etc/httpd/sslcertificate

6. To configure the certificates, run the following command on the command-line interface, as the root user:

   /opt/gls/clarity/config/installCertificates.sh
7. Restore files and configurations.

   • Copy any custom scripts into their folder locations.

   • For PostgreSQL database, copy / merge the database configuration file, i.e. pg_hba.conf and postgresql.conf.

   • Restore crontab from file.

   • Copy / Merge any additional application configurations.

[Optional] Change the Clarity LIMS Hostname

This step is required only if the new instance hostname is different from the old instance hostname.

[Optional] Update Application Configuration

This step is only required if the passwords for glsftp and / or apiuser have changed.

To update the application configuration, complete the following steps:

1. Update glsftp password.

2. Update apiuser password.

3. Update database connection details.

Perform Application Migration

In this step, the clarity-migrator tool is used to perform the changes required to make the database compatible with the new Clarity LIMS version.

On the command-line interface, run the following commands as the glsjboss user:

[Optional] Install NGS, Illumina Preset Protocols (IPP), and Sequencing RPMs

This step is required only if the RPMs have been installed on the existing Clarity LIMS instance.

• Install the latest NGS, IPP, and Sequencing RPMs compatible with the new LIMS version, as listed in clarityrpms.txt.

All existing workflows, protocols, steps, and master steps are restored during the restoration process. After installing the NGS and IPP RPMs, you do not need to install the workflows again.

• If the automation/External Program Plugin (EPP) scripts installed on the new instance are of a later version (e.g. /opt/gls/clarity/extensions/ngs-common/v5/EPP ), to your old instance (e.g. /opt/gls/clarity/extensions/ngs-common/v4/EPP), you must manually update the script location in your automation / EPP command lines. You can do update the script location in the Clarity LIMS interface or the Operations interface.

• If you intend to install NovaSeq API-based integration, NextSeq integration, or MiSeq integration, use the latest package to ensure OS compatibility.

Delete ElasticSearch Indices and Restart Clarity LIMS on the New Instance

On the command-line interface, run the following commands as the root user:

• Make sure that ElasticSearch is running:
• When ElasticSearch service is running, remove the ElasticSearch indexes:
• Restart Clarity LIMS:

Verify the New Instance

The following verification steps are the minimum required to confirm that the various services are up and running properly.

To conduct a thorough verification, perform your verification steps or daily routine on the new Clarity LIMS instance.

ElasticSearch, Search Indexer, and RabbitMQ

1. Log in to BaseSpace Clarity LIMS via https://<FQDN>/clarity and perform a basic search.

2. Check that search results are returned.

3. If no search results are returned, try again later as the search indexes are still building.

Tomcat, Apache, Automation Worker

1. Run a sample through a QC protocol step. Create a temporary workflow if necessary.

2. Make sure that the automation executes successfully and the log files are accessible.

3. Open a browser window and access https://<FQDN>/api/v2/projects.

4. Log in with api user credentials.

5. Check that all projects are returned in the response.

LabLink

1. Open a browser window and access https://<FQDN>/lablink/.

2. Log in with administrator credentials.

3. On the Projects page, make sure that all data are properly displayed.

[Optional] Sequencing Service

To make sure that the service is running properly, you must initiate an actual sequencing run on the instrument.

If necessary, the Clarity LIMS Support team can provide a backup of your Clarity LIMS data. The data are contained in an encrypted file, which can be downloaded from a secure SFTP server.

Prerequisites

To receive the backup data file, provide the Clarity LIMS Support team with a GNU Privacy Guard (GPG) public key.

For instructions on generating a GPG public key, see the following documentation:

• For Microsoft® Windows®, see .

• For Linux® or Mac®, see .

Receiving and Decrypting your Backup Data

1. After the Clarity LIMS Support team has received the GPG public key, they do the following actions:

   • Create a backup file encrypted with the key.

   • Place the backup file on the LIMS SFTP server at sftp.clarity-lims.com.

   • Provide a username and password so you can access your data. The backups are added to the FTP server weekly.

2. After downloading the backup file, there are several tools available to decrypt the data.

   • For Windows, use the gpg4win tool. For details, see the .

   • For Mac/Linux, use the GPG command on the command line. For details, see .

From Versions 4.2 / 4.3 / 5.0 / 5.1 / 5.2 / 6.0 / 6.1 (PostgreSQL) and 4.3 / 5.2 (Oracle) to 6.2

This section provides the steps required to upgrade an existing on-premise deployment of Clarity LIMS to a RedHat Enterprise Linux/Oracle Linux compatible on-premise deployment of Clarity LIMS v6.2.

The installation procedure includes provisioning and configuring the new instance, and installing and then verifying the new Clarity LIMS version.

For installation requirements, see .

If you have questions about the upgrade procedure, contact the Clarity LIMS Support team.

Migration Paths

The following table shows the applicable migration paths.

Prerequisites

Before Illumina can proceed with the upgrade, complete the following prerequisite steps.

Provision the New Instance

We recommend you provision an instance with similar or higher specifications to the current Clarity LIMS instance.

Note the following:

• All standard operating system (OS) security updates must have been applied.

• Upgrades are only supported from Clarity LIMS v4.2/5.0/5.1/6.0/6.1, and v4.3/5.2.0 (Oracle).

Configure the New Instance

• Custom configurations: If you have made any additional configurations that are not part of the Clarity LIMS pre-installation requirements, apply these configurations to the new instance.

• Passwords: Configure all passwords to be same as the existing instance. After you have verified the new instance, you can change passwords as needed.

Verify User Accounts Email Addresses

Make sure that all user accounts have email addresses associated with them. Users must reset their passwords after the upgrade is complete.

Install and Run the Pre-Validation Script on the Source Server

To assist with validating the system before an upgrade, install the UpgradePreValidation RPM on the source server.

• This RPM is installed temporarily, and provides tools to help check the system before an upgrade.

• If validation is successful, you can remove this RPM and proceed with the upgrade.

1. Install the UpgradePreValidation RPM. Make sure you have the correct repo enabled.

   • On the source server, as the root user, run the following command:
2. [Optional] Set up Secret Utility.

   • If ClarityLIMS-SecretUtil was installed previously, run the following command to set up Secret Utility as the glsjboss user:

     NOTE: Using a vault is the safer way of storing application secrets. If using a vault is not possible, the configuration script supports file-based storage.
3. Run the validation script as follows.

   1. Make sure that the Clarity LIMS server is running.

   2. As the root user, run the following command:
   3. Review the output of the script to determine if you can proceed with the upgrade. If the script outlines any issues with the potential upgrade, review the generated log files and contact the Clarity LIMS Support team for further assistance.

4. Remove the PreValidation RPM.

   Remove the PreValidation RPM only after you confirm that you can upgrade. If you are unsure, consult the Clarity LIMS LIMS Support team.

   • As the root user, run the following command:

Back Up the Current Database and Clarity LIMS

Archive the backup in case a rollback is required.

Before performing the backup, stop Clarity LIMS. The following command stops all Clarity LIMS components, including Automation Worker and integration services.

1. Stop Clarity LIMS:

   • On the command-line interface, run the following command as the root user:
2. Back up Postgresql database:

   On the PostgreSQL server, best practice recommends backing up the database using the pg_dump utility.

   The following example assumes the following:

   • The database server and the application server are on the same server.

   • The pg_dump utility is accessible to the glsjboss user.

     Example

     The Postgres DBA uses the following commands to create a database backup in the glsjboss home directory. Substitute the variables as appropriate for the specific environment.

     • As the glsjboss user:
3. Make sure that the following items, and any other files and configurations, are backed up safely:

   • crontab -l

   • custom scripts

   • OS configuration files

   • firewall rules

   • network configuration

   • etc.

   If there are custom changes to any application configurations (to increase performance, security, etc.), restore/configure these items manually later by referencing the backup.

   We recommend that you back up the items into a single zip file and transfer them to the new instance.

   Directories

   /opt/gls/clarity/users/glsftp or /home/glsftp (Clarity LIMS file store location)

   /opt/gls/clarity/customextensions

   /opt/gls/clarity/glscontents

   /etc/httpd/conf.d

   /etc/httpd/sslcertificate

   Files

   .pgpass

   /opt/gls/pgsql/9.x/pg_hba.conf

   /opt/gls/pgsql/9.x/postgresql.conf

   Additional configurations:

   rpm -qa | grep "BaseSpace\|Clarity" > clarityrpms.txt

Upgrade steps

Install Latest Clarity LIMS on New Instance

1. Make sure that the repository file exists in the following location:
2. As the root user, install the RPMs required on the new instance by referencing the content of clarityrpms.txt.

   • If you are upgrading from Clarity LIMS v4.x, some RPMs (eg, Server RPM) are now included under other RPMs.

   • Install any other required RPMs (eg, Python packages) which are not part of the Clarity LIMS setup.

   • Do not install NGS, Illumina Preset Protocols (IPP), and Sequencing RPMs during this step. You install these RPMs later in the installation process.

3. When prompted for user passwords, enter the passwords used in the previous instance.

4. Stop Clarity LIMS. To do so, on the command-line interface, run the following command as the root user:

   /opt/gls/clarity/bin/run_clarity.sh stop

Install and Configure LabLink 2.4

LabLink v2.4 is compatible with Clarity LIMS v6.2.

If upgrading from Clarity LIMS v4.x, Illumina migrates your LabLink-related data with the following exceptions:

• sample submission templates

• customized UI CSS

• lablink property table configurations

Before completing the following steps, make sure that a database named lablink is created with the same database user as Clarity LIMS database.

1. Install the LabLink RPM. Make sure that you have the correct repo enabled.

   • On the new instance, as the root user, run the following command:
2. Run the pending initialization script.

   • As the glsjboss user, run the following command:
3. The script prompts for a Google reCAPTCHA URL, site key, and secret key.

   • Google reCAPTCHA URL: https://www.google.com/recaptcha/

   • Google reCAPTCHA site key and secret key: View these keys from the Google reCAPTCHA Admin Console, under Settings.

     NOTE: If you prefer not to use reCAPTCHA, leave the site-key and secret-key fields blank when running the configuration script. LabLink does not display the reCAPTCHA when these fields are left blank. You can also use your own reCAPTCHA accounts when configuring LabLink.

   • To reconfigure LabLink (without initializing the database), run the following command as the glsjboss user:

     bash /opt/gls/clarity/config/configure_lablink.sh

Restore Backup Onto the New Instance

1. Extract the backup zip file into a suitable location, e.g. /tmp/restore.

   NOTE: If using Oracle database, skip the following steps 2 and 3. Contact the Clarity LIMS Support team for assistance in performing the migration from Oracle to PostgreSQL.

2. If using PostgreSQL database, the DBA imports the database dump into the new database instance. Dropping and recreating the database might be necessary. If you need to do this, use the following command:

   dropdb -U postgres <OL8DB> createdb -U postgres <OL8DB> psql -U postgres -d <OL8DB> -c 'ALTER DATABASE "<OL8DB>" OWNER TO "<OL8User>"'

3. Restore the database exported from the previous instance.

4. Extract and restore the following directories to the same directory on the new instance:

   • Clarity LIMS file store: /opt/gls/clarity/users/glsftp or /home/glsftp

   • /opt/gls/clarity/customextensions

   • /opt/gls/clarity/glscontents

   • /etc/httpd/conf.d

   NOTE: As of RedHat Enterprise Linux/Oracle Linux 8.8, Apache v2.4 is installed. There are several configuration changes in this version. You can use the new configuration, or merge your previous configuration file cautiously to the new configuration file. For details on the changes, refer to: httpd.apache.org.

5. Copy the SSL certificate files to the following location (create the directory if it does not exist):

   /etc/httpd/sslcertificate

6. To configure the certificates, run the following command on the command-line interface, as the root user:

   /opt/gls/clarity/config/installCertificates.sh
7. Restore files and configurations.

   • Copy any custom scripts into their folder locations.

   • For PostgreSQL database, copy / merge the database configuration file, i.e. pg_hba.conf and postgresql.conf.

   • Restore crontab from file.

   • Copy / Merge any additional application configurations.

[Optional] Change the Clarity LIMS Hostname

This step is required only if the new instance hostname is different from the old instance hostname.

[Optional] Update Application Configuration

This step is only required if the passwords for glsftp and / or apiuser have changed.

To update the application configuration, complete the following steps:

1. Update glsftp password.

2. Update apiuser password.

3. Update database connection details.

Perform Application Migration

In this step, the clarity-migrator tool is used to perform the changes required to make the database compatible with the new Clarity LIMS version.

On the command-line interface, run the following commands as the glsjboss user:

[Optional] Install NGS, Illumina Preset Protocols (IPP), and Sequencing RPMs

This step is required only if the RPMs have been installed on the existing Clarity LIMS instance.

• Install the latest NGS, IPP, and Sequencing RPMs compatible with the new LIMS version, as listed in clarityrpms.txt.

All existing workflows, protocols, steps, and master steps are restored during the restoration process. After installing the NGS and IPP RPMs, you do not need to install the workflows again.

• If the automation/External Program Plugin (EPP) scripts installed on the new instance are of a later version (e.g. /opt/gls/clarity/extensions/ngs-common/v5/EPP ), to your old instance (e.g. /opt/gls/clarity/extensions/ngs-common/v4/EPP), you must manually update the script location in your automation / EPP command lines. You can do update the script location in the Clarity LIMS interface or the Operations interface.

• If you intend to install NovaSeq API-based integration, NextSeq integration, or MiSeq integration, use the latest package to ensure OS compatibility.

Delete ElasticSearch Indices and Restart Clarity LIMS on the New Instance

On the command-line interface, run the following commands as the root user:

• Make sure that ElasticSearch is running:
• When ElasticSearch service is running, remove the ElasticSearch indexes:
• Restart Clarity LIMS:

Verify the New Instance

The following verification steps are the minimum required to confirm that the various services are up and running properly.

To conduct a thorough verification, perform your verification steps or daily routine on the new Clarity LIMS instance.

ElasticSearch, Search Indexer, and RabbitMQ

1. Log in to BaseSpace Clarity LIMS via https://<FQDN>/clarity and perform a basic search.

2. Check that search results are returned.

3. If no search results are returned, try again later as the search indexes are still building.

Tomcat, Apache, Automation Worker

1. Run a sample through a QC protocol step. Create a temporary workflow if necessary.

2. Make sure that the automation executes successfully and the log files are accessible.

3. Open a browser window and access https://<FQDN>/api/v2/projects.

4. Log in with api user credentials.

5. Check that all projects are returned in the response.

LabLink

1. Open a browser window and access https://<FQDN>/lablink/.

2. Log in with administrator credentials.

3. On the Projects page, make sure that all data are properly displayed.

[Optional] Sequencing Service

To make sure that the service is running properly, you must initiate an actual sequencing run on the instrument.

This section provides information and instructions to support Clarity LIMS and administration tasks. Topics covered include

If you require assistance with Clarity LIMS administration, contact the Illumina Support Team.

You can configure an alias for the short and long, singular, and plural forms of the term Project, as displayed in the Clarity LIMS interface.

Configuration Tool and Properties

Renaming is achieved by configuring the following properties, using the omxprops-ConfigTool.jar tool at:

Copy

/opt/gls/clarity/tools/propertytool 

Properties

• clarity.alias.project.short.singular - The short term for "Project"

• clarity.alias.project.short.plural - The short term for "Projects"

• clarity.alias.project.full.singular - The full term for "Project"

• clarity.alias.project.full.plural - The full term for "Projects"

Rules for alias

• clarity.alias.project.short.singular—Maximum of eight characters.

• clarity.alias.project.short.plural—Maximum of nine characters.

• If multiple words are used, capitalize each word (eg, Test Requests).

• Short form singular and plural aliases are truncated to eight characters and nine characters respectively. An ellipsis is used to indicate truncation.

Turning Container Name Uniqueness ON

This constraint is set at the database level. Container Uniqueness can be turned ON by using the migrator tool, and running the optional migration step called "ContainerNameUniqueConstraints". This can be done by calling this command:

java -jar migrator.jar ContainerNameUniqueConstraints

Notes:

• Of course, edit the migrator.properties file to ensure that the mode is set to "migrate" not just "validate".

• If the migrator has carried out its work successfully, the database should now have a new index present called 'unique_cnt_name'. (In psql, do \di to see indexes)

• You do not need to restart Tomcat services in order for this change to take effect

Turning Container Name Uniqueness OFF

Container Uniqueness can be turned OFF by running this SQL command:

Copy

drop index  unique_cnt_name;

This statement will fail if there are already any non-unique container names in the database. If it fails, you can find all non-unique names with this statement:

Copy

SELECT name,  COUNT(name) AS namecount FROM container GROUP BY name HAVING COUNT(name) >  1;

For each non-unique name found, you can iterate through all instances of it, and rename them so that they will be unique.

Copy

SELECT name, luid FROM container WHERE where name = 'non-unique_name';

Copy

UPDATE container SET name='non-unique_name-1' WHERE luid=luid_of_container;

Resolving Non-Unique Container Names

When you run the above query and determine there are too many containers to hand-edit, what now?

1. Are any of the containers that have non unique names in a depleted or discarded state? If so, you can probably delete them once the customer gives the all clear:

Copy

select name, luid, stateid from container where name in (SELECT name FROM container GROUP BY name HAVING COUNT(name) >  1) order by name;

now, what do the values for the container stateid mean? You won't find them in the database, they are hard coded as follows

Copy

1, "Empty"

Copy

2, "Populated"

Copy

3, "Depleted"

Copy

4, "Discarded"

Copy

5, "Reagent-Only"

Copy

6, "New"

Copy

7, "Hybridized"

Copy

8, "Scanned"

Copy

9, "Discarded"

Armed with this knowledge, we can refine the query to highlight containers that may be deleted with little consequence:

Copy

SELECT name, luid, stateid FROM container

Copy

WHERE name IN (SELECT name FROM container GROUP BY name HAVING COUNT(name) >  1)

Copy

AND stateid IN (1,4,9) ORDER BY name;

Notes:

1. Containers can only be deleted if they are empty

2. SQL 'IN' statements normally have a limit of 255 records, so if the the sub-select - the one in parenthesis - returns more than 255 records, your mileage may vary

Introduction

Clarity LIMS creates various log files to help with the resolution of issues. During support request investigation, the Support team may ask for the following types of log files:

• Automation Worker Log File

• Browser HTML and JavaScript Logging

Automation Worker log file

Automation Worker creates history and log files, and stores them on laboratory computers in the logs folder of the Automation Worker installation directory.

Windows

If Automation Worker is installed on a Windows machine using the program default, find the logs folder at the following location:

/opt/gls/clarity/automation_worker/node/logs

Linux

If Automation Worker is installed on a Linux server, find the \logs folder at the following location:

/opt/gls/clarity/automation_worker/node/logs

The following log files are available:

• wrapper.log - This log file outputs information on the starting, running, and stopping of the Automatic Informatics service.

• automatedinformatics.log - This log file outputs messages from installed plug-ins, such as automation commands and ADC directory scans.

To monitor the automatedinformatics.log:

• Log on to the server using the glsai user ID and run the following command:

/opt/gls/clarity/automation_worker/node/bin/automatedinformatics.wrapper.sh log 

If the Automation Worker is installed on a server other than the Clarity LIMS server, use the appropriate user credentials.

Browser HTML and JavaScript Logging

In the web browser, if the LIMS interface does not display items/elements correctly, provide the information and error messages to the Clarity LIMS Support team.

Instructions for finding error messages within the browser console are described in the following sections.

Google Chrome

To Start the Chrome Console:

1. Right-click on an element in the browser and select 'inspect element.'

   A sub window opens below the main window in Chrome, showing the source HTML.

2. Select the Console tab, and reload the troublesome page - any JavaScript errors will be reported there. Include these errors in the Support Request ticket.

NOTE: Between stages in a protocol step you may see errors of the following type:

GET http://qaclarity02.gls.lan:8080/clarity/api/work/310/epp?_dc=1375380986790 404 (Not Found)

Such messages are expected. This is the EPP trigger checking that there is no EPP transition to fire on the page change. (This can be annoying for debug purposes, but feel free to include these in the Support Request ticket.)

To Get the JavaScript version:

1. Open up the Console as described in the previous section.

2. Go to the Network tab.

3. Select 'scripts' from the options listed at the bottom of the tab.

   A script named isis-all.js?v=XXXXX displays.

4. Determine the version build number. (In the previous example, XXXXX represents the version build number).

FireFox

To Start the FireFox Console:

1. Right-click on an element in the browser and select 'inspect element.'

   A sub window opens below the main window in Firefox, showing the source HTML

2. Select the Console tab, and reload the troublesome page - any JavaScript errors will be reported there. Include these errors in the Support Request ticket.

NOTE: Between stages in a protocol step you may see errors of the following type:

GET http://qaclarity02.gls.lan:8080/clarity/api/work/310/epp?_dc=1375380986790 404 (Not Found) 

Such messages are expected. This is the EPP trigger checking that there is no EPP transition to fire on the page change. (This can be annoying for debug purposes, but feel free to include these in the Support Request ticket.)

To Get the JavaScript version:

1. Open up the Console as described in the previous section.

2. In the Filter options Search box and type 'isis'.

   A script named isis-all.js?v=XXXXX appears.

3. Hover over this script with your mouse to find the V (version) build number.

Using Log Files for Troubleshooting Purposes

If you are experiencing problems and need to submit a support request, use the following guidelines to determine which log files to send to the Clarity LIMS Support team:

• basespace-lims-*.log: Include if experiencing slowness in the application. (Default path: /opt/gls/clarity/tomcat/current/logs/)

• automatedinformatics.log: Include if you are experiencing problems with an integration or if a process using an EPP string does not work as expected. (Default path: /opt/gls/clarity/automation_worker/)

• wrapper.log: Include if the Automation Worker is unable to start (rarely needed).

• search-indexer.log: Include if there is issue with search feature. (Default path: /opt/gls/clarity/search-indexer/logs/)

• claritylims.log: Include if there is issue with search feature. (Default path: /var/log/elasticsearch/)

• Browser Console and LIMS JavaScript version: Include for any web interface display issues. A simple refresh of the browser page may resolve the issue. However, the Support team would prefer receiving the console log and JavaScript version to investigate and make product improvements.

If there is an API version mismatch, Config Slicer will log a message at the beginning of an import:

Copy

API version mismatch - package: v2,r20 - server: v2,r21. This may result in some differences between configuration in package and on server.

Generally, this message is not a cause for concern.

However, if there are warnings about configuration differences during the import, changes that have been made to the API in between the package version and the server version may be responsible.

In addition, if the package being imported is from v4.x, a log message may appear at the beginning of an import:

Detected package is from a pre-5.0 Clarity system. The package will be upgraded to match new configuration requirements.

In this case, the package is upgraded and the resulting configuration is output to a folder in the same location as the package being imported. This upgraded configuration package is the v5.x representation of the v4.x configuration and can be used directly to troubleshoot error occurring on import. The updated packaged can also be imported directly.

Config Slicer Version Updates

The following changes have been included in the latest release of Config Slicer (v 3.0.51):

General changes:

• The entity summary is now shown after the individual differences have written to the log file, as opposed to before these differences.

• Support was dropped for the following process type attributes which are no longer used:

  • SupportsExternalProgram,

  • ShowInExplorer,

  • ShowInButtonBar,

  • OpenPostProcess,

  • IconConstant

• Support was dropped for the show-in-tables property on Custom Fields (UDFs) because it is no longer used.

• There is now support for the new Clarity LIMS 5.0 distinction between Protocol Step names and Master Step (ProcessType) names.

• It is now possible to slice in and out all the new Master Step settings added in Clarity 5.0, including:

  • Default Process Template

  • Instrument Types

  • Container Types

  • Reagent Kits

  • Reagent Types

  • Control Types

  • Sample Fields

  • Queue Fields

  • Ice Bucket Fields

  • Step Fields

  • Step Properties

• When importing/validating a slice from 4.x into 5.x, if the package contains any Master Steps (process types) or Protocols, it is upgraded to be compatible with the new configuration available in 5.x. This process is automatic, and the updated package is written out to the directory containing the package being imported or the directory that log file is being written to. If errors occur while importing, this updated package can be manipulated directly and imported to fix them.

Note: If upgrading the package fails, import/validation will fail. In general, this will reflect a mistake in the configuration package being imported.

Changes in support of slicing from 4.x to 5.0:

The following changes were specifically added to support slicing between Clarity LIMS 4.x and BaseSpace Clarity LIMS 5.0. All of these changes will be saved into a new configuration package that will be written to the same location as the package being sliced:

• Backwards compatibility for Protocol Step setup configuration.

• Support for updates of parent entities after both the parent and child entities have been imported.

  • This change is required for updating the defaultProcessTemplate and step-fields step properties on Master Steps. Both require that the Master Step (ProcessType), ProcessTemplate, and Master Step Custom Fields (ProcessType UDFs) exist before they can be set on the Master Step.

• Support for setting the qcProtocolStep flag on Master Steps, allowing the correct Master Step type to be displayed in the Lab Work Configuration UI. The setting is propagated up from the Protocol Step to the Master Step.

After slicing, it is recommended to examine the configuration closely for any QC Protocol Steps that previously shared a Master Step with nonQC Protocol Steps. This setting may not transfer as expected and there is potential for misconfigured Protocol Steps. In particular, if the Master Step produces measurements, and even just one child step of a Master Step has qcProtocolStep=true, then the Master Step will get qcProtocolStep=true set on it. Thus, all other steps that use that Master Step have qcProtocolStep=true, whether or not it was set before.

• Support for setting the default container on Steps and Master Steps in such a way that all behaviour is maintained from 4.2

  • If every child step of a Master Step has the default container that was defined as a permitted container on the master (through the 'OutputContainerType' process-type-attribute) then the default will be added as a permitted container on the master step

  • If any child does not have the default container from the Master Step, then the default is removed from the Master and each child that had the container is updated so that the default is the first permitted container (and hence default)

• Extra containers and any step properties that are no longer valid on a step will be migrated to new properties or removed.

• Step-setup file configuration has been moved to the Master Step and it not possible to have a different set of step-setup files on a step than are specified on the Master. The master step owns the list of files. Both the search-result-file-index attribute on each file element and the message element for each file are defined by the Master and must be duplicated on every step. When moving a 4.x configuration with step-setup files to a 5.x configuration, the following events will occur:

  • The set of all the step-setup files found on all the child steps in the slice are added to to the Master Step and each child step.

  • If more than one step in the slice defines step-setup files for the same search-result-file-index, then all the messages for that search-result-file-index will be concatenated by newlines.

  • The enabled attribute will be set to true for the step-setup on all child steps.

  • The locked attribute will be set to false for the step-setup on all child steps.

  • In the case of importAndOverwrite, the step-setup of any existing child steps for an overwritten master step will be included.

• Upon validation after import, the following differences have been reconciled:

  • UDFs that differ by STYLE only

  • missing defaultProcessTemplate

  • missing attemptAutoPlacement

  • missing autoAttachFiles

  • missing qcWithPlacement

Saving passwords in encrypted format is recommended. Use the omxprops to do this action.

To encrypt a password:

Use the following command:

# java -jar /opt/gls/clarity/tools/propertytool/omxprops-ConfigTool.jar encryptPassword <password>

This command returns a text string resembling the following example:

RqHL5XpY0NStVRjd+BngRQ==  

To set an encrypted password:

Set the password by enclosing the text string in the ENC() wrapper.

Consider the following examples:

• When setting an encrypted password in a configuration file:\

  Copy

  rabbitmq.password=ENC(RqHL5XpY0NStVRjd+BngRQ==)

• When using the property tool to set an encrypted password:\

  Copy

  # java -jar /opt/gls/clarity/tools/propertytool/omxprops-ConfigTool.jar set -y ftp.password 'ENC(RqHL5XpY0NStVRjd+BngRQ==)'
   

• Using ENC() is not needed when setting the password in Automation Worker:\

  Copy

  informaticsClientTarget.sftpPassword=RqHL5XpY0NStVRjd+BngRQ==

Clarity LIMS provides Audit Trail, a robust data-tracking system that allows for tracking of the following:

• All user activity (ie, who did what, and when).

• Every action that is written to the database.

Audit Trail has two capture systems, Event Log and Detail Log.

• Event Log—Records familiar BaseSpace Clarity LIMS user actions and presents this information in a format that is easy to read and understand.

• Detail Log—Records exacting information about changes resulting from actions recorded in the Event Log. This includes both updated values and previous values.

Enabling Audit Trail may result in a small performance hit due to the overhead of writing the entries to the database. It is recommended that you periodically archive the Audit Trail database so that it does not become too large.

NOTE: Audit Trail is enabled by default.

Enforcing Unique Sample Names Within a Project

By default, Clarity LIMS allows duplicate sample names within the same project. If you would like to enforce sample name uniqueness within a project, you can do so.

Two scripts have been developed to support this requirement:

• SampleNamePerProjectUniqueConstraintStep: Apply this uniqueness constraint to enforce unique sample names within a project.

• CleanupDuplicatedSampleNamesPerProjectStep: Prior to running the uniqueness constraint, use this optional cleanup script to clean up a database that already contains duplicate sample names.

Both scripts are available via the clarity-migrator.jar tool.

Cleaning up the database

If your database contains projects in which duplicate sample names exist, run the CleanupDuplicatedSampleNamesPerProjectStep script to clean up sample names that would violate the sample uniqueness constraint property. The cleanup script searches the database for sample names that are not unique and renames them.

The cleanup script also renames the corresponding original submitted sample name - since there is a one-to-one correspondence between submitted sample and derived sample names in the LIMS interface.

To clean up the database:

1. As the glsjboss user, change to the clarity-migrator directory:\

   Copy

   Copy

   cd /opt/gls/clarity/tools/database/clarity-migrator

2. Run the clarity-migrator.jar tool, providing the name of the cleanup step as a parameter:\

   Copy

   Copy

   - java -jar clarity-migrator.jar CleanupDuplicatedSampleNamesPerProjectStep

3. The step will run and no validation errors should be reported.

Once cleanup has been performed successfully, you can apply the sample uniqueness constraint.

Enforcing sample uniqueness

After you have cleaned up the database (if this step was required), you can apply the uniqueness constraint.

Applying the sample uniqueness constraint results in a change at the LIMS database / schema level. Once you have applied this change, there is no script available to revert it.

If you need to remove the uniqueness constraint, you will need to submit a request to the Illumina Support team.

To enforce sample uniqueness:

1. As the glsjboss user, change to the clarity-migrator directory:\

   Copy

   Copy

   cd /opt/gls/clarity/tools/database/clarity-migrator

2. Run the clarity-migrator.jar tool, providing the name of the uniqueness constraint step as a parameter:\

   Copy

   Copy

   - java -jar clarity-migrator.jar SampleNamePerProjectUniqueConstraintStep

3. The step will run and no validation errors should be reported.

Results

After enforcing sample uniqueness, if a user attempts to accession or update sample names that already exist in the project - via the user interface or the API - an error message displays. The message describes the problem and advises the user to rename the duplicate samples.

The following sections describe and illustrate what happens if an accessioned or updated sample name conflicts with an existing sample name within the same project.

Postgres database

If an accessioned or updated sample name conflicts with an existing sample name within the same project:

Upload/modify via a sample sheet will result in the error shown below.

The sample with duplicate name is named within the parenthesis of the Detailed error message.

The quoted string in the Detailed error is the database name of the constraint being violated (uk_sample_name_per_project = unique key on sample table for name and project).

Sample management accession/modify will result in the error shown below.

If a user attempts to accession/modify a sample name under similar circumstances via an API operation, the results received would be similar to the content of this error message.

This section explains how to use the LDAP Checker tool a script (ldap-checker.jar) that checks and reports on an LDAP configuration. Instructions for use are also provided in the README.txt file that accompanies the tool.

The ldap-checker script is included with the Clarity LIMS installation and is available at the following location:

/opt/gls/clarity/tools/ldap-checker

The ldap-checker script performs numerous checks of the LDAP configuration and reports on any incorrect items found.

Script Properties

Point the script to one or more files containing (at a minimum) the database connection properties. Alternatively, set these properties from the command line.

The script loads properties from the following sources and in the following order:

1. Any JDBC properties files specified with -f (see the table for options).

   If multiple properties files specify the same property, the last file is used.

2. Any Java system properties specified on the command line using

   Copy

   -D<propertyName>=<propertyValue>

   Properties specified on the command line are only checked if they do not appear in the properties files.

3. The properties table in the database.

   The properties table is only checked if the same property is not already specified in the properties file or on the command line.

After the script has the basic database connection properties, it loads further settings from the corresponding Clarity LIMS database.

The following JDBC properties are required:

• jdbc.driverClassName

• jdbc.url

• jdbc.username

• jdbc.password

Options:

Run the Script

1. Change to the directory containing ldap-checker tool:

   Copy

   /opt/gls/clarity/tools/ldap-checker

2. Run the script. To specify a properties file, use the following example:

   Copy

   java -jar ldap-checker-<version>.jar -f database.properties 

Example Properties File

The tool includes an example database.properties file. This example shows a properties file that is specified with the -f option.

The following options are available:

• Edit this file and use it.

• Provide properties on the command line, using: -D.

  For example:

  Copy

  -Djdbc.url=<url to properties file> 

Examples

Example 1: Using SSL

Specify and provide the path to the keystore:

java -Djavax.net.ssl.trustStore=/path/to/keystore -jar ldap-checker-<version>.jar -f database.properties 

Example 2: Checking Users

To check a set of specific users (even those that have not been provisioned), use the following script:

java -jar ldap-checker-<version>.jar -u usertocheck -f database.properties 

Overriding Properties

To override properties that are typically loaded from the properties table, use command-line system properties or one or more properties files.

Using system property ( -D options must be specified before the -jar option):

java -Djavax.net.ssl.trustStore=/path/to/keystore -Dldap.managerPass=mypassword -jar ldap-checker-<version>.jar -u usertocheck -f database.properties

Using multiple properties files:

java -jar ldap-checker-<version>.jar -u usertocheck -f database.properties custom-ldap.properties

• In this example, Custom-ldap.properties might resemble the following:

ldap.managerDn=CN=GLS Admin,CN=Users,DC=gls,DC=lan

ldap.managerPass=mypassword

If you use, or would like to use, an LDAP server to consolidate directory services, it is possible to integrate LDAP with Clarity LIMS.

The Clarity LIMS LDAP solution allows for the following features:

• User name and password authentication against LDAP to govern access to Clarity LIMS.

• Ongoing unidirectional synchronization of user information (such as first name, last name, title, phone, fax, and email) from LDAP to Clarity LIMS. For example, if your telephone number is changed in the LDAP directory, the information is pushed down to Clarity LIMS, keeping contact information current.

• Automated unidirectional provisioning of user accounts from LDAP to Clarity LIMS. For example, adding a user to a particular group within the LDAP directory automatically results in a local account with LDAP authentication being added to Clarity LIMS.

Providing Information about your LDAP Implementation

Our Field Application Specialist (FAS) team meets with you to discuss the current LDAP implementation. In preparation for this meeting, collect the following information:

• The type of provisioning you would like to use to synchronize Clarity LIMS with LDAP (automatic or manual).

• A list of the LDAP attributes the current system uses to record the following user properties: first name, last name, title, phone number, fax number, and e‐mail address.

NOTE: When integrating Clarity LIMS with LDAP, the LIMS database and the LDAP directory remain as separate and distinct entities.

Supported LDAP Servers

Clarity LIMS is tested with the following LDAP servers:

• ApacheDS 1.5 and later

• Microsoft Active Directory (Windows Server 2003 or later)

• OpenLDAP 2.3.35 and later

Access and Changes

While user provisioning and authentication are handled with LDAP, a Clarity LIMS system administrator completes the following steps:

1. Determine the level of access that a user requires.

2. Modify the userʹs account within the LIMS to provide that access.

Once an LDAP integration with Clarity LIMS is established, all changes to user profiles must be made from the LDAP server.

Provisioning Users

Only automatic user provisioning is available.

With automatic user provisioning, Clarity LIMS users are created automatically by a provisioning tool that periodically synchronizes the LDAP server with the LIMS.

To make use of the LDAP directory services, Clarity LIMS maps to specific LDAP attributes within a defined schema.

However, the directory structure used can vary among installations. Our Field Applications Specialist (FAS) team work with you to complete the following items:

• Analyze a specific LDAP solution and directory organization or assist with the selection and initial configuration of an LDAP service.

• Discuss the user elements that will be synchronized between the LDAP service and Clarity LIMS systems.

• Configure LDAP to connect to your Clarity LIMS systems.

Caching User Authentication Results from your LDAP Server

User authentication is handled in the Clarity LIMS.

In previous versions of Clarity LIMS, a few customers reported slow response time for the REST API when using LDAP users for authentication. As of Clarity LIMS v5.2.x / v4.3.x, the REST API response time has improved by introducing a new feature that caches user authentication results through a new property (api.session.timeout).

To make use of the new feature, do the following actions:

• Make sure that api.session.timeout property is set.

• Include the HTTP Connection & Authorization request headers and session cookie in the HTTP request.

Setting the api.session.Timeout Property

Stored in the Clarity LIMS database table, the api.session.timeout property allows you to specify the period of time for which a user's session should persist, after they have been authenticated.

This property is set during installation or upgrade of the LIMS. The default value is 5 minutes. If necessary, update the value using the omxprops-ConfigTool.jar tool at the following location:

/opt/gls/clarity/tools/propertytool

For example:

$ java -jar /opt/gls/clarity/tools/propertytool/omxprops-ConfigTool.jar
set -y api.session.timeout '15'

For this configuration to take effect, stop and restart Tomcat:

service clarity_tomcat stop
service clarity_tomcat start

Including the HTTP Authorization Request Header and Session Cookie

To persist user authentication, the HTTP request must contain the following HTTP request headers:

• Request Header

  • Connection: Keep-Alive

  • Authorization: Basic <credentials>

The HTTP request headers are required for the initial request, and for any subsequent request to get a valid JSESSIONID. Additional scenarios are described in the following table.

To make sure that a valid authenticated session is provided if the cookie in the request has expired, also provide the following JSESSIONID cookie:

• Cookie

  • JSESSIONID=<a valid JSESSIONID from the initial request>

The following table lists the various combinations of HTTP Authorization request header and JSESSIONID cookie and their expected result. It assumes that the HTTP Connection request header is provided for all scenarios.

Create a Configuration Package File for Import to Another System (or for Backup Purposes)

Use a configuration package file to copy a configuration set from one server to another or to back up a particular working configuration at a particular time.

Required steps:

1. Create a configuration manifest file.

2. Export to configuration package file.

Import (install) a Configuration Package File on a Production or Test Server

This process involves copying a configuration set from one server to another, by importing a configuration package file.

For example, to move a configuration set to a different environment for testing or troubleshooting purposes, or copy a new configuration set (created and tested on one system) onto another system.

Required steps:

1. On the source server, create a configuration manifest file, and then export to configuration package file.

2. On the destination server, import the configuration package file.

Compare Differences between a Working Configuration and a Broken Configuration

There are two approaches:

• Comparing configuration manifest files provides a way to determine if there are processes or UDFs missing from a system. The information in the manifest files only allows comparing process and UDF names, not the specific way in which a process or UDF is configured.

• Comparing configuration package files helps check how specific processes are configured. If the systems being compared are meant to be identical, this method is more appropriate to use.

Required steps:

1. On each system, create a configuration manifest file, or a configuration package file.

2. Run a diff comparison on the two files.

3. Edit the broken manifest file, export it, and import the resulting configuration package file into the system to add the missing entities.

There are several tools that available to compare files:

• Meld (graphical), for Linux, port to MacOS

• Standard Unix diff (Linux, MacOS) (use -q for a quick check).

• FileMerge (OSX with XCode installed) - /Developer/Applications/Utilities/FileMerge.app

Merge the Configuration Sets from Multiple Test Systems to a Production System

Combine configuration sets from multiple systems, merge them into a single configuration package file, and then import the file into a new system.

Required steps:

1. On each source server, create and edit a configuration manifest file.

2. Merge the entities from all files into a single manifest file.

3. Export the resulting file to a configuration package file.

4. On the destination server, import the merged configuration package file.

Back up and Restore a Configuration Set

Copy a configuration set to restore it on another server and use it for testing/troubleshooting purposes.

Required steps:

1. On the server containing the 'broken' source system, create a full manifest file, containing all of the LIMS system configuration.

2. Export the manifest to a configuration package file. Save file to media/disk.

3. On the target server, import the configuration package file created on the source system.

Back up and Upgrade a Configuration Set

To upgrade or add to a configuration set already installed on a server, two configuration package files are needed: one to back up the working configuration set and one containing the new updated configuration that have been created on a test server.

Required steps:

1. On the server you want to upgrade, create a full manifest file and export this to a configuration package file. Save this file as a backup.

2. On the test server, create a manifest file and edit it so that it only includes the entities you want to import.

3. On the server you want to upgrade, import the configuration package file.

Deploy a New Configuration Set from Test to Production Server

You may want to take a configuration that has been created and tested on one system/site (referred to as the source system in the steps below), and deploy it on another system/site (destination system).

Take a configuration that has been created and tested on one system/site (referred to as the source system in the following steps) and deploy it on another system/site (destination system).

Required steps:

1. On the source system, create a configuration package file containing the tested configuration to import.

2. On the destination system, create a full manifest file and export this to a configuration package file. Save this file as a backup.

3. On the destination system, import the configuration package file that was created in step 1.

If running Config Slicer v3.0.x against a configuration package or manifest file that was generated with a pre-3.0 version of Config Slicer, an error message displays.

Scenario 1: Out-of-Date Configuration Package

In this scenario the error message will resemble the following:

Copy

SOLUTION:

Upgrade the configuration package file by running upgrade-config-slice.jar against it.

• This jar file should be located in the same directory as Config Slicer (/tools/config-slicer).

• The upgrade script will save an upgraded copy of the configuration package file (to the same directory), which can be inspected and imported.

Example:

Copy

To upgrade a manifest file at the same time, simply add it to the script as a second argument:

Copy

Scenario 2: Out-of-Date Manifest File with no Corresponding Configuration Package

In this scenario the error message resembles the following:

Copy

SOLUTION:

This upgrade process is a little more involved:

1. Extract the list of process types from the manifest file
.

2. Format them as parameters to Config Slicer's custom manifest generation for process types.

   For example, assuming we extracted the process types Process Type 1 and Process Type 2 in step 1, the following command would be used:

   Copy
3. Run the custom manifest generation.

4. Take the list of analyte UDFs and ResultFile UDTs from the manifest file that was generated and add them to the old manifest file
.

5. Add the following line of text to the top of the old manifest file:

   Copy

The Config Slicer tool is used to move small incremental configuration changes, contained in a configuration set, between Clarity LIMS systems. For example, it moves changes between a test system and a production system.

This configuration tool provides granular export/import functionality that allows the management of configurations that support experimental workflows.

Use this tool to back up, copy, deploy, and restore configuration sets. By making small incremental changes, make sure that the modifications made to the production system are minimal.

Review the following key concepts:

• Configuration set—This item may be created by the Illumina Support team or by the customer. It comprises the items (know as entities) that are added to a Clarity LIMS system to allow for customization for a particular scientific experiment or workflow. The Illumina NGS Extensions Package is a good example. See .

• Configuration manifest file—This text file determines the configuration set to be exported from a system. The manifest file does not contain the actual configuration data. It only drives the extraction of configuration from a system.

• Configuration package file—This XML file contains the top-level entities selected by the configuration manifest file, plus any related child entities. For example, for process types, it includes process type UDFs, process templates (protocols), and output UDFs.

Config Slicer tool Export/Import Process Overview

The Config Slicer tool uses an export/import process to transfer configuration sets from a source to a destination server. This process breaks down into the following tasks:

1. On the source Clarity LIMS server, use the Config Slicer tool to create a configuration manifest file.

2. Edit the manifest file so that only the required custom configuration set is preserved.

3. Use the Config Slicer tool to export the edited manifest file to a configuration package file.

4. On the destination Clarity LIMS server, use the Config Slicer tool to import the configuration package file into the system.

A configuration package file can be imported into multiple systems. Use this feature to create and import multiple custom configuration sets, such as the Illumina TruSeq integration. This functionality also provides the Illumina Support team with a scalable way to keep up with constantly changing protocols.

Supported Entities

A configuration set comprises the entities added to a Clarity LIMS system that allow for customization of the system for a particular scientific experiment or workflow. The following entities are currently supported by the Config Slicer tool:

• Sample UDFs and UDTs

• Container UDFs and UDTs

• Project UDFs and UDTs

• Artifact groups (experiments)

• Reagent types

• Control types

• Reagent kits

• Process types (any configured processes – for example, Pool Samples and Add Multiple Reagents)

  • Process UDFs and UDTs

  • Output UDFs

• Process templates - UDFs, UDTs, and parameter strings only (other entities such as instruments and researchers are not supported)

• Protocols

• Workflows

When performing custom manifest generation by workflow, protocol, or process type, the following entities are not exported: Sample, Container, Project UDTs, and UDFs for Project. Account (Lab) and Client (Researcher) UDFs are never exported by config slicer. These are known issues.

Config Slicer does not export/import nonstep automation, nor does it preserve the order of protocols.

Tools and requirements

When working with Config Slicer on the Clarity LIMS application server, there are no additional prerequisites. The latest version of the config-slicer.jar file is installed as part of Clarity LIMS on the Clarity LIMS server. In a default installation, find the file in the following location:

Copy

To work with Config Slicer on a machine other than the Clarity LIMS server, do the following:

1. Make sure that Java is installed.

2. Copy the /opt/gls/clarity/tools/config-slicer directory, and its contents, from the Clarity LIMS server to the machine. The config-slicer directory and the config-slicer package should contain the following:

   • config-slicer-<version>.jar

   • libs subdirectory (which includes all the libraries referenced by config-slicer-<version>.jar, including groovy-all-2.4.8.jar)

   • upgrade-config-slice.jar