During the Wavestone OT Cyber Day, Loïc Lebain and Benoit Bouffard conducted a workshop in which they noted that IT departments were still struggling to develop a catalogue of cybersecurity solutions for OT. Based on their experience with our customers, they presented an overview of the difficulties with cybersecurity in OT today, and then, presented a method to enable the industrialization of IT services for OT. The stated goal is to set up a Solution Factory to be able to develop a catalogue of ready-to-use cybersecurity products adapted to the industrial world.

 

The Current Perception of Cybersecurity in OT

In industrial environments, cybersecurity does not always have a positive image and is seen as a potential obstacle to business development. Cybersecurity teams are often criticised for defining rules but delegating their implementation without providing a solution or any help for the implementation of requested changes. For example, it is difficult to regularly change the passwords of dozens of generic industrial accounts, even though this rule is standard on a traditional IT perimeter. As a result, OT teams are often left alone to meet the criteria for security policy requirements.

Left alone, industrial operational teams develop “homemade” solutions designed with their very local point of view, at the scale of their site. These solutions are beyond the group’s control and are very specific (dependence on a local supplier, in-house solution designed for the site’s specific network architecture, etc.), and scalability capabilities are not evaluated. All these solutions are developed by expert and passionate teams who can question security practices and standards, but who rarely have in mind any strategic vision, even at the local scale, making the integration of their solutions at the scale of a group of industrial sites nearly impossible.

Short-term solutions… or even dangerous

In the long run, these local solutions have many disadvantages:

• They are not up to production standards and remain in the POC phase.
• They are poorly documented, which makes maintenance difficult.
• Scaling up to a group of industrial sites is nearly impossible in the long term.

As shown below, some of the “homemade” solutions encountered have even proven to be dangerous:

Real-life examples taken from the 2020-2023 industrial sites audits

Standardise cybersecurity integration in operations

Industrial companies stand out by their strong needs of availability and the substantial real-world implications of the operations. Consequently, investments in this sector must align with the magnitude of these challenges which require cybersecurity solutions of very large scale and complexity. IT, cybersecurity, and OT departments must cooperate throughout the development process to ensure that solutions are suitable for operations while meeting the group’s security standards. The goal is to industrialise the development of cybersecurity solutions for the OT perimeter, providing ready-to-use solutions ready to be deployed at scale.

The solution is the development of a catalogue of cybersecurity services in which services are selected and developed at the group level, in collaboration with all the players (Cyber, operations, IT) and integrating the management of the entire life cycle of the solution (maintenance, documentation, decommissioning, etc.). Thus, the cybersecurity department and the IT department can create, with the industrial department, a product management roadmap, with an industrialized process for the creation of solutions.

 

Designing an OT Cybersecurity Solution

The process of creating a solution must address several issues:

• Collect the needs of all stakeholders.
• Transcribing needs
• Ensuring Large-Scale Adoption by all industrial sites.

To ensure the efficiency of the process and the solutions, the development of the different solutions is necessarily long and can extend over a period of 2 to 3 years. Wanting to go faster means exposing oneself to poor coverage of operational needs, which could lead to the development of uncontrolled local solutions or poorly controlled and incomplete deployment.

Providing security solutions: a 6-step process

 Solution Factory Process

1.     Research & Development

The goal of the R&D phase is to find the best solution to meet all cybersecurity needs. Thus, in the event of an audit of the central office, compliance with security policies is guaranteed if the tool is used. During R&D, a few points are crucial:

• Assemble a project team with representatives from IT, cybersecurity as well as the operations, to guarantee the usefulness and usability of the solution.
• Define operational constraints at the right level (availability, resistance in a harsh environment, support, etc.) in order to control costs without compromising the usability of the product.
• Plan maintenance, update and release processes as early as R&D to avoid getting stuck with an imperfect or obsolete product.
• Plan the budget and business model of the product. In particular, who has to pay and what are the operating and investment costs. This helps prevent the project from getting stuck at the deployment step due to budget issues.

During the R&D phase, it is also interesting to start from what already exists. This makes it possible to identify talents or solutions that could be adapted at scale and across an OT perimeter. There are two possible approaches to finding solutions:

• Find solutions that OT teams use locally and scale them up.
• Search for cybersecurity solutions from the IT for IT catalogue and adapt them to the industrial world.

Two methods to take into account the existing situation

2.     Prototype

It is essential to think about the user experience and to take care of the image of the product from the prototype. The prototype is first and foremost a showcase that should facilitate the adoption of the product, but which can also damage its image if it is not practical and functional. When presenting the prototype, it is important to frame the use cases covered, and to have a functional and simple product. The first image of the prototype is the one that the operational staff will remember.

3.     Minimum Viable Product

The MVP phase has two main challenges: to test the product, and to bring together promoters. Communication around the MVP must be neat, and everything must be done to avoid failures. When testing, you should not only test the solution itself, but also all the support functions and the integration with the rest of the production environment.

Because the MVP can be a Single Point of Failure for production, it is also necessary to take into account the needs of high availability and set up bypass mechanisms in case of problems to reassure operational team and facilitate the integration. A MVP can severely damage a product’s reputation in the long run if it fails.

4.     Packaging

The packaging stage allows you to define all the prerequisites for the deployment of the product. It is necessary to define:

• Processes throughout the life cycle such as the management of deployment requests, defining the obligation or not to deploy, maintenance processes, update processes considering operational needs, etc.
• Define responsibilities, but considering that industrial sites must maintain a stronger independence than what is usually done on IT perimeters. There needs to be a clear definition of what is delegated to on-site managers in nominal mode and in the event of an emergency.
• The cost model, including long-term cost, must be clearly defined and compared to external solutions.
• Support should be considered as Support as a Service and all processes and tools should be set up and communicated.

5.     Preparing for maintenance

The last step before the actual deployment is the preparation for operational maintenance. For each product, a Solution Owner must be identified to manage the relationships between users, suppliers and – during the integration – the integrator. This person should be identified internally prior to deployment to ensure that maintenance is operational throughout the lifecycle without having to rely on an external.

Prior to deployment, there are three things that need to be taken care of to prepare for the product lifecycle and promote its widespread adoption:

6.     Deployment

During the deployment of the product, early adopters must be supported as much as possible to maximize the chances of adoption of the project by other sites. Financial incentives, such as discounts for early adopters, can also be put in place. Different scenarios of speed of adoption must be anticipated in order to be able to deploy quickly enough in case of great success, but without cost issues in case of adoption difficulties.

 

Conclusion

In an industrial environment, cybersecurity is still seen as too restrictive, an obstacle to productivity, and too prescriptive. IT departments set up security policies but do not provide solutions to comply with them, which leads to the development of poorly controlled local solutions. To control these risks, one solution is the development of an IT solution catalogue for OT. The development of these solutions is a lengthy process that can take several years, especially when several projects are launched in parallel. To maximize the chances of success, the operational needs must be considered from the R&D phase up until deployment. Integration with operational processes, support processes, and all budget issues must be considered. Finally, the final key to the success of the solution development process is communication. The image of the product must be carefully maintained and controlled to maximize adoption by industrial sites after the start of deployment.