In recent releases of Red Hat OpenShift Service Mesh, we have spoken about our goal of converging OpenShift Service Mesh with upstream Istio. While we have been taking incremental steps toward this in Service Mesh 2.3 and now 2.4, to accelerate this convergence, we will be releasing a version 3 of OpenShift Service Mesh in the coming year.

A new Operator for Istio on OpenShift

As a preview of this effort, we are pleased to announce that there is a new operator for Istio on Red Hat OpenShift for developer preview and early feedback. This new operator - temporarily called the “Sail Operator” (more on this below) will be the foundation for OpenShift Service Mesh 3.

This operator is significantly different from the current OpenShift Service Mesh operator in that:

• It has a single custom resource - “Istio” for configuring an Istio control plane(istiod).
• It uses Istio’s Helm artifact values to customize configuration.
• It only manages Istio, and does not manage complementary projects - such as Kiali, Prometheus, Grafana, Jaeger, or Elasticsearch.
  • Kiali, monitoring, tracing, and other integrations will continue to be supported by independent operators.
• It does not deploy the Maistra.io distribution of Istio that OpenShift Service Mesh 2 is based on, but rather a clean copy of upstream Istio that is rebased with Istio’s main branch about once per week.
• It does not include Maistra.io resources, such as ServiceMeshControlPlane, ServiceMeshMemberRoll, and ServiceMeshMember.
  • Migration tooling will be provided to convert these resources to Service Mesh 3 configuration formats. See the FAQs at the bottom for more details.

Our goal is to evolve the temporarily named “Sail operator” into a community “Istio operator,” contributing it to the Istio ecosystem GitHub organization as an alternative mechanism for installing and managing Istio on Kubernetes that directly uses Istio’s Helm values as its building blocks. While upstream Istio’s supported installation methods include Istioctl and Helm, we believe that an OLM-based operator can provide further enhancements such as automating in-place or canary upgrades of Istio, which are largely manual tasks with Istioctl or Helm.

In the meantime, as this operator is the last stage of a continuous delivery pipeline, it will allow OpenShift users to try out OpenShift Service Mesh 3 as it is developed.

It is important to note that this operator is incomplete and under active development - any and all aspects of it are subject to change. The name “Sail Operator” (“the operator” in the rest of this post) will change, but the description will continue to be a “community operator for Istio”. It should NOT be used for developing production applications, but rather experimentation and prototyping. By making it available as we develop it, we are hoping to gather early feedback from both our customers and the community. See the “Providing Feedback” section for details on how to give feedback.

From Maistra to Istio

OpenShift Service Mesh is based on the Maistra.io downstream project. The Maistra project at the time of Istio 1.0 and prior to OpenShift 4.0. At the time, Istio was still a young project outside of the Cloud Native Computing Foundation (CNCF). OpenShift’s user base required enterprise features - such as multi-tenancy, which were not a focus for the fast moving Istio project. Thus, the Maistra project was used to create a service mesh that, while based on upstream Istio, contained many notable differences developed with OpenShift’s pre 4.0 user base in mind.

A lot has changed during that time.

Istio is now a graduated project in the Cloud Native Computing Foundation (CNCF), reflecting that the project has passed 3rd party security and governance reviews, while obtaining broad industry adoption with a large multi-vendor community. The project has matured significantly and even has experimental features for implementing multi-tenancy. There are also exciting innovations under development - such as ambient mesh, which looks to take Istio into the world of “sidecar-less” service mesh.

Meanwhile, Red Hat has learned a lot about running Istio in large enterprise environments (See: Day 2 Operations E-Book, blogs). We have helped customers deploy Istio to clusters with hundreds of namespaces and thousands of services, and have learned many lessons along the way. While we have introduced features such as multi-cluster federation to our downstream Maistra distribution, we have also heard from many customers and prospects that they want us to support more of the latest Istio features.

We know Red Hat customers value support on open source standards, such as Istio, and maintaining a significantly different Maistra.io distribution of Istio has a significant cost that impacts our ability to bring new Istio features to our customers while taking time away from the Istio community.

Thus, to better serve our customers and the Istio community, the time has come to move away from our Maistra.io distribution of Istio and to deliver an OpenShift Service Mesh that is based directly on the Istio project.

In addition to supporting our customers on the latest Istio features, this will allow us to spend more time contributing to the Istio community, implementing customer RFEs, and supporting cross platform integrations such as cert-manager and Argo Rollouts.

While we are working to deliver an OpenShift Service Mesh 2.5 based on Maistra, we are planning for this to be the final release of OpenShift Service Mesh based on the Maistra.io project. With Service Mesh 3 based directly on Istio, we will make no further updates to the maistra.io project or website once all Service Mesh 2 releases have reached their end of life. For more details on this migration path and timelines, see the FAQ at the bottom.

OpenShift Service Mesh 3

Any major release means new challenges, new configurations, and new procedures to follow. In a word: change. While change is hard, it is also beneficial. Moving to Service Mesh 3 is no different. There will be a learning curve, and an adjustment period. It is our belief that giving customers early access to the new operator while it is under development will help ease the transition, and provide a taste of the benefits Service Mesh 3 will bring. These benefits include but are not limited to:

Convergence with Upstream Istio

The documentation covering the differences between OpenShift Service Mesh 2.x and Istio is lengthy. While there are good reasons for these differences, they also can result in confusion with users - particularly when learning about service mesh for the first time. Service Mesh 3 will remove many of these differences, which in turn will create an easier learning path from community Istio to OpenShift Service Mesh. For more information on specific features, see the FAQ at the bottom.

Up to Date Service Mesh releases

As Maistra contains many differences, rebasing between upstream Istio and Maistra is a time consuming task that can take months. By productizing community Istio, we will be able to release against the most recent version of Istio on a cadence that is more aligned with the OpenShift Container Platform. That cadence is roughly 3 times per year, so OpenShift customers will have access to the latest Istio features within weeks of a release rather than months later.

Common Configuration with Istio

The ServiceMeshControlPlane resource was created to provide guardrails for configuring Istio and related components such as Kiali, Prometheus, Jaeger and Elasticsearch. Unfortunately, maintaining this resource has proven to be unsustainable. This delays our support for new features, and causes friction with users as they must translate Istio configuration into ServiceMeshControlPlane configuration, which is only fully documented in GitHub. Frequently, newer Istio configuration is not yet supported, while the complementary component configuration is limited.

In comparison, the OpenShift Service Mesh 3 operator will use Istio’s community supported Helm configuration - with added validations, converging the user experience between Istio and OpenShift Service Mesh. Related components such as Kiali, monitoring and distributed tracing will continue to be supported as part of OpenShift, but will be configured using their separate operators.

While this will provide better support for our users, we also note that Istio’s configuration management has room for improvement. Using Istio’s configuration for OpenShift Service Mesh means that our efforts to improve this area going forward will be in collaboration with the Istio community for the benefit of all Istio users.

Greater support for upstream Istio features

The differences between OpenShift Service Mesh and Istio mean that supporting many upstream features requires a much larger effort to incorporate. By using upstream Istio configuration, we will be able to more easily support OpenShift customers on features like multi-cluster topologies and Istio on virtual machines outside of Kubernetes.

Easier migration from Community Istio

Common configuration and feature convergence will also enable customers to more easily move between community Istio and OpenShift Service Mesh. We realize that teams frequently develop and prototype with community projects, and we want to ensure that these can be seamlessly moved to the supported Service Mesh product. This also means that community Istio documentation will be much easier to use with OpenShift Service Mesh.

A Community Operator to Preview New Content

In addition to more up-to-date releases, the community operator will allow users to experiment with nightly builds of OpenShift Service Mesh 3.0 and beyond. While these will be unsupported, it will enable OpenShift users to easily experiment with new features before they are released by OpenShift Service Mesh or even Istio. We also believe that a community operator can provide a method for installing and managing Istio that enables more automation across upgrades and potentially other management features in the future. We plan to contribute this operator to the Istio ecosystem organization for community collaboration.

Modular Design with Integrations

OpenShift Service Mesh 2 aimed to offer a “batteries included” solution that provided everything needed for an end-to-end service mesh solution - including its own monitoring and tracing stack. The challenge with this approach is that Service Mesh has become a platform wide feature, rather than a single package. To get the most out of Service Mesh, it must be well integrated with the rest of your application platform - including networking, monitoring, GitOps, certificate management, multi-cluster management, and more. At the same time, the OpenShift platform is modular by design. Each user will install and configure a slightly different set of components to create their own custom application platform to suit their needs. Many customers also integrate with offerings from Red Hat’s broad ecosystem of partners.

Thus, our focus with Service Mesh 3 will be on seamless integrations rather than a tightly bundled architecture. While this will mean integrations with supported OpenShift operators, it will also make it easier for our users to integrate Service Mesh with 3rd party components that support community Istio integrations.

Improve Istio compatibility with OpenShift

As part of this strategy, we are contributing enhancements to upstream Istio to improve its compatibility with OpenShift. Istio currently provides a profile and documentation for running on OpenShift. Our aim is to reduce the number of special considerations for running Istio on OpenShift, potentially removing the need for this special profile. This will benefit OpenShift users by making it easier for them to use community Istio on OpenShift, removing any potential concerns around vendor lockin.

Envoy and OpenSSL for FIPS Compliance

One key difference that we will maintain in OpenShift Service Mesh 3 is our use of OpenSSL as an encryption library in Envoy rather than BoringSSL used upstream. Because OpenSSL is a cryptographic component that Red Hat submits for FIPS testing for validation by NIST’s Cryptographic Module Validation Program, it allows us to have full control of the validation process and provide transparent information on our FIPS compliance status for each release, which is valued by our public sector customers. We are, however, in the process of creating an OpenSSL/BoringSSL compatibility library to substantially reduce the effort of maintaining Envoy with OpenSSL.

Getting Started with the new Operator

The new operator is available today on OpenShift Container Platform in Operator Hub with the temporary name “Sail Operator”. Please review the instructions in the README before installing and configuring the operator. Note that there are still some workarounds for using Istio on OpenShift, though we aim to remove these through upstream contributions.

Once you have validated that istiod is running, you can test drive your new Istio using the Bookinfo sample application. Unlike with OpenShift Service Mesh, no modifications are necessary for Bookinfo to run with this new operator.

Providing Feedback

While this new operator for Istio is experimental and will evolve significantly over the coming weeks and months, we appreciate any and all feedback on this new direction. Community members can provide feedback by creating issues in the maistra/istio-operator repository. OpenShift customers should raise questions and feedback through their account teams - we would be delighted to meet with customers to discuss OpenShift Service Mesh and our plans for version 3.

Note that as a developer preview feature, we are not accepting support cases for this operator at this time.

What Comes Next?

This new operator for Istio will evolve significantly over the coming weeks and months. We will continue to make contributions to upstream Istio to improve the OpenShift experience while creating a productization pipeline and documentation for a Technology Preview release.

While our timelines are subject to change, we are aiming for a technology preview release in late Q4 of this year, with general availability in the first half of 2024.

Frequently Asked Questions

What will the upgrade path from 2.y to 3.y look like?

While this has yet to be finalized, an upgrade from Service Mesh 2 to 3 will at minimum require a conversion of configuration between OpenShift Service Mesh 2 and 3 formats. To minimize this effort, we are planning to provide tooling to help with the conversion of such configuration and to validate the Service Mesh 3 configuration before it is applied to a cluster.

That said, as with most major releases, a rolling “in place” upgrade of services will likely not be possible. It will likely be necessary to create a new Istio control plane and data plane with the converted configuration and newly added applications, while migrating traffic from the existing set of applications to the new ones using an outside load balancer. This would be a similar upgrade as Service Mesh 1.x to 2.0.

While it is still too early to provide in depth migration guidance, we have a substantial Service Mesh 2 user base running mission critical applications, so we will make every effort to provide as smooth of an upgrade path as possible. We will provide more details as we move closer to general availability.

What will happen to Maistra.io (OpenShift Service Mesh) specific features in 3 such as multi-tenancy and federation?

While we will provide a complete list of feature mappings between OpenShift Service Mesh 2 and 3.0 upon general availability, the two features that we are most focused on migrating are multi-tenancy and federation.

OpenShift Service Mesh’s multi-tenancy feature allows multiple service meshes to be created within a single cluster, such that they can be maintained in relative isolation. This is widely used by customers with large clusters that are shared across multiple teams. In Service Mesh 3, we plan to continue supporting customers on multi-tenant use cases using evolving Istio features such as multiple control planes per cluster combined with existing isolation and control features. We have also seen that multi-tenancy use cases can sometimes be managed within a single mesh, using features like Sidecar and the exportTo setting. Service Mesh 3 will document multiple approaches to multi-tenancy.

OpenShift Service Mesh’s federation provides a “zero-trust” or “need to know” approach to connecting service meshes between clusters. This can be particularly useful for sharing services between meshes or enabling high availability across meshes. While upstream Istio offers multiple multi-cluster topologies (which we plan to add support for in Service Mesh 3), we have heard from customers that they value the isolation provided by federation. Thus, we will look at ways to support this use case - through upstream contributions, by separating the functionality into a separate project independent of Istio or a combination of both. In the scope of multi-cluster management, we will also look at how to provide a better experience with Red Hat Advanced Cluster Management and service mesh.

Other features will either be contributed to upstream Istio or deprecated in 2.5 and dropped in 3.0. This has already started, with OpenShift Routing (IOR) and Grafana having been deprecated in 2.4.

Will Istio Ambient Mesh be part of OpenShift Service Mesh 3.0?

Yes, though our support level at the time of general availability will depend on the maturity of Ambient mesh in upstream Istio. It is still evolving rapidly, and we will be working with the community to make it work with OpenShift. We encourage users to watch the Istio community for updates on Ambient Mesh. We will put out a separate blog post when Istio Ambient Mesh is ready for preview on OpenShift.

What new features will be in OpenShift Service Mesh 3 that will not be in 2?

While OpenShift Service Mesh 2 is based on the Istio project, and provides support for most Istio APIs, it does not provide support for some notable features that we are aiming to provide support for in Service Mesh 3. The most notable of these features include istioctl, multi-cluster topologies such as multi-primary, primary-remote and external control planes as well as external services. Our aim is to bring support for these in future OpenShift Service Mesh 3 releases.

There will of course be some limitations to Red Hat’s support. Features classed as experimental or Alpha by the community will often be classed as Developer or Technology Preview. Some APIs, like EnvoyFilter API will likely continue to be unsupported (except where documented) due to the nature of the API for exposing Envoy internals. We will still need to set some boundaries based on our ability to provide customer support SLAs. Those limitations will be substantially less than they were with OpenShift Service Mesh 2.

OpenShift Service Mesh 3’s release notes will include a feature matrix documenting our support level across Istio features for each supported release, similar to the community Istio feature status page.

When will OpenShift Service Mesh 3.0 be generally available?

We are targeting a Technology Preview release in late 2023, with general availability in the first half of 2024. This will depend on our development velocity, testing, and feedback as we progress through these stages of development.

When will support end for OpenShift Service Mesh 2?

As with all OpenShift Service Mesh releases, the end of life date of the final Service Mesh 2 release will be dependent on the general availability date of Service Mesh 3.0 and the next two minor releases (e.g. 3.1, 3.2). Note that the OpenShift Life Cycle Policy page may not yet reflect the planned 3.0 major release - this will be updated closer to general availability. Based on this policy, if Service Mesh 3.0 becomes generally available as described above, OpenShift Service Mesh 2.5 would end its maintenance support phase with the release of OpenShift Service Mesh 3.2. This will ensure that customers have ample time to plan a migration to the new major release - with a minimum of 8 months of overlapping support.

To learn more about Red Hat OpenShift Service Mesh, visit:

https://www.redhat.com/en/technologies/cloud-computing/openshift/what-is-openshift-service-mesh