In today’s economy, the most successful businesses have figured out how to use data to drive efficiency, increase profitability, and develop strategic competitive advantage. But oftentimes, the tsunami of data organizations have at their disposal makes it challenging to harness the true power it can deliver to them. Having data available in a consumable format when and where it is needed can mean the difference between market leadership or missed opportunities. With Red Hat Data Services based upon Red Hat OpenShift Container Storage 4.5 and Red Hat Ceph Storage, organizations can more efficiently manage their data at rest, in motion, and in action to deliver meaningful business insights.
The key features of OpenShift Container Storage 4.5 are:
- External Mode support for shared storage with Red Hat Ceph Storage
- Support for virtual machines and applications running on OpenShift
- Object bucket claims backed with Red Hat Ceph Storage
- Support for disconnected/air-gapped and proxy installations
- Dynamic persistent volume expansion (in Technical Preview)
- Support for Microsoft Azure and Google Cloud Platform (in Technical Preview)
Increasingly, organizations are turning to cloud-native technologies for their applications and data. Red Hat OpenShift is a Kubernetes-based cloud-native platform that enables application agility, scale, and efficiency. Red Hat OpenShift Container Storage dramatically enhances OpenShift in all of these dimensions, enabling fast and reliable access to large amounts of diverse data in hybrid cloud environments. Together, they deliver an integrated experience for a wide range of users, including application developer and DevOps teams, data science and data engineering teams, and cloud infrastructure teams.
Red Hat OpenShift Container Storage is the only container-native data storage solution that is entirely integrated into the OpenShift console, enabling operators to view everything they need in one place. OpenShift Container Storage leads the industry in providing broad support for all types of storage, including file, block, and object. This enables application developers to more consistently and easily handle a wider range of workload types, from structured databases and semi-structured data warehouses to unstructured data lakes. Because OpenShift Container Storage is built for OpenShift, it delivers a consistent experience regardless whether you are developing and deploying on-prem in virtual machines or bare metalor in the private or public clouds.
Expanded Technical Proof Points for Real-World Application and Data Workloads
As more and more organizations accumulate data from sources ranging from device edge to offsite and on-premises data centers as well as public and private clouds, a mounting challenge is in how to ingest and prepare all of that data in a continuous stream so that it is useful and timely. Red Hat OpenShift Container Storage helps organizations prepare, organize, and manage their data whether it is at rest, on the move, or in action, providing valuable insights to their businesses.
OpenShift Container Storage helps organizations make data at rest more accessible, while delivering scalability and resiliency in a modern cloud-native platform. By enabling deterministic performance for data workloads such as archives, data lakes, and databases like PostgreSQL, Data Warehouses, and IBM Db2 Warehouse, OpenShift Container Storage helps customers who are building cloud-native applications deliver a consistent experience with improved agility and operational efficiency. According to a director of Db2 Development for IBM Data and AI:
"With no prior experience with OpenShift Container Storage, our team was able to set up two distinct OpenShift clusters and conduct full Db2 Warehouse Performance validation in less than two weeks. This, by far, exceeded our expectations, as tests involving other Kubernetes storage services proved to be much more difficult to configure and validate.
Additionally, OpenShift Container Storage has outperformed these other cloud-native storage solutions in all tested scenarios when using identical hardware. We have been delighted with the ease of use and outstanding performance of OpenShift Container Storage, prompting us to consider it our preferred data platform for running Db2 Warehouse on OpenShift"
- Piotr Mierzejewski, Director Db2 Development, IBM Data & AI
Increasingly, object storage is becoming a preferred data lake platform because of its ease of use via RESTful APIs and its massive scalability. Red Hat Ceph Storage, which is the data plane for OpenShift Container Storage, delivers on these key requirements. As proof of Red Hat Ceph’s industry-leading scalability, Evaluator Group executed an independent benchmark, which demonstrated deterministic performance while ingesting 10 billion objects:.
“The size and scope of new applications driving the digital transformation requires IT to rethink their data architecture. Evaluator Group set up a Red Hat Ceph 4.1 cluster, creating 10 Billion objects while measuring the systems scale and performance. The system performed very well, supporting high object rates for small objects and high throughput for large objects with predictable, nearly linear performance from zero to 10 Billion objects.”
- Russ Fellows, Senior Analyst, Evaluator Group.
Likewise, the future of data services requires increased automation capabilities across the entire IT infrastructure. Creating a data pipeline that can ingest, prepare, and manage data from its inception at the device edge, for example, and automating its journey throughout the workflow, and automatically preparing and potentially even transforming this data and sharing it securely with other entities help make the constant onslaught of data manageable. By leveraging capabilities in Red Hat OpenShift, Red Hat Middleware, and Red Hat Data Services, organizations are able to create scalable, event-driven architectures while ensuring data quality and reducing bottlenecks.
An example showing pneumonia detection automation enabled by object bucket claims, Kafka topics, and OpenShift Serverless with AI inference and image transformation is one way businesses, researchers, and public sector organizations can accelerate and automate data pipelines. By enabling new business models and moving data from batch to real-time processing, OpenShift Container Storage enables automated processes for data on the move, helping organizations access and find the right data quickly.
At the same time, OpenShift Container Storage makes data more actionable for application developers and data scientists. As many development teams move to continuous integration and continuous deployment (CI/CD) models, tools such as Jenkins and Tekton help to streamline data pipelines. With many developers simultaneously committing code, large assembly dependencies can occur, affecting build times. By providing persistent storage to these pipeline processes and their requirements, OpenShift Container Storage can greatly reduce build and test time dependencies for concurrent builds.
For data analytics, artificial intelligence, and machine learning, data scientists need reliable access to data regardless of the data type. With OpenShift Container Storage, whether the data is in a file repository, in block storage in a database, or in a data lake, data scientists can use industry leading tools, such as Jupyter notebooks, Tensorflow, and Pytorch, to build machine-learning models. They can also accelerate time-to-business results by federating queries with Starburst Presto or Apache Spark, and they can enable true insights with engines like IBM Watson, Microsoft SQL Server Big Data Clusters, or SAP Data Intelligence.
Support for Virtualized Applications Running on OpenShift
As organizations continue on their journey toward a cloud-native future, it is important to maintain support for virtualized applications today. Together with OpenShift Virtualization, OpenShift Container Storage enables deployment of virtualized applications alongside containerized applications on a common infrastructure. OpenShift Container Storage also enables migration of virtual machines between worker nodes by providing shared block storage for high reliability and performance.
External Mode Support for Shared Storage With Red Hat Ceph Storage
With the 4.5 release, Red Hat is further expanding the significance of Ceph Storage by enabling customers to tap into Ceph’s massive scalability for applications requiring large amounts of data. With OpenShift Container Storage External Mode, customers can easily set up and operate a stand-alone storage cluster that simultaneously supports block, file, and object access by one or more OpenShift clusters. This feature also enables centralized OpenShift storage administration.
Object Bucket Claim Support Backed With Red Hat Ceph Storage
With OpenShift Container Storage 4.5, users now also have the ability to issue object bucket claims against Ceph object storage clusters. This capability extends direct access to S3 bucket APIs and enables many of the features and scalability native to Ceph. With this new feature, object bucket notifications, for example, can now be enabled through automated data pipelines in OpenShift.
Support for Disconnected/Air-gapped and Proxy Installations
Security continues to be a key concern for cloud-native applications. OpenShift Container Storage 4.5 introduces two new features to support environments that require higher levels of security. Support for air-gapped clusters is now provided, which means customers can physically isolate their OpenShift Container Storage clusters from public networks. This release also supports proxy-mode cluster access, where system-wide outbound traffic can be routed through a proxy service.
Dynamic Persistent Volume Expansion (in Technical Preview)
With dynamic persistent volume expansion, users are able to grow their storage volumes to dynamically meet their changing needs. This prevents the need for volume migration or application downtime when additional capacity is needed.
Support for Microsoft Azure and Google Cloud Platform (in Technical Preview)
OpenShift Container Storage now also runs on Microsoft Azure and Google Cloud Platform, in addition to AWS, as well as bare metal and virtualized on-premises platforms. With added support for Microsoft Azure and Google Cloud Platform, users can now leverage all the benefits of Microsoft Azure as part of their hybrid cloud strategy.