Retrofitting BYO Firewall Rules on OSD-GCP

OSD-GCP clusters are provisioned with platform-managed firewall rules that use GCP network tags to target instances. The BYO (Bring Your Own) firewall feature replaces these with rules that target WIF service accounts instead. This gives customers full ownership of their firewall rules while maintaining the same network security posture.

This guide walks through retrofitting an existing OSD-GCP cluster from tag-targeted rules to SA-targeted BYO rules, disabling CCM firewall management, and validating Day-2 operations.

Overview

The retrofit procedure:

1. Create SA-targeted BYO rules alongside the existing tag-targeted rules
2. Verify coexistence (both rule sets active, no conflicts)
3. Delete the platform-managed tag-targeted rules
4. Disable CCM firewall management so the Cloud Controller Manager does not recreate rules
5. (Optional) Validate Day-2 operations (scale-up, upgrades) with BYO rules only

Prerequisites

• An existing OSD-GCP cluster deployed with WIF (Workload Identity Federation)
• gcloud CLI with permissions to manage firewall rules in the GCP project
• oc CLI authenticated to the cluster with cluster-admin privileges
• ocm CLI logged in to OCM

Step 1: Set Environment Variables

Only the cluster name and GCP project need to be set manually. All other values are derived dynamically.

Derive the infrastructure ID from OCM:

Derive the VPC network and BYO rule prefix:

Derive the WIF service accounts from the cluster’s compute instances:

Set the GCP health check probe IP ranges. These are Google’s global health check source ranges , used by all GCP load balancers. They do not vary by project or region.

Verify all variables are set:

Step 2: Record the Existing Platform-Managed Firewall Rules

Before making any changes, capture a snapshot of the current firewall rules:

A standard OSD-GCP cluster will have 6 platform-managed rules, all using network tags as their targeting mechanism:

Step 3: Establish a Baseline (Optional but Recommended)

Deploy a test workload to verify networking before, during, and after the migration:

Create a LoadBalancer service to test CCM behavior:

Wait for the LoadBalancer to be assigned an external IP (this may take up to 90 seconds):

Verify all networking paths:

All checks should pass: API healthy, ingress returning Hello OpenShift!.

Step 4: Create BYO Firewall Rules (SA-Targeted)

Create 8 SA-targeted rules that mirror the platform-managed rules plus cover ingress. These rules coexist safely with the existing tag-targeted rules.

BYO Rules Summary

Verify BYO Rules

Confirm all 8 rules were created:

At this point both rule sets (tag-based and SA-based) are active simultaneously. Verify the cluster is still healthy:

Step 5: Delete Platform-Managed Firewall Rules

With BYO rules in place, remove the original tag-targeted rules:

Leave CCM-managed k8s-* rules (for existing LoadBalancer services) in place for now.

Immediately verify the cluster:

If the baseline workload was deployed:

The cluster should continue operating with zero disruption: API responding, all nodes Ready, all operators healthy, and networking paths functional.

Step 6: Disable CCM Firewall Management

With BYO firewall rules, the customer owns all firewall rules. The Cloud Controller Manager (CCM) must be told not to create or delete firewall rules when LoadBalancer services are created or deleted. CCM should still manage the load balancer resources (forwarding rules, target pools); only its firewall operations should be disabled.

Understanding the ConfigMap Sync Chain

The CCM reads its cloud config from openshift-cloud-controller-manager/cloud-conf, but this ConfigMap is managed by a sync controller. Patching it directly will be reverted within seconds.

The correct approach is to patch the source ConfigMap. Changes propagate automatically through a three-stage sync chain:

Apply the Flag

Read the current cloud config, set firewall-rules-management to Disabled, and patch it back:

Verify Propagation

Wait approximately 30 seconds for the sync controllers, then verify the flag propagated to all three ConfigMaps:

All three should show: firewall-rules-management = Disabled

Verify CCM Behavior

Restart the CCM pods to pick up the new config:

Check the CCM logs for the Disabled flag:

When a LoadBalancer service is created with the flag disabled, CCM logs should contain entries with firewall rules are unmanaged, confirming that firewall operations are being skipped. The exact function names and message format may vary across OpenShift versions.

CCM will still provision load balancer resources (forwarding rules, target pools, external IPs) but will skip all firewall rule creation and deletion.

Step 7: Validate Day-2 Operations (Optional)

Worker Scale-Up

Scale a machinepool to verify new nodes are covered by BYO rules:

The new node should become Ready without any firewall changes. SA-targeted rules apply to any instance running with the matching WIF service account.

Scale back down when done:

Z-Stream Upgrade

Upgrade to a newer z-stream release to verify BYO rules and the firewall-rules-management flag survive the upgrade:

Schedule an upgrade:

Monitor progress:

After the upgrade completes, verify:

Confirm BYO firewall rules are still present:

Confirm platform-managed rules are still gone:

Confirm the flag persisted through the upgrade:

The BYO rules should be unchanged, platform-managed rules should remain absent, and the firewall-rules-management = Disabled flag should persist through the upgrade. The new CCM pod should load the flag correctly.

Cleanup

If you deployed the baseline test workload, remove it: