A Guide to Integrating Red Hat OpenStack with Cisco ACI

While working on a Red Hat OpenStack related engagement with one of our customers, we did a Proof of Concept (PoC) with them where the scope was to set up Red Hat OpenStack in their environment and integrate it with a Cisco ACI network solution already available in their environment. So, the main purpose of this write-up is to share our experiences on:

• CISCO ACI Preparation on the under cloud /director node.
• Creation of HEAT Templates for Overcloud Deployment (ACI perspective).
• Issues faced and the solution implemented for OpenStack.

Let’s explore the above-mentioned points in detail with actual commands used and the expected output with screenshots.

CISCO ACI Preparation on the undercloud / director node

For OpenStack to integrate with CISCO ACI, below are the steps required to be executed on the director node to setup Cisco related services on the Overcloud.

From a valid Cisco Account, Download the openstack-tripleo-cisco packages on the Director.

Installation of Cisco ACI Packages and HEAT Templates on the undercloud/director node

[stack@director ~]$ sudo yum install /home/stack/tripleo-ciscoaci-13.0-642.noarch.rpm

Command output:

Run the Cisco Aci python script to download and install cisco related packages and container Images.

Command:

[stack@director ~]$ /opt/ciscoaci-tripleo-heat-templates/tools/build_openstack_aci_containers.py

Command output:

Generate Controller and Compute roles and add Cisco ACI Services in aci_roles.yaml.

Command :

[stack@director ~]$  openstack overcloud roles generate --roles-path /usr/share/openstack-tripleo-heat-templates/roles -o ~/templates/environment-files/roles_data.yaml
    Controller Compute

[stack@director ~]$ cp /home/stack/templates/environment-files/roles_data.yaml /home/stack/templates/environment-files/aci_roles_data.yaml 

[stack@director ~]$ vim /home/stack/templates/environment-files/aci_roles_data.yaml 
~~~ Under controller role added the following lines: ~~~
- OS::TripleO::Services::CiscoAciAIM
- OS::TripleO::Services::CiscoAciLldp

~~~ Under the compute role added the following line: ~~~
- OS::TripleO::Services::CiscoAciLldp

Command output :

Creation of HEAT Templates for Overcloud Deployment (ACI perspective)

The director uses Heat Orchestration Templates (HOT) as a template format for its Overcloud deployment plan. Templates in HOT format are mostly expressed in YAML format. The purpose of a template is to define and create a stack, which is a collection of resources that Heat creates and the configuration per resources. Resources are objects in OpenStack and can include compute resources, network configuration, security groups, scaling rules, and custom resources.

Deploy Command to run the Templates

source ~/stackrc
nohup openstack overcloud deploy --templates /home/stack/templates \
-n /home/stack/templates/environment-files/network_data.yaml \
-r /home/stack/templates/environment-files/aci_roles_data.yaml \
-e /home/stack/templates/environment-files/node-info.yaml \
-e /home/stack/templates/environment-files/overcloud_images.yaml \
-e /home/stack/templates/environment-files/ciscoaci_containers.yaml \
-e /home/stack/templates/environment-files/aci_cs.yaml \
-e /home/stack/templates/environment-files/cinder-dellemc-unity-config.yaml \
-e /home/stack/templates/environment-files/ips-from-pool-all.yaml \
--log-file /home/stack/Overcloud-Logs/osp-deploy-log-`date +%F_%H-%M-%S`.out &

Cisco ACI Container Image Template

parameter_defaults:
   DockerHorizonImage: X.X.X.X:pppp/rhosp13/openstack-horizon-ciscoaci:latest
   DockerCiscoAciAimImage: X.X.X.X:pppp/rhosp13/openstack-ciscoaci-aim:latest
   DockerOpflexAgentImage: X.X.X.X:pppp/rhosp13/openstack-ciscoaci-opflex:latest
   DockerNeutronApiImage: X.X.X.X:pppp/rhosp13/openstack-neutron-server-ciscoaci:latest
   DockerNeutronConfigImage: X.X.X.X:pppp/rhosp13/openstack-neutron-server-ciscoaci:latest
   DockerHeatEngineImage: X.X.X.X:pppp/rhosp13/openstack-heat-engine-ciscoaci:latest
   DockerCiscoLldpImage: X.X.X.X:pppp/rhosp13/openstack-ciscoaci-lldp:latest

Issues faced and the solution implemented for OpenStack

On facing the issues, I raised the case with TSANet Connect (Technical Support Alliance Network) and got the Cisco global teams involved. Cisco technical team came on a couple of calls and their involvement was really helpful as they have very good and practical experience on both Cisco ACI and Red Hat OpenStack side from integration perspective.

Below section details the exact issues and the solution implemented:

1 Error in accessing the Openstack machines from outside

Issue :- Machines created in openstack were not accessible from outside of openstack. Cisco ACI Integration with openstack does not provide a feature to create a network of type “FLAT” in openstack.

Solution Implemented :-

• Providing access to L3OUT Domain in Cisco ACI for openstack in the common tenant.
• Creation of an External Floating IP Network in openstack. source ~/overcloudrc 

(overcloud) [stack@director]$ neutron net-create ext-fip-net --router:external --apic:distinguished_names type=dict ExternalNetwork=uni/tn-common/out-FW-Common-Services-Zone-L3Out/instP-Common-Services-xEPG
(overcloud) [stack@director]$ neutron subnet-create ext-fip-net X.X.X.X/25 --name ext-fip-subnet --disable-dhcp --gateway X.X.X.X

• Creation of a virtual Router in Openstack. source ~/overcloudrc 

(overcloud) [stack@director]$ openstack router create Router1

• Setting the External Gateway of the router to the External FIP Network. source ~/overcloudrc 

(overcloud) [stack@director]$ openstack router set --external-gateway ext-net Router1

• Connecting the Tenant/Private Network to the Router. source ~/overcloudrc 

(overcloud) [stack@director]$ openstack router add subnet Router1 tenant-subnet1

After implementing the above solution the following Network Topology was created.

2 VMs Created on Compute node are NOT being learned by Cisco ACI

Issue :-  For this PoC we had only 2 Compute Nodes. Machines Created on compute node 1 were not being learned by the Cisco ACI. In Other words, CISCO ACI didn’t have any connectivity with the machine network hosted on the Openstack compute node 1. Machines were not able to connect with each other and gateway IP was also not reachable.

Solution Implemented 

• There was a mismatch of configuration of openvswitch on the compute node 1 with the other compute node 0.
• OpenVswitch Bridge (OVS) “br-fabric” on the compute node is connected to Cisco ACI and OVS Bridge “br-ex” is connected to the Cisco Opflex Agent running on compute host. Connectivity between these two OVS Bridge provides the VMs to have external connectivity with the Cisco ACI.
• A Patch port (Can be imagined as a virtual wire) between br-fabric and br-ex was NOT present on the Faulty Compute host ( Compute-1).
• To Create a Patch port, Below set of commands were implemented.

(overcloud) [stack@director ~]$ ssh comp1 -l heat-admin
[heat-admin@overcloud-compute-1 ~]$ sudo ovs-vsctl add-port br-ex br-ex_to_br-fa
[heat-admin@overcloud-compute-1 ~]$ sudo ovs-vsctl set interface br-ex_to_br-fa type=patch 
[heat-admin@overcloud-compute-1 ~]$ sudo ovs-vsctl set interface br-ex_to_br-fa options:peer=br-fa_to_br-ex

After implementing the above solution the VMs created in OpenStack are being learned by the Cisco ACI.

Conclusion

This article will really help those who are either planning this kind of integration between Red Hat OpenStack and Cisco ACI network solution or might do it in future. The information provided here will help you in preparing OpenStack for Cisco ACI, Heat template and probable issues and their solutions. Any feedback is most welcome to enhance it further.