For this exercise we wanted to compare the performance of OpenDaylight SDN Controller when its run over our CloudRouter Docker Container vs as a OSv Application. We would be running the standard wcbench tool to measure ODL Performance. Thanks to the OpenDaylight community for putting out extensive information through their CrossProject Integration wiki pages - https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Performance_Test
Tests will be run on Google Cloud. We plan to use the standard n1-highcpu-8 (8 vCPU, 7.2 GB memory) for this test.
WCBench from https://github.com/dfarrell07/wcbench.git
WCBench is a wrapped version of CBench, the SDN controller benchmark tool. It blasts a controller with OpenFlow packet-in messages and counts the rate of flow mod messages returned.
Install CloudRouter VM (Run the CloudRouter docker container) yum -y install docker service docker start docker pull cloudrouter/odl-fedora docker run --publish-all=true -t -i cloudrouter/odl-fedora /bin/bash cd /opt/opendaylight/opendaylight-helium/ ./karaf opendaylight-user@root>feature:install odl-mdsal-common opendaylight-user@root>feature:install odl-config-api opendaylight-user@root>feature:install odl-config-netty-config-api opendaylight-user@root>feature:install odl-config-core opendaylight-user@root>feature:install odl-config-manager opendaylight-user@root>feature:install odl-config-netty opendaylight-user@root>feature:install odl-mdsal-broker opendaylight-user@root>feature:install odl-flow-model opendaylight-user@root>feature:install odl-flow-services opendaylight-user@root>feature:install odl-openflowjava-protocol opendaylight-user@root>feature:install odl-config-persister opendaylight-user@root>feature:install odl-config-startup opendaylight-user@root>feature:install odl-protocol-framework opendaylight-user@root>feature:install odl-yangtools-models opendaylight-user@root>feature:install odl-yangtools-data-binding opendaylight-user@root>feature:install odl-yangtools-binding opendaylight-user@root>feature:install odl-yangtools-common opendaylight-user@root>feature:install odl-yangtools-binding-generator opendaylight-user@root>feature:install odl-netconf-api opendaylight-user@root>feature:install odl-netconf-mapping-api opendaylight-user@root>feature:install odl-netconf-util opendaylight-user@root>feature:install odl-netconf-impl opendaylight-user@root>feature:install odl-config-netconf-connector opendaylight-user@root>feature:install odl-netconf-netty-util opendaylight-user@root>feature:install odl-netconf-monitoring opendaylight-user@root>feature:install odl-openflowplugin-southbound opendaylight-user@root>feature:install odl-openflowplugin-flow-services opendaylight-user@root>feature:install odl-openflowplugin-drop-test opendaylight-user@root> opendaylight-user@root>dropallpacketsrpc on |
# Start the image # ssh to the karaf port. # ssh -o PreferredAuthentications=keyboard-interactive -o PubkeyAuthentication=no -p 8101 karaf@<Google-Cloud-Instance-IP-ADDR> ( default password is 'karaf' ) # Install features opendaylight-user@root>feature:install odl-mdsal-common opendaylight-user@root>feature:install odl-config-api opendaylight-user@root>feature:install odl-config-netty-config-api opendaylight-user@root>feature:install odl-config-core opendaylight-user@root>feature:install odl-config-manager opendaylight-user@root>feature:install odl-config-netty opendaylight-user@root>feature:install odl-mdsal-broker opendaylight-user@root>feature:install odl-flow-model opendaylight-user@root>feature:install odl-flow-services opendaylight-user@root>feature:install odl-openflowjava-protocol opendaylight-user@root>feature:install odl-config-persister opendaylight-user@root>feature:install odl-config-startup opendaylight-user@root>feature:install odl-protocol-framework opendaylight-user@root>feature:install odl-yangtools-models opendaylight-user@root>feature:install odl-yangtools-data-binding opendaylight-user@root>feature:install odl-yangtools-binding opendaylight-user@root>feature:install odl-yangtools-common opendaylight-user@root>feature:install odl-yangtools-binding-generator opendaylight-user@root>feature:install odl-netconf-api opendaylight-user@root>feature:install odl-netconf-mapping-api opendaylight-user@root>feature:install odl-netconf-util opendaylight-user@root>feature:install odl-netconf-impl opendaylight-user@root>feature:install odl-config-netconf-connector opendaylight-user@root>feature:install odl-netconf-netty-util opendaylight-user@root>feature:install odl-netconf-monitoring opendaylight-user@root>feature:install odl-openflowplugin-southbound opendaylight-user@root>feature:install odl-openflowplugin-flow-services opendaylight-user@root>feature:install odl-openflowplugin-drop-test opendaylight-user@root> opendaylight-user@root>dropallpacketsrpc on |
# Install CloudRouter VM # yum -y install git # git clone https://github.com/dfarrell07/wcbench.git # yum -y install python-matplotlib # cd wcbench # ./wcbench.sh -c ## to install CBench and its deps # ./wcbench.sh -i ## to get a local install of ODL. But we won't use it. # Edit wcbench.sh and set CONTROLLER_PORT and CONTROLLER_IP to the correct server when you run the test. |
[root@cloudrouter-perf-2 wcbench]# ./wcbench.sh -t 2 -r Set MS_PER_TEST to 120000, TESTS_PER_SWITCH to 1, CBENCH_WARMUP to 0 Collecting pre-test stats Running CBench against ODL on 10.240.238.181:49190 Collecting post-test stats Collecting time-irrelevant stats Average responses/second: 17171.55 |
[root@cloudrouter-perf-4 wcbench]# ./wcbench.sh -t 2 -r Set MS_PER_TEST to 120000, TESTS_PER_SWITCH to 1, CBENCH_WARMUP to 0 Collecting pre-test stats Running CBench against ODL on 10.240.78.172:6633 Collecting post-test stats Collecting time-irrelevant stats Average responses/second: 12137.07 |
Based on this test run its clear that we are noticing a significant performance drop when running OpenDaylight on OSv versus docker.
Caveat:: This performance test has not been peer reviewed yet and we are looking forward to help from the OSv community.