CloudRouter -- OpenDaylight performance comparison between Docker and OSv
Introduction
For this exercise we wanted to compare the performance of OpenDaylight SDN Controller when its run over our CloudRouter Docker Container vs as a OSv Application. We would be running the standard wcbench tool to measure ODL Performance. Thanks to the OpenDaylight community for putting out extensive information through their CrossProject Integration wiki pages - https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Performance_Test
Test Hardware
Tests will be run on Google Cloud. We plan to use the standard n1-highcpu-8 (8 vCPU, 7.2 GB memory) for this test.
Test Tools
WCBench from https://github.com/dfarrell07/wcbench.git
WCBench is a wrapped version of CBench, the SDN controller benchmark tool. It blasts a controller with OpenFlow packet-in messages and counts the rate of flow mod messages returned.
Test Environment Preparation
Server1 (Running ODL on docker):
Install CloudRouter VM (Run the CloudRouter docker container) yum -y install docker service docker start docker pull cloudrouter/odl-fedora docker run --publish-all=true -t -i cloudrouter/odl-fedora /bin/bash cd /opt/opendaylight/opendaylight-helium/ ./karaf opendaylight-user@root>feature:install odl-mdsal-common opendaylight-user@root>feature:install odl-config-api opendaylight-user@root>feature:install odl-config-netty-config-api opendaylight-user@root>feature:install odl-config-core opendaylight-user@root>feature:install odl-config-manager opendaylight-user@root>feature:install odl-config-netty opendaylight-user@root>feature:install odl-mdsal-broker opendaylight-user@root>feature:install odl-flow-model opendaylight-user@root>feature:install odl-flow-services opendaylight-user@root>feature:install odl-openflowjava-protocol opendaylight-user@root>feature:install odl-config-persister opendaylight-user@root>feature:install odl-config-startup opendaylight-user@root>feature:install odl-protocol-framework opendaylight-user@root>feature:install odl-yangtools-models opendaylight-user@root>feature:install odl-yangtools-data-binding opendaylight-user@root>feature:install odl-yangtools-binding opendaylight-user@root>feature:install odl-yangtools-common opendaylight-user@root>feature:install odl-yangtools-binding-generator opendaylight-user@root>feature:install odl-netconf-api opendaylight-user@root>feature:install odl-netconf-mapping-api opendaylight-user@root>feature:install odl-netconf-util opendaylight-user@root>feature:install odl-netconf-impl opendaylight-user@root>feature:install odl-config-netconf-connector opendaylight-user@root>feature:install odl-netconf-netty-util opendaylight-user@root>feature:install odl-netconf-monitoring opendaylight-user@root>feature:install odl-openflowplugin-southbound opendaylight-user@root>feature:install odl-openflowplugin-flow-services opendaylight-user@root>feature:install odl-openflowplugin-drop-test opendaylight-user@root> opendaylight-user@root>dropallpacketsrpc on
Server 2 (Running ODL on OSv):
# Start the image # ssh to the karaf port. # ssh -o PreferredAuthentications=keyboard-interactive -o PubkeyAuthentication=no -p 8101 karaf@<Google-Cloud-Instance-IP-ADDR> ( default password is 'karaf' ) # Install features opendaylight-user@root>feature:install odl-mdsal-common opendaylight-user@root>feature:install odl-config-api opendaylight-user@root>feature:install odl-config-netty-config-api opendaylight-user@root>feature:install odl-config-core opendaylight-user@root>feature:install odl-config-manager opendaylight-user@root>feature:install odl-config-netty opendaylight-user@root>feature:install odl-mdsal-broker opendaylight-user@root>feature:install odl-flow-model opendaylight-user@root>feature:install odl-flow-services opendaylight-user@root>feature:install odl-openflowjava-protocol opendaylight-user@root>feature:install odl-config-persister opendaylight-user@root>feature:install odl-config-startup opendaylight-user@root>feature:install odl-protocol-framework opendaylight-user@root>feature:install odl-yangtools-models opendaylight-user@root>feature:install odl-yangtools-data-binding opendaylight-user@root>feature:install odl-yangtools-binding opendaylight-user@root>feature:install odl-yangtools-common opendaylight-user@root>feature:install odl-yangtools-binding-generator opendaylight-user@root>feature:install odl-netconf-api opendaylight-user@root>feature:install odl-netconf-mapping-api opendaylight-user@root>feature:install odl-netconf-util opendaylight-user@root>feature:install odl-netconf-impl opendaylight-user@root>feature:install odl-config-netconf-connector opendaylight-user@root>feature:install odl-netconf-netty-util opendaylight-user@root>feature:install odl-netconf-monitoring opendaylight-user@root>feature:install odl-openflowplugin-southbound opendaylight-user@root>feature:install odl-openflowplugin-flow-services opendaylight-user@root>feature:install odl-openflowplugin-drop-test opendaylight-user@root> opendaylight-user@root>dropallpacketsrpc on
Client:
# Install CloudRouter VM # yum -y install git # git clone https://github.com/dfarrell07/wcbench.git # yum -y install python-matplotlib # cd wcbench # ./wcbench.sh -c ## to install CBench and its deps # ./wcbench.sh -i ## to get a local install of ODL. But we won't use it. # Edit wcbench.sh and set CONTROLLER_PORT and CONTROLLER_IP to the correct server when you run the test.
Test Results
wcbench results from docker
[root@cloudrouter-perf-2 wcbench]# ./wcbench.sh -t 2 -r Set MS_PER_TEST to 120000, TESTS_PER_SWITCH to 1, CBENCH_WARMUP to 0 Collecting pre-test stats Running CBench against ODL on 10.240.238.181:49190 Collecting post-test stats Collecting time-irrelevant stats Average responses/second: 17171.55
wcbench results from OSv:
[root@cloudrouter-perf-4 wcbench]# ./wcbench.sh -t 2 -r Set MS_PER_TEST to 120000, TESTS_PER_SWITCH to 1, CBENCH_WARMUP to 0 Collecting pre-test stats Running CBench against ODL on 10.240.78.172:6633 Collecting post-test stats Collecting time-irrelevant stats Average responses/second: 12137.07
Conclusion:
Based on this test run its clear that we are noticing a significant performance drop when running OpenDaylight on OSv versus docker.
Caveat:: This performance test has not been peer reviewed yet and we are looking forward to help from the OSv community.