This guide shows you how to set up LiveDFS atop an OpenStack cloud platform. The setup consists of two parts: (i) Setting up OpenStack compute nodes and (ii) Deploying LiveDFS on the OpenStack image service.
OpenStack Compute Setup
This guide only shows the setup of our testbed. To see the detailed information of how to administer OpenStack Compute, please visit OpenStack Compute Administration Manual.
Our testbed contains three machines: one Cloud Controller (CC), one Compute Node(node) and one Image Server (Glance). This configuration is for our testing purpose only. You can add multiple compute nodes to simulate a more realistic cloud environment. The network configuration is as follows:
Cloud Controller 192.168.0.1 Compute Node 192.168.0.2 Image Server 192.168.0.3
First every machine should have a Ubuntu 10.04 LTS installed as currently LiveDFS only operates on Linux kernel 2.6.32 and LiveDFS is known not compatible with newer kernels. We are porting LiveDFS to newer kernels with more features. We will post our work when it is ready.
Run the following commands in the terminal to install OpenStack Compute in Cloud Controller:
sudo apt-get install python-software-properties sudo add-apt-repository ppa:openstack-release/2011.2 sudo apt-get update sudo apt-get install -y rabbitmq-server sudo apt-get install -y python-greenlet python-mysqldb sudo apt-get install -y nova-common nova-doc python-nova nova-api sudo apt-get install -y nova-network nova-objectstore nova-scheduler sudo apt-get install -y euca2ools unzip
Next Run the following commands in the terminal to install MySQL in Cloud Controller:
bash
MYSQL_PASS=nova
NOVA_PASS=notnova
cat <<MYSQL_PRESEED | debconf-set-selections
mysql-server-5.1 mysql-server/root_password password $MYSQL_PASS
mysql-server-5.1 mysql-server/root_password_again password $MYSQL_PASS
mysql-server-5.1 mysql-server/start_on_boot boolean true
MYSQL_PRESEED
sudo apt-get install -y mysql-server
sudo sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
sudo service mysql restart
sudo mysql -uroot -p$MYSQL_PASS -e 'CREATE DATABASE nova;'
sudo mysql -uroot -p$MYSQL_PASS -e "GRANT ALL PRIVILEGES ON *.* TO 'nova'@'%' WITH GRANT OPTION;"
sudo mysql -uroot -p$MYSQL_PASS -e "SET PASSWORD FOR 'nova'@'%' = PASSWORD('$NOVA_PASS');"
exit
After that copy the following settings to /etc/nova/nova.conf
--dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --verbose --sql_connection=mysql://nova:notnova@192.168.0.1/nova --rabbit_host=192.168.0.1 --ec2_api=192.168.0.1 --ec2_url=http://192.168.0.1:8773/services/Cloud --s3_host=192.168.0.1 --network_manager=nova.network.manager.FlatDHCPManager --fixed_range=10.0.0.0/16 --network_size=8 --glance_host=192.168.0.3 --image_service=nova.image.glance.GlanceImageService
After that you should restart all the services in the cloud controller by the following command:
restart libvirt-bin; restart nova-network; restart nova-api; restart nova-objectstore; restart nova-scheduler
We now start to configure the OpenStack Compute using root:
nova-manage db sync nova-manage user admin ldfs nova-manage project create LiveDFS ldfs nova-manage network create 10.0.0.0/24 1 256 mkdir -p /root/creds nova-manage project zipfile LiveDFS ldfs /root/creds/novacreds.zip unzip /root/creds/novacreds.zip -d /root/creds/ cat /root/creds/novarc >> ~/.bashrc
Next change the following in /root/.bashrc :
export S3_URL="http://192.168.0.1:3333"
export EC2_PRIVATE_KEY=${NOVA_KEY_DIR}/creds/pk.pem
export EC2_CERT=${NOVA_KEY_DIR}/creds/cert.pem
export NOVA_CERT=${NOVA_KEY_DIR}/creds/cacert.pem
After that you can run
. .bashrc
You now finished setting up the Cloud Controller. After that you need to setup the compute node:
sudo apt-get install python-software-properties sudo add-apt-repository ppa:openstack-release/2011.2 sudo apt-get update sudo apt-get install -y nova-common nova-doc python-nova nova-network nova-compute
All compute nodes share the same /etc/nova/nova.conf with the Cloud Controller:
--dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --verbose --sql_connection=mysql://nova:notnova@192.168.0.1/nova --rabbit_host=192.168.0.1 --ec2_api=192.168.0.1 --ec2_url=http://192.168.0.1:8773/services/Cloud --s3_host=192.168.0.1 --network_manager=nova.network.manager.FlatDHCPManager --fixed_range=10.0.0.0/16 --network_size=8 --glance_host=192.168.0.3 --image_service=nova.image.glance.GlanceImageService
After that you should restart all the services in the compute node by the following command:
restart libvirt-bin; restart nova-network; restart nova-compute;
We keep in sync with the database so that the cloud controller knows the compute node's existence
nova-manage db sync
Next we start to install Openstack Image Service (Glance) in the Image Server, which LiveDFS resides on. Run the following command to install Image Service:
sudo apt-get install python-software-properties sudo add-apt-repository ppa:openstack-release/2011.2 sudo apt-get update sudo apt-get install glance
All the installations are finished. Have fun with OpenStack!
Deploy LiveDFS on OpenStack Image Service
Deploying LiveDFS on OpenStack Image Service is easy. OpenStack Image Service uses a folder to store images so that we can mount that folder by LiveDFS. The default location of storing images is as follows:
/var/lib/glance/images
First you need to compile LiveDFS module and insert it into the kernel (as root)
tar zxvf liveDFS.tar.gz cd liveDFS make insmod liveDFS.ko
Next you need to format a clean partition (e.g. /dev/sdb1) and mount it as LiveDFS
tar zxvf mkfs.livedfs.tar.gz ./mkfs.livedfs /dev/sdb1 mount -t livedfs /dev/sdb1 /var/lib/glance/images chown glance:root /var/lib/glance/images