Naming VMs based on Group name

One of my customer asked me to name his VMs based on the group name provisioning that VM.  Example:  redgroup-001

Turns out that you cannot use the usual naming method as services are always provisioned as “super-administrator” in Cloudforms 3.0.

I added the following code in my “CustomizeRequest” to get this done:

  1.   #
  2.   # Changing VM name based on Group:  groupname_###
  3.   #
  4.  
  5.   miq_request = prov.miq_request
  6.   user = miq_request.requester
  7.   group = user.miq_group
  8.  
  9.   log(:info, “User: #{user.name} Group: #{group.description}”)
  10.   vm_name = “#{group.description}_”
  11.    
  12.     # Loop through 0-999 and look to see if the name already exists in CFME
  13.     for i in (1..999)
  14.       new_vm_name = “#{vm_name}#{i}”
  15.       if $evm.vmdb(‘vm’).find_by_name(new_vm_name).nil?
  16.         log(:info, “Using VM:<#{new_vm_name}> name”)
  17.         break
  18.       end
  19.     end
  20.   prov.set_option(:vm_target_name, new_vm_name)
  21.   log(:info, “Adding Option prov.set_option(:vm_target_name, #{prov.get_option(:vm_target_name)}) to Provisioning ID:<#{prov.id}>”)
  22.   prov.set_option(:vm_target_hostname, new_vm_name)
  23.   log(:info, “Adding Option prov.set_option(:vm_target_hostname, #{prov.get_option(:vm_target_hostname)}) to Provisioning ID:<#{prov.id}>”)
  24.   prov.set_option(:vm_name, new_vm_name)
  25.   log(:info, “Adding Option prov.set_option(:vm_name, #{prov.get_option(:vm_name)}) to Provisioning ID:<#{prov.id}>”)

Executing troubleshooting commands inside a namespace

Here is how you can execute some troubleshooting commands as if you were running them on a virtual router inside Neutron.

[root@serverX ~(keystone_admin)]# neutron router-list
[root@serverX ~(keystone_admin)]# ip netns
[root@serverX ~(keystone_admin)]# ip netns exec qrouter-10bf634d-3228-4041-8f3a-4d5e0e603c07 ping
8.8.8.8
[root@serverX ~(keystone_admin)]# ip netns exec qrouter-10bf634d-3228-4041-8f3a-4d5e0e603c07 netstat
-rn

*  Kernel namespaces are used to secure virtual networks from each other.    This is great from a security perspective but sometimes makes troubleshooting harder.  

My RDO installation procedure

This is OLD.  You should look at this updated article:

http://www.marcoberube.com/archives/346

 

RDO is a community of people using and deploying OpenStack on Red Hat Enterprise Linux, Fedora and distributions derived from these (such as CentOS, Scientific Linux and others).

This procedure describes how to install RDO Icehouse release on CentOS 6.5.  By following this procedure, you will install all Openstack modules on one host but keep your options open if you eventually want to add an additional compute node.

Requirements:
  • Centos 6.5 basic server installation
  • 20GB+ of disk space:  Depends of what you want to do but more is better as you will have to host instances and images as well.
  • 2 network interface:   eth0 for your public network; eth1 for your private (SDN) network.   Both interfaces should have static IP addresses.
Diagram of what we are doing
 Simple Openstack Architecture

* eth1 on a private network is only required if you are planning to grow this environment to more than one host.  If all you need is all-in-one, replace “gre” by “local” in the upcoming network configuration files.

Add RDO channel, install openstack-packstack and update all packages:
yum install -y http://rdo.fedorapeople.org/rdo-release.rpm
yum install -y openstack-packstack
yum -y update
reboot

* Packstack is a puppet based Openstack installer.  Easy way to install Openstack

** Reboot is required if your kernel has been updated

Generate an SSH key
ssh-keygen

Create a packstack answer file (configuration file for the automated installation)

packstack --gen-answer-file=/root/answers.txt

Updating the following settings in your answer file

CONFIG_HEAT_INSTALL=y
CONFIG_NEUTRON_OVS_TENANT_NETWORK_TYPE=gre
CONFIG_NEUTRON_OVS_TUNNEL_RANGES=1:1000
CONFIG_NEUTRON_OVS_TUNNEL_IF=eth1
CONFIG_KEYSTONE_ADMIN_PW=<your_password>
Start your packstack installation
packstack --answer-file=/root/answers.txt

 * This will take a while.  I fixed many installation errors just by running that install command a second time.  Not sure why (network timeouts maybe..??) but there is so many puppet manifest, I guess it’s easy to have one fail on you and running it one more time often fix the problem.

 Neutron configuration

Ok, this is where everybody has issues.  First thing to understand is that you need an external bridge (ifcfg-br-ex) on your host for Neutron to work properly.  It’s like adding a virtual switch between eth0 and the physical interface.

This new file should have been created during the installation:

/etc/sysconfig/network-scripts/ifcfg-br-ex

We need to move the IP address from eth0 to this bridge and connect eth0 to that bridge instead.

Update this file so it looks like this:

DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
IPADDR=<ip address from eth0>
NETMASK=<netmask from eth0>
GATEWAT<gateway from eth0>
ONBOOT=yes

Now, edit /etc/sysconfig/network-scripts/ifcfg-eth0 so it looks like this:

DEVICE=eth0
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex
ONBOOT=yes

* Be careful here, many problems are coming from typos in these files.

Finally, let’s edit /etc/neutron/plugin.ini and make some updates:

[ml2]
type_drivers = gre
tenant_network_types = gre

 * My understanding is that this should have been updated by packstack as we used GRE in our answer file.  That said, at the moment I tested this procedure, that didn’t work.  This is probably due to the fact that Icehouse is now using the ML2 plugin on top of Openvswitch instead of using Openvswitch directly.

 Restart your network
service network restart

* If you lost your network connectivity, something went wrong.  Go back to your console access to validate that ifcfg-eth0 and ifcfg-br-ex are configured properly.

You can use the following Openvswitch command to validate that you have a bridge br-ex and and port named “eth0” connected to that bridge.

ovs-vsctl show
Validating that Openstack is running fine

Once your network connectivity is working, use the following command to check that openstack is running properly:

source /root/keystonerc_admin
openstack-status

This should give you a very long output providing status information on all your modules.

Removing default Neutron configuration

Packstack configures some public network and router by default.  Unless you are very lucky and the public network configured as the exact same IP range as your real public network, let’s remove these settings and start fresh:

neutron router-gateway-clear router1
neutron subnet-delete public_subnet

Now, let’s configure a new router and public subnet.

neutron subnet-create --name public_subnet --enable_dhcp=False --allocation-pool=start=192.168.122.10,end=192.168.122.20 --gateway=192.168.122.1 public 192.168.122.0/24
neutron router-gateway-set router1 public

* Obviously, change all IP address settings based on your public network settings.   Neutron will not use DHCP to allocate public floating IP addresses, they will be manually assigned from the allocation pool described in this command.  

Get to Horizon web interface in your browser
http://<your_br-ex_ip>/

Login: admin
Password:  <password set in your answer file>

* if you haven’t set your keystone admin password in your answer file, you can find the password automatically generated for you in /root/keystonerc_admin

Setting up virtual network for the admin tenant

First, let’s look at what our network currently looks like.

>> Click on Project/Network/Network Topology

Screen Shot 2014-06-30 at 10.57.29 AM

 

Ok, we have a public network and a subnet assigned to this network.   We don’t want to run instances directly on this network.  Instead, we will create a virtual private network and a virtual router to route our traffic.

>> Click on “Create Network”
Network Name = private

>> Click on “Subnet”
Subnet name = private_subnet
Network Address = 10.0.0.0/16 (or whatever you like)
Gateway IP = 10.0.0.1

>> Click on “Subnet Details”
Enable DHCP = True
DNS Name Servers = 8.8.8.8

>> Click on “Create”

Screen Shot 2014-06-30 at 11.06.32 AM

Now, we need a router to route our traffic from private to public.

>> Click on “Create Router”
Router Name = router1
>> Click on “Create Router”

Aim your mouse over that new router and click on “View router details”

>> Click on “+ Add Interface”
Subnet = Private
IP Address = 10.0.0.1
>> Click on “Add Interface”

>> Click on “Routers” in the left menu to get back to your list of routers : /Project/Network/Routers

>> Click on “Set Gateway”
External Network = public
>> Click on “Set Gateway”

If you look at your “network topology” again, everything should now be connected:

Screen Shot 2014-06-30 at 11.12.08 AM

Yeah!!!  Let’s boot our first instance… shall we????

>> Click on /Project/Compute/Images
An image named “cirros” should already be loaded.  Click on “Launch” to start this image.
Instance name = <your instance name?>
>> Click on “Networking”
Drag “private” in the box of “selected networks”
>> Click “Launch” !!!!

I will not be getting into any details on how to use Openstack here.  This was only an installation procedure.   But if you look at your instances, click on your new instance and check the console, you should see your instance running.   Your instance should have grabbed an IP address from DHCP and be able to reach your external network.

Additional documentation

Here is a good document describing how Neutron works and help you troubleshoot common issues:
http://docs.openstack.org/openstack-ops/content/network_troubleshooting.html

 Coming soon:

– Adding a second compute node

 

Export a sub-section of your automation model

In Cloudforms 3.0, you can only export the full automation model from the web interface.  That said, what if you want to take a backup of only a sub-section.

Here is how to do it from the appliance command line:

# cd /var/www/miq/vmdb/
# script/rails runner script/rake evm:automate:export NAMESPACE=<your_namespace> CLASS=<class_name> FILE=<filename>.xml

To export all classes under an namespace, just remove your class argument.

Call or SMS notification using Twilio

By default, Cloudforms notify a user by email that a new VM or a Service has been fully provisioned.  What if you would like to send a text message (SMS) or call somebody?  Twilio let you call or SMS somebody from a Restful API.

Step 1)  Create a Twilio account

Twilio | Try Twilio Free

Please note that if you are using the FREE service, you will get a twilio trial message before your SMS or CALL.  To remove this message, you will have to pay for this service.  That said, it’s very cheap!

Step 2)  Install required Twilio Ruby Gems on your appliance
gem install twilio-ruby
gem install twilio-twimlbin
Step 3) Import my Twilio class in your automation model

Download here (unzip before importing in Cloudforms):  Twilio.xml.zip

** By default, my Twilio class will be imported under /POC/Integration namespace.  Edit the XML file if want to change the location.

Here is what the Twilo-call code looks like.  Very simple…

  1. ###################################
  2. #
  3. # CFME Automate Method: twilio-call
  4. #
  5. # by Marco Berube
  6. #
  7. # Notes: Twilio is a service to send call or sms request from a restful API.
  8. #            More informatin at:  https://www.twilio.com/api
  9. #
  10. ###################################
  11.  
  12. begin
  13.   # Method for logging
  14.   def log(level, message)
  15.     @method = ‘twilio-call’
  16.     $evm.log(level, “#{@method}: #{message}”)
  17.   end
  18.   # dump_root
  19.   def dump_root()
  20.     log(:info, “Root:<$evm.root> Begin $evm.root.attributes”)
  21.     $evm.root.attributes.sort.each { |k, v| log(:info, “Root:<$evm.root> Attribute – #{k}: #{v}”)}
  22.     log(:info, “Root:<$evm.root> End $evm.root.attributes”)
  23.     log(:info, “”)
  24.   end
  25.  
  26.   # dump all root attributes to the log
  27.   dump_root
  28.  
  29. require ‘twilio-twimlbin’
  30. require ‘twilio-ruby’
  31.  
  32. sid = $evm.object[‘sid’]
  33. token = $evm.object[‘token’]
  34. from_number = $evm.object[‘from’]
  35. to_number = $evm.object[‘to’]
  36. to_number ||= $evm.root[‘dialog_twilio_to’] 
  37. body = $evm.object[‘message’]
  38.  
  39.  
  40. response = Twilio::TwiML::Response.new do |r|
  41.   r.Say body, :voice => ‘woman’
  42. end
  43.  
  44. twiml = Twimlbin.new
  45. twiml.create(response.text)
  46.  
  47. # set up a client to talk to the Twilio REST API 
  48. @client = Twilio::REST::Client.new sid, token 
  49.  
  50. @client.account.calls.create({
  51.   :to => to_number, 
  52.   :from => from_number, 
  53.   :url => twiml.external_url,  
  54.   :method => ‘GET’,  
  55.   :fallback_method => ‘GET’,  
  56.   :status_callback_method => ‘GET’,    
  57.   :record => ‘false’
  58. })
  59.  
  60.   # Exit method
  61.   log(:info, “Twilio notification call request completed”)
  62.   exit MIQ_OK
  63.  
  64.   # Error Rescue
  65. rescue => err
  66.   log(:error, “[#{err}]\n#{err.backtrace.join(“\n“)}”)
  67.   exit MIQ_STOP
  68. end
Step 4) Update your Twilio attributes for your CALL and SMS instance. 

screenshot-twilio

Sid / Token =  your username/password to access your Twilio account from a Restful API.  Log in to your Twilio account to find this information

From = Your Twilio account phone number.  Log in to your Twilio account to find this information

To = Default phone number that this class will call to notify the user.  You can dynamically change this phone number from your service dialog by using a variable called “twilio_to”.  This call will overwrite the default number from dialog_twilio_to

Message = Your notification message

* Obviously, there is a lot of potential improvements here.    I am currently using this only on demos, so I didn’t spend much time on this.  But you could get the user phone number from active directory or provide the VM name in your notification message.

Step 5) Replace your default Email class by this Twilio/Call or SMS at the end of your provisioning state machine.