cassandra cluster status when a node goes down

1)
When all 3 nodes in the cluster up

# nodetool status

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address       Load       Tokens  Owns (effective)  Host ID                               Rack
UN  10.140.15.59  75.4 KB    256     100.0%            a07836a2-8c56-4da7-803d-4389f87a4854  rack1
UN  10.140.15.61  256.7 KB   256     100.0%            00c51f6a-2d08-4b27-81f9-b9754b6b0b66  rack1
UN  10.140.15.60  90.74 KB   256     100.0%            d89d9cd8-d154-43d9-97ec-6b827c9a7ced  rack1

2)
When the node 10.140.15.61 is down

# nodetool status

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address       Load       Tokens  Owns (effective)  Host ID                               Rack
UN  10.140.15.59  75.4 KB    256     100.0%            a07836a2-8c56-4da7-803d-4389f87a4854  rack1
DN  10.140.15.61  256.7 KB   256     100.0%            00c51f6a-2d08-4b27-81f9-b9754b6b0b66  rack1
UN  10.140.15.60  90.74 KB   256     100.0%            d89d9cd8-d154-43d9-97ec-6b827c9a7ced  rack1

How to setup cassandra zookeeper and rabbitmq cluster

1)
Nodes:
ct1-multinode-cont : 11.120.15.59
ct2-multinode-cont : 11.120.15.60
ct3-multinode-cont : 11.120.15.61

2)
zookeeper cluster:

a)
config:

$vim /etc/zookeeper/conf/zoo.cfg

server.59=11.120.15.59:2888:3888
server.60=11.120.15.60:2888:3888
server.61=11.120.15.61:2888:3888

b)
Check cluster status

b1)
$ /usr/share/zookeeper/bin/zkServer.sh status
JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
Mode: follower

b2)
$ /usr/share/zookeeper/bin/zkServer.sh status
JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
Mode: leader
 
b3)
From  11.120.15.61



$ /usr/share/zookeeper/bin/zkServer.sh status
JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
Mode: follower

c)
docs:
http://myjeeva.com/zookeeper-cluster-setup.html

http://hanishblogger.blogspot.in/2013/11/zookeeper-setup-in-distributed-mode.html

3)
cassandra cluster:

a)
config:

$/etc/cassandra/cassandra.yaml
cluster_name: 'contrail'

seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: 11.120.15.59,11.120.15.60,11.120.15.61
b)
Check cluster status  
 
# nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address       Load       Tokens  Owns (effective)  Host ID                               Rack
UN  11.120.15.59  75.4 KB    256     100.0%            a07836a2-8c56-4da7-803d-4389f87a4854  rack1
UN  11.120.15.61  256.7 KB   256     100.0%            00c51f6a-2d08-4b27-81f9-b9754b6b0b66  rack1
UN  11.120.15.60  90.74 KB   256     100.0%            d89d9cd8-d154-43d9-97ec-6b827c9a7ced  rack1


c)
docs:
https://docs.datastax.com/en/cassandra/2.0/cassandra/initialize/initializeSingleDS.html

4)
rabbitmq cluster config:

a)
config:
http://blog.hemantthorat.com/setup-rabbitmq-clusters-on-ubuntu/#.Vm-7a5PoEc0

b)
Check cluster status  
 
$ sudo rabbitmqctl cluster_status
Cluster status of node 'rabbit@ct1-multinode-cont' ...
[{nodes,[{disc,['rabbit@ct1-multinode-cont','rabbit@ct2-multinode-cont',
                'rabbit@ct3-multinode-cont']}]},
 {running_nodes,['rabbit@ct2-multinode-cont','rabbit@ct3-multinode-cont',
                 'rabbit@ct1-multinode-cont']},
 {cluster_name,<<"rabbit@localhost">>},
 {partitions,[]}]

Nova boot with glance image fails Corrupt image download. Checksum was

1)
#nova boot --flavor 2 --image 61bb862c-69f1-49cc-9d37-1dbbf3eb841e --nic net-id=146520dc-2d6b-4ba1-b803-2edd1906a6f3 vm2

#nova boot --flavor 2 --image 61bb862c-69f1-49cc-9d37-1dbbf3eb841e --nic net-id=146520dc-2d6b-4ba1-b803-2edd1906a6f3 v1 --availability-zone nova:cp1-multinode-cont

2)
Fix:
#vim /usr/lib/python2.7/dist-packages/glanceclient/common/http.py

class HTTPClient(object):

    def __init__(self, endpoint, **kwargs):
        if self.endpoint.startswith("https"):
            #compression = kwargs.get('ssl_compression', True)
            compression = True

* Comment the line "compression = kwargs.get('ssl_compression', True)", then set "compression = True"

* Restart glance-api, glance-registry, cinder-api, cinder-volume, cinder-scheduler

3)
From /var/log/upstart/nova-compute.log

2015-08-07 01:43:02.100 18716 ERROR nova.compute.manager [req-66173648-5d32-4ab4-b647-6f571e5a2b76 5f51a7daa18f4465936fbb440ad05471 76cfebd8f2974dad94fd8770da958663 - - -] [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e] Instance failed to spawn
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e] Traceback (most recent call last):
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2473, in _build_resources
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     yield resources
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2345, in _build_and_run_instance
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     block_device_info=block_device_info)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2384, in spawn
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     admin_pass=admin_password)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2782, in _create_image
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     instance, size, fallback_from_host)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5900, in _try_fetch_image_cache
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     size=size)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 231, in cache
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     *args, **kwargs)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 480, in create_image
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     prepare_template(target=base, max_size=size, *args, **kwargs)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     return f(*args, **kwargs)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 221, in fetch_func_sync
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     fetch_func(target=target, *args, **kwargs)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 507, in fetch_image
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     max_size=max_size)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 87, in fetch_to_raw
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     max_size=max_size)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 77, in fetch
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     IMAGE_API.download(context, image_href, dest_path=path)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/image/api.py", line 182, in download
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     dst_path=dest_path)
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 362, in download
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     for chunk in image_chunks:
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]   File "/usr/lib/python2.7/dist-packages/glanceclient/common/utils.py", line 398, in integrity_iter
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]     (md5sum, checksum))
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e] IOError: [Errno 32] Corrupt image download. Checksum was d41d8cd98f00b204e9800998ecf8427e expected 133eae9fb1c98f45894a4e60d8736619
2015-08-07 01:43:02.100 18716 TRACE nova.compute.manager [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e]
2015-08-07 01:43:02.104 18716 INFO nova.compute.manager [req-66173648-5d32-4ab4-b647-6f571e5a2b76 5f51a7daa18f4465936fbb440ad05471 76cfebd8f2974dad94fd8770da958663 - - -] [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e] Terminating instance
2015-08-07 01:43:02.109 18716 INFO nova.virt.libvirt.driver [-] [instance: cd3fa491-fd04-4be8-87f1-27738ee8285e] During wait destroy, instance disappeared.

cinder create volume from image JSONDecodeError: Expecting value: line 1 column 1 (char 0)

1)
# cinder create --image-id 8f980b10-b491-41c3-bd48-a84001929840 --display-name cirros1 1
ERROR: Internal Server Error (HTTP 500) (Request-ID: req-410318bc-9a39-4cbc-8401-2037ec5d1d53)

2)
Fix:
#vim /usr/lib/python2.7/dist-packages/glanceclient/common/http.py

class HTTPClient(object):

    def __init__(self, endpoint, **kwargs):
        if self.endpoint.startswith("https"):
            #compression = kwargs.get('ssl_compression', True)
            compression = True

* Comment the line "compression = kwargs.get('ssl_compression', True)", then set "compression = True"

* Restart glance-api, glance-registry, cinder-api, cinder-volume, cinder-scheduler

3)
/var/log/upstart/cinder-api.log

2015-08-06 14:23:26.296 36113 DEBUG glanceclient.common.http [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -] curl -i -X GET -H 'User-Agent: python-glanceclient' -H 'Content-Type: application/octet-stream' -H 'Accept-Encoding: gzip, deflate, compress' -H 'Accept: */*' -H 'X-Auth-Token: {SHA1}e6717209a08a3b895581176f415a3b34c8264e40'  --cert None --key None glance+https://image.jiocloud.com:9292/v2/images/8f980b10-b491-41c3-bd48-a84001929840 log_curl_request /usr/lib/python2.7/dist-packages/glanceclient/common/http.py:122
2015-08-06 14:23:26.369 36113 DEBUG glanceclient.common.http [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -]
HTTP/1.1 200 OK
content-length: 699
via: 1.1 image.jiocloud.com:9292
access-control-expose-headers: Accept, Content-Type, X-Auth-Token, X-Subject-Token
server: Apache/2.4.7 (Ubuntu)
access-control-allow-headers: Accept, Content-Type, X-Auth-Token, X-Subject-Token
connection: close
date: Thu, 06 Aug 2015 14:23:26 GMT
access-control-allow-origin: *
access-control-allow-methods: GET POST OPTIONS PUT DELETE PATCH
content-type: application/json; charset=UTF-8
x-openstack-request-id: req-7670140c-8334-4148-b9f3-d3d9845c11c9
 log_http_response /usr/lib/python2.7/dist-packages/glanceclient/common/http.py:135
2015-08-06 14:23:26.371 36113 DEBUG taskflow.engines.action_engine.runner [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -] Exiting old state 'SCHEDULING' in response to event 'wait_finished' on_exit /usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/runner.py:201
2015-08-06 14:23:26.371 36113 DEBUG taskflow.engines.action_engine.runner [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -] Entering new state 'WAITING' in response to event 'wait_finished' on_enter /usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/runner.py:205
2015-08-06 14:23:26.371 36113 DEBUG taskflow.engines.action_engine.runner [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -] Exiting old state 'WAITING' in response to event 'examine_finished' on_exit /usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/runner.py:201
2015-08-06 14:23:26.372 36113 DEBUG taskflow.engines.action_engine.runner [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -] Entering new state 'ANALYZING' in response to event 'examine_finished' on_enter /usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/runner.py:205
2015-08-06 14:23:26.372 36113 WARNING cinder.volume.api [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -] Task 'cinder.volume.flows.api.create_volume.ExtractVolumeRequestTask;volume:create' (14e19665-b4a7-4f40-8ce9-1aa00d1e374a) transitioned into state 'FAILURE' from state 'RUNNING'
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api Traceback (most recent call last):
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 67, in _execute_task
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     result = task.execute(**arguments)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/api/create_volume.py", line 417, in execute
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     self._check_image_metadata(context, image_id, size)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/api/create_volume.py", line 251, in _check_image_metadata
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     image_meta = self.image_service.show(context, image_id)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/cinder/image/glance.py", line 237, in show
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     _reraise_translated_image_exception(image_id)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/cinder/image/glance.py", line 235, in show
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     image = self._client.call(context, 'get', image_id)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/cinder/image/glance.py", line 176, in call
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     return getattr(client.images, method)(*args, **kwargs)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 102, in get
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     resp, body = self.http_client.get(url)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 261, in get
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     return self._request('GET', url, **kwargs)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 248, in _request
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     body_iter = resp.json()
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/requests/models.py", line 741, in json
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     return json.loads(self.text, **kwargs)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 488, in loads
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     return _default_decoder.decode(s)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 370, in decode
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     obj, end = self.raw_decode(s)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api   File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 389, in raw_decode
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api     return self.scan_once(s, idx=_w(s, idx).end())
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2015-08-06 14:23:26.372 36113 TRACE cinder.volume.api
2015-08-06 14:23:26.374 36113 DEBUG taskflow.engines.action_engine.runner [req-ddb757de-85f5-4a1e-9999-36bff3afae7f - - - - -] Discarding failure 'Failure: simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)' (in response to event 'executed') under completion units request during completion of node 'cinder.volume.flows.api.create_volume.ExtractVolumeRequestTask;volume:create==1.0' (intention is to REVERT) analyze /usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/runner.py:181

vagrant virtualbox VM creation command VBoxManage

You can see this command if you run "ps -aux | grep VBoxManage" just after "vagrant up".

$/usr/lib/virtualbox/VBoxManage import /root/.vagrant.d/boxes/ubuntu-VAGRANTSLASH-trusty64/20151117.0.0/virtualbox/box.ovf --vsys 0 --vmname ubuntu-cloudimg-trusty-vagrant-amd64_1448828432010_1244 --vsys 0 --unit 6 --disk /root/VirtualBox VMs/ubuntu-cloudimg-trusty-vagrant-amd64_1448828432010_1244/box-disk1.vmdk

vagrant multinode setup

1)
Ubuntu 14.04.3 LTS
#sudo -i

2)
#export no_proxy='127.0.0.1,169.254.169.254,localhost'
#export http_proxy='http://100.140.192.30:10000/'
#export https_proxy='http://100.140.192.30:10000/'

3)
#apt-get update

4)
a)
#echo "deb http://download.virtualbox.org/virtualbox/debian trusty contrib" | sudo tee -a /etc/apt/sources.list

b)
#cat /etc/apt/sources.list | grep virtualbox
deb http://download.virtualbox.org/virtualbox/debian trusty contrib

5)
#apt-get update
GPG error: http://download.virtualbox.org trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54422A4B98AB5139

6)
#apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 54422A4B98AB5139

Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.yjM7CTppoR --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv-keys 54422A4B98AB5139
gpg: requesting key 98AB5139 from hkp server keyserver.ubuntu.com
gpg: key 98AB5139: public key "Oracle Corporation (VirtualBox archive signing key) " imported
gpg: Total number processed: 1
gpg:               imported: 1

7)
#apt-get update

8)
#apt-get install linux-headers-3.13.0-63-generic

9)
#apt-get install virtualbox-5.0

10)
Install vagrant
#wget --no-check-certificate https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.deb
#dpkg -i vagrant_1.7.4_x86_64.deb

11)
Install librarian-puppet
#apt-get install ruby
#gem install librarian-puppet-simple --no-ri --no-rdoc

12)
#apt-get install git
#git clone https://github.com/sajuptpm/puppet-mycloud

13)
#cd puppet-mycloud

14)
Install all puppet-mycloud dependencies from file "$pwd/Puppetfile" to "$pwd/modules"
#export pwd=`pwd`
#export git_protocol=https
#librarian-puppet install --puppetfile=$pwd/Puppetfile  --path=$pwd/modules

15)
Initialize Vagrant setup
#./vagrant_parallel_provision.sh initialize

* This command will provision a dhcp server and a separate vboxnet adapter.
* This command will provision "httpproxy1" server, that contains dhcp and proxy server.
* Before running the next command , ensure puppet has completed the provision of httpproxy server.

16)
Fix for Connection timeout:

httpproxy1: SSH address: 127.0.0.1:2222
httpproxy1: SSH username: vagrant
httpproxy1: SSH auth method: private key
httpproxy1: Warning: Connection timeout. Retrying...
httpproxy1: Warning: Connection timeout. Retrying...

* Check "/etc/resolv.conf" for correct "nameserver" entry.

17)
#bash vagrant_parallel_provision.sh up >> vagrant.log 2>&1 &

* The above commands , creates a new token for consul.The token and the "vboxnet" for your cluster can be found in the "vagrant_keys".

18)
In order to recreate the  vagrant environment:
#bash vagrant_parallel_provision.sh reset >> vagrant.log 2>&1 &
This will reprovision the entire setup in a new network and a new consul token.    

19)
In order to reprovision the environment by making changes in puppet-mycloud code;
#bash vagrant_parallel_provision.sh provision >> vagrant.log 2>&1 &

20)
In order to destroy the entire system
#bash vagrant_parallel_provision.sh destroy>> vagrant.log 2>&1 &

21)
In order to destroy and bring up a single server in the cluster
#vagrant destroy gcp1
#source vangrant_keys
#vagrant up gcp1

22)
In order to get the status of vagrant servers:
#vagrant status

23)
In order to login to a specific server:
#vagrant ssh gcp1

24)
In order to cleanup the environment:
#bash vagrant_parallel_provision.sh cleanup

under cloud ironic setup How to register a node to ironic

1)
Find ILO of machine which you want to configure
#nmap -sP 100.214.122.64/27

* 100.214.122.64/27 --> ILO network

2)
Goto ILO Web Interface of the machine
https://100.214.122.93/

3)
How to do ILO Reset to get new ILO IP from DHCP Server (isc-dhcp-server) running on UC ?
Ans:
Goto ILO Web Interface of the machine
https://100.214.122.93/
Then Select "Network" --> iLO Dedicated Network Port --> IPv4 --> Select "Enable DHCPv4" ---> Then "Submit" and "Reset".

4)
How to Check Storage ?
Ans:
Goto ILO Web Interface of the machine
https://100.214.122.93/
Then Select "Information" --> System Information --> Storage

5)
How to Enable "Network Boot" for "NIC-4" or "em4" ?

a)
First Open Remote Console::
Goto ILO Web Interface of the machine
https://100.214.122.93/
Then Select "Remote Console" --> "Remote Console" --> "Java Integrated Remote Console (Java IRC)" --> launch

b)
Then Reboot::
Select "Power Switch" --> "Cold Boot"

c)
Then Goto BIOS Setup and Enable "Network Boot" for "NIC-4" ?

F9 --> Enter --> Wait for ROM Utility --> Select "System Options" --> "Embedded NICs" --> "NIC 4 Boots Options" --> Press "Enter" --> Select "Network Boot"

OR

F11 --> Enter --> F9 --->  Wait for ROM Utility --> Select "System Options" --> "Embedded NICs" --> "NIC 4 Boots Options" --> Press "Enter" --> Select "Network Boot"

d)
Then Goto BIOS Setup and Disable "Network Boot" for "NIC 1", "NIC 2" and "NIC 3".

6)
RAID Array Configuraion

a)
Open RAID Array Configuraion Interface
Reboot --> Then press "F9" --> Then press "F5"

b)
Create "RAID 1" for Compute Node.
Select "RAID Array Configuraion (Slot 1)" --> "Configure" --> "Create Array" --> Select All HDDs --> "Create Array" --> "RAID 1" --> "Create Logical Drive" --> Finish

b1)
Set Bootable Logical Drive/Volume.
"Set Bootable Logical Drive/Volume" --> "Local - Logical Drive 1" --> Select "Primary Bootable Logical Drive/Volume" --> OK --> Finish

c)
Create "RAID 0" for Storage Node.
Select "RAID Array Configuraion (Slot 1)" --> "Configure" --> "Create Array" --> Select All HDDs --> "Create Array" --> "RAID 0" --> "Create Logical Drive" --> Finish

c1)
Set Bootable Logical Drive/Volume.
"Set Bootable Logical Drive/Volume" --> "Local - Logical Drive 1" --> Select "Primary Bootable Logical Drive/Volume" --> OK --> Finish

7)
Register that machine "100.214.122.93" to ironic.

a)
Goto UC and source "openrc_admin".

b)
Check nodes which are already registered to ironic.
root@uc1:~#ironic node-list

c)
Register the node "100.214.122.93" to ironic.
root@uc1:~#/usr/bin/python2.7 -m jiocloud.enroll --ilo_username myusername --ilo_password mypassword --os_username admin --os_tenant openstack --os_password mypassword --os_auth_url https://identity-uc.mycloud.com:5000/v2.0 --nic 4 --ilo_address 100.214.122.93

d)
Check newly registered node
root@uc1:~#ironic node-list

8)
Configure the ironic node.

a)
Configure for compute (based on requirement).
root@uc1:~#ironic node-update 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx replace properties/cpu_arch=x86_64.g1.compute

* 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx --> form $ironic node-list

b)
Configure for storage (based on requirement).
root@uc1:~#ironic node-update 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx replace properties/cpu_arch=x86_64.g1.storage

c)
Set "maintenance" to False
root@uc1:~#ironic node-update 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx replace maintenance=False

d)
Verify updated configurations
root@uc1:~#ironic node-show 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx

9)
Test node provision:
a)
Power off the node:
root@uc1:~#ironic node-set-power-state 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx off

b)
provision the node:

For Compute Node:
root@uc1:~#nova boot --flavor g1.compute --image Ubuntu-trusty --key-name mykey1 --nic net-id=f6e86850-32e9-4366-a03b-2b2a9ac481f0 compute-node1
* Right now, we can't provision VM on a specific ironic node  with option "--availability-zone nova:xxxx"

For Storage Node:
root@uc1:~#nova boot --flavor g1.storage --image Ubuntu-trusty --key-name mykey1 --nic net-id=f6e86850-32e9-4366-a03b-2b2a9ac481f0 storage-node1

c)
Verify:
root@uc1:~#nova list
root@uc1:~#ironic node-list

10)
Power Off/On (Optional):
root@uc1:~#ironic node-set-power-state 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx off
root@uc1:~#ironic node-set-power-state 2d32d1dd-5363-47d4-a5d6-dc3a1fe7f496xx on

11)
Validate
#ironic node-validate [node-id]

12)
How to Delete nova vm and ironic node
$nova delete [vm-id]
$ironic node-set-provision-state [node-id] deleted
$ironic node-delete [node-id]

Register quanta node
#################

1)
Create chassis
#ironic chassis-create

2)
Create node
#ironic help node-create

#ironic node-create -d pxe_ipmitool -c 2c860c23-20ef-4b21-bbf3-656047beb85c -i ipmi_address=100.234.122.92 -i ipmi_username=admin -i ipmi_password=admin -i ipmi_terminal_port=0 -p cpus=16 -p memory_mb=131072 -p local_gb=678 -p cpu_arch=x86_64

#ironic node-list

3)
Create port

3a)
Find MAC address

a)
Open ILO web interface.
Then Open Remote console
Then reboot the system via ILO web interface "Remote Control" --> Server Power control --> Reset Server --> Click on "Perform action"
Then Press "F11 or F12", then "Ctrl + S" to enter into setup menu (Bios)

b)
Enable PXE boot (Network boot)
"Advanced" --> "OnBoard Device Configuration" --> "Onboard LAN Port" --> "Enable with PXE"

3b)
#ironic help port-create

#ironic port-create -a 2C:60:0C:28:3B:92 -n 0aa9238b-39ed-45a6-a803-f20755748882

#Ironic port update, if you want to change the MAC address
#ironic port-update 18580aeb-81e0-4936-baab-d23a36e3c01a replace address=2c:60:0c:28:3b:91

4)
Ironic node Update
#ironic node-update 0aa9238b-39ed-45a6-a803-f20755748882 replace properties/cpu_arch=x86_64.g1.compute

#ironic node-update 0aa9238b-39ed-45a6-a803-f20755748882 replace maintenance=False

5)
Power-off the system
#ironic node-set-power-state 0aa9238b-39ed-45a6-a803-f20755748882 off

6)
Test nova boot
#nova boot --flavor g1.compute --image Ubuntu-trusty --key-name mykey1 --nic net-id=f6e86850-32e9-4366-a03b-2b2a9ac481f0 compute-node1-quanta

#nova list

#ironic port-list

nova-compute, fails with Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""

============ISSUE==========

nova-compute, fails with Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""

============FIX==========

* In compute node, open /etc/oslo/matchmaker_ring.json and remove invalid nodes from "conductor": and other sections.

$cat /etc/oslo/matchmaker_ring.json

{
  "cert": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "consoleauth": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "conductor": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "cinder-scheduler": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "compute": [
    "gcp1-test-dev-test"
  ],
  "cinder-volume": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "cinder-volume:oc1-test-dev-test": [
    "oc1-test-dev-test"
  ],
  "cinder-volume:ocdb1-test-dev-test": [
    "ocdb1-test-dev-test"
  ],
  "scheduler": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ]
}

================ERROR1=============

$/var/log/nova/nova-compute.log

2015-06-11 16:11:36.901 55059 TRACE nova.servicegroup.drivers.db
2015-06-11 16:11:36.904 55059 WARNING nova.openstack.common.loopingcall [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] task > run outlasted interval by 50.01 sec
2015-06-11 16:11:36.906 55059 WARNING oslo_messaging._drivers.impl_zmq [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:11:36.923 55059 ERROR nova.servicegroup.drivers.db [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] Recovered model server connection!
2015-06-11 16:11:46.906 55059 WARNING oslo_messaging._drivers.impl_zmq [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:12:04.374 55059 INFO nova.openstack.common.service [req-ea1d6e3f-e2e1-4edf-b5a4-1103fb679032 - - - - -] Caught SIGTERM, exiting
2015-06-11 16:12:56.826 18384 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:12:56.866 18384 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2015-06-11 16:12:56.885 18384 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:12:56.888 18384 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2015-06-11 16:12:56.963 18384 WARNING oslo_messaging._drivers.impl_zmq [req-d4cefc49-632f-4150-8d76-51a6786a4622 - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:24:40.795 58030 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:24:40.834 58030 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2015-06-11 16:24:40.853 58030 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:24:40.855 58030 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2015-06-11 16:24:40.930 58030 WARNING oslo_messaging._drivers.impl_zmq [req-c3c40f37-5f62-4c5a-9fce-14a87cd262e6 - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:25:40.935 58030 CRITICAL nova [req-c3c40f37-5f62-4c5a-9fce-14a87cd262e6 - - - - -] Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""
2015-06-11 16:25:40.935 58030 TRACE nova Traceback (most recent call last):
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/bin/nova-compute", line 10, in
2015-06-11 16:25:40.935 58030 TRACE nova     sys.exit(main())
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/cmd/compute.py", line 72, in main
2015-06-11 16:25:40.935 58030 TRACE nova     db_allowed=CONF.conductor.use_local)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/service.py", line 277, in create
2015-06-11 16:25:40.935 58030 TRACE nova     db_allowed=db_allowed)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/service.py", line 157, in __init__
2015-06-11 16:25:40.935 58030 TRACE nova     self.conductor_api.wait_until_ready(context.get_admin_context())
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 292, in wait_until_ready
2015-06-11 16:25:40.935 58030 TRACE nova     timeout=timeout)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/baserpc.py", line 62, in ping
2015-06-11 16:25:40.935 58030 TRACE nova     return cctxt.call(context, 'ping', arg=arg_p)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-06-11 16:25:40.935 58030 TRACE nova     retry=self.retry)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-06-11 16:25:40.935 58030 TRACE nova     timeout=timeout, retry=retry)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1033, in wrap
2015-06-11 16:25:40.935 58030 TRACE nova     return func(self, *args, **kws)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1065, in send
2015-06-11 16:25:40.935 58030 TRACE nova     return self._send(target, ctxt, message, wait_for_reply, timeout)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1055, in _send
2015-06-11 16:25:40.935 58030 TRACE nova     pooled=True)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 856, in _multi_send
2015-06-11 16:25:40.935 58030 TRACE nova     envelope, allowed_remote_exmods, pooled)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 781, in _call
2015-06-11 16:25:40.935 58030 TRACE nova     msg = msg_waiter.recv()
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 228, in recv
2015-06-11 16:25:40.935 58030 TRACE nova     return self.sock.recv_multipart(**kwargs)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 359, in recv_multipart
2015-06-11 16:25:40.935 58030 TRACE nova     return _Socket_recv_multipart(self, flags, copy, track)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/zmq/sugar/socket.py", line 265, in recv_multipart
2015-06-11 16:25:40.935 58030 TRACE nova     parts = [self.recv(flags, copy=copy, track=track)]
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 338, in recv
2015-06-11 16:25:40.935 58030 TRACE nova     self._eventlet_recv_event.block()
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 104, in block
2015-06-11 16:25:40.935 58030 TRACE nova     self._hub.switch()
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-06-11 16:25:40.935 58030 TRACE nova     return self.greenlet.switch()
2015-06-11 16:25:40.935 58030 TRACE nova Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""
2015-06-11 16:25:40.935 58030 TRACE nova
2015-06-11 16:26:26.905 64005 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:26:26.934 64005 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2015-06-11 16:26:26.948 64005 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:26:26.950 64005 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2015-06-11 16:26:27.007 64005 WARNING oslo_messaging._drivers.impl_zmq [req-c2be6b2c-2e4b-48f2-b2d2-395df63250e9 - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:27:27.011 64005 CRITICAL nova [req-c2be6b2c-2e4b-48f2-b2d2-395df63250e9 - - - - -] Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""


================ERROR2=============


2015-06-11 16:09:26.889 55059 TRACE nova.servicegroup.drivers.db
2015-06-11 16:09:26.892 55059 WARNING nova.openstack.common.loopingcall [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] task > run outlasted interval by 50.01 sec
2015-06-11 16:09:26.895 55059 WARNING oslo_messaging._drivers.impl_zmq [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:10:14.236 55059 ERROR nova.openstack.common.periodic_task [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Error during ComputeManager._run_pending_deletes: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task Traceback (most recent call last):
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/periodic_task.py", line 224, in run_periodic_tasks
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     task(self, context)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6543, in _run_pending_deletes
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     context, filters, expected_attrs=attrs, use_slave=True)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 161, in wrapper
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     args, kwargs)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 335, in object_class_action
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     objver=objver, args=args, kwargs=kwargs)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     retry=self.retry)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     timeout=timeout, retry=retry)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1033, in wrap
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return func(self, *args, **kws)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1065, in send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return self._send(target, ctxt, message, wait_for_reply, timeout)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1055, in _send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     pooled=True)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 856, in _multi_send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     envelope, allowed_remote_exmods, pooled)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 781, in _call
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     msg = msg_waiter.recv()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 228, in recv
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return self.sock.recv_multipart(**kwargs)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 359, in recv_multipart
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return _Socket_recv_multipart(self, flags, copy, track)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/zmq/sugar/socket.py", line 265, in recv_multipart
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     parts = [self.recv(flags, copy=copy, track=track)]
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 338, in recv
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     self._eventlet_recv_event.block()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 104, in block
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     self._hub.switch()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return self.greenlet.switch()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""


================ERROR3=============


2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task
2015-06-11 16:10:14.240 55059 DEBUG nova.openstack.common.periodic_task [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks /usr/lib/python2.7/dist-packages/nova/openstack/common/periodic_task.py:219
2015-06-11 16:10:14.240 55059 DEBUG nova.openstack.common.periodic_task [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Running periodic task ComputeManager._run_image_cache_manager_pass run_periodic_tasks /usr/lib/python2.7/dist-packages/nova/openstack/common/periodic_task.py:219
2015-06-11 16:10:14.242 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Acquired file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after waiting 0.000s acquire /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:212
2015-06-11 16:10:14.242 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" acquired by "do_register_storage_use" :: waited 0.001s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444
2015-06-11 16:10:14.244 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Releasing file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after holding it for 0.002s release /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:227
2015-06-11 16:10:14.244 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" released by "do_register_storage_use" :: held 0.002s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
2015-06-11 16:10:14.245 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Acquired file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after waiting 0.000s acquire /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:212
2015-06-11 16:10:14.246 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" acquired by "do_get_storage_users" :: waited 0.001s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444
2015-06-11 16:10:14.247 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Releasing file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after holding it for 0.001s release /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:227
2015-06-11 16:10:14.248 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" released by "do_get_storage_users" :: held 0.002s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
2015-06-11 16:10:14.250 55059 WARNING oslo_messaging._drivers.impl_zmq [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:10:14.252 55059 DEBUG oslo_concurrency.lockutils [-] Lock "fd97cc9e-dae2-4878-80a7-c2049ec3fcf5" released by "query_driver_power_state_and_sync" :: held 60.021s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
2015-06-11 16:10:14.253 55059 ERROR nova.compute.manager [-] [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5] Periodic sync_power_state task had an error while processing an instance.
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5] Traceback (most recent call last):
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6042, in _sync
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     query_driver_power_state_and_sync()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return f(*args, **kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6039, in query_driver_power_state_and_sync
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._query_driver_power_state_and_sync(context, db_instance)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6079, in _query_driver_power_state_and_sync
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     use_slave=True)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6095, in _sync_instance_power_state
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     db_instance.refresh(use_slave=use_slave)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 192, in wrapper
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._context, self, fn.__name__, args, kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 340, in object_action
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     objmethod=objmethod, args=args, kwargs=kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     retry=self.retry)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     timeout=timeout, retry=retry)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1033, in wrap
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return func(self, *args, **kws)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1065, in send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return self._send(target, ctxt, message, wait_for_reply, timeout)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1055, in _send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     pooled=True)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 856, in _multi_send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     envelope, allowed_remote_exmods, pooled)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 781, in _call
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     msg = msg_waiter.recv()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 228, in recv
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return self.sock.recv_multipart(**kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 359, in recv_multipart
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return _Socket_recv_multipart(self, flags, copy, track)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/zmq/sugar/socket.py", line 265, in recv_multipart
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     parts = [self.recv(flags, copy=copy, track=track)]
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 338, in recv
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._eventlet_recv_event.block()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 104, in block
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._hub.switch()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return self.greenlet.switch()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5] Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""

puppet how to manually run a class

1)
how to manually run a class
------------------------------

$echo 'include rjil::system::apt' | puppet apply
$echo 'include rjil::system::apt' | puppet apply --debug
$echo 'include rjil::system::apt' | puppet apply --debug ---noop

2)
How to add entry in /etc/hosts
---------------------------------------

$echo "include rjil::jiocloud::dns" | puppet apply

notify {"ip=${ip}, cname=${cname}, name=${name}":}  === print
notify {"ip=${ip}, ip_real=${ip_real}":}

Notice: ip=false, cname=keystone.service.consul, name=identity.jiocloud.com
Notice: ip=false, ip_real=10.140.15.69

/etc/puppet/modules.overrides/rjil/manifests/jiocloud/dns/entry.pp

3)
How consul check keystone service
-------------------------------------------------------
a)
$vim puppet-rjil/manifests/keystone.pp

  rjil::jiocloud::consul::service { "keystone":
    tags          => ['real'],
    port          => 5000,
  }

b)
$vim puppet-rjil/manifests/jiocloud/consul/service.pp

  file { "/etc/consul/$name.json":
    ensure => "present",
    content => template('rjil/consul.service.erb'),
  } ~> Exec <| title == 'reload-consul' |>

c)
vim /etc/consul/keystone.json

{
  "service": {
    "name": "keystone",
    "port": 5000,
    "tags": [
      "real"
    ],
    "check": {
      "script": "/usr/lib/jiocloud/tests/service_checks/keystone.sh",
      "interval": "10s"
    }
  }
}

* Consul will read this json file and execute the script in the "check" section, then register the service if check is success.

d)
$vim /usr/lib/jiocloud/tests/service_checks/keystone.sh

#!/bin/bash
set -e
/usr/lib/nagios/plugins/check_http -S -H 10.140.15.66 -p 5000

4)

ceph ubuntu repo PGP PUBLIC KEY

https://git.ceph.com/?p=ceph.git;a=blob_plain;f=keys/release.asc


1)
puppet hiera Example:
===========

rjil::system::apt::repositories:
  ceph:
    location: 'http://ceph.com/debian-hammer'
    release: 'trusty'
    repos: 'main'
    include_src: false
    key: '17ED316D'
    key_content: |
      -----BEGIN PGP PUBLIC KEY BLOCK-----
      Version: GnuPG v1.4.11 (GNU/Linux)
      mQINBFX4hgkBEADLqn6O+UFp+ZuwccNldwvh5PzEwKUPlXKPLjQfXlQRig1flpCH
      E0HJ5wgGlCtYd3Ol9f9+qU24kDNzfbs5bud58BeE7zFaZ4s0JMOMuVm7p8JhsvkU
      C/Lo/7NFh25e4kgJpjvnwua7c2YrA44ggRb1QT19ueOZLK5wCQ1mR+0GdrcHRCLr
      7Sdw1d7aLxMT+5nvqfzsmbDullsWOD6RnMdcqhOxZZvpay8OeuK+yb8FVQ4sOIzB
      FiNi5cNOFFHg+8dZQoDrK3BpwNxYdGHsYIwU9u6DWWqXybBnB9jd2pve9PlzQUbO
      eHEa4Z+jPqxY829f4ldaql7ig8e6BaInTfs2wPnHJ+606g2UH86QUmrVAjVzlLCm
      nqoGymoAPGA4ObHu9X3kO8viMBId9FzooVqR8a9En7ZE0Dm9O7puzXR7A1f5sHoz
      JdYHnr32I+B8iOixhDUtxIY4GA8biGATNaPd8XR2Ca1hPuZRVuIiGG9HDqUEtXhV
      fY5qjTjaThIVKtYgEkWMT+Wet3DPPiWT3ftNOE907e6EWEBCHgsEuuZnAbku1GgD
      LBH4/a/yo9bNvGZKRaTUM/1TXhM5XgVKjd07B4cChgKypAVHvef3HKfCG2U/DkyA
      LjteHt/V807MtSlQyYaXUTGtDCrQPSlMK5TjmqUnDwy6Qdq8dtWN3DtBWQARAQAB
      tCpDZXBoLmNvbSAocmVsZWFzZSBrZXkpIDxzZWN1cml0eUBjZXBoLmNvbT6JAjgE
      EwECACIFAlX4hgkCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEOhKwsBG
      DzmUXdIQAI8YPcZMBWdv489q8CzxlfRIRZ3Gv/G/8CH+EOExcmkVZ89mVHngCdAP
      DOYCl8twWXC1lwJuLDBtkUOHXNuR5+Jcl5zFOUyldq1Hv8u03vjnGT7lLJkJoqpG
      l9QD8nBqRvBU7EM+CU7kP8+09b+088pULil+8x46PwgXkvOQwfVKSOr740Q4J4nm
      /nUOyTNtToYntmt2fAVWDTIuyPpAqA6jcqSOC7Xoz9cYxkVWnYMLBUySXmSS0uxl
      3p+wK0lMG0my/gb+alke5PAQjcE5dtXYzCn+8Lj0uSfCk8Gy0ZOK2oiUjaCGYN6D
      u72qDRFBnR3jaoFqi03bGBIMnglGuAPyBZiI7LJgzuT9xumjKTJW3kN4YJxMNYu1
      FzmIyFZpyvZ7930vB2UpCOiIaRdZiX4Z6ZN2frD3a/vBxBNqiNh/BO+Dex+PDfI4
      TqwF8zlcjt4XZ2teQ8nNMR/D8oiYTUW8hwR4laEmDy7ASxe0p5aijmUApWq5UTsF
      +s/QbwugccU0iR5orksM5u9MZH4J/mFGKzOltfGXNLYI6D5Mtwrnyi0BsF5eY0u6
      vkdivtdqrq2DXY+ftuqLOQ7b+t1RctbcMHGPptlxFuN9ufP5TiTWSpfqDwmHCLsT
      k2vFiMwcHdLpQ1IH8ORVRgPPsiBnBOJ/kIiXG2SxPUTjjEGOVgeA
      =/Tod
      -----END PGP PUBLIC KEY BLOCK-----

How to github Checking out pull requests locally

1)
fetch pull request 761 from remote "jiocloud_remote" and save in local branch "temp_branch".
$git fetch jiocloud_remote pull/761/head:temp_branch

2)
Then got to that branch where you want to apply changes in this pull request.
$git checkout mydevbranch

3)
Merge the Pull Request
$git merge -m 'Merging Pull Request' temp_branch

4)
If there is any conflict during merge then do following steps.
a) Fix conflict
b) $git add fixed_file
c) $git commit -a

5)
Done

How to Install Ubuntu Packages from a Specific Repository

1)
$apt-cache policy python-ironicclient
python-ironicclient:
  Installed: (none)
  Candidate: 0.5.1-0ubuntu1~cloud0
  Version table:
     1:2014.2.245 0
        400 http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing/main amd64 Packages
     0.5.1-0ubuntu1~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main amd64 Packages
     0.1.2-0ubuntu1 0
        500 http://in.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

2)
$apt-get install python-ironicclient/trusty-testing

OR

$apt-get -t trusty-testing python-ironicclien

Apt-Pinning Assign highest priority to a repository

1)
$vim /etc/apt/sources.list.d/rustedhalo.list
deb http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing main

2)
$apt-cache policy python-ironicclient
python-ironicclient:
  Installed: 0.5.1-0ubuntu1~cloud0
  Candidate: 1:2014.2.245 ====imp
  Version table:
     1:2014.2.245 0
        500 http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing/main amd64 Packages
 *** 0.5.1-0ubuntu1~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main amd64 Packages
        100 /var/lib/dpkg/status
     0.1.2-0ubuntu1 0
        500 http://in.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

3)
$vim /etc/apt/preferences.d/my_test_pref

Package: *
Pin: release a=trusty-testing
Pin-Priority: 400

4)
$apt-get update

5)
$apt-cache policy python-ironicclient
python-ironicclient:
  Installed: 0.5.1-0ubuntu1~cloud0
  Candidate: 0.5.1-0ubuntu1~cloud0 ====imp
  Version table:
     1:2014.2.245 0
        400 http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing/main amd64 Packages
 *** 0.5.1-0ubuntu1~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main amd64 Packages
        100 /var/lib/dpkg/status
     0.1.2-0ubuntu1 0
        500 http://in.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

How to fix unmet dependencies of a deb package

1)
$cd /home/saju/pkgoverride

$wget http://jiocloud.rustedhalo.com/ubuntu/pool/main/r/rjil-cicd/rjil-cicd_2014.2.234_all.deb

2)
http://serverfault.com/questions/250224/how-do-i-get-apt-get-to-ignore-some-dependencies
a)
Unpack:
$ar x rjil-cicd_2014.2.234_all.deb

b)
Unpack control archive:
$tar xzf control.tar.gz

c)
Fix dependencies in control (use a text editor)
$vim control

d)
Repack control.tar.gz:
$tar c {post,pre}{inst,rm} md5sums control | gzip -c > control.tar.gz

e)
Repack deb:
$ar rcs newpackage.deb debian-binary control.tar.gz data.tar.xz

f)
$dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

3)
$vim /etc/apt/sources.list.d/overrides.list
deb [trusted=yes] file:/home/saju/pkgoverride ./

4)
$apt-get update

5)
$apt-cache policy rjil-cicd

6)
$apt-get install rjil-cicd

How to OpenContrail Check Compute node's peer with Controller Node

How to Contrail Check Compute node's peer with Controller Node

1)
Method-1:

Check Compute Agent's Xmpp Connection Status.
It should show an entry which contains IP of Controller node.

http://ip-of-compute-node:8085/Snh_AgentXmppConnectionStatusReq

Example:

Here IP of compute nodes 10.140.15.57 and 10.140.15.59
Here IP of controller node is 10.140.15.60

2)
Method-2:

Check Bgp Neighbor List.
It should contains entry for all Compute nodes.

http://ip-of-contrail-controller-node:8083/Snh_BgpNeighborReq

Example:

* Here IP of Compute nodes which are peered with controller is 10.140.15.57 and 10.140.15.59
* Here IP of controller node is 10.140.15.60
* I have created a VM in cp1 node and for that VM private IP assigned from "net_tempest" network and floating IP assigned from "public" network.So we can see routing instance of these two networks here (see the last picture).

3)
If you are not seeing entries in above urls, then check for "discovery server" IP in contrail-vrouter-agent.conf

$vim /etc/contrail/contrail-vrouter-agent.conf

# IP address of discovery server
server=10.140.15.78

4)

How to git cherry-pick commit from another/upstream repository

1)
Clone forked repo
#git clone https://github.com/JioCloud/contrail-third-party.git
#cd contrail-third-party.git

2)
Check all existing branches
#git branch
  branch1
  branch2
* master

3)
Add upstream remote (upstream repo) and name it "upstream_remote"
#git remote add upstream_remote https://github.com/Juniper/contrail-third-party.git

4)
Check all remotes
#git remote -v
upstream_remote    https://github.com/Juniper/contrail-third-party (fetch)
upstream_remote    https://github.com/Juniper/contrail-third-party (push)
origin    https://github.com/JioCloud/contrail-third-party (fetch)
origin    https://github.com/JioCloud/contrail-third-party (push)

5)
Fetch the latest version of master (fetch all branches) from remote "upstream_remote"
#git fetch upstream_remote

6)
Goto the branch where you want to apply the commits.
$git checkout R1.10
* This "R1.10" branch is in "origin" remote

7)
cherry-pick the commit from newly added  upstream_remote and apply.
$git cherry-pick [commit-id]

* You can use $git log command to find the commit-id from upstream_remote branches.
* Example:
   $git log -n 5 upstream_remote/branch_name

OpenStack zeromq send_multipart code trace from oslo.messaging._drivers.impl_zmq to eventlet.green.zmq to pyzmq

OpenStack zeromq send_multipart code trace from oslo.messaging._drivers.impl_zmq to eventlet.green.zmq to pyzmq

1)
https://github.com/JioCloud/oslo.messaging/blob/stable/icehouse/oslo/messaging/_drivers/impl_zmq.py#L216

zmq = importutils.try_import('eventlet.green.zmq') <=====1

class ZmqSocket(object):
    """A tiny wrapper around ZeroMQ.

    Simplifies the send/recv protocol and connection management.
    Can be used as a Context (supports the 'with' statement).
    """
    def __init__(self, addr, zmq_type, bind=True, subscribe=None):
        self.sock = _get_ctxt().socket(zmq_type) <=====4

    def send(self, data, **kwargs):
        if not self.can_send:
            raise RPCException(_("You cannot send on this socket."))
        self.sock.send_multipart(data, **kwargs) <====8

def _get_ctxt():
    if not zmq:
        raise ImportError("Failed to import eventlet.green.zmq")
    global ZMQ_CTX
    if not ZMQ_CTX:
        ZMQ_CTX = zmq.Context(CONF.rpc_zmq_contexts) <=====2
    return ZMQ_CTX

2)
https://github.com/eventlet/eventlet/blob/master/eventlet/green/zmq.py#L302

__zmq__ = __import__('zmq')
_Socket = __zmq__.Socket
_Socket_send_multipart = _Socket.send_multipart<=====10

class Context(__zmq__.Context): <=====3
    """Subclass of :class:`zmq.core.context.Context`
    """
    def socket(self, socket_type): <=====5
        """Overridden method to ensure that the green version of socket is used

        Behaves the same as :meth:`zmq.core.context.Context.socket`, but ensures
        that a :class:`Socket` with all of its send and recv methods set to be
        non-blocking is returned
        """
        if self.closed:
            raise ZMQError(ENOTSUP)
        return Socket(self, socket_type) <=====6

class Socket(_Socket): <=====7
    @_wraps(_Socket.send_multipart)<=====9
    def send_multipart(self, msg_parts, flags=0, copy=True, track=False): <=====12
        """A send_multipart method that's safe to use when multiple
        greenthreads are calling send, send_multipart, recv and
        recv_multipart on the same socket.
        """
        if flags & NOBLOCK:
            return _Socket_send_multipart(self, msg_parts, flags, copy, track) <=====13/or

        # acquire lock here so the subsequent calls to send for the
        # message parts after the first don't block
        with self._eventlet_send_lock:
            return _Socket_send_multipart(self, msg_parts, flags, copy, track) <=====13/or

def _wraps(source_fn): <=====11
    """A decorator that copies the __name__ and __doc__ from the given
    function
    """
    def wrapper(dest_fn):
        dest_fn.__name__ = source_fn.__name__
        dest_fn.__doc__ = source_fn.__doc__
        return dest_fn
    return wrapper

3)
https://github.com/zeromq/pyzmq/blob/4b8c8a680bb86fc6e0a82447abd7821f046a026a/zmq/sugar/socket.py#L47

import zmq
from zmq.backend import Socket as SocketBase <======17

class Socket(SocketBase, AttributeSetter):
    def send_multipart(self, msg_parts, flags=0, copy=True, track=False): <=====14
        for i,msg in enumerate(msg_parts):
            if isinstance(msg, (zmq.Frame, bytes, _buffer_type)):
                continue
            try:
                _buffer_type(msg)
            except Exception as e:
                rmsg = repr(msg)
                if len(rmsg) > 32:
                    rmsg = rmsg[:32] + '...'
                raise TypeError(
                    "Frame %i (%s) does not support the buffer interface." % (
                    i, rmsg,
                ))
        for msg in msg_parts[:-1]: <======15
            self.send(msg, SNDMORE|flags, copy=copy, track=track) <======16
        # Send the last part without the extra SNDMORE flag.
        return self.send(msg_parts[-1], flags, copy=copy, track=track)   

__all__ = ['Socket'] <=====

4)
https://github.com/zeromq/pyzmq/blob/master/zmq/backend/__init__.py

from .select import public_api, select_backend

backend = os.environ['PYZMQ_BACKEND'] <====a (Find Backend 'cython' or 'cffi')
if backend in ('cython', 'cffi'):
    backend = 'zmq.backend.%s' % backend

_ns = select_backend(backend) <====b (Load backend api modules)
globals().update(_ns)

__all__ = public_api

5)
https://github.com/zeromq/pyzmq/blob/master/zmq/backend/select.py

public_api = [  <====c (Backend api modules list)
    'Context',
    'Socket',
    ... ...
    ]
   
def select_backend(name):
    mod = __import__(name, fromlist=public_api) <====d (Load backend api modules)

6)
Here, selected backend is "cython"
https://github.com/zeromq/pyzmq/tree/master/zmq/backend/cython

7)
https://github.com/zeromq/pyzmq/blob/master/zmq/backend/cython/socket.pyx#L619

cdef class Socket: <======18
    cpdef object send(self, object data, int flags=0, copy=True, track=False): <======19
        _check_closed(self)
       
        if isinstance(data, unicode):
            raise TypeError("unicode not allowed, use send_string")
       
        if copy:
            # msg.bytes never returns the input data object
            # it is always a copy, but always the same copy
            if isinstance(data, Frame):
                data = data.buffer
            return _send_copy(self.handle, data, flags) <======20
        else:
            if isinstance(data, Frame):
                if track and not data.tracker:
                    raise ValueError('Not a tracked message')
                msg = data
            else:
                msg = Frame(data, track=track)
            return _send_frame(self.handle, msg, flags)
           
cdef inline object _send_copy(void *handle, object msg, int flags=0): <======21
    rc = zmq_msg_init_size(&data, msg_c_len)
    _check_rc(rc)
   
    while True:
        with nogil:
            memcpy(zmq_msg_data(&data), msg_c, zmq_msg_size(&data))
            rc = zmq_msg_send(&data, handle, flags) <======22
            if not rc < 0:
                rc2 = zmq_msg_close(&data)
        try:
            _check_rc(rc)
        except InterruptedSystemCall:
            continue
        else:
            break
    _check_rc(rc2)