contrail generate python source from instance_service.thrift

1)
$sudo apt-get install thrift-compiler

2)
$git clone https://github.com/Juniper/contrail-controller
$cd contrail-controller/src/vnsw/agent/openstack

3)
contrail-controller/src/vnsw/agent/openstack$ ls
instance_service_server.cc  instance_service_server.h  instance_service.thrift  SConscript  test

4)
contrail-controller/src/vnsw/agent/openstack$ thrift --gen py instance_service.thrift

5)
contrail-controller/src/vnsw/agent/openstack$ ls
gen-py  instance_service_server.cc  instance_service_server.h  instance_service.thrift  SConscript  test

6)
contrail-controller/src/vnsw/agent/openstack$ ls gen-py/
__init__.py  instance_service

7)
contrail-controller/src/vnsw/agent/openstack$ ls gen-py/instance_service/
constants.py  __init__.py  InstanceService.py  InstanceService-remote  ttypes.py

Trying to register the VirtualBox kernel modules using DKMSError! Your kernel headers for kernel 3.13.0-63-generic cannot be found

1)
$ sudo -E apt-get install virtualbox-5.0
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libgsoap4 libvncserver0
Use 'apt-get autoremove' to remove them.
Recommended packages:
  libsdl-ttf2.0-0 pdf-viewer
The following NEW packages will be installed:
  virtualbox-5.0
0 upgraded, 1 newly installed, 0 to remove and 56 not upgraded.
Need to get 0 B/63.1 MB of archives.
After this operation, 154 MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously unselected package virtualbox-5.0.
(Reading database ... 85359 files and directories currently installed.)
Preparing to unpack .../virtualbox-5.0_5.0.8-103449~Ubuntu~trusty_amd64.deb ...
Unpacking virtualbox-5.0 (5.0.8-103449~Ubuntu~trusty) ...
Processing triggers for ureadahead (0.100.0-16) ...
Processing triggers for shared-mime-info (1.2-0ubuntu3) ...
Processing triggers for mime-support (3.54ubuntu1.1) ...
Setting up virtualbox-5.0 (5.0.8-103449~Ubuntu~trusty) ...
addgroup: The group `vboxusers' already exists as a system group. Exiting.
Stopping VirtualBox kernel modules ...done.
Uninstalling old VirtualBox DKMS kernel modules ...done.
Trying to register the VirtualBox kernel modules using DKMSError! Your kernel headers for kernel 3.13.0-63-generic cannot be found.
Please install the linux-headers-3.13.0-63-generic package,
or use the --kernelsourcedir option to tell DKMS where it's located
 ...failed!
  (Failed, trying without DKMS)
Recompiling VirtualBox kernel modules ...failed!
  (Look at /var/log/vbox-install.log to find out what went wrong)
$

2)
$ sudo -E apt-get install linux-headers-3.13.0-63-generic
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libgsoap4 libvncserver0
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  linux-headers-3.13.0-63
The following NEW packages will be installed:
  linux-headers-3.13.0-63 linux-headers-3.13.0-63-generic
0 upgraded, 2 newly installed, 0 to remove and 56 not upgraded.
Need to get 9,579 kB of archives.
After this operation, 76.8 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://in.archive.ubuntu.com/ubuntu/ trusty-security/main linux-headers-3.13.0-63 all 3.13.0-63.103 [8,874 kB]
Get:2 http://in.archive.ubuntu.com/ubuntu/ trusty-security/main linux-headers-3.13.0-63-generic amd64 3.13.0-63.103 [705 kB]
Fetched 9,579 kB in 0s (43.6 MB/s)                      
Selecting previously unselected package linux-headers-3.13.0-63.
(Reading database ... 86137 files and directories currently installed.)
Preparing to unpack .../linux-headers-3.13.0-63_3.13.0-63.103_all.deb ...
Unpacking linux-headers-3.13.0-63 (3.13.0-63.103) ...
Selecting previously unselected package linux-headers-3.13.0-63-generic.
Preparing to unpack .../linux-headers-3.13.0-63-generic_3.13.0-63.103_amd64.deb ...
Unpacking linux-headers-3.13.0-63-generic (3.13.0-63.103) ...
Setting up linux-headers-3.13.0-63 (3.13.0-63.103) ...
Setting up linux-headers-3.13.0-63-generic (3.13.0-63.103) ...
Examining /etc/kernel/header_postinst.d.
run-parts: executing /etc/kernel/header_postinst.d/dkms 3.13.0-63-generic /boot/vmlinuz-3.13.0-63-generic
$

3)
$ sudo -E apt-get install virtualbox-5.0
Reading package lists... Done
Building dependency tree      
Reading state information... Done
virtualbox-5.0 is already the newest version.
The following packages were automatically installed and are no longer required:
  libgsoap4 libvncserver0
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 56 not upgraded.
$

4)
$ sudo -E apt-get purge virtualbox-5.0
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libaudio2 libcaca0 libgsoap4 libice6 libqt4-declarative libqt4-network
  libqt4-opengl libqt4-script libqt4-sql libqt4-xml libqt4-xmlpatterns
  libqtcore4 libqtdbus4 libqtgui4 libsdl1.2debian libsm6 libvncserver0 libvpx1
  libxmu6 libxt6 qtcore4-l10n
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
  virtualbox-5.0*
0 upgraded, 0 newly installed, 1 to remove and 56 not upgraded.
After this operation, 154 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 110957 files and directories currently installed.)
Removing virtualbox-5.0 (5.0.8-103449~Ubuntu~trusty) ...
Purging configuration files for virtualbox-5.0 (5.0.8-103449~Ubuntu~trusty) ...
Processing triggers for mime-support (3.54ubuntu1.1) ...
Processing triggers for shared-mime-info (1.2-0ubuntu3) ...
$

5)
$ sudo -E apt-get install virtualbox-5.0
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libgsoap4 libvncserver0
Use 'apt-get autoremove' to remove them.
Recommended packages:
  libsdl-ttf2.0-0 pdf-viewer
The following NEW packages will be installed:
  virtualbox-5.0
0 upgraded, 1 newly installed, 0 to remove and 56 not upgraded.
Need to get 0 B/63.1 MB of archives.
After this operation, 154 MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously unselected package virtualbox-5.0.
(Reading database ... 110179 files and directories currently installed.)
Preparing to unpack .../virtualbox-5.0_5.0.8-103449~Ubuntu~trusty_amd64.deb ...
Unpacking virtualbox-5.0 (5.0.8-103449~Ubuntu~trusty) ...
Processing triggers for ureadahead (0.100.0-16) ...
Processing triggers for shared-mime-info (1.2-0ubuntu3) ...
Processing triggers for mime-support (3.54ubuntu1.1) ...
Setting up virtualbox-5.0 (5.0.8-103449~Ubuntu~trusty) ...
addgroup: The group `vboxusers' already exists as a system group. Exiting.
Stopping VirtualBox kernel modules ...done.
Uninstalling old VirtualBox DKMS kernel modules ...done.
Trying to register the VirtualBox kernel modules using DKMS ...done.
Starting VirtualBox kernel modules ...done.
$

GPG error: http://download.virtualbox.org trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54422A4B98AB5139

1)
$ cat /etc/apt/sources.list
# Repos managed by puppet.
deb http://download.virtualbox.org/virtualbox/debian trusty contrib

2)
$ sudo -E apt-get update
Get:1 http://download.virtualbox.org trusty InRelease [7,152 B]
Ign http://download.virtualbox.org trusty InRelease <====IMP
....
Hit http://in.archive.ubuntu.com trusty/multiverse amd64 Packages
Fetched 7,152 B in 4s (1,451 B/s)           
Reading package lists... Done
W: GPG error: http://download.virtualbox.org trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54422A4B98AB5139
$

3)
$ sudo -E apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 54422A4B98AB5139
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.KEq2FDuXPb --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d/puppetlabs-keyring.gpg --keyring /etc/apt/trusted.gpg.d/puppetlabs-nightly-keyring.gpg --keyring /etc/apt/trusted.gpg.d/rustedhalo.gpg --keyserver keyserver.ubuntu.com --recv-keys 54422A4B98AB5139
gpg: requesting key 98AB5139 from hkp server keyserver.ubuntu.com
gpg: key 98AB5139: public key "Oracle Corporation (VirtualBox archive signing key) " imported
gpg: Total number processed: 1
gpg:               imported: 1
$

4)
$ sudo -E apt-get update
Get:1 http://download.virtualbox.org trusty InRelease [7,152 B]
Get:2 http://download.virtualbox.org trusty/contrib amd64 Packages [1,545 B]  <====IMP
....
Hit http://in.archive.ubuntu.com trusty/multiverse amd64 Packages
Fetched 8,697 B in 4s (1,760 B/s)           
Reading package lists... Done
$

scons example

https://github.com/Juniper/contrail-installer/blob/master/contrail.sh

In contrail.sh

TARGET=${TARGET:-production}
SCONS_ARGS="--opt=$TARGET"

###fetch-packages###
sudo scons $SCONS_ARGS controller/src/vnsw
sudo scons $SCONS_ARGS vrouter

###Build/Install###
sudo scons $SCONS_ARGS --root=/ controller/src/vnsw install
sudo scons $SCONS_ARGS --root=/ vrouter install
sudo scons $SCONS_ARGS --root=/ openstack/nova_contrail_vif install

https://bitbucket.org/scons/scons/wiki/SconsTutorial1

http://www.scons.org/doc/HTML/scons-man.html

Contrail Never Ending polling from ifmap-server by contrail-controller, VM details not coming at contrail-control node from ifmap-server, VM's link local address missing at compute node

Fix
======
Restart cassandra
$service cassandra restart
OR
Check cassandra connection Pool

Symptoms
=========
1)
Never Ending polling by contrail-controller from ifmap-server
$tail -f /var/log/contrail/contrail-control.log

100.1.0.024    100.1.0.1100.1.0.275de43f2-93da-481e-a0d1-c4aa96a3f113true100.1.0.11100.1.0.254    truepublic_subnet controller/src/ifmap/client/ifmap_channel.cc 584
2015-06-23 Tue 15:33:45:315.863 UTC  ct1-test-dev-test [Thread 140311640958720, Pid 59613]: SANDESH: Send FAILED: 1435073625315829 IFMapPeer [SYS_DEBUG]: IFMapChannelProcResp: Http header length is 75 Content length is 1351 Total bytes read are 75 controller/src/ifmap/client/ifmap_channel.cc 679
2015-06-23 Tue 15:33:45:316.137 UTC  ct1-test-dev-test [Thread 140311334471424, Pid 59613]: SANDESH: Send FAILED: 1435073625316120 IFMapPeer [SYS_DEBUG]: IFMapServerConnection: 0 bytes in reply_. 1426 bytes in reply_str. PollResponse message is:
 HTTP/1.1 200
Content-Type: application/soap+xml
Content-Length: 1351


99.0.0.024    99.0.0.199.0.0.28f442855-b940-46b3-9e70-ae596e3ce644truetrueprivate_shared_subnet controller/src/ifmap/client/ifmap_channel.cc 584
2015-06-23 Tue 15:33:45:656.135 UTC  ct1-test-dev-test [Thread 140311137343232, Pid 59613]: SANDESH: Send FAILED: 1435073625656087 IFMapPeer [SYS_DEBUG]: IFMapChannelProcResp: Http header length is 75 Content length is 1280 Total bytes read are 75 controller/src/ifmap/client/ifmap_channel.cc 679
2015-06-23 Tue 15:33:45:656.338 UTC  ct1-test-dev-test [Thread 140311321876224, Pid 59613]: SANDESH: Send FAILED: 1435073625656320 IFMapPeer [SYS_DEBUG]: IFMapServerConnection: 0 bytes in reply_. 1355 bytes in reply_str. PollResponse message is:
 HTTP/1.1 200
Content-Type: application/soap+xml
Content-Length: 1280

2)
VM details not coming at contrail-control node from ifmap-server
$tail -f /var/log/contrail/contrail-control.log

3)
VM's Link local address missing at compute node
$route -n

OpenContrail Compile and Build with debug enabled

https://github.com/Juniper/contrail-controller/blob/8cc4abe1d8422ce515673a79bc614b27c4fb088e/lib/hiredis/SConscript#L14
env.Append(CCFLAGS = ['-g'])

https://github.com/Juniper/contrail-controller/blob/c12548ab0adaf3706987a954529e5315f0c5336d/src/query_engine/SConscript#L153
env.AppendUnique(CCFLAGS='-g')

http://www.scons.org/doc/0.96/HTML/scons-user/c972.html

http://pages.cs.wisc.edu/~driscoll/software/scons/variables.html

OpenContrail How to Check Flow Records

1)
pkt Introspect (Compute Node)
-------------------------------
a)
FetchAllFlowRecords

http://compute-node-ip:8085/Snh_FetchAllFlowRecords #==== VIMP

* Ping to Public_IP or Private_IP to see the flow records

sip : 10.140.15.10  #==== IP of GCP node
dip : 100.1.0.11 #==== Public IP
action : 4   
drop_reason : Interface unavialable  #==== ISSUE (Interface of VM unavialable)
short_flow : yes (Interface unavialable) #==== ISSUE (Interface of VM unavialable)

b)
KDropStats

http://compute-node-ip:8085/Snh_KDropStatsReq

ds_flow_action_drop    : 683


2)

Debugging with GDB C++

1)
$gdb
* Open gdb

2)
(gdb) attach [process-id]
* Attach to a process

* Detach process from GDB before quiting GDB.
(gdb) detach [process-id]
(gdb) quit

3)
(gdb) info files

4)
(gdb) info function

5)
(gdb) b FlowEntry::DoPolicy
* Add breakpoint

6)
(gdb) info breakpoints
(gdb) info break
* A backtrace is a summary of how your program got where it is. It shows one line per frame, for many frames, starting with the currently executing frame (frame zero), followed by its caller (frame one), and on up the stack.
* https://sourceware.org/gdb/onlinedocs/gdb/Backtrace.html

7)
(gdb) del 2
* Delete a breakpoint

8)
(gdb) run/r
* Let it run and stop at breakpoint.

9)
(gdb) bt
OR
(gdb) backtrace
* Print the location

10)
(gdb) next
* If you want to execute the entire function with one keypress, type "next" or "n".

11)
(gdb) step
* To execute one line of code, type "step" or "s". If the line to be executed is a function call, gdb will step into that function and start executing its code one line at a time.

12)
* Print variable
(gdb) p var1
(gdb) print var1

* Run following command to format the print output
(gdb) set print pretty

13)
* Print current line and line number with few lines before and after
(gdb) list *$rip

14)
* Print current line
(gdb) frame

15)
* Print current line and more info with the function name
(gdb) where

16)
* Display a few lines of your source program around the current location
(gdb) l
OR
(gdb) list

17)
* print the data type of a variable
* You can even use ptype to look at structures
(gdb) ptype var1
OR
(gdb) pt var1

OpenContrail TypeError: string indices must be integers, not str self._create_prop(bch, obj_ids['uuid'], 'autonomous_system', field)

====ERROR1====

$vim /var/log/upstart/contrail-api.log
$vim /var/log/contrail/contrail-api.log

-------------
Traceback (most recent call last):
  File "/usr/bin/contrail-api", line 9, in
    load_entry_point('vnc-cfg-api-server==0.1dev', 'console_scripts', 'contrail-api')()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1592, in server_main
    main()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1551, in main
    vnc_api_server = VncApiServer(args_str)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 437, in __init__
    self._db_init_entries()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1089, in _db_init_entries
    config_version=CONFIG_VERSION))
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1140, in _create_singleton_entry
    self._db_conn.dbe_create(obj_type, obj_ids, obj_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1705, in wrapper2
    ret = func(self, obj_type, obj_ids, obj_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1723, in dbe_create
    (ok, result) = method(obj_ids, obj_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/gen/vnc_cassandra_client_gen.py", line 4033, in _cassandra_global_system_config_create
    self._create_prop(bch, obj_ids['uuid'], 'autonomous_system', field)
TypeError: string indices must be integers, not str

====ERROR2====

$vim /var/log/upstart/contrail-api.log
$vim /var/log/contrail/contrail-api.log

-------------
Traceback (most recent call last):
  File "/usr/bin/contrail-api", line 9, in
    load_entry_point('vnc-cfg-api-server==0.1dev', 'console_scripts', 'contrail-api')()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1595, in server_main
    main()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1554, in main
    vnc_api_server = VncApiServer(args_str)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 437, in __init__
    self._db_init_entries()
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1092, in _db_init_entries
    VirtualNetwork(cfgm_common.IP_FABRIC_VN_FQ_NAME[-1]))
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 1143, in _create_singleton_entry
    self._db_conn.dbe_create(obj_type, obj_ids, obj_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1705, in wrapper2
    ret = func(self, obj_type, obj_ids, obj_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1723, in dbe_create
    (ok, result) = method(obj_ids, obj_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/gen/vnc_cassandra_client_gen.py", line 9941, in _cassandra_virtual_network_create
    parent_uuid = self.fq_name_to_uuid(parent_method_type, parent_fq_name)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 976, in fq_name_to_uuid
    raise NoIdError('%s %s' % (obj_type, fq_name))
NoIdError: Unknown id: project [u'default-domain', u'default-project']

====FIX====

[] rmr  /fq-name-to-uuid

Fix OpenStack Nova instance stuck in BUILD scheduling state

1)
Check "/etc/oslo/matchmaker_ring.json" and remove invalid nodes
$vim /etc/oslo/matchmaker_ring.json

2)
Restart nova-conductor
$service nova-conductor restart

3)
Try to launch a new VM

nova-compute, fails with Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""

============ISSUE==========

nova-compute, fails with Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""

============FIX==========

* In compute node, open /etc/oslo/matchmaker_ring.json and remove invalid nodes from "conductor": and other sections.

$cat /etc/oslo/matchmaker_ring.json

{
  "cert": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "consoleauth": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "conductor": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "cinder-scheduler": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "compute": [
    "gcp1-test-dev-test"
  ],
  "cinder-volume": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ],
  "cinder-volume:oc1-test-dev-test": [
    "oc1-test-dev-test"
  ],
  "cinder-volume:ocdb1-test-dev-test": [
    "ocdb1-test-dev-test"
  ],
  "scheduler": [
    "oc1-test-dev-test",
    "ocdb1-test-dev-test"
  ]
}

================ERROR1=============

$/var/log/nova/nova-compute.log

2015-06-11 16:11:36.901 55059 TRACE nova.servicegroup.drivers.db
2015-06-11 16:11:36.904 55059 WARNING nova.openstack.common.loopingcall [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] task > run outlasted interval by 50.01 sec
2015-06-11 16:11:36.906 55059 WARNING oslo_messaging._drivers.impl_zmq [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:11:36.923 55059 ERROR nova.servicegroup.drivers.db [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] Recovered model server connection!
2015-06-11 16:11:46.906 55059 WARNING oslo_messaging._drivers.impl_zmq [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:12:04.374 55059 INFO nova.openstack.common.service [req-ea1d6e3f-e2e1-4edf-b5a4-1103fb679032 - - - - -] Caught SIGTERM, exiting
2015-06-11 16:12:56.826 18384 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:12:56.866 18384 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2015-06-11 16:12:56.885 18384 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:12:56.888 18384 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2015-06-11 16:12:56.963 18384 WARNING oslo_messaging._drivers.impl_zmq [req-d4cefc49-632f-4150-8d76-51a6786a4622 - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:24:40.795 58030 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:24:40.834 58030 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2015-06-11 16:24:40.853 58030 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:24:40.855 58030 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2015-06-11 16:24:40.930 58030 WARNING oslo_messaging._drivers.impl_zmq [req-c3c40f37-5f62-4c5a-9fce-14a87cd262e6 - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:25:40.935 58030 CRITICAL nova [req-c3c40f37-5f62-4c5a-9fce-14a87cd262e6 - - - - -] Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""
2015-06-11 16:25:40.935 58030 TRACE nova Traceback (most recent call last):
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/bin/nova-compute", line 10, in
2015-06-11 16:25:40.935 58030 TRACE nova     sys.exit(main())
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/cmd/compute.py", line 72, in main
2015-06-11 16:25:40.935 58030 TRACE nova     db_allowed=CONF.conductor.use_local)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/service.py", line 277, in create
2015-06-11 16:25:40.935 58030 TRACE nova     db_allowed=db_allowed)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/service.py", line 157, in __init__
2015-06-11 16:25:40.935 58030 TRACE nova     self.conductor_api.wait_until_ready(context.get_admin_context())
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 292, in wait_until_ready
2015-06-11 16:25:40.935 58030 TRACE nova     timeout=timeout)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/nova/baserpc.py", line 62, in ping
2015-06-11 16:25:40.935 58030 TRACE nova     return cctxt.call(context, 'ping', arg=arg_p)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-06-11 16:25:40.935 58030 TRACE nova     retry=self.retry)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-06-11 16:25:40.935 58030 TRACE nova     timeout=timeout, retry=retry)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1033, in wrap
2015-06-11 16:25:40.935 58030 TRACE nova     return func(self, *args, **kws)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1065, in send
2015-06-11 16:25:40.935 58030 TRACE nova     return self._send(target, ctxt, message, wait_for_reply, timeout)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1055, in _send
2015-06-11 16:25:40.935 58030 TRACE nova     pooled=True)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 856, in _multi_send
2015-06-11 16:25:40.935 58030 TRACE nova     envelope, allowed_remote_exmods, pooled)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 781, in _call
2015-06-11 16:25:40.935 58030 TRACE nova     msg = msg_waiter.recv()
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 228, in recv
2015-06-11 16:25:40.935 58030 TRACE nova     return self.sock.recv_multipart(**kwargs)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 359, in recv_multipart
2015-06-11 16:25:40.935 58030 TRACE nova     return _Socket_recv_multipart(self, flags, copy, track)
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/zmq/sugar/socket.py", line 265, in recv_multipart
2015-06-11 16:25:40.935 58030 TRACE nova     parts = [self.recv(flags, copy=copy, track=track)]
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 338, in recv
2015-06-11 16:25:40.935 58030 TRACE nova     self._eventlet_recv_event.block()
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 104, in block
2015-06-11 16:25:40.935 58030 TRACE nova     self._hub.switch()
2015-06-11 16:25:40.935 58030 TRACE nova   File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-06-11 16:25:40.935 58030 TRACE nova     return self.greenlet.switch()
2015-06-11 16:25:40.935 58030 TRACE nova Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""
2015-06-11 16:25:40.935 58030 TRACE nova
2015-06-11 16:26:26.905 64005 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:26:26.934 64005 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2015-06-11 16:26:26.948 64005 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db __init__ /usr/lib/python2.7/dist-packages/nova/servicegroup/api.py:68
2015-06-11 16:26:26.950 64005 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2015-06-11 16:26:27.007 64005 WARNING oslo_messaging._drivers.impl_zmq [req-c2be6b2c-2e4b-48f2-b2d2-395df63250e9 - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:27:27.011 64005 CRITICAL nova [req-c2be6b2c-2e4b-48f2-b2d2-395df63250e9 - - - - -] Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""


================ERROR2=============


2015-06-11 16:09:26.889 55059 TRACE nova.servicegroup.drivers.db
2015-06-11 16:09:26.892 55059 WARNING nova.openstack.common.loopingcall [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] task > run outlasted interval by 50.01 sec
2015-06-11 16:09:26.895 55059 WARNING oslo_messaging._drivers.impl_zmq [req-928ef316-ceb4-4e69-8e08-a965dcafe37f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:10:14.236 55059 ERROR nova.openstack.common.periodic_task [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Error during ComputeManager._run_pending_deletes: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task Traceback (most recent call last):
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/periodic_task.py", line 224, in run_periodic_tasks
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     task(self, context)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6543, in _run_pending_deletes
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     context, filters, expected_attrs=attrs, use_slave=True)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 161, in wrapper
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     args, kwargs)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 335, in object_class_action
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     objver=objver, args=args, kwargs=kwargs)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     retry=self.retry)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     timeout=timeout, retry=retry)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1033, in wrap
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return func(self, *args, **kws)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1065, in send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return self._send(target, ctxt, message, wait_for_reply, timeout)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1055, in _send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     pooled=True)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 856, in _multi_send
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     envelope, allowed_remote_exmods, pooled)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 781, in _call
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     msg = msg_waiter.recv()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 228, in recv
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return self.sock.recv_multipart(**kwargs)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 359, in recv_multipart
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return _Socket_recv_multipart(self, flags, copy, track)
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/zmq/sugar/socket.py", line 265, in recv_multipart
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     parts = [self.recv(flags, copy=copy, track=track)]
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 338, in recv
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     self._eventlet_recv_event.block()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 104, in block
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     self._hub.switch()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task   File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task     return self.greenlet.switch()
2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""


================ERROR3=============


2015-06-11 16:10:14.236 55059 TRACE nova.openstack.common.periodic_task
2015-06-11 16:10:14.240 55059 DEBUG nova.openstack.common.periodic_task [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks /usr/lib/python2.7/dist-packages/nova/openstack/common/periodic_task.py:219
2015-06-11 16:10:14.240 55059 DEBUG nova.openstack.common.periodic_task [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Running periodic task ComputeManager._run_image_cache_manager_pass run_periodic_tasks /usr/lib/python2.7/dist-packages/nova/openstack/common/periodic_task.py:219
2015-06-11 16:10:14.242 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Acquired file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after waiting 0.000s acquire /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:212
2015-06-11 16:10:14.242 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" acquired by "do_register_storage_use" :: waited 0.001s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444
2015-06-11 16:10:14.244 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Releasing file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after holding it for 0.002s release /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:227
2015-06-11 16:10:14.244 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" released by "do_register_storage_use" :: held 0.002s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
2015-06-11 16:10:14.245 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Acquired file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after waiting 0.000s acquire /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:212
2015-06-11 16:10:14.246 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" acquired by "do_get_storage_users" :: waited 0.001s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444
2015-06-11 16:10:14.247 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Releasing file lock "/var/lib/nova/instances/locks/nova-storage-registry-lock" after holding it for 0.001s release /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:227
2015-06-11 16:10:14.248 55059 DEBUG oslo_concurrency.lockutils [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] Lock "storage-registry-lock" released by "do_get_storage_users" :: held 0.002s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
2015-06-11 16:10:14.250 55059 WARNING oslo_messaging._drivers.impl_zmq [req-db4c5598-dda4-455a-b2de-e9f3693cf03f - - - - -] rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing is deprecated. It is suggested to change the value to ring.
2015-06-11 16:10:14.252 55059 DEBUG oslo_concurrency.lockutils [-] Lock "fd97cc9e-dae2-4878-80a7-c2049ec3fcf5" released by "query_driver_power_state_and_sync" :: held 60.021s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
2015-06-11 16:10:14.253 55059 ERROR nova.compute.manager [-] [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5] Periodic sync_power_state task had an error while processing an instance.
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5] Traceback (most recent call last):
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6042, in _sync
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     query_driver_power_state_and_sync()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return f(*args, **kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6039, in query_driver_power_state_and_sync
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._query_driver_power_state_and_sync(context, db_instance)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6079, in _query_driver_power_state_and_sync
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     use_slave=True)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6095, in _sync_instance_power_state
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     db_instance.refresh(use_slave=use_slave)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 192, in wrapper
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._context, self, fn.__name__, args, kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 340, in object_action
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     objmethod=objmethod, args=args, kwargs=kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     retry=self.retry)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     timeout=timeout, retry=retry)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1033, in wrap
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return func(self, *args, **kws)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1065, in send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return self._send(target, ctxt, message, wait_for_reply, timeout)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 1055, in _send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     pooled=True)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 856, in _multi_send
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     envelope, allowed_remote_exmods, pooled)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 781, in _call
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     msg = msg_waiter.recv()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_zmq.py", line 228, in recv
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return self.sock.recv_multipart(**kwargs)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 359, in recv_multipart
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return _Socket_recv_multipart(self, flags, copy, track)
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/zmq/sugar/socket.py", line 265, in recv_multipart
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     parts = [self.recv(flags, copy=copy, track=track)]
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 338, in recv
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._eventlet_recv_event.block()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/green/zmq.py", line 104, in block
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     self._hub.switch()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]   File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5]     return self.greenlet.switch()
2015-06-11 16:10:14.253 55059 TRACE nova.compute.manager [instance: fd97cc9e-dae2-4878-80a7-c2049ec3fcf5] Timeout: Timeout while waiting on RPC response - topic: "", RPC method: "" info: ""

Find service/client connected to contrail vrouter agent port 9090

1)
Check the service/client which connected to vrouter agent port 9090
#netstat -pant | grep 9090
tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      56238/contrail-vrou
tcp        0      0 127.0.0.1:9090          127.0.0.1:52856         ESTABLISHED 56238/contrail-vrou
tcp        0      0 127.0.0.1:52856         127.0.0.1:9090          ESTABLISHED 64005/python

2)
Check the service/client which connected to vrouter agent port 9090
$lsof -i :9090
COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
contrail- 15486 root   51u  IPv4 13415815      0t0  TCP *:9090 (LISTEN)
contrail- 15486 root   53u  IPv4 13432972      0t0  TCP localhost:9090->localhost:41731 (ESTABLISHED)
nova-comp 55059 nova   52u  IPv4 13439027      0t0  TCP localhost:41731->localhost:9090 (ESTABLISHED)

2a)
How to map an ephemeral port to a pid
$sudo lsof -i :41731
$sudo lsof -i tcp:41731 <--- pid="" show="" span="">

$sudo fuser 41731

3)
Find the service/client using PID
$ps -aux | grep 55059
root     42138  0.0  0.0  10460   936 pts/6    S+   16:00   0:00 grep --color=auto 55059
nova     55059  1.1  0.0 2827008 88796 ?       Ssl  02:28   9:23 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf

4)
Stop nova compute
$service nova-compute stop
nova-compute stop/waiting

5)
Check the service/client which connected to vrouter agent port 9090
$lsof -i :9090
COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
contrail- 15486 root   51u  IPv4 13415815      0t0  TCP *:9090 (LISTEN)

6)
Find vrouter agent process
$ps -aux | grep 15486
root     15486  1.6  0.0 2434356 88660 ?       Sl   13:59   2:16 /usr/bin/contrail-vrouter-agent --config_file /etc/contrail/contrail-vrouter-agent.conf

OpenStack Client Test for OpenContrail vrouter Agent

https://github.com/Juniper/contrail-controller/blob/master/src/vnsw/agent/openstack/test/example_port_client.py

Replace imports to get it working

from contrail_vrouter_api.gen_py.instance_service import InstanceService
from contrail_vrouter_api.gen_py.instance_service import ttypes

OpenStack kilo opencontrail vif driver fix NovaException: vif_type parameter must be present for this vif_driver implementation

Fix
===========

1)
a)
#vim /etc/nova/nova.con
nova_contrail_vif.contrailvif.ContrailNetworkAPI

b)
Restart nova-compute
#service  nova-compute restar

c)
#nova boot ---

2) OR
 
JUNO/KILO release doesn't support libvirt_vif_driver configuration in nova.conf
vif_driver is set to LibvirtGenericVIFDriver. plug/unplug/get_config api from
this class doesn't support opencontrail. Till opencontrail vif_driver is
upstreamed, overwrite the vif_driver with VRouterVIFDriver


https://github.com/Juniper/contrail-nova-vif-driver/blob/88a1fb67919056818ecfff2d462120f3c4693831/nova_contrail_vif/contrailvif.py#L42

a)
$vim /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py

##SM
from nova_contrail_vif.contrailvif import VRouterVIFDriver =====

class LibvirtDriver(driver.ComputeDriver):

    def __init__(self, virtapi, read_only=False):
        ##SM
        self.vif_driver = VRouterVIFDriver() =====
        #self.vif_driver = libvirt_vif.LibvirtGenericVIFDriver()

b)
$service  nova-compute restart
$tail -f /var/log/nova/nova-compute.log

Error
===========

$/var/log/nova/nova-compute.log

2015-06-10 19:44:05.482 7765 ERROR nova.compute.manager [req-820a22fd-b835-45e3-b654-805e68adbd2e 8029c48a21654b51bea88039dbd140a8 ada49dd0f0a8429ca2b91cc120b8437c - - -] [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b] Instance failed to spawn
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b] Traceback (most recent call last):
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2461, in _build_resources
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]     yield resources
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2333, in _build_and_run_instance
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]     block_device_info=block_device_info)
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2382, in spawn
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]     write_to_disk=True)
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4183, in _get_guest_xml
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]     context)
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4054, in _get_guest_config
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]     flavor, virt_type)
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 368, in get_config
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]     _("vif_type parameter must be present "
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b] NovaException: vif_type parameter must be present for this vif_driver implementation
2015-06-10 19:44:05.482 7765 TRACE nova.compute.manager [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b]
2015-06-10 19:44:05.486 7765 INFO nova.compute.manager [req-820a22fd-b835-45e3-b654-805e68adbd2e 8029c48a21654b51bea88039dbd140a8 ada49dd0f0a8429ca2b91cc120b8437c - - -] [instance: 675c279e-3380-47d4-9050-a24d8b0ff59b] Terminating instance

how to get openstack kilo ubuntu PGP PUBLIC KEY

1)
$sudo add-apt-repository cloud-archive:kilo

2)
$apt-key list

3)
$apt-key export [key_from_above_command] 

4)
Example:
$apt-key export EC4926EA

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1

mQINBFAqSlgBEADPKwXUwqbgoDYgR20zFypxSZlSbrttOKVPEMb0HSUx9Wj8VvNC
r+mT4E9wAyq7NTIs5ad2cUhXoyenrjcfGqK6k9R6yRHDbvAxCSWTnJjw7mzsajDN
ocXC6THKVW8BSjrh0aOBLpht6d5QCO2vyWxw65FKM65GOsbX03ZngUPMuOuiOEHQ
Zo97VSH2pSB+L+B3d9B0nw3QnU8qZMne+nVWYLYRXhCIxSv1/h39SXzHRgJoRUFH
vL2aiiVrn88NjqfDW15HFhVJcGOFuACZnRA0/EqTq0qNo3GziQO4mxuZi3bTVL5s
GABiYW9uIlokPqcS7Fa0FRVIU9R+bBdHZompcYnKAeGag+uRvuTqC3MMRcLUS9Oi
/P9I8fPARXUPwzYN3fagCGB8ffYVqMunnFs0L6td08BgvWwer+Buu4fPGsQ5OzMc
lgZ0TJmXyOlIW49lc1UXnORp4sm7HS6okA7P6URbqyGbaplSsNUVTgVbi+vc8/jY
dfExt/3HxVqgrPlq9htqYgwhYvGIbBAxmeFQD8Ak/ShSiWb1FdQ+f7Lty+4mZLfN
8x4zPZ//7fD5d/PETPh9P0msF+lLFlP564+1j75wx+skFO4v1gGlBcDaeipkFzeo
zndAgpegydKSNTF4QK9iTYobTIwsYfGuS8rV21zE2saLM0CE3T90aHYB/wARAQAB
tD1DYW5vbmljYWwgQ2xvdWQgQXJjaGl2ZSBTaWduaW5nIEtleSA8ZnRwbWFzdGVy
QGNhbm9uaWNhbC5jb20+iQI3BBMBCAAhBQJQKkpYAhsDBQsJCAcDBRUKCQgLBRYC
AwEAAh4BAheAAAoJEF7bG2LsSSbqKxkQAIKtgImrk02YCDldg6tLt3b69ZK0kIVI
3Xso/zCBZbrYFmgGQEFHAa58mIgpv5GcgHHxWjpX3n4tu2RM9EneKvFjFBstTTgo
yuCgFr7iblvs/aMW4jFJAiIbmjjXWVc0CVB/JlLqzBJ/MlHdR9OWmojN9ZzoIA+i
+tWlypgUot8iIxkR6JENxit5v9dN8i6anmnWybQ6PXFMuNi6GzQ0JgZIVs37n0ks
2wh0N8hBjAKuUgqu4MPMwvNtz8FxEzyKwLNSMnjLAhzml/oje/Nj1GBB8roj5dmw
7PSul5pAqQ5KTaXzl6gJN5vMEZzO4tEoGtRpA0/GTSXIlcx/SGkUK5+lqdQIMdyS
n8bImU6V6rDSoOaI9YWHZtpv5WeUsNTdf68jZsFCRD+2+NEmIqBVm11yhmUoasC6
dYw5l9P/PBdwmFm6NBUSEwxb+ROfpL1ICaZk9Jy++6akxhY//+cYEPLin02r43Z3
o5Piqujrs1R2Hs7kX84gL5SlBzTM4Ed+ob7KVtQHTefpbO35bQllkPNqfBsC8AIC
8xvTP2S8FicYOPATEuiRWs7Kn31TWC2iwswRKEKVRmN0fdpu/UPdMikyoNu9szBZ
RxvkRAezh3WheJ6MW6Fmg9d+uTFJohZt5qHdpxYa4beuN4me8LF0TYzgfEbFT6b9
D6IyTFoT0LequQINBFAqSlgBEADmL3TEq5ejBYrA+64zo8FYvCF4gziPa5rCIJGZ
/gZXQ7pm5zek/lOe9C80mhxNWeLmrWMkMOWKCeaDMFpMBOQhZZmRdakOnH/xxO5x
+fRdOOhy+5GTRJiwkuGOV6rB9eYJ3UN9caP2hfipCMpJjlg3j/GwktjhuqcBHXhA
HMhzxEOIDE5hmpDqZ051f8LGXld9aSL8RctoYFM8sgafPVmICTCq0Wh03dr5c2JA
gEXy3ushYm/8i2WFmyldo7vbtTfx3DpmJc/EMpGKV+GxcI3/ERqSkde0kWlmfPZb
o/5+hRqSryqfQtRKnFEQgAqAhPIwXwOkjCpPnDNfrkvzVEtl2/BWP/1/SOqzXjk9
TIb1Q7MHANeFMrTCprzPLX6IdC4zLp+LpV91W2zygQJzPgWqH/Z/WFH4gXcBBqmI
8bFpMPONYc9/67AWUABo2VOCojgtQmjxuFn+uGNw9PvxJAF3yjl781PVLUw3n66d
wHRmYj4hqxNDLywhhnL/CC7KUDtBnUU/CKn/0Xgm9oz3thuxG6i3F3pQgpp7MeMn
tKhLFWRXo9Bie8z/c0NV4K5HcpbGa8QPqoDseB5WaO4yGIBOt+nizM4DLrI+v07y
Xe3Jm7zBSpYSrGarZGK68qamS3XPzMshPdoXXz33bkQrTPpivGYQVRZuzd/R6b+6
IurV+QARAQABiQIfBBgBCAAJBQJQKkpYAhsMAAoJEF7bG2LsSSbq59EP/1U3815/
yHV3cf/JeHgh6WS/Oy2kRHp/kJt3ev/l/qIxfMIpyM3u/D6siORPTUXHPm3AaZrb
w0EDWByA3jHQEzlLIbsDGZgrnl+mxFuHwC1yEuW3xrzgjtGZCJureZ/BD6xfRuRc
mvnetAZv/z98VN/oj3rvYhUi71NApqSvMExpNBGrdO6gQlI5azhOu8xGNy4OSke8
J6pAsMUXIcEwjVEIvewJuqBW/3rj3Hh14tmWjQ7shNnYBuSJwbLeUW2e8bURnfXE
TxrCmXzDmQldD5GQWCcD5WDosk/HVHBmHlqrqy0VO2nE3c73dQlNcI4jVWeC4b4Q
SpYVsFz/6Iqy5ZQkCOpQ57MCf0B6P5nF92c5f3TYPMxHf0x3DrjDbUVZytxDiZZa
XsbZzsejbbc1bSNp4hb+IWhmWoFnq/hNHXzKPHBTapObnQju+9zUlQngV0BlPT62
hOHOw3Pv7suOuzzfuOO7qpz0uAy8cFKe7kBtLSFVjBwaG5JX89mgttYW+lw9Rmsb
p9Iw4KKFHIBLOwk7s+u0LUhP3d8neBI6NfkOYKZZCm3CuvkiOeQP9/2okFjtj+29
jEL+9KQwrGNFEVNe85Un5MJfYIjgyqX3nJcwypYxidntnhMhr2VD3HL2R/4CiswB
Oa4g9309p/+af/HU1smBrOfIeRoxb8jQoHu3
=xg4S
-----END PGP PUBLIC KEY BLOCK-----

openstack glance image-create error Unexpected EOF

Fix
===

https://review.openstack.org/#/c/142952/4/glanceclient/common/http.py

/usr/lib/python2.7/dist-packages/glanceclient/common/http.py


Error
=====

 $glance --debug image-create --progress --name cirros-0.3.3 --disk-format qcow2 --container-format bare --is-public True --file cirros-0.3.2-i386-disk.img
curl -i -X POST -H 'Accept-Encoding: gzip, deflate, compress' -H 'x-image-meta-container_format: bare' -H 'Accept: */*' -H 'X-Auth-Token: {SHA1}c18891d276885a33f8799b559148e8f8fd6ba27e' -H 'x-image-meta-size: 12336128' -H 'x-image-meta-is_public: True' -H 'User-Agent: python-glanceclient' -H 'Content-Type: application/octet-stream' -H 'x-image-meta-disk_format: qcow2' -H 'x-image-meta-name: cirros-0.3.3'  --cert None --key None https://image.jcloud.com:9292/v1/images
[=============================>] 100%
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/glanceclient/shell.py", line 637, in main
    args.func(client, args)
  File "/usr/lib/python2.7/dist-packages/glanceclient/v1/shell.py", line 232, in do_image_create
    image = gc.images.create(**fields)
  File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 288, in create
    data=image_data)
  File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 262, in post
    return self._request('POST', url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 206, in _request
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 455, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 558, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 353, in send
    low_conn.send(i)
  File "/usr/lib/python2.7/httplib.py", line 811, in send
    self.sock.sendall(data)
  File "/usr/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 291, in sendall
    return self.connection.sendall(data)
SysCallError: (-1, 'Unexpected EOF')
(-1, 'Unexpected EOF')
 

puppet how to manually run a class

1)
how to manually run a class
------------------------------

$echo 'include rjil::system::apt' | puppet apply
$echo 'include rjil::system::apt' | puppet apply --debug
$echo 'include rjil::system::apt' | puppet apply --debug ---noop

2)
How to add entry in /etc/hosts
---------------------------------------

$echo "include rjil::jiocloud::dns" | puppet apply

notify {"ip=${ip}, cname=${cname}, name=${name}":}  === print
notify {"ip=${ip}, ip_real=${ip_real}":}

Notice: ip=false, cname=keystone.service.consul, name=identity.jiocloud.com
Notice: ip=false, ip_real=10.140.15.69

/etc/puppet/modules.overrides/rjil/manifests/jiocloud/dns/entry.pp

3)
How consul check keystone service
-------------------------------------------------------
a)
$vim puppet-rjil/manifests/keystone.pp

  rjil::jiocloud::consul::service { "keystone":
    tags          => ['real'],
    port          => 5000,
  }

b)
$vim puppet-rjil/manifests/jiocloud/consul/service.pp

  file { "/etc/consul/$name.json":
    ensure => "present",
    content => template('rjil/consul.service.erb'),
  } ~> Exec <| title == 'reload-consul' |>

c)
vim /etc/consul/keystone.json

{
  "service": {
    "name": "keystone",
    "port": 5000,
    "tags": [
      "real"
    ],
    "check": {
      "script": "/usr/lib/jiocloud/tests/service_checks/keystone.sh",
      "interval": "10s"
    }
  }
}

* Consul will read this json file and execute the script in the "check" section, then register the service if check is success.

d)
$vim /usr/lib/jiocloud/tests/service_checks/keystone.sh

#!/bin/bash
set -e
/usr/lib/nagios/plugins/check_http -S -H 10.140.15.66 -p 5000

4)

ceph ubuntu repo PGP PUBLIC KEY

https://git.ceph.com/?p=ceph.git;a=blob_plain;f=keys/release.asc


1)
puppet hiera Example:
===========

rjil::system::apt::repositories:
  ceph:
    location: 'http://ceph.com/debian-hammer'
    release: 'trusty'
    repos: 'main'
    include_src: false
    key: '17ED316D'
    key_content: |
      -----BEGIN PGP PUBLIC KEY BLOCK-----
      Version: GnuPG v1.4.11 (GNU/Linux)
      mQINBFX4hgkBEADLqn6O+UFp+ZuwccNldwvh5PzEwKUPlXKPLjQfXlQRig1flpCH
      E0HJ5wgGlCtYd3Ol9f9+qU24kDNzfbs5bud58BeE7zFaZ4s0JMOMuVm7p8JhsvkU
      C/Lo/7NFh25e4kgJpjvnwua7c2YrA44ggRb1QT19ueOZLK5wCQ1mR+0GdrcHRCLr
      7Sdw1d7aLxMT+5nvqfzsmbDullsWOD6RnMdcqhOxZZvpay8OeuK+yb8FVQ4sOIzB
      FiNi5cNOFFHg+8dZQoDrK3BpwNxYdGHsYIwU9u6DWWqXybBnB9jd2pve9PlzQUbO
      eHEa4Z+jPqxY829f4ldaql7ig8e6BaInTfs2wPnHJ+606g2UH86QUmrVAjVzlLCm
      nqoGymoAPGA4ObHu9X3kO8viMBId9FzooVqR8a9En7ZE0Dm9O7puzXR7A1f5sHoz
      JdYHnr32I+B8iOixhDUtxIY4GA8biGATNaPd8XR2Ca1hPuZRVuIiGG9HDqUEtXhV
      fY5qjTjaThIVKtYgEkWMT+Wet3DPPiWT3ftNOE907e6EWEBCHgsEuuZnAbku1GgD
      LBH4/a/yo9bNvGZKRaTUM/1TXhM5XgVKjd07B4cChgKypAVHvef3HKfCG2U/DkyA
      LjteHt/V807MtSlQyYaXUTGtDCrQPSlMK5TjmqUnDwy6Qdq8dtWN3DtBWQARAQAB
      tCpDZXBoLmNvbSAocmVsZWFzZSBrZXkpIDxzZWN1cml0eUBjZXBoLmNvbT6JAjgE
      EwECACIFAlX4hgkCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEOhKwsBG
      DzmUXdIQAI8YPcZMBWdv489q8CzxlfRIRZ3Gv/G/8CH+EOExcmkVZ89mVHngCdAP
      DOYCl8twWXC1lwJuLDBtkUOHXNuR5+Jcl5zFOUyldq1Hv8u03vjnGT7lLJkJoqpG
      l9QD8nBqRvBU7EM+CU7kP8+09b+088pULil+8x46PwgXkvOQwfVKSOr740Q4J4nm
      /nUOyTNtToYntmt2fAVWDTIuyPpAqA6jcqSOC7Xoz9cYxkVWnYMLBUySXmSS0uxl
      3p+wK0lMG0my/gb+alke5PAQjcE5dtXYzCn+8Lj0uSfCk8Gy0ZOK2oiUjaCGYN6D
      u72qDRFBnR3jaoFqi03bGBIMnglGuAPyBZiI7LJgzuT9xumjKTJW3kN4YJxMNYu1
      FzmIyFZpyvZ7930vB2UpCOiIaRdZiX4Z6ZN2frD3a/vBxBNqiNh/BO+Dex+PDfI4
      TqwF8zlcjt4XZ2teQ8nNMR/D8oiYTUW8hwR4laEmDy7ASxe0p5aijmUApWq5UTsF
      +s/QbwugccU0iR5orksM5u9MZH4J/mFGKzOltfGXNLYI6D5Mtwrnyi0BsF5eY0u6
      vkdivtdqrq2DXY+ftuqLOQ7b+t1RctbcMHGPptlxFuN9ufP5TiTWSpfqDwmHCLsT
      k2vFiMwcHdLpQ1IH8ORVRgPPsiBnBOJ/kIiXG2SxPUTjjEGOVgeA
      =/Tod
      -----END PGP PUBLIC KEY BLOCK-----

How to github Checking out pull requests locally

1)
fetch pull request 761 from remote "jiocloud_remote" and save in local branch "temp_branch".
$git fetch jiocloud_remote pull/761/head:temp_branch

2)
Then got to that branch where you want to apply changes in this pull request.
$git checkout mydevbranch

3)
Merge the Pull Request
$git merge -m 'Merging Pull Request' temp_branch

4)
If there is any conflict during merge then do following steps.
a) Fix conflict
b) $git add fixed_file
c) $git commit -a

5)
Done

How to Install Ubuntu Packages from a Specific Repository

1)
$apt-cache policy python-ironicclient
python-ironicclient:
  Installed: (none)
  Candidate: 0.5.1-0ubuntu1~cloud0
  Version table:
     1:2014.2.245 0
        400 http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing/main amd64 Packages
     0.5.1-0ubuntu1~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main amd64 Packages
     0.1.2-0ubuntu1 0
        500 http://in.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

2)
$apt-get install python-ironicclient/trusty-testing

OR

$apt-get -t trusty-testing python-ironicclien

Apt-Pinning Assign highest priority to a repository

1)
$vim /etc/apt/sources.list.d/rustedhalo.list
deb http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing main

2)
$apt-cache policy python-ironicclient
python-ironicclient:
  Installed: 0.5.1-0ubuntu1~cloud0
  Candidate: 1:2014.2.245 ====imp
  Version table:
     1:2014.2.245 0
        500 http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing/main amd64 Packages
 *** 0.5.1-0ubuntu1~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main amd64 Packages
        100 /var/lib/dpkg/status
     0.1.2-0ubuntu1 0
        500 http://in.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

3)
$vim /etc/apt/preferences.d/my_test_pref

Package: *
Pin: release a=trusty-testing
Pin-Priority: 400

4)
$apt-get update

5)
$apt-cache policy python-ironicclient
python-ironicclient:
  Installed: 0.5.1-0ubuntu1~cloud0
  Candidate: 0.5.1-0ubuntu1~cloud0 ====imp
  Version table:
     1:2014.2.245 0
        400 http://jiocloud.rustedhalo.com/ubuntu/ trusty-testing/main amd64 Packages
 *** 0.5.1-0ubuntu1~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/kilo/main amd64 Packages
        100 /var/lib/dpkg/status
     0.1.2-0ubuntu1 0
        500 http://in.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

apt-get install WARNING: The following packages cannot be authenticated!

If you recently added PPA or repo like "$add-apt-repository cloud-archive:kilo"
Then run following commands to fix authentication WARNING

$apt-key update 

$apt-get update

Ubuntu How to rerun cloud-init manually

1)
$rm -rf /var/lib/cloud/*
OR
$rm -rf /var/lib/cloud/sem/* /var/lib/cloud/instance /var/lib/cloud/instances/*

2)
$cloud-init init

$cloud-init modules -m final

3)
$tail -f /var/log/cloud-init.log

$tail -f /var/log/cloud-init-output.log

How to run cloud-init manually

1)
$rm -rf /var/lib/cloud/*
OR
$rm -rf /var/lib/cloud/sem/* /var/lib/cloud/instance /var/lib/cloud/instances/*

2)
$cloud-init init

$cloud-init modules -m final

3)
$tail -f /var/log/cloud-init.log

$tail -f /var/log/cloud-init-output.log

How to fix unmet dependencies of a deb package

1)
$cd /home/saju/pkgoverride

$wget http://jiocloud.rustedhalo.com/ubuntu/pool/main/r/rjil-cicd/rjil-cicd_2014.2.234_all.deb

2)
http://serverfault.com/questions/250224/how-do-i-get-apt-get-to-ignore-some-dependencies
a)
Unpack:
$ar x rjil-cicd_2014.2.234_all.deb

b)
Unpack control archive:
$tar xzf control.tar.gz

c)
Fix dependencies in control (use a text editor)
$vim control

d)
Repack control.tar.gz:
$tar c {post,pre}{inst,rm} md5sums control | gzip -c > control.tar.gz

e)
Repack deb:
$ar rcs newpackage.deb debian-binary control.tar.gz data.tar.xz

f)
$dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

3)
$vim /etc/apt/sources.list.d/overrides.list
deb [trusted=yes] file:/home/saju/pkgoverride ./

4)
$apt-get update

5)
$apt-cache policy rjil-cicd

6)
$apt-get install rjil-cicd

opencontrail remove and insert vrouter module

1)
$vim insertscript.sh

sudo rmmod vrouter
sleep 10

sudo ifdown eth3
sleep 10

sudo ifup eth3
sleep 10

sudo ifdown vhost0
sleep 10

sudo ifup vhost0
sleep 10

2)
chmod 777 insertscript.sh

3)
nohup ./insertscript.sh &

How to install openstack kilo liberty in ubuntu 14.04

https://wiki.ubuntu.com/ServerTeam/CloudArchive

git filter commit logs by Author and print it in single line

$git log -n 15 --pretty=oneline --author=saju myremote/contrailv2

018839692b324484aec12cd68c8fcaa13ad6c83f Added dan's changes to test upgrade
a3e907f7220471597a64d4fbfbd611390ede4a38 Added options for env specific repos
ec6f169a0549e0ecf1a7eea0fcaaedcc11cf0028 Disabling contrail-dns
6ca130da7fc11085a33a8250e69588401cf07f28 Enabled contrailv2 repos
42c8c208c4bbe50a9f47bc8e9f0dfa1b90a4a9c0 Enabled contrailv2 in AT

How To Install Stable OpenStack Liberty/Juno/Kilo with Neutron and Swift in Ubuntu 15.04 14.04

1)

Clone
$git clone https://github.com/openstack-dev/devstack.git -b stable/liberty

2)

CD to cloned devstack directory

$cd devsack

3)

Create a file named "localrc" with following content.
 

##For neutron

disable_service n-net 

enable_service q-svc q-agt q-dhcp q-l3

##For swift

enable_service s-proxy s-object s-container s-account

4)

Run stack.sh and install openstack
$./stack.sh

Time to switch to OpenStackClient unified CLI

How To Install Stable OpenStack Liberty In 2 Steps Using Devstack

1)
$git clone https://github.com/openstack-dev/devstack.git -b stable/liberty

2)
$cd devsack
$./stack.sh

How To Install Stable OpenStack Liberty In Ubuntu Using Devstack

1)
$git clone https://github.com/openstack-dev/devstack.git -b stable/liberty

2)
$cd devsack
$./stack.sh