Fix
======
Restart cassandra
$service cassandra restart
OR
Check cassandra connection Pool
Symptoms
=========
1)
Never Ending polling by contrail-controller from ifmap-server
$tail -f /var/log/contrail/contrail-control.log
100.1.0.0 24 100.1.0.1 100.1.0.2 75de43f2-93da-481e-a0d1-c4aa96a3f113 true 100.1.0.11 100.1.0.254 true public_subnet
2015-06-23 Tue 15:33:45:315.863 UTC ct1-test-dev-test [Thread 140311640958720, Pid 59613]: SANDESH: Send FAILED: 1435073625315829 IFMapPeer [SYS_DEBUG]: IFMapChannelProcResp: Http header length is 75 Content length is 1351 Total bytes read are 75 controller/src/ifmap/client/ifmap_channel.cc 679
2015-06-23 Tue 15:33:45:316.137 UTC ct1-test-dev-test [Thread 140311334471424, Pid 59613]: SANDESH: Send FAILED: 1435073625316120 IFMapPeer [SYS_DEBUG]: IFMapServerConnection: 0 bytes in reply_. 1426 bytes in reply_str. PollResponse message is:
HTTP/1.1 200
Content-Type: application/soap+xml
Content-Length: 1351
99.0.0.0 24 99.0.0.1 99.0.0.2 8f442855-b940-46b3-9e70-ae596e3ce644 true true private_shared_subnet
2015-06-23 Tue 15:33:45:656.135 UTC ct1-test-dev-test [Thread 140311137343232, Pid 59613]: SANDESH: Send FAILED: 1435073625656087 IFMapPeer [SYS_DEBUG]: IFMapChannelProcResp: Http header length is 75 Content length is 1280 Total bytes read are 75 controller/src/ifmap/client/ifmap_channel.cc 679
2015-06-23 Tue 15:33:45:656.338 UTC ct1-test-dev-test [Thread 140311321876224, Pid 59613]: SANDESH: Send FAILED: 1435073625656320 IFMapPeer [SYS_DEBUG]: IFMapServerConnection: 0 bytes in reply_. 1355 bytes in reply_str. PollResponse message is:
HTTP/1.1 200
Content-Type: application/soap+xml
Content-Length: 1280
2)
VM details not coming at contrail-control node from ifmap-server
$tail -f /var/log/contrail/contrail-control.log
3)
VM's Link local address missing at compute node
$route -n
======
Restart cassandra
$service cassandra restart
OR
Check cassandra connection Pool
Symptoms
=========
1)
Never Ending polling by contrail-controller from ifmap-server
$tail -f /var/log/contrail/contrail-control.log
2015-06-23 Tue 15:33:45:315.863 UTC ct1-test-dev-test [Thread 140311640958720, Pid 59613]: SANDESH: Send FAILED: 1435073625315829 IFMapPeer [SYS_DEBUG]: IFMapChannelProcResp: Http header length is 75 Content length is 1351 Total bytes read are 75 controller/src/ifmap/client/ifmap_channel.cc 679
2015-06-23 Tue 15:33:45:316.137 UTC ct1-test-dev-test [Thread 140311334471424, Pid 59613]: SANDESH: Send FAILED: 1435073625316120 IFMapPeer [SYS_DEBUG]: IFMapServerConnection: 0 bytes in reply_. 1426 bytes in reply_str. PollResponse message is:
HTTP/1.1 200
Content-Type: application/soap+xml
Content-Length: 1351
2015-06-23 Tue 15:33:45:656.135 UTC ct1-test-dev-test [Thread 140311137343232, Pid 59613]: SANDESH: Send FAILED: 1435073625656087 IFMapPeer [SYS_DEBUG]: IFMapChannelProcResp: Http header length is 75 Content length is 1280 Total bytes read are 75 controller/src/ifmap/client/ifmap_channel.cc 679
2015-06-23 Tue 15:33:45:656.338 UTC ct1-test-dev-test [Thread 140311321876224, Pid 59613]: SANDESH: Send FAILED: 1435073625656320 IFMapPeer [SYS_DEBUG]: IFMapServerConnection: 0 bytes in reply_. 1355 bytes in reply_str. PollResponse message is:
HTTP/1.1 200
Content-Type: application/soap+xml
Content-Length: 1280
2)
VM details not coming at contrail-control node from ifmap-server
$tail -f /var/log/contrail/contrail-control.log
3)
VM's Link local address missing at compute node
$route -n
* I have temp fixed this issue by restarting following services
ReplyDelete$service cassandra restart
$service contrail-schema restart
$service ifmap-server restart
* Check contrail-control log while restarting above services. May be you see continuous polling in log from contrail-control to ifmap-server, that case restart ifmap-server again.
$tail -f /var/log/contrail/contrail-control.log