环境
我在 2 个网络服务器之间有一个 rails 3.2.x 应用程序负载平衡:web1 和 web2。每个网络服务器都指向 mysql 数据库的集群 IP。该数据库是集群的 2 台服务器(按机架空间)。在故障转移之前,一次有 1 个数据库处于活动状态。
错误
在我们最新的故障转移中,web1 开始发送ActiveRecord::ConnectionTimeoutError
(Middleware/Rack/ActiveRecord::QueryCache#call
下面的堆栈跟踪),而 web2 没有错误。请注意,这发生在数据库稳定并且 100% 切换到另一个节点之后。
理论
我的理论是会话或缓存或连接引用了旧节点,一旦切换它就找不到所述连接。这看起来确实很奇怪,因为两个数据库节点都是集群的,在同一个集群 IP 上服务并被复制。
临时解决方案
我们的临时解决方案是重新启动 web1 rails 服务器,这很有效。
问题
1)我想知道我是否可以解释这里可能发生的事情
2)是否有配置设置可以防止将来发生这种情况。
堆栈跟踪
错误消息 ActiveRecord::ConnectionTimeoutError: could not get a database connection within 5 seconds (waited 5.000268907 seconds). 当前最大池大小为 5;考虑增加它。
...record/connection_adapters/abstract/connection_pool.rb: 258:in block (2 levels) in checkout'
…record/connection_adapters/abstract/connection_pool.rb: 242:in
loop' ...record/connection_adapters/abstract/connection_pool.rb: 242:in block in checkout'
/usr/local/lib/ruby/1.9.1/monitor.rb: 211:in
mon_synchronize' ...record/connection_adapters/abstract/connection_pool.rb: 239:in checkout'
…record/connection_adapters/abstract/connection_pool.rb: 102:in
block in connection' /usr/local/lib/ruby/1.9.1/monitor.rb: 211:in mon_synchronize'
…record/connection_adapters/abstract/connection_pool.rb: 101:in
connection' …record/connection_adapters/abstract/connection_pool.rb: 410:in retrieve_connection'
…nnection_adapters/abstract/connection_specification.rb: 171:in
retrieve_connection' …nnection_adapters/abstract/connection_specification.rb: 145:inconnection'
…s/activerecord-3.2.19/lib/active_record/query_cache.rb: 67:in
通话中救援' ...s/activerecord-3.2.19/lib/active_record/query_cache.rb: 61:in call'
…record/connection_adapters/abstract/connection_pool.rb: 479:in
call' ...ack-3.2.19/lib/action_dispatch/middleware/callbacks.rb: 28:in block in call'
…s/activesupport-3.2.19/lib/active_support/callbacks.rb: 405:in
_run__862588711471957735__call__1324336831638990183__callbacks' ...s /activesupport-3.2.19/lib/active_support/callbacks.rb:405:在__run_callback'
…s/activesupport-3.2.19/lib/active_support/callbacks.rb: 385:in
_run_call_callbacks' ...s/activesupport-3.2.19/lib/active_support/callbacks.rb: 81:in run_callbacks'
…ack-3.2.19/lib/action_dispatch/middleware/callbacks.rb: 27:in
call' ...ack-3.2.19/lib/action_dispatch/middleware/remote_ip.rb: 31:in call'
…/gems/airbrake-3.1.14/lib/airbrake/rails/middleware.rb: 13:in
call' ....19/ lib/action_dispatch/middleware/debug_exceptions.rb: 16:in call'
…2.19/lib/action_dispatch/middleware/show_exceptions.rb: 56:in
call' ...by/1.9.1/gems/railties-3.2.19/lib/rails/rack/logger.rb: 32:in call_app'
…by/1.9.1/gems/railties-3.2.19/lib/rails/rack/logger.rb: 16:in
block in call' ...ivesupport- 3.2.19/lib/active_support/tagged_logging.rb: 22:in tagged'
…by/1.9.1/gems/railties-3.2.19/lib/rails/rack/logger.rb: 16:in
call' ...ck-3.2.19/lib/action_dispatch/middleware/request_id.rb: 22:in call'
…/ruby/1.9.1/gems/rack-1.4.5/lib/rack/methodoverride.rb: 21:in
call' .../bundle/ruby/1.9.1/gems /rack-1.4.5/lib/rack/runtime.rb: 17:in call'
…9.1/gems/airbrake-3.1.14/lib/airbrake/user_informer.rb: 16:in
_call' ...9.1/gems/airbrake-3.1.14/lib/airbrake/user_informer.rb: 12:in call'
…le/ruby/1.9.1/gems/railties-3.2.19/lib/rails/engine.rb: 484:in
call' ...by/1.9.1 /gems/railties-3.2.19/lib/rails/application.rb: 231:incall'
…gems/railties-3.2.19/lib/rails/railtie/configurable.rb: 30:in
method_missing' ...r-3.0.13/lib/phusion_passenger/rack/request_handler.rb: 96:in `process_request'