|
안녕하세요. 서비스중에 failover가 발생했습니다.
항상 db stop/ os shutdown으로 failover 테스트만 해보다가 운영중에 실제 failover가 발생했는데,
failover 원인을 못찾겠어서 문의드립니다.
해당시간에 db down이나 네트워크 OS이슈는 없었습니다.
pgpool로그를 확인해봤는데 "detecting postmaster down error" 메시지의 원인을 모르겠습니다.
down 에러 직전에 쿼리응답을 기다리고 있었다는 구문이 남았는데, 쿼리는 문제가 없어보이는데,
down 직전의 쿼리가 원인이라고 봐야할까요 ?
failover시간에 pgpool 로그 같이 올립니다.
혹시 제가 더 찾아볼만한게 있으면 조그만한거라도 답변 부탁드려요
* 버전 : postgresql 9.6.22 / pgpool-II version 4.1.8 (karasukiboshi)
* pgpool 구성 : m -s 디비서버에 각각 pgpool 데몬 띄움
* failover config : failover_on_backend_error = on 사용
2022-01-28 13:45:46: pid 72195: DETAIL: destination = 0 for query= "/*
Query~~~(매우 자추 호출퇴는 초기화 DDL 구문들 )
2022-01-28 13:45:46: pid 72195: DEBUG: waiting for query response
2022-01-28 13:45:46: pid 72195: DETAIL: waiting for backend:0 to complete the query
2022-01-28 13:45:46: pid 32884: DEBUG: detecting postmaster down error
2022-01-28 13:45:46: pid 32884: DETAIL: receive admin shutdown error from a node
2022-01-28 13:45:46: pid 32884: LOG: reading and processing packets
2022-01-28 13:45:46: pid 32884: DETAIL: postmaster on DB node 0 was shutdown by administrative command
2022-01-28 13:45:46: pid 32884: LOG: received degenerate backend request for node_id: 0 from pid [32884]
2022-01-28 13:45:46: pid 64032: LOG: new IPC connection received
2022-01-28 13:45:46: pid 64032: LOG: watchdog received the failover command from local pgpool-II on IPC interface
2022-01-28 13:45:46: pid 64032: LOG: watchdog is processing the failover command [DEGENERATE_BACKEND_REQUEST] received from local pgpool-II on IPC interface
2022-01-28 13:45:46: pid 64032: LOG: we have got the consensus to perform the failover
2022-01-28 13:45:46: pid 64032: DETAIL: 1 node(s) voted in the favor
2022-01-28 13:45:46: pid 64032: DEBUG: removing failover request from 1 nodes with ID:567447
2022-01-28 13:45:46: pid 64032: DEBUG: sending watchdog packet to socket:9, type:[7], command ID:0, data Length:54
2022-01-28 13:45:46: pid 63529: DEBUG: function call walker, function name: "pg_terminate_backend"
2022-01-28 13:45:46: pid 63529: DEBUG: function call walker, function name: "pg_backend_pid"
:
:
|
