JDBC Catalog connection pool not released - always reach to limit

[lvchaoin]

Hi There,

We are using Doris v2.1.2, and have some JDBC catalog created for SQL Server, the connection pool seems still has some problem since this issue was reported on version 1.2.4. We have increased the connection pool to 500, but after about 10 days, it reached to the maximum connections, normally it won't need some many connections, cause we have only a little bit business on it right now. My question is how to check what and how many active JDBC connection is being used in real-time so that we can analyze the problem ?

Thanks a lot !

2024-06-01 11:06:06,417 [slow_query] |Client=192.168.0.78:33484|User=admin|Ctl=internal|Db=test_db|State=ERR|ErrorCode=1105|ErrorMessage=errCode = 2, detailMessage = (192.168.0.94)[INTERNAL_ERROR]UdfRuntimeException: Initialize datasource failed:  CAUSED BY: GetConnectionTimeoutException: wait millis 5000, active 500, maxActive 500

[zy-kkk]

We changed the Jdbc connection pool dependency from Druid to HikariCP in Doris 2.1.3, which may solve this problem.

[lvchaoin]

It seems the problem still existed :(

CAUSED BY: SQLTransientConnectionException: HikariPool-2 - Connection is not available, request timed out after 5000ms.

[lvchaoin]

workaround feedback to dev team, we think it should be able to resolve.

We setup the connection pool size to 500, and after 5 days, the error start to occur, and also we observed the BE memory usage increased a lot, then we tried the following actions:
1、 alter catalog CATALOG_SUMIN set properties ('connection_pool_max_size' = '500') -- run this sentence but not change the pool size, it doesn't work
2、 alter catalog CATALOG_SUMIN set properties ('connection_pool_max_size' = '200') -- it works, and also we observed the memory usage start to decrease.

Workaround solution, we will set up the schedule to change the connection_pool_max_size periodically with different size number, we tried this solution half month ago, and this time again prove the problem. Hope the information useful for problem solving.

Memory usage trend chart:

[raventan95]

@zy-kkk We also faced the same problem. We use catalog a lot, and we face this problem every 2-3 days. We are using the latest 2.0.15.
What is the solution?