Error connecting to database: [1040]: Too many connections

This is used for general discussion that is not necessarily server-related.
Post Reply
dustinandband
A semi-regular
A semi-regular
Posts: 22
https://www.youtube.com/channel/UC40BgXanDqOYoVCYFDSTfHA
Joined: Mon Oct 05, 2015 7:38 pm

Error connecting to database: [1040]: Too many connections

Post by dustinandband »

I've been getting this sourcemod error today (september 1st) on my servers:

Code: Select all

Error connecting to database: [1040]: Too many connections
We have our database and website hosted on NFO.
Our L4D2 game servers have somewhere around 4-8 plugins on each server (11 servers total) that just keeps an open DB connection (the DB handle is declared globally and not closed). Aside from that we just have 4 websites that also keep an ongoing connection.

I did some queries in phpmyadmin to see how many simultaneous connections we had (most processes listed here):

Code: Select all

show variables like "max_connections";	result = 1,000
SELECT COUNT(*) FROM information_schema.PROCESSLIST;	result = 79
Since there's only 79/1,000 simultaneous connections, I'm thinking there might be an issue with disk space possibly? We weren't getting these errors before today (same amount of servers + same plugins) and I noticed the websites where down this morning (around 6AM PST) for a little while, I figured it was cause of maintenance.

Just trying to narrow down the issue so anything helps. Thanks

DB host: survival.site.nfoservers.com
Website host: hosted30.nfoservers.com
Naleksuh
This is my homepage
This is my homepage
Posts: 298
Joined: Thu Jul 25, 2019 12:35 am

Re: Error connecting to database: [1040]: Too many connections

Post by Naleksuh »

Hmm, I had this once before but it was a SourceMod error, if you're 100% sure you are doing the SourceMod work properly that's good.

Keep in mind that there are multiple users sharing the MySQL server, and all of those count towards the connection limit. You can check the available disk space through SSH, how much is there on your machine?
User avatar
TimeX
Staff
Staff
Posts: 1730
Joined: Thu Jul 22, 2004 12:24 am
Location: Big Bear, CA

Re: Error connecting to database: [1040]: Too many connections

Post by TimeX »

That error means you are running into our imposed limit, as stated in our ToS. Additionally, other customers on the node will not affect this, as it is per-IP.

From the ToS:
Clients may not: Have open more than 20 remote (non-local) connections to the MySQL server per remote IP address.

You will need to check how many connections are open from the game server that is having problems, and then either try to reduce it, or change the functionality so that the connections do not stay open.
TimeX
dustinandband
A semi-regular
A semi-regular
Posts: 22
Joined: Mon Oct 05, 2015 7:38 pm

Re: Error connecting to database: [1040]: Too many connections

Post by dustinandband »

Thanks, that helps
TimeX wrote: Wed Sep 01, 2021 7:25 pm From the ToS:
Clients may not: Have open more than 20 remote (non-local) connections to the MySQL server per remote IP address.
Is it possible that limit got changed today to 7 or 8? I haven't seen those errors before today and the screenshot indicates 7 connections per game server IP address. I checked the error logs of 5 different servers - this specific error started today.
  • The server config hasn't changed. (Each server rsyncs to a github repo once a day - update log since Aug 4th).
  • We haven't added any additional servers (though the TOS is regarding connections is per individual IP address so that wouldn't matter).
Maybe it's also possible that the total amount of active DB connections for all users on this server exceeds the max_connections limit of 1,000 ?
User avatar
TimeX
Staff
Staff
Posts: 1730
Joined: Thu Jul 22, 2004 12:24 am
Location: Big Bear, CA

Re: Error connecting to database: [1040]: Too many connections

Post by TimeX »

That limit of 10 connections wouldn't have changed.

Is your image showing what is current, or is that historical? If other addons or plugins are opening additional connections, and then closing, they may not be showing, if that only shows current connections.

I personally am not able to check on the total connection limit, but we generally have that set much higher than would typically be encountered, so running into that limit is possible, but not likely.

With the server not changing, does that include usage, meaning player numbers are also about the same?
TimeX
Naleksuh
This is my homepage
This is my homepage
Posts: 298
Joined: Thu Jul 25, 2019 12:35 am

Re: Error connecting to database: [1040]: Too many connections

Post by Naleksuh »

Yeah I kind of agreed with dustinandband here that it's looking like it is the overall limit and not regular per-client limit.

Also, the game server and web server will have different IP, so that'd make them be seperate. Are all your game servers running on different IPs from each other? Or one IP and multiple ports?
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Error connecting to database: [1040]: Too many connections

Post by Edge100x »

The limit TimeX is thinking of is actually enforced outside of MySQL and would not give this error. As far as I know, the "Too many connections" error only occurs when MySQL itself hits max_connections.

If you're still seeing it, please have your support request escalated to me, and I can investigate further. My guess would be that another customer was locally opening a large number of connections at once.
dustinandband
A semi-regular
A semi-regular
Posts: 22
Joined: Mon Oct 05, 2015 7:38 pm

Re: Error connecting to database: [1040]: Too many connections

Post by dustinandband »

Edge100x wrote: Thu Sep 02, 2021 1:34 pm The limit TimeX is thinking of is actually enforced outside of MySQL and would not give this error. As far as I know, the "Too many connections" error only occurs when MySQL itself hits max_connections.

If you're still seeing it, please have your support request escalated to me, and I can investigate further. My guess would be that another customer was locally opening a large number of connections at once.
I've seen the error yesterday (Sep 1st) but not today on any servers. Will keep a lookout in case it happens again.

I was figuring that was the reason. I guess technically someone could go crazy with hosting a bunch of servers all connected to the same database and be using up a lot of those connections. I'm using 79 (out of the 1,000 allowed concurrent connections) with 11 servers so I can see that being possible.

Appreciate the input Edge100x. Thanks
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Error connecting to database: [1040]: Too many connections

Post by Edge100x »

Yes, it doesn't sound like it was you.

Quite possibly, another customer's site was misconfigured or broken, or another customer was seeing an attack that led his site to slow down and hold open many connections. If you see it again, please let us know through the support system -- hopefully we can catch it in the act.
dustinandband
A semi-regular
A semi-regular
Posts: 22
Joined: Mon Oct 05, 2015 7:38 pm

Re: Error connecting to database: [1040]: Too many connections

Post by dustinandband »

Happening again

host:
hosted30.nfoservers.com
http://survival.site.nfoservers.com/DemoArchive/
http://survival.site.nfoservers.com/sourcebans/

Code: Select all

Warning: mysqli_real_connect(): (HY000/1040): Too many connections in /usr/www/survival/public/sourcebans/includes/adodb/drivers/adodb-mysqli.inc.php on line 124
When this happens - all of our database related server plugins stop working.
Any chance there could be an automated system in the future to detect this instead of having customers report it manually?

Thanks
User avatar
TimeX
Staff
Staff
Posts: 1730
Joined: Thu Jul 22, 2004 12:24 am
Location: Big Bear, CA

Re: Error connecting to database: [1040]: Too many connections

Post by TimeX »

In this situation, a problem with MySQL on the hosted30 node was detected by the system, and it is currently being worked on.
TimeX
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Error connecting to database: [1040]: Too many connections

Post by Edge100x »

Please don't hesitate to open a support request directly about account-specific issues like this. We will respond within minutes and begin investigating right away. In the forum, we may not see it as quickly, and we can't take some troubleshooting actions without customer permission (using a logged-in account or verified email address).
Naleksuh
This is my homepage
This is my homepage
Posts: 298
Joined: Thu Jul 25, 2019 12:35 am

Re: Error connecting to database: [1040]: Too many connections

Post by Naleksuh »

Edge100x wrote: Fri Sep 23, 2022 8:58 pm Please don't hesitate to open a support request directly about account-specific issues like this. We will respond within minutes and begin investigating right away. In the forum, we may not see it as quickly, and we can't take some troubleshooting actions without customer permission (using a logged-in account or verified email address).
I think they were under the impression it was a software problem. I also thought it was a software problem, until TimeX confirmed it was a host issue. However, this would make automated systems scanning for problems much harder, and also not easily run by NFO. Software owners could create them themselves, but might be affected by the same problems they're trying to spot
Post Reply