A local Mac can connect remote Ubuntu 22.04 by ssh for about 16 minutes, and disconnected. The local machine screen said Connection to xx.xx.xx.xx closed by remote host. Then I tried to connect the remote server from different PC of different IP locations by ssh, but the screens of any PC showed kex_exchange_identification: read: Connection reset by peer Connection reset by xx.xx.xx.xx`. Then I rebooted the remote server, and all the PC can connect to the server, but the identical problem happened again after 16 minutes. Any ideas?
Add a comment
|
1 Answer
That is very little information to go by. Can you ping the server when ssh fails? Can you access remotely any other services running on that server (i.e. HTTP, SMTP, ...) when ssh no longer works?
Depending on the results, the most likely explanation is either server crash (e.g. kernel panic) or (even more likely) hugely excessive I/O or CPU load. Both are often caused by remote exploits, especially if you haven't gone out of your way to set reasonable resource limits (number of concurrent processes, max memory usage evaluation etc)
Best would be if you could access remote server console out-of-band (virtual machine, iLO, dRAC, ...) and see what does it say (dmesg, ps, etc) and if it is responsive there at all when the problem occurs as well as immediately before.
Failing having such out-of-band access, to debug the issue, I'd shutdown all services except sshd (e.g. sudo service cron stop etc.) for an hour (check with ps auxf if you've got all of them) and check if the problem repeats.
If it doesn't repeat and ssh is stable then (most likely scenario), I'd start turning services on one by one (with at least 1h delay between) until you can isolate which is the service that causes the issue (e.g. webserver running some vulnerable script that gets exploited by remote attacker).
(Also, it should go without saying, make sure your system is up-to-date regarding security updates -- but that is usually only going to help if the server hasn't been exploited yet. You an also try to look at the system logs post-mortem, but if the issue is high load, they likely won't contain useful information).
