LinuxHostingSupport

If your emails are bouncing back and IP is getting blacklisted, it is clear your server is used for spamming purpose. To find out if spam emails are sent using a PHP script OR by a client, refer

How to find a Spammer on a Plesk Server?

Sometimes email accounts are hacked and are used for sending spam email. The header of such emails contain “Network” in the received line instead of the UID of the domain.

Below we will see how to trace such accounts.

Now, read the mail queue and you will notice, large number of emails are sent to strange email accounts

# /var/qmail/bin/qmail-qread
 1 Jan 2013 01:50:32 GMT  #768553  1214
        remote  someone@domain1.com
        remote  someone123@domain1.com
        remote  someone@domain2.com
        remote  someone123@domain2.com
        ****list continue...****

Now you need to find out the IP that is sending the emails, so use QmHandle tool to read the message header by passing the message ID to it (in above case its “768553”)

# qmHandle -m768553 | less
 Received: (qmail 20390 invoked from network); 1 Jan 2013 01:50:32
 Received: from unknown (HELO User) (1.1.1.1)

This email is invoked from ‘Network’ and the offending IP is 1.1.1.1. Now, search the IP in the server logs i.e. /var/log/messages

# grep 1.1.1.1 /var/log/messages
 Jan  1 12:12:00 smtp_auth: SMTP connect from unknown@ [1.1.1.1]
 Jan  1 12:12:00 smtp_auth: smtp_auth: SMTP user [USER] :
 /var/qmail/mailnames/[DOMAIN]/[USER] logged in from unknown@ [1.1.1.1]

As you can see above, the logs will display the email account accessed by the hacker from IP 1.1.1.1.

Now let’s take a look at the password of the email account we found in the above logs

# /usr/local/psa/admin/bin/mail_auth_view | grep user@domain
 +--------------+--------+------------+
 |    address   |  flags |  password  |
 +--------------+--------+------------+
 |  user@domain |        |   qazxsw   |
 +--------------+--------+------------+

The password isn’t great and no wonder why the email account is compromised.

Now, change the password of the email account from Plesk, restart the IMAP server and monitor the server logs to see the difference

# tail -f /var/log/messages | grep grep 1.1.1.1
 Jan  1 12:20:08 smtp_auth: SMTP connect from unknown@ [1.1.1.1]
 Jan  1 12:20:08 smtp_auth: FAILED: [USER] - password incorrect
 from unknown@ [1.1.1.1]

As you can see above, the hacker from IP 1.1.1.1 can no longer access the email account.

Here are Various ways to find a Spammer on a Plesk server.

If emails are sent using a PHP script on a Plesk server, there are following 2 ways to determine the PHP script.

1) The following command will display the PHP scripts running in real-time. You have to execute the below script at the time the emails are been sent from your server rapidly.

Execute the below command as it is:

# lsof +r 1 -p `ps axww | grep httpd | grep -v grep | awk \
 ' { if(!str) { str=$1 } else { str=str","$1}}END{print str}'` \
 | grep vhosts | grep php

This will continuously display the path to the PHP files as they will be accessed and executed.

2) This method is used when you are not around and still wanted to trace the folder or the domain of the PHP script that is sending emails is running from.

a) Create a /var/qmail/bin/sendmail-wrapper file with the following contents

#!/bin/sh
 (echo X-Additional-Header: $PWD ;cat) | tee -a /var/tmp/mail.send \
|/var/qmail/bin/sendmail-qmail "$@"

Grant executable permission on the sendmail wrapper and replace it with the old sendmail file of Qmail as stated below:

# chmod a+x /var/qmail/bin/sendmail-wrapper
# mv /var/qmail/bin/sendmail /var/qmail/bin/sendmail-qmail
# ln -s /var/qmail/bin/sendmail-wrapper /var/qmail/bin/sendmail

b) Create a log file /tmp/mail.send and grant read/write permissions to all.

# touch /var/tmp/mail.send
# chmod a+rw /var/tmp/mail.send

c) Wait for a few hours and revert back the sendmail files

# rm -f /var/qmail/bin/sendmail
# ln -s /var/qmail/bin/sendmail-qmail /var/qmail/bin/sendmail

then go through the log file /tmp/mail.send. The log file contains “X-Additional-Header” lines that will display the path to the folder name the PHP scripts are residing in. Example:

X-Additional-Header: /home/vhosts/domain1.com

To locate all the domains the PHP scripts that are sending emails are residing in, execute:

# grep X-Additional /var/tmp/mail.send | grep \
 `cat /etc/psa/psa.conf | grep HTTPD_VHOSTS_D \
 | sed -e 's/HTTPD_VHOSTS_D//' `

If no script is listed, it means mail() php function was not used to send emails.

If you feel emails are saturated in the Plesk Qmail mail queue, there is a possibility that your Plesk server is been used for sending spam emails.

On a Plesk server relaying is not allowed by default so following are the ways spamming is mostly done. They are explained below point wise.

1) Using CGI by a user
2) PHP scripts. Also refer the article to locate the directories of the PHP scripts that are sending emails.
3) By a compromised email account

First, lets look at the the mail queue

# /var/qmail/bin/qmail-qstat
messages in queue: 22507
messages in queue but not yet preprocessed: 0

As you can see above, there are a large amount of emails in the mail queue. The source of these emails could either be a PHP/CGI script OR an authorized email account on the server.

Let’s start with reading the message headers with ‘qmail-qread’

# /var/qmail/bin/qmail-qread
5 Nov 2012 11:50:17 GMT #768752 1231 
remote user1@domain1.com
remote user2@domain1.com
remote user1@domain2.com

This will list the sender and recipient of all the emails in the mail queue.

In the above example #768752 is the message ID, now find out the location of this email to read the complete header

# find /var/qmail/queue/mess/ -name 768752
/var/qmail/queue/mess/0/768752

Above is the complete path to the mail file, now open the file and look for the “Received” line.

# cat /var/qmail/queue/mess/0/768752 | more

The “Received” line indicates from where the message was received OR invoked.

1) If the message is sent via CGI by a user, it will display the UID of the user as below:

Received: (qmail 26193 invoked by uid 10001); 5 Nov 2012 11:50:17

Now, search the UID 10001 in the passwd file to find the domain name

# grep 10001 /etc/passwd

This will display the domain name the UID 10001 belongs to.

2) The “Received” line indicates the UID of user Apache (i.e. 48)  if email is sent via a PHP script

Received: (qmail 26193 invoked by uid 48); 5 Nov 2012 11:50:17 +000

In such a case, you have to monitor the PHP scripts in real-time i.e. scripts that are running when emails are been sent.

Execute the below command as it is when the mail queue is growing rapidly

# lsof +r 1 -p `ps axww | grep httpd | grep -v grep | \
awk ' { if(!str) { str=$1 } else { str=str","$1}}END{print str}'` \
| grep vhosts | grep php

The above command won’t display the location of the php scripts, so please refer the article to locate the folders of the PHP scripts that are sending emails.

3) Many a time email accounts are compromised and used for sending bulk/spam emails from other locations. In such a case, “Received” line contains “invoked from network”

Received: (qmail 26193 invoked from network); 5 Nov 2012 11:50:17

Refer the article to find the compromised email accounts on a Plesk server.

Screen is used to run interactive programs in the backgroud while we can logout from the server. We can also re-attach to the existing screen session to check the progress of the running programs.

Sometimes, server admin need to run commands under a different user so they “su” to another user and use screen.

root@server [~]# su - user1
user1@server [~]# screen
Cannot open your terminal '/dev/pts/0' - please check.

As you can see, after changing the identity to ‘user1’, you cannot run screen and instead it exits with the error message

Cannot open your terminal '/dev/pts/0' - please check.

This indicates that the ‘user1’ don’t have access to the /dev/pts/0 file.

This is because the terminal is owned by the user (root) who opens the session so even if you su to another user (user1), the terminal will still be owned by the original user (root) hence the error.

Here are the permission and ownership of the terminal:

# ls -la /dev/pts/0
crw--w----  1 root tty 136, 0 Oct 28 04:34 /dev/pts/0

As you can see the ‘user1’ have no permission to read and write to the file. The file is only readable by root and writable by root and tty group.

There are 4 different solutions as stated below out of which 1st and 2nd are not recommended. They are a security risk and only recommended if you want to perform a very small tasks.

With the first 2 solutions, you may end up giving unprivileged access to a privileged login if you don’t revert the changes.

1) Set read/write permissions for ‘all’ on the terminal device in question which is /dev/pts/0 in our case. This way you can su to any user and run a screen session under his session.

# chmod a+rw /dev/pts/0

2) Set read permission to ‘tty’ group and then add the user ( in our case ‘user1’ ) to the ‘tty’ group in /etc/group file.

# chmod g+r /dev/pts/0

Open /etc/group file and search for tty:x:5: , at the end of the line add the username so he will be a part of the ‘tty’ group

tty:x:5:user1

Make sure you remove the user from the tty group once you complete your task.

3) This is the safest solution and is recommended. Set a strong password for user1, SSH directly with the user and run the screen session under it.

4) This is an alternate solution for the 3rd method.

a) Start a screen session as root
b) change to user1 with su command
c) execute your scripts/command
d) detach (don't terminate) from the screen using Ctrl a+d

I have a server with ‘root’ access but when I tried to SFTP in with the SSH Explorer client, it came up with a “subsystem request for sftp failed” error message. When trying to SFTP in, the SSH (/var/log/secure) logs came up with the following error:

sshd: subsystem request for sftp
sshd: error: subsystem: cannot stat /usr/libexec/sftp-server: No 
such file or directory
sshd: subsystem request for sftp failed, subsystem not found

A user can SFTP in only when SSH access is enabled for the user, in my case, I was using ‘root’ as user and was able to SSH in fine.

On further investigation, I found the ‘Subsystem’ parameter for sftp is defined in the SSH configuration file /etc/ssh/sshd_config.

# grep Subsystem /etc/ssh/sshd_config
Subsystem  sftp  /usr/libexec/sftp-server

This indicates that SFTP is looking /usr/libexec/sftp-server file which is missing

# ls -la /usr/libexec/sftp-server
/bin/ls: /usr/libexec/sftp-server: No such file or directory

The solution is to find the actual location of the ‘sftp-server file and

1) Creating a symlink to it in the /usr/libexec/ directory
OR
2) Edit the SSH configuration file and specify the correct path to the ‘sftp-server’ file in the ‘Subsystem’ parameter.

Locate the sftp-server file:

# find / -name sftp-server
/usr/libexec/openssh/sftp-server

and create a symlink to the actual file

# ln -s /usr/libexec/openssh/sftp-server /usr/libexec/sftp-server

Restart the SSHD service

# service sshd restart

You should be able to SFTP in successfully.