Amazon EC2 give users an option to setup apps without bothering much about servers. But, when you are with Amazon, you are bound to follow their usage policy.
Any resource abuse will immediately send an Amazon EC2 abuse report. And, this needs quick action to avoid service disruption.
That’s why, we often get requests from Amazon customers to find and fix the root cause of abuse activity as part of our Server Management Services.
Today, we’ll see how 1onlyhost’ Engineers identify and fix underlying problem based on Amazon EC2 abuse report.
Details of EC2 abuse report
Just like any other cloud service provider, Amazon network also implements strict usage policy for security reasons. This helps to avoid any problems with AWS stack due to abuse activity on a single server. And, users never experience any downtime due to resource abuse in the network.
It is the primary responsibility of AWS users to ensure that their instances or application running on AWS stack do not do any malicious activity.
Moreover, AWS team runs constant check on their network and alert users about stack abuse.
At this point of time, it’s worth to check on how the abuse report actually look like.
A sample report forwarded to us by a customer appears as:
We’re writing to follow up on your outstanding EC2 abuse reports. We’ve observed Intrusion Attempts activity from account xxxx370, and haven’t received a response from you regarding this. Please take corrective
measures immediately and reply to this email to notify us that you’ve done so; failure to respond to this notice within 24 hours may result in your instances being isolated
As AWS team give just 24 hours, it becomes really critical to check the server immediately and do a proper fix. Again, in case of abuse, finding the real root cause becomes the trickiest thing.
How we fixed abuse on Amazon EC2
Now, we’ll see how our Security Engineers fixed the abuse on customer’s Amazon EC2 instance.
When the customer approached us with the abuse report, he was in a panic situation. He forwarded all the details given by the AWS abuse team.
So, we began troubleshooting straightaway.
1. Analyzing the report
As the first step of investigation, we analyzed the report sent by Amazon team. It had detailed log file that included the domain name, IP address involved in the suspicious attack, etc.
The IP 34.xx.xx.40 has just been banned by Fail2Ban after
1 attempts against apache-critico.
Domain: xxx.com (5.xx.172.xx)/fabc/sites/xxx.com/web/htdocs/logs/access:34.xx.xx.40 - - [13/May/2019:05:06:11 +0200] "GET /books.php?lang=ab'&id_n=8'\" HTTP/1.1" 200 206265 "-" "-" "-"
/fabc/sites/xxx.com/web/htdocs/logs/access:34.xx.xx.40 - - [13/May/2019:05:06:13 +0200] "GET /books.php?id_n=8&lang=ab HTTP/1.1" 200 206262 "-" "-" "-"2. Identifying the attack
Our next step was to identify the type of attack from the domain. On detailed investigation, we found that the script in the domain was trying to initiate a login-attack on a remote website.
We found the entire details from the Mod_security logs of the EC2 instance.
3. Applying fix
After finding the abuse account, the next step was to apply a fix and stop the abuse. Here, the server was initiating outbound connections to a remote server. This is not intended as per the usual setup.
Therefore, our Dedicated Engineers blocked all the outbound port 80 connections from the server. We did this by tweaking the firewall of the EC2 instance from the AWS console.
We blocked the outbound connections to the specific IP mentioned in the report. Also, we confirmed that no outbound connections are established from the server:
"Failed to connect to xx.43.xx.162 port 80
"Failed to connect to 194.xx.xx.48 port 80
Additionally, we did a complete scan on the instance to find the infected files if any.
4. Reporting back to Amazon
The final step lies in informing AWS team about the actions taken on the server. They would evaluate the fix and avoid further actions on the abuse incident.
In this case, after sending the details to AWS abuse team, the server was saved from further followup actions.
[Do you know that abuse usually happens in un-managed servers? We offer 24×7 server management to avoid attacks.]
Conclusion
In short, none of the service providers allow abusing of resources. When you get Amazon EC2 abuse report, it’s really critical to take immediate action to avoid account termination. Today, we saw how Dedicated Engineers identified and fixed the abuse account on EC2 instance.
