AWS S3 bucket logs not ingesting to Microsoft Sentinel

Question

I have configured the AWS S3 data connector in Microsoft Sentinel. Ref: https://learn.microsoft.com/en-us/azure/sentinel/connect-aws?tabs=s3. I have created a S3 bucket and Simple queue service as documented on the connector page. Furthermore, I have also created an assumed role and have configured the data connector which has the current status as connected. Now when I am uploading a log file in the S3 bucket. I am uploading a gzipped json file in the s3 bucket and have selected AWS CloudTrail in the data connector. I am able to see the push and pull rate in the SQS but I am not receiving any data in Sentinel.

Answer 1

Hi @Deep Thakkar ,

Based on your description and the fact that messages are being pushed, sounds like the connection to the data source is working, but one of these issues applies:

• The data can't be read from the SQS bucket
• The AWS service is not exporting the right logs
• Microsoft Sentinel is lacking permission for this KMS to decrypt the files
• Event notifications aren't defined correctly.

I would recommend checking these possibilities:

1. Make sure that Microsoft Sentinel has permission for this KMS to decrypt the files. Review the required KMS permissions for the GuardDuty and CloudTrail logs. You need all three policies to enable the AWS Sentinel account's assumed role to read messages in the queue, allow GuardDuty to send logs to S3 and read the data using KMS, and allows CloudTrail to encrypt the logs it sends to S3.
2. Ensure that you have specified which supported event types Amazon S3 should send the notification to.
3. Check that the notification is defined from the specific folder that includes the logs, and is defined with the .gz suffix
4. Verify that there are no errors in the health logs by running this query:

SentinelHealth
| where TimeGenerated between (ago(startTime)..ago(endTime))
| where SentinelResourceKind  == "AmazonWebServicesS3"
| where Status != "Success"
| distinct TimeGenerated, OperationName, Se

Make sure that the health feature is enabled:

SentinelHealth | take 20

For additional troubleshooting, see: AWS S3 troubleshooting. Note also that it's normal for it to take up to 30 minutes for the data to ingest.

If everything checks out but you're still not receiving the logs, I would recommend disconnecting and connecting back with the global admin tenant permissions. I'm happy to go over the logs and screenshots of your configuration if you would like to share more details.

If the information was helpful to you, please Accept the answer. This will help us and improve searchability for others in the community who may be researching similar questions. Otherwise let us know if you have further questions.