Self Hosted Integration Studio Out of Memory errors but system has plenty of memory

Question

Greetings - I am fairly new to supporting a portion of a Azure Data Factory. One of our servers that is running the Integration Runtime has a current version of runtime and plenty of CPU and memory available but it is throwing out this error a couple of times a day when trying to run a copy db task.

ErrorCode=UserErrorOutOfMemory,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The available memory of the Integrated Runtime (self-hosted) is too small, please increase your machine memory.

Before upping the size on the Azure VM that is running this, would it make sense to reduce the current jobs from the current number of 16 down to say 12? It is quite possible that our team needs to make the copy/reloading of data more efficient as well.

Also does anyone know of any documentation around migrating to OpenJDK or another version of Java to support Integration Runtime on Windows server?

Answer 1

Okay to anyone else that has this issue. Digging deeper into the event logs pointed to a low virtual memory issue. The page file was system managed and the reason we had our issue was there was not enough free space on the drive where the page file was located to both expand the page file and to allow Integration Studio to do it's thing. We expanded this drive and this appears to have fixed the issue.

Answer 2

@mpls

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Issue:

I am fairly new to supporting a portion of a Azure Data Factory. One of our servers that is running the Integration Runtime has a current version of runtime and plenty of CPU and memory available but it is throwing out this error a couple of times a day when trying to run a copy db task.

ErrorCode=UserErrorOutOfMemory,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The available memory of the Integrated Runtime (self-hosted) is too small, please increase your machine memory.

Before upping the size on the Azure VM that is running this, would it make sense to reduce the current jobs from the current number of 16 down to say 12? It is quite possible that our team needs to make the copy/reloading of data more efficient as well.

Also does anyone know of any documentation around migrating to OpenJDK or another version of Java to support Integration Runtime on Windows server?

Solution:

After digging deeper into the event logs, we discovered that the issue was related to low virtual memory. Specifically, the page file was system-managed, and there wasn't enough free space on the drive where the page file was located. This lack of space prevented the page file from expanding as needed, which in turn affected the performance of the Integration Runtime.

To resolve this, we expanded the drive where the page file is located, and I’m pleased to report that this appears to have fixed the issue. The Integration Runtime is now functioning properly without any further memory-related errors.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

Hope this helps. Do let us know if you have any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.