Redis timeout error

Question

Redis timeout error

Neeta Parmar 0

Hello team,

We have implemented Redis cache in our application.

Application thrown error like as below.

User's image

We have also increased Min values for IOCP and Worker as shown below screen.

User's image

But still application thrown errors as above.

Please suggest solution for the same.

thanking you

gyrus Team

Adithya Prasad K 90 Reputation points Microsoft External Staff

2025-03-10T14:59:10.0133333+00:00
Hi Neeta Parmar,
Thank you for reaching out and posting the query.
It looks like you're experiencing a StackExchange.Redis.RedisTimeoutException related to your Redis cache implementation in your application. Here’s a structured analysis of the issue at hand and some steps you can take to troubleshoot and potentially resolve it.
Exception: StackExchange.Redis.RedisTimeoutException
Message: Timeout performing EVAL (5000ms)
Context: Indicates that the application is unable to perform a Redis operation within the specified period, leading to a timeout.
Timeout Configuration:
The timeout is set at 5000ms for the EVAL command.
Worker threads (200) and IOCP threads (200) have been configured.

**
Error Message Details**:

High async operations (async-ops: 1).
IOCP values are noted: ‘(Busy=22, Free=32745)’ suggesting some contention or blocking issue.
The server is at GyrusAimProdCache1.redis.cache.windows.net:6380.

Thread Management:

The configuration for thread management is being done dynamically via GetMinThreads().
It logs current values of worker and IOCP threads but highlights that the min number didn't change
Increase EVAL Timeout

The default timeout might be too short, especially if your function is performing complex operations. Consider increasing it:
var options = ConfigurationOptions.Parse("your_redis_connection_string"); options.AbortOnConnectFail = false; // Keep trying to connect options.SyncTimeout = 10000; // Increase timeout to 10 seconds options.AsyncTimeout = 10000; // Increase async timeout to 10 seconds ConnectionMultiplexer.Connect(options);

Analyze Redis Queries

Review the content of the Lua script you are executing with EVAL. Complex or inefficient scripts can lead to higher execution times.

Monitor your Redis server for command performance.

Redis Server Performance

Ensure that your Redis server has adequate resources (CPU, Memory).

Check if there are any spikes in CPU usage or memory limits being reached.

Look into scaling your Redis instance if you are frequently experiencing performance issues.

Application Load Engineering

If the application is under heavy load, consider load balancing or scaling out copies of your application.

Implement a rate limiter or circuit breaker pattern to gracefully handle potential overload situations.

Review Connection Configuration

Ensure that your Redis connection pool has adequate size configured to handle the number of concurrent requests your application needs.

Use ConnectionMultiplexer with appropriate settings for MaxClients.

Exception Handling

Implement better error handling around Redis calls to handle timeouts and retries more effectively.

Time-sensitive operations with Redis can face complications under high load or inefficient query execution. Following the above steps should help you identify the root cause of the RedisTimeoutException and improve your application's stability.

If issues persist after implementing these suggestions, consider deeper profiling and analysis of Redis performance along with application metrics.

I would request you to refer the below mentioned links for more information
https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-configure?tabs=access-keys#connection-pooling
https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/monitor-cache?tabs=basic-standard-premium#available-metrics-and-statistics
https://stackexchange.github.io/StackExchange.Redis/Configuration

I hope this information helps. Please do let us know if you have any further queries. 
Adithya Prasad K 90 Reputation points Microsoft External Staff

2025-03-11T08:43:47.55+00:00

Hi Neeta Parmar,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Neeta Parmar 0 Reputation points

2025-03-11T13:08:52.26+00:00

Hello Aditya,

Thank you for quick reply.

I will check and try to implement solutions suggested by you.

Let you know if require.

Thanking you
Adithya Prasad K 90 Reputation points Microsoft External Staff

2025-03-12T06:45:15.1733333+00:00

Hi Neeta Parmar, We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Neeta Parmar 0 Reputation points

2025-03-12T09:04:57.09+00:00
Hi Aditya,

Please find my notes with bold text.

Analyze Redis Queries

Review the content of the Lua script you are executing with EVAL. Complex or inefficient scripts can lead to higher execution times.

Monitor your Redis server for command performance.

We have not implemented complex code. Also have monitored Redis server not didn't find any error logs etc.

3.                 Redis Server Performance

Ensure that your Redis server has adequate resources (CPU, Memory).

Check if there are any spikes in CPU usage or memory limits being reached.

CPU and Memory appear to have been fine over the last 30 days, with CPU being less than 65% (usually much less) and memory below 20% according to the metrics page.

Look into scaling your Redis instance if you are frequently experiencing performance issues.

We have considered this, but it is expensive and cannot easily be downsized once we scale it, so we want to look into alternatives first.

4.                 Application Load Engineering

If the application is under heavy load, consider load balancing or scaling out copies of your application.

Implement a rate limiter or circuit breaker pattern to gracefully handle potential overload situations.

Similar to above, though unlike the Redis server, the Web App can be easily scaled down if we change our minds. It’s still costly though.

5.                 Review Connection Configuration

Ensure that your Redis connection pool has adequate size configured to handle the number of concurrent requests your application needs.

Use ConnectionMultiplexer with appropriate settings for MaxClients.

In current application, we have implemented Redis using provider

<globalization culture="en-US" uiCulture="en-US" />

<sessionState mode="Custom" customProvider="MySessionStateStore">

<providers>

<add name="MySessionStateStore" type="Microsoft.Web.Redis.RedisSessionStateProvider"

...............................

With the below configuration.

operationTimeoutInMilliseconds="5000"

retryTimeoutInMilliseconds="16000"

connectionTimeoutInMilliseconds ="3000"

pollingMaxExpiredSessionsPerSecond="20"

If possible we will implement We will implement use of ConnectionMultiplexer to set MaxClients.

6.                 Exception Handling

Implement better error handling around Redis calls to handle timeouts and retries more effectively.

We have implemented global application-level error handling in notepad. Will monitor the logs to find our root cause of the Redis issue.

Thanks

Neeta

Your answer

Adithya Prasad K 90 Reputation points Microsoft External Staff

2025-03-11T08:43:47.55+00:00

Hi Neeta Parmar,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Neeta Parmar 0 Reputation points

2025-03-11T13:08:52.26+00:00

Hello Aditya,

Thank you for quick reply.

I will check and try to implement solutions suggested by you.

Let you know if require.

Thanking you
Adithya Prasad K 90 Reputation points Microsoft External Staff

2025-03-12T06:45:15.1733333+00:00

Hi Neeta Parmar, We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Share via

Redis timeout error

Your answer