In another installment of the troubleshooting IAM system issues series, this week we have a post about debugging poor web agent performance. As always, we present the reported issue, what we found during troubleshooting, and how the problem was remedied.
| Issue | Poor Web Agent Performance: Cache & Connection Management |
| Symptoms | An existing customer recently reached out to us because they were experiencing slower than expected performance from a newly installed web server and web agent. Based on the information that they gathered from the Chrome developer tools, they believed that the page rendering times were much higher than their other web servers. |
| Troubleshooting | The new web server is part of a High Availability (HA) set and it was unclear if they were hitting the new server during their test. To get a better idea of what was happening, we asked them to perform two tests. First, we had them update their host file to point to one of the older web servers in their environment. We then had them perform a simple login test while using the Chrome developer tools to record the transaction. Next, we had them modify the host file to point to the new web server. Again they performed a simple login test and recorded the transaction. After exporting the HTTP Archive (HAR) file from Chrome, we had the customer use JMeter to run two small load tests (with 200 virtual users); in order to target each web server individually, we requested that they modify the host file as done before. During these two tests, we requested that they run the following command on the policy servers to capture the number of agent connections being established from each server during the test: (netstat -an | grep 44443 | grep {web server IP} | grep -c ESTABLISHED}. Once done, they forwarded the HAR exports to us, as well as the web agent logs and the netstat test totals. After examining the HAR data, we confirmed that the new web server’s performance was indeed slower than the older servers (despite having more memory, more processors, and faster storage). The netstat totals showed that there was a significant difference in the number of connections opened during the load test. |
| Condition | Existing SiteMinder system with recent web server addition. |
| Cause | We examined the web agent log files and saw that the HCO and ACO were set to the default values for the following: MaxResourceCacheSize, MaxSessionCache, MaxSocketsPerPort, and ResourceCacheTimeout. The customer had created new ACO and HCO objects for the new server (instead of copying the existing ones) and did not modify them to match the existing objects. |
| Remedy | We requested that the customer match the values previously used for the other web agents. Once the changes were completed and web agents restarted, we had the customer run a simple login test in their browser with the Chrome developer tools. The data captured showed a significant improvement in page rendering speed for the new agent. |
As always, we hope that you have found this information useful. If you need IAM assistance, reach out to SIS today and we would be happy to assist you. And subscribe to our newsletter to be notified about the posting of future articles and other SIS news.
