Debugging Windows 2000 COM Applications

Debugging Windows 2000 COM Applications - Documents
(Page 3 of 4 )

Server II Not Responding

I. Configuration
“Server I” and “Server II” are configured to serve website pages in a coordinated effort. The servers are active and achieve balanced delivery of web pages to users under Windows 2000 Advanced Server Load Balancing (WLB).

II. Situation
Server ”I” is the primary server in WLB and delivering 100% uptime. Server II, a secondary server in WLB, is giving 60% uptime and daily requires a staff administrator to periodically resolve “Server Error” errors manually. During office hours, errors cause a disruption of services to staff personnel. Over weekends, services are not provided to external users.

NOTE: Servers ”I” and ”II” are equal in hardware, OS configuration, IIS Configuration and Application code.

III. Problem
Server ”II” consistently stops delivery of services to users.

IV. Error
“Page cannot be displayed, 500 internal server error.”
• System Log – No messages.
• Application Log – No messages.
• IIS Log – Server busy message.
• Performance Monitor – No activity greater then 50% CPU or RAM for longer than 45 seconds.
• System is not “locked-up”. There are no visible performance issues.

V. Resolution
* Unacceptable
• Reboot will restore use to users.
• Stop and start of IIS out of process component will restore use to users.

VI. Server Hardware (Server ”I” and “II”)
Compaq Proliant 6400R- Model 6/500 (1024k cache)
• 4- Pentium III Xeon tm 500-mhz processors
• 2- gigabyte EDO 50ns memory
• Smart array 3200 Controller
• 2- NC3122 PCI Dual 10/100
• 4- 9.1 GB Wide Ultra2 Universal 10k drives
• 3- 18.2 GB Wide Ultra2 Universal 10k drives

VII. Server Software
Compaq Network Teaming and Configuration
• Compaq Load Balancing Team – Compaq Smart Start Version 5.2
Windows 2000 Advanced Server – Service Pack 2 (sp2)
• OS Build –
• Service Pack – Service Pack 2 (sp2)
• Hot Fixes – None
• Windows 2000 IIS Server
Configure Window Load Balancing (Advanced Server) for Server ”I” and Server II
• Affinity setting on primary (Server I): Port 80: None, Port 443: Class C
Server ”II” software has been reloaded according current Server ”I” documentation
• 2 rebuilds, QA, Passed both times.

VIII. TESTING (To reproduce and correct error)
Status
o Error can not be induced manually
1. Error can not be manually reproduced

o Server ”II”- System and Application Information
1. System Auditing – Server ”II”& ”I” are configured the same
2. Application – Components/website copied and setup from Server ”I” to II

o Server ”I”& ”II” Compaq Diagnostic testing, Version 5.175
All tests passed successfully
2. CPU
3. Memory
4. Printing Devices
5. Parallel Interface
6. Graphics Controller
7. Smart Array Controller
8. Serial Interface
9. CDROM

Scenario 1
Reproduce error conditions.

o Server ”II” Test process
1. Directly access Server ”II” by IP
2. Execute lengthy processes, reports, logins, and thoroughly navigate website.
3. Review the Windows Performance Monitor of CPU and RAM during daily usage. Activate and monitor “Profile Trace” in supporting SQL server for unduly large queries or memory requirements.

Scenario 2
The idea of scenario 2 testing was to activate Server ”II” (the secondary WLB server) as a single/standalone primary server providing website services and test its responsiveness.
o Server ”II” Test process
4. Turn off Server ”I”– leave off
5. Uninstall the WLB from Server II
6. Change IP address of Server ”II” to 159.XXX.X.XXX in Compaq team, TCP/IP properties.
7. Server ”II”IP is 159.XXX.X.XXX normally. Server ”I”IP is 159.XXX.XX.XXX, IP of registered domain name is 159.XXX.X.XXX
8. Reset Server ”II”IP to 159.XXX.X.XXX
9. Reboot Server II
10. Test access to website (http://XXX.state.XX.us/) from the inside network and outside the network.
11. Test Failed. Server ”II” is unavailable.
12. Server ”I” was brought back on line, and Server ”II” was reconfigured as WLB secondary server.
13. All services restored to pre-test status.

o Result
1. Server responds as standalone server

Resolution
o Error cannot be reproduced or corrected
IX. WLB Document
Login as Local Administrator
 Stop website
  Go to Network Properties
  Compaq Team – Properties
   Check mark in WNLB
    Properties
     Virtual IP – 10.XX.X.XX (Prod Staff View)
      Test 10.XXX.X.XXX
     Full Internet name: XXX.XX.us
     Host properties
      Priority ID = 1
       1 for first server on team
       2 for second team server
     Check mark in “Initial Cluster” Act…
Dedicated Address – Physical address of the server (NIC card)
Port Rules –
    80 – 80 – TCP – Multiple – EQUAL – NONE
     443 – 443 – TCP – Multiple - EQUAL – Class C
     OK
    TCP Properties
     Advanced
      IP listed should be: 1 Physical IP of server
         1 Virtual IP of team
    Example: Platz – 10.XXX.X.XXX
      10.XX.X.XX (Production Virtual)
No Reboot Required
X. END – No Resolution

Server ”II” Debug Process
Please execute items 1, 2 and 3 to prepare for collection of information when the server crashes. Additionally ”I” will need the information from 1 and 2 as soon as you generate it for analysis. Item 4 should be executed again (now) to make sure it functions properly. After IIS fails, accomplish items 4, 5, and 6.

Please do Now
1) Machine Information. Please extract the files from the IISTools.zip into a directory and run the IISTools.bat file on Server II. This will generate an iistools.txt file.
a) <<iistools.zip>>
b) Put file in USERS/MBURNE1/HANG
2) Application Information. Please run the following applications on Server ”II” (so that we can see if the VB DLLs are crashing) and put file in USERS/USER1/HANG.
a) Windows 2000
i) <<vbchkw2k.zip>>
a. You can run this as follows: vbchkw2k /sc > DLLoutput.txt
b. Put file in USERS/USER1/HANG.
3) Before the hang occurs. Copy the folder named: DEBUG2 from the directory (Technetz) USERS/USER1 to the destination server. The files will work on your server after you have done this without rebooting the server.
a) Open Task Manager and get the PID for Inetinfo (Write it down).
b) Get the PID for every instance of DLLHOST.EXE (Write them down).
c) From the Command Prompt execute this command within the DEBUG2 folder for each PID: userdump <PID>
d) This will put a file with *.dmp into the folder where userdump.exe resides. After each command, rename the resulting file uniquely.
e) Item ‘a’ to ‘d’ above are testing. To insure the process works, delete all *.dmp files from the folder.
With ERROR

Once the hang has occurred
4) The following steps will allow us to get a memory dump of the IIS process as well as a debug log that will directly correspond to the memory dump.
a) Open Task Manager and get the PID for Inetinfo (Write it down).
b) Get the PID for every instance of DLLHOST.EXE (Write them down).
c) From the Command Prompt execute this command within the DEBUG2 folder for each PID: userdump <PID>. RENAME the result file every time the command is executed.
d) This will put a file with *.dmp into the folder where userdump.exe resides. After each command, rename the resulting file uniquely.
e) Get a Perfmon with the following (with all available counters for all of them):
(1) ASP
(2) Processor
(3) Process
(4) WebService
(5) Memory
(6) Thread
ii) Put All *.dmp files and Perfmon files in USERS/USER1/HANG.

5) Restore Services by Start and Stop of OUT-OF-PROCESS Service.
6) Put all files in Users/USER1/hang
blog comments powered by Disqus