Upon arriving at work yesterday morning, I was greeted by a blue screen with a KERNEL_STACK_INPAGE_ERROR message and a STOP: 0X00000077 code. As I rebooted the box, I received an UNMOUNTABLE_BOOT_VOLUME with STOP: 0XC0000185. Some quick research on another computer showed that these codes were indicative of a bad hard drive. And Dell Diagnostics returned error code 0142 with status byte 78 when it checked the hard drive. I called Dell with error and I should have my replacement hard drive today. Ironically the previous day, I cleaned up my box, placing all my essential files onto my share on the san. There were a few older backups and software installations that I didn’t save, but I can always redownload the software. It has been very annoying and counter productive the last two days to use an older underpowered workstation that I found under a desk. It is frustrating not to have all the tools I normally use frequently not installed. This situation has made me think harder about experimenting with VDI. Having a virtual desktop stored on our san would be very helpful. I could install the tools I typically use and set it up just the way I want to without have to worry about hardware failure. This has been the first drive fail on a workstation I have used that was no longer usable at all. I attempted to read the drive with a Knoppix cd and was unsuccessful. Makes me wish I could have RAID on my workstation similar to the servers I manage. That way I could avoid having to resetup my work environment.
Author Archive for frank
During a weekend install of new firewalls in my main production datacenter we also had an IBM Proventia Intrusion Prevention System installed. The device is set for learning mode for a while before it will start blocking nefarious traffic. I was alerted today by the datacenter which also manages the IPS that we had a large number of events for HTTP_UserAgent_Too_Long. After requesting more detail on the events I was sent a document with several hundred pages of information. This info included source IP, http request, User Agent, and more.
The event triggered for User Agents that were longer than 200 characters, which is also Microsoft’s recommendation for maximum user agent lengths. The vulnerability is in some http servers that will cause a buffer overflow and thus denial of service when the user agent is too long. I found a tool described here and downloadable here to generate large http requests. I ran this against a test web server with IIS and found that it never crashed the server even though I generated user agent string of several million characters. Java seemed to crash on the client before I ever affected the web server. (I did find that I received a URI too long when I put about 19,500 characters in the get statement.) So I determined that my servers were not affected by this vulnerability.
I noticed in the events that were sent to me that the user agents all looked legitimate. Most were between 200 and 250 characters, so they weren’t far over the limit. They were all Internet Explorer strings with many .Net versions displayed. So I spent some time googling user agent strings and found several good articles and sites that I have linked to below.
The User Agent is a string that is sent during an http request that provides the web server with information on the browser being used and the platform it is running on. Microsoft provides a good summary here. You can see what the user agent of your browser is by going to: http://whatsmyuseragent.com/ . When I went to this site using firefox (my default browser) I received this: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4. Checking my user agent from IE revealed: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648) I found a good article that discusses .net and the user agent string. One point that I noticed was that .net truncates the string to 256 characters.
I also found this site that describes how to change the user agent string in various browsers. This site is a searchable database of user agents. And this is an interesting article on a string sniffer that collects strings into a database. Another interesting site lists IP addresses and their known user agents and whether they are a bot or browser.
Yesterday I found a fix to a problem I have been having with my Symantec antivirus server. For a while now, every time I tried to unlock the server group in System Center I received this message: “Error: Can’t communicate with the Server Goup. Verify Network Connectivity and that machines are operating within the Group! If problem persists, try clearing the Server Group cache and re-discovering all Server Groups.” after a brief timeout. I ignored it for a while since I didn’t really need to do anything to the group and the server was running fine otherwise. But yesterday I found this forum thread and this article from Symantec describing a fix to the problem. Apparently there is a registry DWORD value at “HKEY_LOCAL_MACHINE\SOFTWARE\Intel\LanDesk\VirusProtect6\CurrentVersion\ScSComms\LocalData” called “LoginCaCertIssueSerialNum”. This is a counter that increments each time the group is unlocked. Once the value exceeds 256, the group becomes unable to be unlocked. Setting the value back to 1 fixed the problem immediatly for me. I found this right after I had opened a case with Symantec, and a tech called me right after I fixed the problem on my own. Of course he had the same solution, but I asked him why this is set up like this and he had no answer. According to Symantec’s article this is fixed in version 10.0.2, and my server is just behind that. It needs to be upgraded anyway. I have seen so many strange bugs in Symantec software, but they aren’t the only ones with issues.
In the morning I will be flying up to Toronto for my first business trip out of the country. I will be setting up a new datacenter for the company I work for in a Sungard facility in Mississauga, a suburb of Toronto Ontario Canada. Due to Canadian laws, we must keep all our customer’s data in Canada in order to do business in there. So we are putting in an exact replica of our production system but with canadian customer data. I will be putting in 6 physical servers and plenty of networking equipment. 3 are ESX hosts with 4 virtual servers each. This is gonna be interesting to see how virtualizing our system works out. From my experience setting up the servers in our office, I think it is going to work great giving us enhanced performance and flexibility while lowering our equipment costs.Â
There is one problem with this trip though. We shipped the equipment on the second of July. It cleared customs in the middle of last week, but DHL has no idea where the stuff is now. Hopefully we can find the equipment tomorrow, so that we have time to install before the end of the week. I have another trip planned to our datacenter in Phoenix beginning on Sunday so it is going to be an interesting couple of weeks. Of course my papers and finals for my online classes fall in these two weeks too. But the experience will be good…as long as we can locate the servers.
I had to make a small change to the network of a QA box I manage. So I logged in, via remote desktop using a domain admin account and went to the open the network connection. Which was missing. An ipconfig confirmed that I did have a network address, and of course it was working since I was accessing the machine remotely. Very puzzling. A reboot did not solve the problem either. Since it is working, I’m not gonna mess with it. I took this screenshot to document the problem.
I have updated the theme of the site. The theme is K2, a slick theme/framework. I had been using K2 since the beginning of the site, however I was several versions behind. The theme is a pretty basic theme that is fully customizable with CSS. I am not a very creative designer, so I doubt I will ever make it look that slick. This theme has an easy option to change the header image, so I’ll probably start with that eventually. This theme is now wider. The old one was optimized for 800×600 screens, this one is now optimized for 1024×768. Since 97% of the visitors to my blog had at least 1024 or higher this should make it easier for pictures and text to be displayed. Hopefully I will figure out how to customize it more. Any input on the theme is appreciated.
After my previous observation of Endpoint’s memory usage, I came across a disk space issue. On both test Windows 2003 clients, I found that the space on C: was completely gone. Using some space analysis tools I found that on both servers, c:\program files\common files\Symantec shared\virusdefs contained several gigs of temp files. Some research on Symantec’s forum’s showed me that this was a common problem. I also found on this blog that this should be fixed in MR2. I have not checked my version number, but I am assuming that I don’t have MR2 installed. I haven’t yet had a chance to install and test, but have instead removed Endpoint from one of the test servers, and my own desktop where I was testing. Performance on my machine instantly improved. My machine is a brand new Dell, with dual core and 2gb of ram. If it can slow that down so drastically, I am definitly not going to drag older machine’s down with Endpoint. Hopefully MR2 improves these problems.
There is a very useful Windows command that Microsoft didn’t include in any version of windows but should have. This is the uptime command. There is a knowledge base article here. It was released back in the Windows NT days, but still works on all current versions of windows. I have found this to be an easy way to find out the uptime of the windows servers I maintain without have to log on to the box.
Ok, not really, but I do know if somebody is looking at my site. When I stopped keeping my site up to date about a year and a half ago, I let the mint stats that I was keeping break. This past February I connected the site with google analytics. This is the website stats tool that google runs that they bought from Urchin a few years ago. It is a pretty good free stats tool, which seems to integrate well with adwords (which I don’t use). Now though, I wanted a little more real time stats as well as more flexibility. So I went back to Mint which is a great stats program. It does cost $30, but it is well worth the money. I would highly recommend it to anyone who runs their own website. There are numerous plugins that can be added to enhance the functionality, or you can write your own. My installation can also be hosted on my server, which I like. That gives me more control over the settings and allows me direct access to the database if I want to run queries. I am already getting addicted to watching my stats and seeing when I have visiters. Not that I have a lot of visiters, but I do have a couple of posts that get regular traffic. Watching my stats will probably help push me to post more useful articles. Please note that I do not give out the stats information nor do I have any way of identifying who you are while visiting my site.
I’m not sure if anyone really subscribes to and reads my feed (not much new stuff to read) but I noticed today that since upgrading wordpress a week ago, my feed was broken with feedburner. I downloaded the latest version of the feedburner plugin (I didn’t know google now owned feedburner) and modified the settings and my feed is back working now. Feel free to check it more often as I am now going to be more actively putting content on the site and interacting with it more.