Tag: linux
Server Upgrades... lost entries...
My good friend, Rob, hosts my site for me, in return for helping with server maintenance. After being on Gentoo for the past three years, though, we decided it was time to switch to something a little easier to maintain, so last night we wiped the system partitions and installed Ubuntu server.
I'll say this: the setup is much faster! However, we had a few gotchas that surprised us — it didn't setup our RAID array out-of-the-box, which led to a good hour of frustration as we tried to verify that the install wouldn't wipe it, and then to verify that we could re-assemble it. (We succeeded.) Additionally, we second-guessed a few things we shouldn't have, which led to needing to back out and reconfigure. But what was over a 12 hour install with Gentoo we accomplished in a matter of a few hours with Ubuntu server — so it was a huge success that way.
Unfortunately, our mysqldump of all databases… wasn't, a fact we discovered only after importing it into the new system. I ended up losing my blog database and PEAR channel database. Fortunately, the PEAR channel has not changed at all in the past year, so we had an old backup that worked, and I had a snapshot of my blog database from three weeks ago I was able to use. As a result, there are a few missing entries, but for the most part, all works. If you commented on one of those missing entries, my apologies.
Now that the install is done, I'm also finalizing some design changes to my blog — it's time to leave the black and white for more colorful grounds. Look for a revamp in the coming weeks!
Apache HOSTNAME on Clusters
In an effort to debug issues on a cluster, I was trying to determine which machine on the cluster was causing the issue. My idea was that I could insert a header token identifying the server.
My first idea was to add the directive Header add X-Server-Ip "%{SERVER\_ADDR}e
in my httpd.conf
. However, due to the nature of our load
balancer, Apache was somehow resolving this to the load balancer IP address on
all machines of the cluster — which was really, really not useful.
I finally stumbled on a good solution, however: you can set environment
variables in apachectl
, and then pass them into the Apache environment using the
PassEnv
directive from mod_env
; once that's done, you can use the environment
variable anywhere.
In my apachectl
, I added the line export HOSTNAME=\
hostname``. Then, in my
httpd.conf
, I added first the line PassEnv HOSTNAME
, followed by the
directive Header add X-Server-Name "%{HOSTNAME}e"
. Voilá! I now had the
hostname in the header, which gave me the information I needed for debugging.
Gutsy Gibbon review
Early in the week, I decided to avoid the release rush and go ahead and update my laptop to Ubuntu's Gutsy Gibbon release. Overall, it's quite good, with one caveat I'll elaborate on later.
Back on Linux Again
A little over a year ago, I stopped using Linux as my primary desktop due to the fact that a number of programs we were using were Windows dependent. Despite getting coLinux running, I've never been completely satisfied with the setup. I missed being able to paste with my middle-mouse button, and I was constantly having character encoding issues pasting back and forth between PuTTY and windows apps, couldn't access mail easily between my coLinux and Windows partitions, and overall felt that I was losing out on some productivity by not having a native linux environment as my primary OS.
Last week, we had an infrastructure change at work, and I basically realized that my Windows + coLinux setup was going to get in the way of productivity -- and that, at this point, there were now Windows applications tying me to that OS. So, I decided it was time to go back to Linux.
coLinux Recovery
As I've written previously, I use coLinux in order to have a Linux virtual machine running on my Windows XP install. It runs Debian unstable (SID), which gives me all the apt-get love I could want.
Except when an apt-get based install goes bad, that is, like it did Saturday evening. This is the tale of how I got it back up and running.
XP + Cygwin + coLinux == Productivity
I wrote earlier of my experiences using Windows XP, a move I've considered somewhat unfortunate but necessary. I've added a couple more tools to my toolbox since that have made the environment even better.
File_SMBPasswd woes
I've been cobbling together a system at work for the last couple months to allow a single place for changing all network passwords. This includes a variety of database sources, as well as passwd files and smbpasswd files. I've been making use of PEAR's File_Passwd and File_SMBPasswd, and they've greatly simplified the task of updating passwords for those types of systems. However, I've encountered some issues that I never would have expected.
I have the web user in a group called 'samba', and I have the smbpasswd file owned by root:samba. I then set the smbpasswd file to be group +rw. Simple, right? The web user should then be able to update the smbpasswd file without a problem, right? Wrong.
I kept getting errors, and on investigation continually found that the smbpasswd file permissions had reverted to 0600 — i.e., only the root user could access it. I tried using 'chattr -i' on the off-chance that the file had been made immutable (which didn't make sense, as I was able to see the permissions change). No luck.
Based on observations of when the permissions reverted, it appears that the various SMB processes will reset the permissions! An example is when someone attempts to mount a resource from the server; this accesses the smbpasswd file to perform authentication — and at this point the file permissions change. I can find no documentation to support this; these are simply my observations.
So, to get around the behaviour, I created a script that will set the file permissions to what I want them, and then gave sudo privileges to the samba group for that script. This script is then called via system() in the update script just before processing.
It's a hack, and could be made more secure, but it works.
Server upgrades, samba, and permissions, oh my!
Last week, we performed a long needed upgrade to the company file/print/intranet server. Basically, we'd been on a Redhat 8 system, and there were upgrades we were wanting to perform, and couldn't due to lack of libraries. We could have possibly compiled from source in some occasions… but that would likely have shuttled us into a similar dependency hell as using Redhat in the first place.
So, we decided to re-install the OS, and switch to Gentoo in the process. We've found that Gentoo is a great distro for servers — it allows us to tailor the install to the server purpose, and simultaneously provides a clean upgrade path via portage.
Things went primarily without a hitch. We lost a few databases due to a bad DB backup (argh! there went the wiki!), but that was the primary extent of the damage.
When investigating the sytem post-install, I discovered some connectivity issues with Samba. Basically, when connecting via a *nix-based machine, we were getting symlinks reported as being local to the connecting machine, not the server. This meant that symlinks on the server weren't being followed — which caused major issues for those connecting via FTP, Mac, or Linux.
PHP, Cgiapp, and extensibility
At work this week, Rob was doing some monitoring of our bandwidth usage. We have SNMP on each of our servers now, and he uses MRTG to create bandwidth usage graphs that are updated every five minutes or so. He's been monitoring since late last year.
Before January, we had two systems going. The first, legacy, system hosted the majority of the content from garden.org, and was done using Tango 2000, a web application server that ran on top of IIS and Windows NT 4. I say 'ran', because Tango 2000 was the last version to ship; the company that made it stopped supporting it a year later. This meant we could not upgrade our server's OS to Windows 2000 or 2003, nor could we switch to a more secure web server, etc. It was a time bomb waiting to happen.
The second system is a basic LAMP system — Linux + Apache + MySQL + PHP. Rob began migrating applications to it shortly after he started at NGA 3 years ago, one application at a time. Mostly, new applications were placed on it, though in May 2003, he and the other programmer who was there at the time started migrating old applications to the techology. Part of the reason I was hired was to continue this migration.
The migration was time consuming, and plenty of other projects often got in the way. However, starting last July, we made a big push to get it all ported over — before the old WinNT server fails on us. In January, we were able to rollout the new garden.org, which runs on this new technology.
A big reason we were able to finish is because of Cgiapp. I originally ported it to PHP last year around this time, and knew that while I wanted to develop new applications using it, I wasn't so sure I could sell Rob on it.
Amazingly, it didn't take much to convince him. We had already started using Smarty for templates just before this, and were also using OOP in new development. Cgiapp just helped unify these technologies and to provide a nice, standard framework with which to program.
This last can not be emphasized enough. We started developing all applications in three places: an API for data access, a Cgiapp-based application, and our templates. Either one of us could pick up development of an application from the other without having to spend a day or two familiarizing ourselves with the idiosyncracies of what the other had decided was the programming paradigm of the day. Sure, we still have our own programming styles, but the framework makes it easy to debug or extend each others programs painlessly.
Now, back to the bandwidth reports: Rob has noticed that our bandwidth usage has been growing steadily on the new server since we switched garden.org over — a 45 degree line. At one point this week, our outgoing bandwidth was almost 3 T1s — and we were having no performance issues whatsoever. This simply would not have been possible on the old system — nor without Cgiapp. We've managed to produce both a hardware architecture and a programming framework that has proved immensely scalable — which will in turn save the organization money.
I love open source! How else can you create such high-performing software without paying through the nose for it?
Get Firefox!
Those who know me know that I love linux and open source. One particular program that firmly committed me to open source software is the Mozilla project — a project that took the Netscape browser's codebase and ran with it to places I know I never anticipated when I first heard of the project.
What do I like about Mozilla? Well, for starters, and most importantly, tabbed browsing changed the way I work. What is tabbed browsing? It's the ability to have multiple tabs in a browser window, allowing you to switch between web pages without needing to switch windows.
Mozilla came out with a standalone browser a number of months back called, first Phoenix, then Firebird, and now Firefox. This standalone browser has a conservative number of basic features, which allow for a lean download — and yet, these basic features, which include tabbed browsing and disabling popups, far surpass Internet Explorer's features. And there are many extensions that you can download and integrate into the browser.
One such extension is a tabbed browsing extension that makes tabbed browsing even more useful. With it, I can choose to have any links leaving a site go to a new tab; or have bookmarks automatically load in a new tab; or group tabs and save them as bookmark folders; or drag a tab to a different location in the tabs (allowing easy grouping).
Frankly, there's few things I can find that Firefox can't do.
And, on top of that, it's not integrated into the operating system. So, if you're on Windows, that means if you use Firefox, you're less likely to end up with spyware and adware — which often is downloaded and installed by special IE components just by visiting sites — ruining your internet experience.
So, spread the word: Firefox is a speedy, featureful, SECURE alternative to Internet Explorer!