The sale of WebCom was both bitter and sweet. The sale represented independence and success for many involved, but it also was the beginning of the end. WebCom was bootstrapped from what money Chris had and some surplus equipment that we got from a customer in exchange for free hosting. That equipment lasted us until late 1995 when we needed to transition from a 486 running Windows NT 3.51 and Microsoft SQL Server, to a Sun Enterprise 1000e running Sybase SQL Server.
I mentioned before that Chris and Thomas organized the company with a 67%/33% split, eventually I would have 1%, taken from Chris’ portion, and Neal [the CFO] got 10% IIRC, of which I think Chris and Thomas gave up 5% each. After we moved to 2880 Soquel Ave, Thomas started working on his exit from the company. That exit would precipitate one of the biggest threats we ever had as a company.
Mental Illness
This next part is difficult for me to discuss, it ends in a tragedy that might have been prevented and deprived the world of a beautiful human being. Mental illness is a real issue, it affects many people and it affects everyone in different ways. At some point throughout our lives, all of us will experience symptoms of mental illness, but if we seek help and treatment, we will be able to recover or manage our symptoms. Chris had bipolar disorder. I figured out a little over a year after I started working at WebCom that Chris suffered from some sort of seasonal disorder, I assumed it was Seasonal Affective Disorder, but it went much deeper. Around September Chris would become increasingly difficult to work with, he would be very critical, and he would yell, make threats, and generally be unpleasant. The littlest things could set him off, but it seemed that once he had his major blowup, he was super easy going and wanted to make it up to you.
I have a strong personality, some would say I’m stubborn, I say I have a strong personality. I was bullied a lot as a child because I was overweight, but I was smart too. My Mother was my primary caregiver and she grew up in a redneck family where they didn’t take crap from nobody. When kids would bully me, I’d use my words against them, so I had a defense mechanism for dealing with bullies. When Chris would have an episode it would trigger my bully defense and I would stand up to him, this often resulted in a reprimand in my employee file or him telling me to leave work, effectively suspending me. My way of dealing with these situations was to absorb his negative energy and apply an equal and opposite force. This is what Chris needed and after a while I gained the reputation of being the only person that could survive these episodes unscathed. Ironically, some of these episodes would result in him giving me a substantial raise. It’s really confusing for a kid to butt heads with his boss, be reprimanded, then rewarded.
Not everyone at WebCom was as strong and bullheaded as me, that took a toll on work relationships, including Chris’ relationship with Thomas. Sometime during early 1998 Thomas was antsy to get out of WebCom, this set the stage for a very poor hiring decision at WebCom. Thomas was the Sr. System Administrator, he did most of the hardware purchasing and was the person at the end of the pager hook when things broke. Carrying the pager would end up making him frazzled a lot of the time, so we started interviewing for a Sr. System Administrator. I was asked to sit in and observe the interviews we conducted to replace Thomas, but I was only asked my thoughts about technical ability.
The Hunt for a Sr. Sysadmin
The first guy we interviewed drove a BMW M5 and was a consummate Silicon Valley type, he seemed full of himself but didn’t pass the sniff test. The second guy we interviewed seemed to be a lot like the typical Santa Cruz sysdmin type, he was a Linux hater and BSD lover. He was spunky, a bit full of himself, and a fast talker. He’d been around long enough to know how to do the work, but also been around long enough to learn how to be a good at bending the truth.
We hired this second guy, despite some of my concerns, and his obvious agenda to push BSD at us when we were a Sun shop with a commitment towards Linux in future endeavors. Not long after this guy was hired, he talked the boss into hiring his wife as another sysadmin. It was a little odd that we had a husband and wife team for sysadmins, since we’d gotten by with just 1 Sr. Sysadmin, and before long the operations group had 4 people.
I won’t name names, unless he gives me as a reference, then I’ll burn him down, but our new Sr. and his wife ran a pornography website as a side business. She did all the web design and he ran the hardware. This wasn’t entirely unusual for the area, but it was getting a little away from our usual hiring practices. What wasn’t immediately obvious is that he’d engineered a way of running his porn site while on company time, and most of the money spent on our operations team was actually paying them to run their own porn site.
Yes, Sr. did do the normal sysadmin jobs, but it seemed like stuff was always broken and he was always putting out fires. What’s best for the business should be what’s best for the employees — repeatedly putting out fires instead of working towards long term solutions will keep Sr. employed and give him an excuse for keeping the rest of his staff employed, but it means the business has outages and downtime; things run less smoothly.
Circling back to the beginning of this post, in late 1998 we accepted an offer by Verio to sell WebCom for $8mn. Sr. and his wife believed that he was going to get some form of security or option as part of his compensation. I never saw the hiring agreement, but I’d speculate it said something to the effect of maybe they would get options in the future. When Sr. learned about the sale of WebCom, the only way it was possible for him to know is that he was monitoring the CEO’s email (Chris).
Betrayal
One day I showed up to work and the Bat Cave was empty, there were no admins around and the lights were off. Chris called me into his office to tell me that Sr. and his wife had changed all of the admin/root passwords on every system the night before and they demanded a $10,000 payment in exchange for the password list. We frantically looked around to see if we had any root shells open, to regain control of our critical infrastructure, but he’d killed all open shells and locked us out.
There was discussion amongst Chris and Neal about what to do, the plan was to give him a cashier’s check, then cancel it as soon as we had changed the passwords. It was a weird and surreal experience, he and his family were sitting out in the parking lot just waiting for us to make a decision. We got the passwords back and changed them all, Sr. and his wife just left their jobs for a paltry $10,000.
The fallout from the betrayal was swift and complete, Chris immediately promoted me to Sr. System Administrator in addition to my existing role, eventually the role would be renamed Director of System Architecture. I got a BIG raise from System Software Engineer to Sr. System Administrator.
I had to vet the remaining Junior employees, they weren’t entirely unaware of what was going on and Sr. had told them not to show up to work the day he and his wife changed the passwords. Chris asked me to “hack” into the email of one of the Juniors because he stored it on his desktop Linux machine instead of our mail servers. He immediately knew someone had been in his machine when he showed up for work and was unhappy and distrustful of me after that. He later went on to become the Sr. System Administrator and stuck with WebCom until the end.
It took about 6 months to completely iron out all of the problems Sr. had been hiding and patching up, by that time the role of Sr. was made easier by reducing the pager fatigue. We went through another Sr. that Verio foisted upon us, we hired a sales guy from Santa Cruz Electronics to be a tech and Junior, and we promoted one of the Tech Support guys to Junior. After the first RIF there was just 2 operations people around, a night Sysadmin and the Junior cum Senior Sysadmin.
That last Senior became a good friend of mine and was best man at my wedding, however I’ve lost touch with him since he canceled his phone number, didn’t renew his domain, and stopped sending me Christmas cards. I miss talking to him and would like to reconnect.
Yeah, the person in question was problematic, to say the least. He had an ungraceful exit from a subsequent startup I was involved with as well that involved $50,000 worth of computer equipment, in $500 chunks (that being his purchasing authority), being routed directly to his home. This was discovered as we were auditing the books as a part of shutting down the company… the CEO and his outsourced accounting firm were completely asleep at the wheel, which was really annoying, as running a tight ship and spotting a similar situation was one of the things he bragged about when we hired him. I’m still in contact with the wife of the person in question, who left him after a long history of abusive behavior.
Chris was unquestionably unmedicated bipolar, although it took me leaving the company to have enough perspective to realize it; I’d grown up dealing with that (my Dad had it), and unconsciously deployed all the same coping tactics I’d used as a kid when dealing with his irrational behavior… the sad thing is, once he left WebCom, Chris actually got himself medicated with Paxil, but wasn’t properly supervised by medical professionals (he went off on a road trip in his van), so a known side effect, suicidal ideation during the first six months after starting on it, wound up killing him.
I left WebCom because Chris had systematically sabotaged every single acquisition opportunity to date (the way he spiked the GlobalCenter acquisition, in particular, was just egregiously obnoxious),
while the company’s operating margins were headed towards zero, and our competitors had access to capital at a fraction of effective interest rates we were paying. I believe Verio sold junk bonds to the tune of $100 million at 8% interest at one point; the effective interest rate on our last major purchase of equipment from Sun (the E4001, hardware storage array, and tape backup) was around 30%. In my mind, this all equated to the enterprise value of the company evaporating to nothing in no time if we didn’t do something quick (i.e. sell). Chris was also a micro-manager and control freak, so the twin options of external investment and professional management were also off the table. These liabilities were unquestionably hitting our bottom line in terms of revenue growth and profitability. The hiring of a CFO was a major change, and I recall the gentleman in question commenting that cleaning up after Chris was like dealing with a circus elephant.
Ironically, as soon as I left, Chris got serious about selling the company; I heard that he basically didn’t want to operate it without me. In point of fact, he was pressuring me to finalize the exit or change my mind all the way up to the time I signed the final papers, and I later learned that, only a few days before I did so, he recieved an acquisition offer from Verio, but did not disclose that to me. Which kind of pisses me off, as it screwed me out of about a million dollars, but whatever. I probably would’ve pissed away that on stupid investments as well, LOL.
You talked about WebCom SMTP in an earlier post; you didn’t mention that Chris holed up in his office for two weeks solid, doing nothing but coding on it, then handed it over to you and declared it was 85% done. Really, whatever the technical merits of it were, in my opinion, reimplementing one of the basic protocols of the Internet, and investing two years of your work time, that of our lead programmer and smartest engineer, was utterly irrational for a company our size. Pure Sun Sendmail was clearly a mess, but the smart thing would’ve been to replace it with some other scalable, open-source solution (there were several options at the time), but Chris had to maintain control over everything, so that was just not an option. Especially after he invested two weeks of his life coding on an in-house solution.
Chris wrote the initial draft of the schema and stored procedures for WebCom SMTP, enough to get the listener component working, I wrote all of the C code and revised a good portion of the schema and stored procedures to actually make them work as we neared production. We had quite a few hours of strategy sessions discussing the best way to store email messages inside the database server. Development of WSMTP was completed over several years in 3 major phases. Rightly or wrongly, the plan and choices worked well enough to outpace the message handling capacity of any competing solution available.
The main reason we could handle more message volume is that the discrete steps of processing messages were broken up into parallelizable processes, we effectively pipelined email processing. The part that came first was the WSMTP listener, because NMS was falling on its face and we needed a way to intercept and proxy messages around NMS so it wouldn’t handle outgoing customer email. In those days we put WSMTP on the e4000, ran NMS on the Sun Ultra1, and routed outgoing mail to the Solaris x86 box.
The Solaris x86 box became an impediment so we replaced it with Linux, which more than doubled its speed with no other changes. That was the first production Linux server we had. When we went to production with WSMTP we had a bunch of Windows NT 4 boxes running Sybase, they weren’t particularly robust. I happened to go to the Linuxworld Expo not long after that and Sybase was giving out free copies of Sybase 11.5 for Linux. There was no license agreement we had to sign, it was just a set of packages you could install. I think this was intended for evaluation purposes, but in their haste they forgot to define limitations. I built up another DB server and named it “Redeemer”, after the weapon in Unreal Tournament, but most people thought it had to do with Linux redeeming us 😀 I installed Sybase 11.5 on that box and figured out that the simultaneous connection limit was 412 connections, so we set it to 412. Somewhere around that time I also installed Sybase 11.5 on one of the Intel Apollo boxes and ported the WebCom Counter schema to that, so counter.webcom.com, aka abacus, was a standalone Linux box running Apache and Sybase.
Soon after Sybase 11.5 made its debut in WSMTP, all of the NT boxes were quickly transitioned over, this lessened the administrative overhead of those DB servers and picked up some speed. When Verio took over, Chris Merz got absorbed by Verio after the first RIF and went into their DBA group. He was responsible for bringing up Sybase 11.9.2 on Intel hardware for them, after which we were treated to some spanking fast Intel P3 XEON boxes. WebCom ended with 1 of those boxes taking most of the DB load (we could load balance connections at the client level) for WSMTP.