THE TECHNICAL SUPPORT DEPARTMENT FROM HELL!!!!

WARNING : May be too intense for some readers. Parental guidance is suggested. Some names have been changed to protect me from Google.com.

The system I am describing below actually consists of two machines. One is the voice mail machine we commonly call "the Blue Thing™," and the other is a standard PC running Windows NT. Every night, the Blue Thing uses FTP to transfer any changes that have been made to voice mail to the PC. And that's where the trouble begins...

2:00pm Friday : The IBX programmer reports that PC-Talk, a program that lets people check their voicemail from their computer, has had its name/extension database wiped out. I check it, and sure enough, it's only showing extensions and not the names assigned to those extensions.

2:10pm : I reboot the PC. Same problem.

2:20pm : I run the Names Download applet. It tells me there is an FTP Write Error, so I run it again. And again. Same problem.

2:30pm : Knowing full well that The Blue Thing Company always blames "IIS misconfigurations," I double-check our IIS setup against BTC's specs. Just as I suspected, everything was perfect. The PC checks out OK, so I inform our IBX programmer that since I don't know a damn thing about the Blue Thing (hey, I'm an admin, not a telecom guy) we'll need to bring in BTC support.

3:00pm : BTC wants to know if anything was done to the PC before it stopped working. Since I hadn't even touched it in three weeks, I tell them "no," carefully avoiding any mention of the hotfixes I applied to it three weeks before.

3:30pm : After going around in circles with BTC, our now-frustrated IBX programmer grabs me and demands to know the last thing I did to the PC. I tell her about the hotfixes and ask her "please, do NOT tell BTC that. They are immediately going to tell you that the hotfixes broke it, which is B.S. because it's been working *fine* with the hotfixes for three weeks now."

3:45pm : Arguing from her office. I start having a very, very bad feeling about all this...

3:50pm : I'm asked to go up into our switch room and talk with BTC support. The BTC technician has me check IIS settings, and then says "okay, well, Sandy tells me you installed some hotfix or something. Can you uninstall it?"

I tell him no, our security department wouldn't like that very much, and for that matter neither would I. Is he sure that it's a problem with the hotfix, and not something else?

4:00pm : He's sure it's the hotfixes. He wants me to stay on the line and uninstall them one by one, until the system starts working again. I tell him "no", because a. I applied a hotfix bundle that consisted of several dozen hotfixes so we'd be on the phone all week, b. the thing worked just dandy for 3 weeks after they were applied, and c. removing them one at a time would most likely render the PC unbootable.

I ask him if there's something else we could check.

4:10pm : He's still insisting I remove the hotfixes, and telling me "just don't tell your security department." I tell him I wouldn't have to tell them anything, they'd figure it out as soon as my machine started attacking them with Nimda and Code Red.

4:30pm : He has me reading off the name of every single file on the PC-Talk install CD, because he can't find the program he thinks will do something or other that I don't understand. I don't think he does, either, but I patiently read off a couple hundred filenames like "7ffy397.oc_".

5:15pm : I decide that my first gut feeling was correct. The guy is a moron. I tell him "look, it's 55 degrees in this switch room, my skin is cracking from the dry air, and I can't really see because my contacts have welded to my eyeballs. I have to get off the phone now. Send someone out Monday." He schedules a call for Monday, says something nasty like "this will probably never get fixed because you can't stick things out" and hangs up.

9:00am Monday : I show up at the office. Sandy tells me that BTC technicians logged into the Blue Thing over the weekend and verified that the problem was on the PC - "they're getting an FTP write error," she says.

9:10am : I ftp into the PC using the BTC machine's username and password, and upload 100 megs in 15 seconds without a problem. I tell Sandy that the FTP service is working fine.

10:00am : I remove IIS and reinstall it from CD, sending it back to the days of Service Pack 3. I go back to my cubicle expecting the security department to have my head on a stick by lunchtime.

3:00pm : We get BTC on the phone again. They still insist it's an IIS problem - "we're getting an FTP Write Error. So it has to be your PC."

When I explain that IIS has been uninstalled and reinstalled, they tell me it must be one of the other hotfixes and to pull them all off.

3:10pm : BTC support is now telling Sandy that I am a bad system administrator because I install hotfixes without knowing what they do and that I'm not a team player because I won't work with them.

3:20pm : My boss is now involved, and wanting to know why I don't just remove the hotfixes. I tell him the same thing I told BTC, and add that I'm not real sure the BTC techs have a clue about what they're doing. He seems to be satisified with that.

3:30pm : I tell Sandy to get a BTC technician on-site ASAP, because their phone support tech keeps having me verify that IIS is working, and then making up some random thing to check. Furthermore, I am completely sick of dealing with them, and if this turns out to have *anything* to do with IIS or the hotfixes, The Boss can fire me and I won't say a word.

3:40pm : A BTC technician is scheduled to arrive at 7am Tuesday. Sandy tells me to show up at my normal time and try to avoid the BTC people.

8:00am Tuesday morning : Sandy calls me at home and asks if it is at all possible to remove the hotfixes. I tell her it's not a hotfix problem, but to go ahead and reinstall Service Pack 6a.

8:10am : My boss calls wanting to know why there is absolutely nothing in my logbook about the PC for the past three weeks. I tell him that it's because I haven't *done* anything to it in the past three weeks.

8:50am : I arrive at work. The BTC technician is insisting that it's a problem with IIS because she's getting an FTP Write error.

9:10am : I disable IIS entirely and replace it with FTP Serv-U so I can watch every step of the process. I start the names download, watch the Blue Thing log in, starts transmitting data at 10 bytes a second, and then - lo and behold - THE FUCKING BLUE THING CLOSES THE SOCKET!!!!! THE BLUE THING DOES THIS, NOT MY PC!!! AND FURTHERMORE, ALTHOUGH THEY HAD ME CHECK ANONYMOUS FTP ACCESS 18 ZILLION TIMES, THEIR MACHINE NEVER EVER LOGS IN ANONYMOUSLY!!!

9:20am : I point these things out to the BTC technician. She insists that FTP Serv-U is lying and it doesn't matter anyway because it's not IIS.

9:40am : Our department head is now involved, and the BTC technician is explaining to her that I am throughly incompetant. I completely ignore everyone around me and start pinging the Blue Thing and discover - surprise, surprise - it's dropping packets. The BTC technician insists it's because IIS isn't running. I point out that since the Blue Thing runs a proprietary OS, whether or not it's dropping packets has NOTHING to do with IIS. I ask her to check the network interface on the Blue Thing. She tells my boss and the department head that I don't know anything and that because she's getting an FTP Write error it's obviously all my fault but I just can't admit it and she's never met anyone so incompetent.

9:50am : I grab my friend from the networking department and have him bring in a traffic analyzer, on the basis that if the network interface on the Blue Thing is going bad we'll be able to see that and at least prove to my boss that something is wrong with it. The BTC technician is pissed because I won't let her do her job. My boss is wisely letting me stomp about and take care of business without any interference.

10:00am : The traffic analyzer is up. Something is jabbering at 10Mbps, and the Blue Thing is one of the very few machines left that still uses 10baseT. "I'm surprised you could get a ping through this mess at all," my friend says. I start grinning in anticipation of being proved right.

10:10am : It's not the Blue Thing. Something else is jabbering, and the broadcasts are going all over the place - we have a bona fide broadcast storm. The BTC technician is still mumbling that I don't know what I'm doing and I shouldn't have applied the hotfixes. My friend and I give each other the look that means "everything we're saying right now is flying over this dumbasses' head."

10:20am : As we're tracking down the source, we find a loop in the network. We remove the cable that makes the loops, and the storm stops.

10:25am : PC-Talk successfully downloads the names directory and returns to operation. The BTC technician is now throughly pissed off at me. I just smile at her. She stomps out of the office and leaves for her next job. Everyone in the office is glad.

10:40am : I reapply all the hotfixes, reboot the system, and verify that it's still working properly. I then stop IIS and put FTP Serv-U back on, and retest. The system STILL works properly. I disable anonymous access and retest - it STILL WORKS!!!!

3:00pm : My report on the incident is complete. I point out that it never even occurred to BTC support to even LOOK for network problems, that at NO point did they ever suspect that something could be wrong at the Blue Thing, and that most of the things they had me check (such as anonymous FTP access) didn't even APPLY to their software. I mention that the next time something happens to PC-Talk, calling BTC should be considered the absolute last resort. I remember to thank my friend by name.

This message has been edited by damien_s_lucifer on October 16, 2001 at 09:23 PM

hope all goes well tomorrow

------------------
Screw this crap, I've had it. I ain't no Mr. Cool.
< =rwa= >

Hmm...

------------------
When 1337 hax0rs start impaling each other with swords and typing code with a hook on one hand, then they can modify the term "pirate."

Same thing with most tech support...

This message has been edited by justcozz on October 16, 2001 at 10:15 PM

I feel your pain, though...I spent two hours on the phone trying to explain to and advanced OS technition (I'll repeat that for those of you who missed it: and advanced OS technition how her company's operating system works.

------------------

quote:

---

Originally posted by Dark-Angel99:
How come {name removed} doesn't like you? I find you really funny :D

---

Never argue with an idiot...he may be doing the same thing

"Okay. I've disabled DMA mode."
"What does the drive do?"
"It opens and closes the tray, just like it did when the IDE cable wasn't plugged in."
"Okay, now I need you to start Netscape..."

explaining how the OS works to an advanced support tech? Heh...

I do have ONE example of a tech support guy that knew what the hell he was talking about. I forget his name, but he worked for Fujitsu - yes, boys, Fujitsu. When I was still doing return authorizations, they consistently had the *best* customer service of ANY company, technical or otherwise...

Me : "You know, I sell a ton of Fujitsu drives, and it is really rare that one fails."

Him : "That's really good to hear. You know, ever since the engineers replaced all the drive bearings with hydro-magnetic fluid, our returns have dropped to almost nothing. That's some neat stuff, works on the Bernoulli principle..."

I'm making up the terminology 'cause I forget exactly what it was. But it didn't sound like marketing BS, so I went and read their white papers... and sure enough, the dude KNEW WHAT HE WAS TALKING ABOUT!!!!!

Amazing....

This message has been edited by damien_s_lucifer on October 16, 2001 at 11:37 PM

------------------
simultaneity is not absolute. So just because you think i'm wrong, from my frame of reference i'm right!

I give tech support.

Admittedly, I don't at the moment, I'm being 'da consultant' as the other one is off and i'm next in line for 'da bigazz promotion'.

One of the vISPS i support is < Free-Online >, a Sheffield-based ISP.

For some reason every weekend a pack of ravenous 85-year old men and women phone up with one of two errors:

1) "My e-mail isn't working." This is such an understatement. 50\% of the time it's not e-mail related. It's the fact that they can't even get connected so when they launch &*\%(* outlook express it gives them an error.

WELL DUH!

"are you dialled up to us? You can't send mail through our relay server unless you're on our network, you know."

"Dialled up? Oh, is that where my computer makes all those funny noises?"

"Yes, thats it. Are you even dialled up when you try to send mail?"

"No...?"

The other 50\% of the time it's a problem with OE. Everyone who calls uses OE, with the exception of 5\% of the M$phobes who use *spit* Eudora. Maybe 1\% use linux, and they hardly ever call in, because they know what they're doing.

Observe:

Me : "Ok, what client do you use to retreive mail?"

Them : "I use free-online..."

Me : "No, FOL is the name of your ISP, that's us. What e-mail program do you use, you know, like Outlook Express."

Them : "I use free-online..."

Me : *Bangs head on desk*

2) The worst ever problem to talk an 85-year-old through ... the DNS error.

Me : "Good evening, you're through to Mike, how can I help?"

Them : "Is there something wrong with your server? The internet won't work."

Me : "Ok, there's no service issues here. In what way do you mean, 'not working'? Do you get an error message?"

Them : "You'll have to slow down ... i'm a silver surfer you know ... i'm actually 85 ... all this is new to me ..."

I then have to talk the octogenarian through, at length, how to tell me what the error is. It's they get connected with no throughput. So ... I think, DNS error. So, I have to laboriously explain how to open a dos box and type "ping < www.free-online.net > ... if they get nothing, I tell them to ping the IP. If they get something, bingo. Now imagine how hard it is to either tell them to manually input the DNS servers.

The other case is reboot.

Me : "How long has it been since you rebooted your machine?"

Them : "Restart Windows? Oooh ... about a week now."

Me : "And you're running Windows ME... ok."

Me : "Restart your machine. It should be working when you've done that. If it's not, give us a call back. Bye."

Them : "Don't leaeaaaave me!"

Me : *bangs head on desk*

Now I'm acting STC (senior technical consultant) I have to help out the people on the desk (we have 2 new starters in our team, so it's a lot of work) and speak to BT.

Speaking to BT is the most frustrating thing one can possibly imagine.

I might explain it later ... lol.

<rant suspended>

------------------
-- incubus
As I chase the leaves like the words I never find ...

I was using Outlook Express this morning, and all of a sudden, it crashed.

When I started it back up again, it couldn't find my mail server.

Why is it doing this Mike?

You think Microsoft is a horrible, proprietary company? Heh. They are *nothing* compared to telecom equipment.

------------------
Net Syndrome - < www.netsyndrome.net >
Catch The Sickness

quote:

---

Originally posted by miNus:
I was using Outlook Express this morning, and all of a sudden, it crashed.

---

Gee. That's a surprise

incubus may spit on it, but I like < Eudora. > No crashes... not susceptible to 90\% of the viruses floating around... good filtering... mmm

------------------
"Take a cable and simulate a sex act on each end. Press some combonation of play and record buttons." - askheaves

quote:

---

Originally posted by Observer:
After reading all that, I was left thinking, "What would the Bastard do in your situation?"

Hmm...

---

I'm sure the bastard would make absolutely certain that the halon system was not the cause of the problem. If you know what I mean.

quote:

---

Originally posted by demonk:
The [b]EXACT reason why I will not call any tech support until I've exausted every source of help from around me.[/B]

---

------------------
<º(((><

* edit: I can't type.

This message has been edited by TallAssAzn on October 19, 2001 at 09:04 PM

quote:

---

Originally posted by RenegadeSnark:
I'm sure the bastard would make absolutely certain that the halon system was not the cause of the problem. If you know what I mean.

---

------------------
Net Syndrome - < www.netsyndrome.net >
Catch The Sickness

Communication is very important in a large organization

I'm going to get a hat made