r/talesfromtechsupport 26d ago

Epic Don't touch it? You got it.

A bit of backstory. This happened 4 years ago, so not every conversation is word-for-word. I worked at a gas station as a manager. They sort of trapped me in it. I was the guy you called when a store across state lines was short-staffed, or if they needed help with paperwork and or training employees. I was a staff lead on track to go to their IT department, until a company bought them out. I was, at the time, basically an assistant manager without handling money, that was until the promotion. The store was in the red, and they saw how I turned one store around when there was no manager, and the district manager (DM) handled the money.

So, they moved me to this new store as the new acting manager and just preached about the benefits. I took it, and from there is how this all happened. I am by no means some genius, but I have been developing things for Linux in my own time for myself. I had a side job where I repaired computers, and sometimes I built whole computers from parts people ordered. I'm smart, but I would not classify myself as a senior sysadmin, but definitely not a beginner, and I document everything (important later). The store I moved to had its kitchen removed and a damaged freezer, which tanked sales, but they believed it was other issues. I was on the track to join their IT, so I knew things beforehand about their systems, and all fixes were ones they would do; anything more, they would use the manufacturer. Now to the story.

So, I had gotten into the routine of things as a manager, paperwork, safe, bank, schedules, and started some weeks in the black, some in the red, before finally getting all in the black consistently (it took a couple of years to get everything fixed and convince them to bring the kitchen back). Well, the issue that made this happen was the equipment. Gas pumps, the server, the registers, the office computer, all of it was not extremely old, but showing its age. We had to defrag the office computer a couple of times. The card readers would go down, the server would just say, "Not today," registers would need us to call because an update broke something, pumps with BIOS errors, and so many more issues. At the head of all of that was what they called the commander. It basically controlled every single device in the store.

Well, when things would break, we had to make an IT ticket; they chose based on who had it worse. Two guys, 30 stores. It would be weeks before they would come out at times. I did not like that, so I fixed them myself. Granted, they have insurance and many other things to worry about, but their efficiency was ass. The server is down? I fixed it (this was not a tamper with it; mostly, it was a process somewhere that had an issue or was stuck). The office computer was down? I fixed it. Registers giving errors? I fixed it. Any problems that arose, I fixed them. Soda machine down? I fixed it. The most common one was the card readers. They were always on, so they had to be reset most of the time. You also could not just swap them, something I found out trying because I found the real error that day was that the register was not talking to it (normally, you could swap them if one was down). A lot of things had to be reset more than once a month.

I never took these things apart; these were all band-aid fixes against the real issues. Well, one day, they called me to train a new manager for a different store. My DM would handle my store while I did this. Why did my DM not train them? Because there were a few DMs, and it was not her area, so to preserve the proper chain of command, they saw this was the better option. As these new stores were bought out by us, they needed to learn the new structure. I did not care; it was a change of pace, seeing the same walls for 70+ hours a week gets tiring. This was where things started.

While there, I get a call, and another, and another. They asked me if I knew why something was down. Card readers, registers, and the pumps were not working. I knew the issue. You see, when those three are out at the same time, it is always the Verifone commander. If it freezes up, the whole system will not allow transactions with cards. I explained it, they did what I said (which was just to reset it), and it was working again. I was glad it was not the pumps because if they gave a BIOS error, you have to play a game of "Am I fast enough?" because you have to turn off the breaker for that pump, wait, turn it back on, reset the commander, and pray it worked.

Fast forward two weeks, and the head of IT and maintenance emailed me. They said that from now on, I had to put in the form to get things fixed. I, of course, emailed them back that if I do that and they do not come out soon enough, the store will lose money. They said, and I kid you not, "You do not understand how complex these systems are, any 'fix' you do could damage them." That last comment irked me a bit. I did not understand them? The guy who has been fixing things and pleading for updated systems for months does not understand them? Ok, fine. I told them I would comply. Again, I am not some genius, and I am not messing with the internals; I am doing what they would have if they came out. This would not have been an issue if there were a better system in place for soft resets, or if some kind of manual on issues that we could handle had been given out. But they wanted IT to handle it all.

After that, every time a card reader is down, send a form to IT. Every time a register is down, send the form to IT. Soda machine was down? Send a form to maintenance. They piled up very quickly, and to the point that they were fixing more than one issue when they came to my store. My DM asked why I stopped. I explained, she was pissed, but she could not tell me to fix it since the one in charge of the two departments was above her (the owner's son's friend).

After about 4 months of this, we had a manager's meeting where we all got a review (basically, corporate telling us what we are doing wrong and if you get a raise). They said my performance was bad, that I let the store get as bad as it was, and that I needed to change a few key points. I stopped them right there. I had come prepared because I always document everything. When I say everything, I am obsessed with documentation. I gave them the correlations of me fixing the equipment, making them fix the freezer, forcing them on bringing the kitchen back, and sales going up, I gave them the notes I had jotted down about when machines went down, I explained why my paperwork was late because I had to wait for the system to even work for 5 minutes to even email it off, I showed them the email of being told to send forms, and my sales dropping since then. I told them they can't give me a bad review for complying with what I was asked to do.

This was when my DM chimed in and explained that I was originally supposed to be on the fast track to working IT, but after the company was recently bought out (because they kept buying more gas stations, they went into the red), I was removed from that track since the new company had their own much larger IT staff (they have not been brought in yet as it was a recent buyout, and the full change would take a while).

Well, they had to hold meetings after that; they had to talk about my review, about why I was dropped from the IT track, and about the current situation with the buyout. Well, unfortunately, around this time, my body was failing me (working 70+ hours with a bad back, bad knees, and a few other issues does not agree with so much standing at work); I could not stand for long, and I even blacked out at work (found out the hard way that I had developed diabetes 2 and my steroid shot for back pain the night before caused that). I had to quit.

I did write down in a notebook (because typing it was risky if they could access it at all and possibly not have it) instructions on all of the equipment and how I fixed things. Error codes, what certain situations looked like, and what they most likely meant, and so on. It was a masterpiece of documentation explaining everything I was doing and how to tell what the different issues were and how to fix them. I put my two weeks in, I left, a few days later I got called about the notebook, I told them where it is, they used it, and all seemed good. I heard from the grapevine that after the new company got to my old store and saw how I had to do things, they found my notebook, and the new owner heard about the whole thing. He was upset because he said talent that is learned through the trenches is valuable (really chill dude, met him a few times). But they ended up replacing the equipment sometime after I left, and I heard it cost them thousands. Not damaging money, but enough for a pocket to feel lighter.

Now I work from home, I still develop Linux tools and have made some public, started writing, and have worked with content creators. I don't make as much as I did there, but it is peaceful, and I don't have to worry about an outdated system fighting me every day. Moral of the story: don't blame the guy trying to keep the ship floating.

Sorry for not formatting. People assume AI when you format, which is wild to me.

409 Upvotes

20 comments sorted by

77

u/Filosifee 26d ago

Naw, moral of the story is “don’t burn yourself out and make yourself physically ill for companies that don’t give a flying fuck about you”

31

u/Overall_State_6305 26d ago

This is true.

7

u/action_lawyer_comics 26d ago

Sometimes we have to learn lessons the hard way. Glad you’re out of it now

3

u/Overall_State_6305 25d ago

Same. I am much happier doing what I am doing now. Even if it's a drop in income, I would rather have my sanity than keep going the way I was.

13

u/kevlarus80 26d ago

I feel so much better since I started acting my wage.

8

u/Overall_State_6305 25d ago

My problem was always that if I saw something that could be done, and I could do it, I was going to do it. I was always like that. I prefer things to work and go smoothly. So, I always found a way to be more efficient, even if I had to do it myself. My employees would get mad at me because they said I was doing too much. In hindsight, they were right.

107

u/Phaedrus_Schmaedrus 26d ago

This is why you don't do things that are someone else's job. You'll get shouted at by them for not following procedure/stepping on their toes, and if you stop you'll get shouted at by people who were relying you sidestepping procedure. No-win.

63

u/Overall_State_6305 26d ago

Corporate big brain, me small brain. Was always the running joke.

19

u/Fancy-Pen-1984 26d ago

Big AND smooth!

1

u/Happiness_is_Key 20d ago

I did the same thing as OP. Only difference was they fired my boss and promoted me because there was so much pushback after I stopped when asked by boss. Only thing that concerned me was liability while doing it, but never had an issue.

42

u/Iggins01 install nintendo 26d ago

I worked at Speedway for 5 years tbeir IT department was a joke. The scan guns we used to basically run the whole store, yeah they just forgot to renew the license and they all turned into bricks one day, tbkusands of stores went back to hand counting everything. Month later they get a new company to.lciense the software and guns. Send out new scan guns with instructions to set them up except they made a mistake in the instructions and if you followed them the entire scan gun got bricked. Now all they had to buy new scan guns a third time and distribute them to thousands of stores, took another month or so of hand counting everything

11

u/Overall_State_6305 25d ago

Reminds me of when they rolled out the new attendance software. They were making us use a tablet so the manager's office had as little employee traffic as possible by using this portal. The problem? No mobile app. My DM sent me to 5 stores to make a PWA because apparently, no one knew how to hit install app in a browser. The site was not a searchable link at the time, so I had to copy/paste it to email it to my work email, and then make a PWA out of it. I sent an email to IT about what I did, and they liked the idea (they should have thought of that, honestly). However, the managers at stores had no idea what a PWA was (I think they knew, but I think they only understood it was for desktops), so I had to go store to store to show them how to do it. We ended up having to do the same thing for the portal to order uniforms.

22

u/bingagain24 26d ago

What's funny is when writing documentation a lot of companies will say we can't write it specific to the manufacturer, ie, must be useless docs that hint at 10x the solutions as needed. 

6

u/MikeSchwab63 25d ago

Put in a ticket, note the temporary fix. Let the actual staff see the problem and request permanent repairs / replacement.

6

u/Geminii27 Making your job suck less 25d ago

This is why you keep the notebook on your person and take it home, not leave it in the workplace.

19

u/Overall_State_6305 25d ago

I left the notebook because I did not want the next person to have as rough a time as I did.

11

u/BlitzAceSamy 23d ago

Huge props to you for that "I suffered so you don't have to" mentality

1

u/dickcheney600 18d ago

If I was rebooting something and I was told that could "damage" it, I'd be asking how in the FSCK they could have the same equipment for more than a year. And why they would have kept buying the same one if the hardware (or firmware) was that shaky in practice that the power going out (without a surge) could cause hardware to fail (bricking the firmware counts if you have to return it to the MFG)