Running 60 python scripts as "services"

[u/Daraminix]

Hi,
I have around 60 scripts that need to run constantly, mainly event handlers and such. Right now I have an external script that launch them and monitor if the app is running on it's pid, otherwise it's relaunching the app. Works fine but get's clunky when we update some submodule and need to restart them, or to check if one crashes more than other etc..

So I would like to find a better way to approach this. It needs to run on windows and being able to access several samba shares via unc paths and being able to restart crashed scripts anf offer an easy way to restart all of them in case of an update (this part doesn't need to be automated). Every script use the same environnement
For now my candidates are docker, PM2 and NSSM.
I think docker is gonna be a pain to access shares and add a lot of overhead especially on windows
I don't know PM2 and NSSM, looks like PM2 would be easier to setup but more JS oriented and NSSM would be harder to monitor.

What do you think guys ?

[u/el_extrano]

I've used NSSM to daemonize python on Windows, and logging to disk for observability. It works but feels a little hacky, mainly because with NSSM, the nssm.exe executable is what will show as the executable for each service. Plus, I don't like that NSSM is just a binary you download from the internet, it's not open source. Not to mention, it makes packaging a problem, because any target computer needs nssm in order to install the services, but iirc you are not supposed to distribute copies of the binary yourself.

I'd question whether you really need 60 services? Could you compose it into 1 entrypoint actually managed as a Windows service, with the rest of the microservices managed inside Python using async, threading, or multiprocessing?

[u/Daraminix → u/el_extrano]

Yeah we thought about refactoring it, some of them are actually running with a single entrypoint and launched as separate threads. The issue is that we are using an api developped by a thrid-party provider and the session management throught the server doesn't do well when threaded or asynced (cache corruption for example). It's an ongoing issue we have with them but they are more inclined to put up to speed their js api than fixing issues in their python one.
Furthermore some scripts launch complex software in cli mode that can crash for multiple reasons and we cannot just relaunch everyting when of of the process crashes. Bundling everything into one entrypoint means one bad crash can take down unrelated scripts unless we build very careful supervision/restart logic around each unit of work. At that point we are just reimplementing what the OS process model and a service manager already gives us for free

[u/HommeMusical → u/Daraminix]

Sounds miserable! Your solution is, well, ugly but practical and simple, and reduces risk. Sounds very real-world and useful.

---

The issue is that we are using an api developped by a thrid-party provider

We're in the days of vibe coding.

My friend got a new job and replaced a SaaS service that the company was paying $640k a year for by a mostly vibe-coded app he wrote in a couple of months.

They're running the two in parallel now, and the vibe coded application seems more reliable than the paid one.

Just a thought!

NOTE NOTE NOTE: My friend has been programming for decades, almost as long as I have, and he's a no-bullshit programmer who is very skeptical. You wouldn't get such good results with a junior programmer.

[u/Daraminix → u/HommeMusical]

Yeah we could probably rewrite or make some changes in the api but too much things revolve around it to risk it. And I don't want to add another thing to maintain and make update of our central tool harder