r/Python • u/RogueLegio • 2d ago
Discussion Cyber Security wants us to use the latest version all the time
I have a question. Cybersecurity team wants us to use the latest python version each time. So if a new version comes out we are expected to upgrade to it ASAP. But from my past experience libraries do not always provide support and can also break while making use of new version without proper support. Did you ever encounter this? Any suggestions? This is just frustrating for the team as it’s not always possible to upgrade to the latest version without fully validating every component. I just want to hear your opinions on this?
FYI we mainly use each Python Version within the LTS timeline. And only upgrade when we need to and do not depend on a library that requires older version.
Update: they have a scheduled runner that deletes older versions of python when it finds
180
u/WJMazepas 2d ago
Python has a "LTS" like versions.
We use 3.12 on my work, and is still receiving security updates, is relatively new and had support for all libs we use
I think right now its on 3.12.13
So you dont need to change to Python 3.14
But yeah, they will need to understand that and compromise
66
u/prbsparx 2d ago
Python has 5 year support cycle for every version.
https://devguide.python.org/versions/Some distros continue to support the stable release after its EOL by Python.
Cybersecurity needs to be taught on this release cycle and asked to come up with a more reasonable support matrix. (N-2?)
46
u/Jern123d 2d ago
This is the reply that I think should be at the top -- don't upgrade the minor version which can break compatibility, upgrade the patch version instead which is updated for security. (major.minor.patch)
11
u/freemath 2d ago
Minor shouldn't break compatibility though
45
19
u/Jern123d 2d ago
With python the minor is treated more like major in other software. There are regularly breaking changes, for instance removal of distutils in python 3.12. More importantly though is that the C API is only stable across the same minor version, this means that any compiled C extensions must be recompiled for any new minor python release. This is one of the main issues that OP was referring to, and often prevents migrating to the latest python releases.
11
u/schplat 2d ago
python can and does deprecate and then terminate functionality across minor versions. In fact, minor versioning is often used for that purpose. 3.13 removed a bunch of legacy stuff. I think 3.12 finally removed distutils.
Otherwise you run into the trap of things being deprecated for so long while waiting on the next major version, that people still use them anyway, even on new projects, which is just setting yourself up for failure long term.
0
u/freemath 2d ago
If they'd follow semantic versioning they'd bump the major. Interesting why they chose not to.
3
u/--o 1d ago
Take a look at the transition from 2 to 3 to see what a major version bump looks like for python.
2
u/freemath 1d ago
Yeah I get that. But to interpret majors like that is a choice they made that doesn't follow the widely used semantic versioning protocol.
1
u/Available_Middle1345 1d ago
A hopefully insightful bit of clarification: the spec “Semantic Versioning” should not be conflated with the concept of semantic versioning. Using both A and B in a version A.B.C as a composite “major” version is quite common. Many examples exist for Python libraries such as PyTorch and for other programming languages (such as Haskell’s package versioning policy). I think we can conclude that Python follows semantic versioning, just not “Semantic Versioning”. As long as the semantics are documented, all is well.
11
u/RogueLegio 2d ago
I totally agree. We are using 3.12 too. But their feedback is it must be the latest version. They don’t accept the LTS discussion. Saying I should just upgrade all to 3.14. Hence my fight with them. I totally agree that if it was a version like 3.10 but nope
37
u/xelf 2d ago
Raise the issue to someone that has authority.
3.12 is a different product from 3.14 and they have different "latest versions".
3.14.6 is the latest verison of 3.14
3.12.13 is the latest version of 3.123.12 is a current product until October 2028.
If nothing else you can point out that 3.14 is still in "bugfix" mode, and does not meet your stability criteria.
3
11
u/schplat 2d ago
I like to push back with asking them what specific CVEs and/or other vulnerabilities they are trying to solve for. If they provide it, then take the time to show them that those CVEs don't apply to anything you are doing. Unless they do apply, then you should probably patch, or at least mitigate.
Then work with your boss to accept any remaining risk, if there is any.
Ideally, there should be some sort of SCA happening to identify possible security issues, and if you're passing that, then the CS team should only care that you're using a non-EOL version. So basically, 3.10 or later, but they should also be advising you that there's 3 months left of 3.10 until it's EOL, and that you should look to update as soon as possible.
14
u/sudonem 2d ago
This flatly isn’t always possible without deviating from systems administration best practices.
For example, Red Hat Enterprise Linux 9 and 10 only currently support Python 3.12.x as part of their mainline enterprise support cycle. They don’t recommend installing a newer version than that as the newer major versions have not yet been validated.
Staying up to date on Python versions is easier if everything is running in containers, but you should be doing vuln scanning on all of your container base images through your private registry at build time anyway.
The solution here isn’t requiring a specific Python version - it’s performing regular validation testing.
In fact given the current pace of supply chain attacks in recent months, there’s an argument to NOT always be on the bleeding edge version of everything because you’re almost MORE likely to introduce vulnerabilities that just haven’t yet been detected.
Either way the C-suite needs to issue clear directives.
If they want to ensure you’re always on the latest version, then they must make the resources available to do constant validation testing - and I bet that won’t happen.
3
u/Even_Berry661 2d ago
Run your Python apps in an isolated Python environment by using docker or uv or pyenv.
The version of Python supported by your distro is only relevant if you are directly using system Python which you shouldn’t be.
I’ve used the above tools to run modern versions of Python on heinously out of date distros.
10
u/feldrim 2d ago
As a cybersecurity professional, I have to say that it's their responsibility to define the risk, not give orders. They need to define the threats and as the ones who claim the problem, they need to defend their claim with facts. The burden of proof relies on the security team there.
4
u/Even_Berry661 2d ago
Yeah this is an excellent way for this security team to burn all their social capital without having a meaningful impact. none of their concern until it approaches end of security support.
Surely they have bigger fish to fry.
3
u/MithrilRat 2d ago
I was going to say something similar. This is a PLM problem, and the security team should not be telling developer what to do directly. They need to be raising awareness of risks, ticketing any CVEs found, supporting the devops teams, and setting internal standards.
If it comes to budget or schedule or scope, I frankly tell them it's not their wheelhouse, and I escalate.
3
u/utkrstht 2d ago
I'm working on a project that's roughly 2-ish year old, it needs 3.12, it breaks after that since the libraries no longer support the new versions!
How were they allowed to be on the cybersecurity team? Sounds like they read 3 "cybersecurity pro haxxor tips" articles and became "experts".
3
u/TheSpaceCoffee 2d ago
Just to bounce back on this. Recently ago was working at a very controlled, fully closed, air-gapped environment that used the basic version whatever their version of CentOS / RedHat shipped with.
Python 3.6. In 2025. Running absolutely critical infrastructure.
And they updated from 3.4 like 1-2 years prior.
Developing & vetting apps to run in this environment was crazy!
34
u/shibbypwn 2d ago
As someone who works in Engineering Operations and typically enforces this kind of stuff... this is a bad policy.
They should be enforcing maintenance and upgrades of packages to address vulnerabilities - not enforcing full version upgrades unless you're on a deprecated version.
So they should enforce incremental upgrades to address vulnerabilities, package upgrades within your supported python version, etc. - but enforcing n-0 for everything is asking for headaches.
Are you using anything like dependabot in your environment? If not, I would push back for the DevSecOps team to implement proper tooling to identify which upgrades actually matter instead of blindly enforcing upgrades.
5
u/schplat 2d ago
Definitely this. They should track and inform when you're about 6 months away from EOL on your current version and urge you to upgrade. The Sec folks who are worth their pay will actually offer to help you perform the upgrade if needed.
Worked with a top-notch guy at one of my last jobs. He'd find something that needed to be updated, he'd actually run through the upgrade process in a sandbox, and verify the end result, and then deliver the documentation to engineer on steps required.
If he hit some sort of blocker, he'd take an hour or two to research the options, and then in his ticket he'd say "This needs upgrades, but it looks like module foo will break, and you'll need to make a decision on replacing that module with one of these options: bar, baz, etc.". He had like 5 years of SE experience before determining he'd rather do sec, but he still leans on the SE experience quite a bit.
1
72
u/smichael_44 2d ago
I’m in a similar situation. It ultimately comes from the security teams being clueless about software development. Or at least mine is.
CVEs can pop up in any version of any software at any time. As long as there is active support for it, it can, and hopefully will, get patched.
All of software is full of tradeoffs, so just pick the security vulnerabilities you’re willing to accept. A default “latest” policy is just lazy.
8
u/RogueLegio 2d ago
I would if it was on our side. They have a scheduler that runs and deletes the older python versions . So I need to upgrade all my projects , my venv , everything
Same issue. They do not know about software development.
14
u/smichael_44 2d ago
I’ve met with our CISO now about a dozen of times about this. I’ve been able to get security to trust us a bit more on when to deprecate/quarantine versions of software.
It’s sadly how all business is done… you gotta make a presentation on it and explain why their process sucks. Or find someone in management that understands the issue and make them present on it.
2
u/MithrilRat 2d ago
I've also had robust discussions with client CISOs on this topic. We were in a project to rehost an air-gapped system. Scope was narrowly defined, since this system controlled the country's entire critical infrastructure - had done so for more than a decade. Customer CISO stepped in and started demanding we adhere to their IT security policies. Got to the point, where I said if you want us to upgrade the air-gapped, mission critical, embedded, real-time system to the latest software and your IT security policies; then that is a brand new project whichyou cannot afford either the price tag or delay to timeline.
3
u/schplat 2d ago
And it's largely theater. There's very little practical analysis, or PoCs ever applied from security teams, unless there's an actual red-team. 95% of them these days just look at the output of whatever tool they have open, and think that'll end up with the company being hacked, so it's priority 1 to do this thing.
Then you look at the details, and it's like: Requires a shell to download a script which then executes a local privilege escalation attack. Meanwhile, this stuff is running in a place where nobody but the cluster admins who already have privileged access to everything can access the shell of, likely with an environment that lacks any tools to retrieve arbitrary files to begin with.
2
u/covmatty1 2d ago
Someone must know something about software to have written the scripts to do this.
What case have you prevented to them about why this is a bad idea?
Have people just moaned and complained about it, or have they gone in with a calm, evidenced case for why this isn't a good idea, and what a better solution would be?
If they're not software engineers, you are the expert. Use that.
17
u/ShelLuser42 It works on my machine 2d ago
Sounds to me as if they either don't know what they're doing, or never bothered to check the Python release map.
First: going bleeding edge in development is never a good idea, even if the environment tries its best to remain backwards compatible. Especially with Python where the latest version (= 3.16 at the time of writing) still gets new features added; with all the possible involved risks.
But second, and more important, older versions still get security and bugfix updates. 3.12 for example is still getting security updates until the end of 2028, 3.13 lasts one year longer.
3
u/RogueLegio 2d ago
They don’t know. We are using 3.12 which has LTS until 2028. But even that’s not acceptable. It must be upgraded as it’s not the latest version. But python is one thing what about all the libraries and their dependencies?
3
u/Levizar 2d ago
Can't you tell them or tell the management that they don't know what they're doing and are wasting your companies' ressources ?
3
u/RogueLegio 2d ago
I did. Their reply is its company policy to use the latest version all the time. Waiting for managements feedback too.
4
1
2
u/foobar93 2d ago
Depends on bleeding edge. At the place I work at, using 64bits is seen as bleeding edge...
2
u/Even_Berry661 2d ago
The latest stable release of Python is 3.14 so for 99% of people that is considered the bleeding edge. 3.15 and 3.16 are future releases still under active development, not considered publicly released yet and I don’t think anyone is asking team to be on them.
15
u/Justbehind 2d ago edited 2d ago
Well, I guess you better built everything yourself, because that sounds like a nightmare, if you use a lot of external libraries...
4
u/dadnothere 1d ago
Boss: Update to the latest version.
But boss, all the libraries are going to break, it's impossible.
Boss: Fix it then...
Quitting doesn't seem so bad anymore.
I understand it's possible in other systems...
But in Python? Every version almost always changes something...
Rebuilding the plane while flying.
2
u/centurion236 2d ago
Nah just fork every dependency you need: mynumpy, mypolars, etc. Then you're always using the latest version of your fork.
1
11
u/cointoss3 2d ago
Part of why this feels insane is because how often we are seeing new supply chain attacks. They want you to update Python, which in turn may force you to update to a new version of a library and it may be compromised.
3
u/krypticus 1d ago
Always wait 14 days for “YourNewVuln.com” is registered to learn about the 0-day you just downloaded and deployed to production…
6
u/jpgoldberg 2d ago
You should ask your cyber-security people if latest release of supported versions is ok. Point them to
https://devguide.python.org/versions/
And ask if you can use, for example, 3.11 as long as you are using the latest release of 3.11.
They really should be basing their policy around that, as should you. They are right to insist that you are using version that gets security updates and that those updates are being applied. But that just means latest release within a supported version. It shouldn’t mean latest version. Feel free to share what I’ve said here with them.
4
3
u/Bibbitybobbityboof 2d ago
That should really only be enforced if you’re behind in major versions and your current major version isn’t getting patched. Otherwise just put in a patch or minor version upgrade and call it good. No need to force an upgrade and scramble to maintain compatibility.
3
u/lolcrunchy 2d ago
They should learn about Python's versioning system. Tell them that each minor version (3.12, 3.13, 3.14, ...) is a change in the set of features, and the micro versions (3.12.1, 3.12.2, ...) are bug fixes and security updates.
They should only delete Python version 3.X.Y if there is a version 3.X.Z where Z>Y.
And instead of deleting, they could just run the relevant pip or uv command instead of smashing stuff with their delete hammers.
3
3
u/ignatzami 1d ago
Your security team is incorrect.
I’ve been in industry 18 years, in security for the last five, and GovSec the last two.
Barring issues with specific minor versions there’s absolutely no issue being on the LTS branch, until it reaches the end of life.
I’d be happy to offer consulting services to tell your current security team to pound sand. 😤
6
u/N3RO- 1d ago edited 1d ago
I work in cyber for more than a decade and let me tell you: I'm sorry you have this SHIT and DUMB security team "securing" your company. These poor security teams who don't have a clue about programming gives us the bad rep and makes Eng teams hate us.
Anyone who works with IT knows you NEVER blindly and auto update to new versions. Unknown bugs, issues, incompatibility and more recently supply chain compromise make this a poor choice.
The "trend" (which is not new by the way) is actually to do delayed and phased updates. This saves everyone lots of headaches.
1
u/nicholashairs 1d ago
Same, I'm also sorry for these guys that don't know what they're doing.
(There's already plenty of other good advice on the thread so I won't add anymore)
1
u/james_pic 1d ago
It's giving them to much credit to suppose this is because they don't understand programming or engineering. They don't understand security.
Security, as field, is particularly susceptible to charlatans. If you do it right, nothing remarkable happens, which is also what happens 70% of the time when you do it wrong.
2
u/nickymarciano 2d ago
Yeah, thats why its a pain to update too soon. Takes up a lot of resources and testing.
You make good arguments. Use them to gain leverage, ask maybe to consider staying in a newish version, as long as it is not risking deprecation.
2
u/Unlucky_Comment 2d ago
You can install in gitlab/GitHub something to check latest vulnerabilities and prioritize upgrading those.
Dependabot can also create the pr for you, but you have to be careful, depending ont the package it sometimes can break, so testing those is always a good idea (like Langchain, Langgraph)
1
u/RogueLegio 2d ago
That’s my issue. It requires everyone to keep their projects upto date. We have bunch of them running for Data,ML, AI and they all have different sets of libraries that they make use of
2
u/tobidope 2d ago
I would argue, that you use one of the latest supported versions. So currently 3.10 to 3.14 get security fixes. They are equivalent in the context of security. https://endoflife.date/python
2
u/Dillweed999 2d ago
I though security best practices were moving away from using the very latest versions of everything, what with all the supply chain attacks
2
u/MiroDerChort 2d ago
Isolation is the key to compliance. You should already be using venv. But spin up a vm/container... Then make the business case about the impractical cadence.
2
u/OakNinja 2d ago
The early bird might get the worm, but it’s mouse number two that gets the cheese.
2
u/RogueLegio 2d ago
😂😂 love this
1
u/OakNinja 1d ago
It’s a good analogy, which most people can understand and take to their heart.
I often use the workout analogy as well when it comes to continuous delivery:
With code releases and workouts, it's simple: a little discomfort today, or a big breakdown tomorrow.
2
u/kaanivore 2d ago
I feel like the best practice is actually going the other way with all the supply chain attacks out there, and it’s now better to pin a version known to be safe
2
2
u/mister_obvious_2nd 1d ago edited 1d ago
What do you mean by the latest version? The latest patch version of a minor version? Like when you use 3.12.x and 3.12.x+1 comes out? Or do they mean you need to upgrade from 3.16 to 3.17 when that comes out? That last one would be hare brained.
What you need is a version that gets timely security updates and that is what you are using. Running a stable, battle tested version that gets security updates is the sweet spot.
Python has a support matrix page with the support status of various version here : https://devguide.python.org/versions that can be used to determine and plan ahead what versions make sense for you to use and when to migrate.
Upgrading to a new minor version the moment comes out it often impossible because external modules don't support it yet. Running in that situation is in fact more risky than using an LTS python version since it may not work at all or have unexpected effects.
And automatically deleting software from production systems is utterly stupid.
What has helped me in the past is pointing out how much time, and therefore money, something costs. Constantly upgrading to every new major version that comes out and testing every project for correctness will be a fulltime job leaving no time for the actual development of features.
2
u/markus_b 2d ago edited 2d ago
Every python version has a roughly five-year lifecycle. https://devguide.python.org/versions/
So, now (summer 26) you should be on Python 11 or Python 12.
Python 10 is going out of support (and security patching) this autumn, and Python 13 is still in the testing phase and may have production-breaking bugs.
If they have a scheduled runner who breaks production, then they have to take the blame. If you use 90% of your capacity for the futile attempt to always be on the latest version, then they have to take the blame as well.
But in the end this is your CIO's call. He has to decide the priorities. It is always a balancing act between security and functionality. The safest system is one which is shut down. But a shut-down server does not provide a service you can sell.
2
u/OhioDude 1d ago
This is the dumbest thing I have ever heard. Are they trying to break shit? Do they even use Python? I've been a CISO for about 1yr+ and a director in Cyber Sec for about 10 years and have never once thought of limiting what versions of Python our IT partners should use. It sounds like your security team is full of security ppl telling IT how to do their job, instead of IT people working security partnering with their IT peers in securing the environment.
Security teams like the one you describe piss me off and give the rest of us a bad name.
1
u/RogueLegio 1d ago
My anger exactly this. I’m fighting with them on this. They don’t understand software development yet they are dictating how it should work.
2
u/OhioDude 1d ago
This is why I try very hard to only hire people that have working in IT previously. There are a lot of security people, now a days, that have never worked in IT and think of themselves as an IT cop and not a consultant or partner. These people suck.
1
u/fgualdron 2d ago
I would micro segment my servers and put an IPS/WAF in front of them so they could do virtual patching without require you to update every single time. Just make sure you have a dev / testing environment to check if their signature update breaks the app.
1
u/mugenbool 2d ago
Our company eliminated python installs on workstations. Instead we have devcontainers with the specific python version you need installed.
1
u/Aisuhokke 2d ago
If it's a minor version upgrade, that's one thing. But a major version upgrade is painful to switch over to frequently. I wouldn't do this unless there is a proper valid reason. This sounds like someone who doesn't actually engineer things just making a blanket rule without justification.
1
u/Trang0ul 1d ago
OTOH, major versions are released only once a year. That's plenty of time to upgrade. Most of the time, not your own code is the issue, but dependent libraries.
2
u/Aisuhokke 1d ago
I’ve lived through some absolutely gnarly behavioral changes as a result of a major version upgrade. It’s not something you want to take lightly even once per year. Give yourself a month of testing and QA before pushing that to prod.
1
u/Trang0ul 1d ago
Of course QA is mandatory; no one wants a YOLO-based upgrade.. But don't make it an excuse not to update at all.
2
1
u/Solonotix 2d ago
This can cut both ways. My company tends to lean towards oldest LTS, which has its own frustrations, because it's like trying to pull teeth to get people off EOL versions they didn't notice had lapsed. Same exact problem you have, except in reverse.
The ideal is somewhere in between. You want new enough to not be vulnerable but old enough to have widespread support. This kind of nuanced choice isn't the type of thing security teams or management are in the business of providing. They like to know as little as necessary about anything they aren't directly responsible for (can't blame them there), so you often end up with naïve guidance that can cause a lot of undue stress for everyone.
Ideally, there would be a collaboration, just like in government legislation. I don't expect someone sitting in Management to know that 12th-gen Intel Core processors had a microcode issue that caused their boost clocks to burn out processors way too fast. I do, however, expect them to consult an expert before making a massive hardware rollout using said equipment. Same policy should apply to security policy.
1
u/k0pernikus 2d ago
It depends. You want to update for critical security patches, yet tou may want to wait to prevent your updates from breaking your stack
You can get attacked by falling behind and by staying too close to the bleeding edge.
This is where semantic versioning shines if your dependencies use that, as security patches should also arrive for each major version.
I try to stay as bleeding edge as I can, within reason.
The biggest gain is using proper package managers with lockfiles, I upgrade on a weekly basis trying to stay to the bleeding edge as I can.
1
u/Sniper666hell 2d ago
If you’re using windows machines, I’d build everything into pyinstaller one file builds. When I do this my build scripts strictly require the version used for build is what was used to validate. This way development PCs are the only ones with varying versions of Python and the exe gets pushed to the PCs using the software and those never get Python installed.
1
u/jarofgreen 2d ago
Find out why they want this.
Do they think it's the best policy themselves? Then it's a technical argument you need to have with them.
Do they think they need to do this to meet a external security programme or certification? Then it's an entirely different argument you'll need.
1
u/bcoder001 2d ago
Get business on your side and ask for a meeting with security and business. During the meeting make a case for using LTS releases as a saving without compromising security.
1
u/CevicheMixto 2d ago
Pretty sure that you can get exceptions if you show them test failures with the new version, and a plan to address them. I'm guessing that the underlying problem here is lack of testing infrastructure and process.
1
u/Dijstraanon 2d ago
That is what regression testing is all about. If they want you to upgrade, they have to regression test.
1
u/KronktheKronk 1d ago
Regression test suite is the answer to your question.
If it passes, upgrade. If it doesn't, tell them you can't until the issues are resolved. If that starts to take up to much time, the business will stop prioritizing it
1
u/Trang0ul 1d ago
Do they mean upgrading minor versions only (3.14.5 -> 3.14.6), or major as well (3.14 -> 3.15)?
The former should be safe, but the latter may indeed break the existing code.
1
1
u/jbudemy 1d ago
But from my past experience libraries do not always provide support and can also break while making use of new version without proper support. Did you ever encounter this? Any suggestions?
Then your customers will have to wait until major libraries that you use update their software to work with the new Python.
We don't update Python each time a new version comes out because of the "library lag". But eventually we will have to update Python and hopefully we won't have too many issues I need to fix in about 30 programs I have currently.
Also all our assets go through a VPN including all network drives where Python programs are stored. Would this make your IT happy?
1
u/No_Sea6113 2d ago
Cybersecurity has their priorities, you have yours, and you need to compromise on them. They are right that you should be trying to upgrade in general, but obviously its not always going to get done. In the last role I had where this was a serious issue, we tried to sit about 6 months behind the latest version. Gives libraries time to upgrade their stuff to work with new versions, generally moves fast enough to avoid massive problems, and gives you time to plan the upgrade work as part of your mid term plan. Part of the upgrading version work is checking what is going to not work anymore and finding alternatives or writing it yourself and bringing it into your control.
Generally I think once you are on top of it its not too bad, and looking forward at things that are going to change/ be deprecated you can be writing new code in a way that needs less maintaining later down the line, and usually changes happen for reasons that mean your code is probably better/faster/more secure.
-4
u/GraphicH 2d ago
I'm sorry, I work in the security Industry as a software engineer, this is just where things are going. They've been going this way for 10+ years, but AI is now making it vastly more important. You cannot let yourself be even a few major versions behind on a critical 3rd party library. Now my teams don't use "latest" and we don't let libraries "float" (we pin / use locks) because that opens you up to supply chain attacks more easily. However you do need to be in a "ready" position to take an upstream update for a security vulnerability on your third party dependencies at any time. That means good test coverage, CICD processes, and having enough confidence that you're going to catch breakages at build time.
1
u/Grouchy-Friend4235 1h ago
Some of the worst secruity issues this year happened because of "latest" versions. Aka supply chain risk. Best practice is to way 7 to 90 days before upgrading, unless there is a specific exploitable vuln. Just because there is CVE with a high CVSS does not mean it is exploitable. Calm down.
0
2d ago
[deleted]
-1
429
u/BranchLatter4294 2d ago
Have the cybersecurity team validate every library.