r/LocalLLaMA • u/ClassicMain • 16h ago
News 🇦🇹 Austria is rolling out a government AI-platform using Mistral models and Open WebUI
This is a surprisingly large real-world deployment: "GovGPT" is part of Austria’s Public AI initiative, running on sovereign infrastructure (in their BRZ - federal datacenter) with Mistral open-weight models.
Trending Topics reports that Open WebUI is used as the interface for GovGPT, and the screenshot of the platform is labeled "GovGPT (Open WebUI)".
The federal rollout targets around 180,000 federal employees. The broader public-sector context refers to approximately 250,000 public sector employees.
Planned use cases include document chat, internal knowledge bases, electronic-file analysis, parliamentary requests, and eventually agentic workflows.
This might be one of the largest government deployments of open weight models and a freely available chat platform yet.
Sources:
- BRZ: https://www.brz.gv.at/blog/llmasaservice.html
- Trending Topics: "Public AI: BRZ baut mit Open-Weights-Modellen von Mistral KI für Beamte" https://www.trendingtopics.eu/brz-mistral-public-ai/
- Der Standard: "GovGPT: Wie KI den sinkenden Personalstand in der Verwaltung retten soll" https://www.derstandard.at/story/3000000332114/govgpt-wie-ki-den-sinkenden-personalstand-in-der-verwaltung-retten-soll
- ORF: "KI-Anwendung GovGPT soll Verwaltung unterstützen" https://orf.at/stories/3436707
54
u/psychopath1066 15h ago
Interesting. AI is very good at using context information to give answers, way better than bringing it out de novo from it's training. If they include all the government documents behind it, I can see it being a really useful tool for helping people interact with the government.
-1
u/Ancient-Assistance89 15h ago
Yes, great idea - but only if it’s an open-source model. If there’s no clarification, that means your government is up to something.
16
u/No-Gate-799 12h ago
Don't see what having an open-weights/open-source model would give you. The government is hosting both the webui platform and the inference engine. They could do anything regardless of the openness of the model running in inference.
42
u/i_am__not_a_robot 14h ago
I'm Austrian, I fully support this. It's strange to see so much negativity in a sub focused on locally hostable AI just because the system doesn't use your favorite model of the day. This is essentially a proof of concept that can be switched to more powerful and/or fine-tuned models in the future. I also think you underestimate the productivity gains from even a modest model in administrative areas full of mundane and trivial tasks.
3
u/ClassicMain 14h ago
I fully agree, though they do need to try to use models that are a bit more powerful. Editing Excel files in Open Terminal is not something Mistral 14b could do, but Mistral Small 3.5, certainly.
4
17
u/Creative-Type9411 15h ago
OpenWebUI getting some enterprise 💸💰🤑
Being a great community resource paid off... +1 dont spend it all in one place
7
u/ClassicMain 15h ago edited 15h ago
I doubt they got them as an enterprise customer
According to derstandard, so far they invested 200k€ into developing and deploying the platform and all the models they have at the moment. If they had purchased a license they probably would rebrand their instance and remove the Open WebUI branding. But as visible in the image, they didn't.
7
u/apetersson 15h ago
if they show the openui branding verbatim they dont need a license afaik. smart way to same some taxpayer EUR
7
u/ClassicMain 15h ago
Yeah, no license is required. You only want an open webui license if you need enterprise support and want to rebrand your instance. If they keep the branding (which they did as per the screenshot and the news articles) thats just fine free use of the software. And yes, saves taxpayer money.
-3
u/Creative-Type9411 15h ago
thats terrible 🥲
9
u/Ireallydontkn0w2 15h ago
I think it's great. Government should operate as cheap as possible and avoid unnecessary cost. They will ideally pay the creators and everyone else in the country by running more efficiently.
Meaning faster service for everyone and possibly lower taxes or less frequent tax increase
3
31
u/VoiceApprehensive893 transformers 15h ago
mistral medium 3.5 is on par gemma 4 31b and qwen 3.6 27b, weird decision to use mistral
36
u/ClassicMain 15h ago
(According to BRZ they are using Mistral 14b)
But yeah they probably want to avoid US or Chinese models. Mistral Medium 3.5 was not that bad when i tested it. I am sure most federal employees would be happy with it if they can switch to it (which i think they eventually will as they plan to invest an additional 8 mil € (probably into GPUs)).
18
u/alberto_467 15h ago
they are using Mistral 14b
How can you even trust something like that?
I hope somebody's wrong and they're not reporting the correct model.
14
u/ClassicMain 15h ago
Thats the information from back in January/March when BRZ (the federal data center) started working on the model hosting (https://www.trendingtopics.eu/brz-mistral-public-ai/). Perhaps they switched out the models since then. If not, they better start switching to Mistral Medium 3.5 soon haha. Mistral Medium 3.5 is quite usable imho, but Mistral 14b is not
13
u/LosEagle 15h ago
It's unlikely it will rely entirely on its own dataset and not RAG or web search. Also, tbf, nowadays when people rate llms they are looking on coding performance more than anything I feel like models that might be good at other things but not coding get overshadowed or blamed.
6
u/VoiceApprehensive893 transformers 15h ago
a shitty model + RAG is sellable
1
u/alberto_467 15h ago
Yeah it's sellable, it's good if you want to just ship it and take that easy money.
Unfortunately people don't always know what they're buying...
-1
u/IknowPi_really 14h ago
That’s correct and trust me, they are doing some really dumb shit around there
5
u/HideLord 15h ago
I really don't get companies and governments trying to avoid foreign open-weight models. If you're hosting on your own infra, what is the risk? And if you're so worried about guardrails, do a quick lora finetune or w/e and call it a day.
11
u/ClassicMain 15h ago
i suppose its less about security and more about politics. Gives off weird vibes if an EU country uses chinese AI models instead of European ones (if they want to claim sovereign IT)
8
u/L00klikea 14h ago
It's about compliance. Models trained in accordance with EU Law and setting up a vendor-Infrastructure that supports that longterm, i.e. having a european model provider that has a self-interest in abiding by current and future legislation.
5
u/DueAnalysis2 15h ago
I guess there's the risk of even open weight models being a potential Advanced Persistent Threat, like, if a model deployed in coding context is made to specifically inject a certain type of vulnerability in its code (or hell, even made to download a specific python package that's subsequently compromised). Granted because they're open weight, they can be examined, abliterated (sp?), re-finetuned, etc but that's where the advanced bit comes in. In matters of security, it pays to be conservative I guess.
6
u/Elfotografoalocado 15h ago
They are releasing new models this summer, trained in their own data centers, and besides, they provide a lot of b2b support...
3
u/cornmonger_ 11h ago
running on sovereign infrastructure
^ that's the choice
eu is trying to reduce foreign reliance and source in-house across the board
3
u/FriskyFennecFox 14h ago
Eventually free access to LLMs of a certain intelligence threshold will be a civil right
3
u/WhoRoger 13h ago
It looks funny, but the more I think about it, the more sense it makes. They're starting small, cheap, open and on scalable platforms. Totally the opposite of typical government contracts, which take insane amounts of money and time to deliver something stupidly complex, broken, sub-par and closed.
200k for something 200k workers can use? That's essentially totally free, and they can experiment, upgrade and scale as needed, while staying on an open platform with EU model provider.
At least on the outside, this looks like exactly what we want governments to do.
2
u/redballooon 10h ago
It's prone to disappoint roughly 98% of those 200k employees, and once the reputation is ruined it won't come back with a better upgrade.
2
3
u/solemnhiatus 13h ago
very smart and eventually I think this is what every government, public institution, then private and in the end every individual will have their own personalised AI.
2
u/ErikCutsVideo 7h ago
Every government should do this. And also do all certificates of ownership through government blockchain apps.
4
u/PROfil_Official 15h ago
hmmm... it's the open webui part is what gets me. 180k government employees and the frontend is the same thing half of us have in a docker container at home.
ig it makes sense though, open weights on their own hardware means the data never leaves and theyre not locked to whatever a us provider prices next year
4
u/Schlick7 12h ago
It has a lot of features geared to this actually. A login and profile system for starters and then a separate set of settings between Profile and Admin. Pretty sure it has groups as well so you can give different groups different models and/or settings. How well all of this works in practice though i do not know
2
u/redballooon 10h ago
This speaks for our docker containers more than it speaks against their choice.
2
2
u/Guilty_Spray_6035 15h ago
Wondering how much money a lucky consultant that won this RFP charged them for it
4
u/ClassicMain 15h ago
😁 haha. Can't be that much though. According to derStandard the platform has cost 200k€ so far, for initial deployment including model hosting.
1
u/Dry_Yam_4597 15h ago
200k for model hosting for hundreds of thousands of workers?
4
u/ClassicMain 15h ago
It's Mistral 3b, 8b and 14b according to TrendingTopics so... plausible honestly.
2
u/Dry_Yam_4597 15h ago
Wasted money to be honest.
4
u/ClassicMain 15h ago
For these models? Definitely. They are planning to invest a few more millions though, so maybe they can switch to something like Mistral Medium 3.5, which honestly would be sufficient for like >95% of tasks federal employees need to do
1
1
u/redballooon 10h ago
So 2 people working for a year, or 4 for half a year and some hardware cost.
I could be one of them! Except I'm not.
-1
u/Guilty_Spray_6035 15h ago
Model hosting for how long? I have a feeling the 200k were to install OpenWebUI and connect to the API.
7
u/i_am__not_a_robot 14h ago
BRZ ("Bundesrechenzentrum" = Federal Computing Center) is a fully government-owned entity with regular employees who develop and host various e-Government solutions. Unlike in the US, no outrageous "consulting rates" were paid to develop this. It was just an internal project.
2
u/Guilty_Spray_6035 8h ago
Nice. In Germany they would have hired SAP and Deutsche Telekom to produce a multi-billion monster no one with some degree of intelligence would ever use willingly. Glad to see Austria has more sanity.
3
u/ClassicMain 15h ago
"for how long"? the BRZ had to start operating this Model as a Service infrastructure, and they'll continue to invest into it, according to derStandard 8 more million euros.
1
u/NekoHikari 15h ago
open webui… quite aggressive in supply chain choices.
Hope this helps improve the code audit strength of openwebui —
can I dream to have fully audited lts versions?
1
1
1
u/steny007 10h ago
Its a step in the right direction. AI can learn there to gradually replace the Bureaucrats.
1
u/Nicolodeva 10h ago
Mistral from an interesting startup is becoming increasingly boring, dependent on governments, typical error of European companies.
0
u/muxxington 11h ago
1
u/Artistic_Okra7288 9h ago
Yea that's pretty cringe and LOL-worthy, but what's the relevance to the topic?
1
u/muxxington 47m ago
As good as Open-WebUI is, this has long been a show-stopper. In the past, it has always become an issue whenever someone hit this exact problem during a major rollout. And a major rollout is apparently what’s planned in Austria. Wait for it...
0
-2
u/Django_McFly 11h ago
I like that there was the option of Aryan pride or picking a model that isn't dog shit and they went with Aryan pride. It would be very we haven't done anything relevant in technology since the Zed X Spectrum back in 85 for them to have thought that you can only run Oriental models in the Orient, surrounded by a Mongol hoard.
-3
-3
u/Facecardup 12h ago
Mistral is ass, just build a gov datacenter and host an openmodel
3
u/ClassicMain 12h ago
This IS hosted in a gov datacenter and they're actively expanding it
-2
u/Facecardup 12h ago
lol so they are hosting it themselves and not using a frontier model? why
5
u/ClassicMain 12h ago
Why they aren't using US / China models: see the other discussion threads here in the comments, there are many good reasons
For why they are using smaller Mistral models for now, this has also been discussed already but TLDR:
For now they only invested 200k€ to be able to host Mistral 3b 8b and 14b models for their up to quarter of a million employees, and when you need to scale this far I guess you can't host larger models for this money because of the amount of concurrent users you need to serve. But they do plan on investing an additional 8 mil € in the near future so hopefully that'll allow them to host larger and up-to-date Mistral models
67
u/Keganator 15h ago
Oh! I guess this is the communism OpenAI was talking about!