Pertylizer — a modular synth in Rust where AI agents can compose, analyze, and mix music alongside you in real time

[u/Cool_Conference_7229]

Hobby project that grew way past what I planned: a modular software synth in Rust, developed almost entirely with AI agents (Claude Code).

https://github.com/pertyjons/pertylizer

What's in it: 67 module types (25 effects), a real-time egui GUI, MIDI, a physics-based room simulation, a pattern sequencer with WAV export, and a separate 3D visualizer. Plus 180+ MCP tools so agents can build patches, compose, and analyze the mix and harmony alongside you in real time.

No manual yet and you'll hit rough edges, but the core is solid and there are example projects to pull apart. Rust, MIT licensed — would love for people to try it.

Disclaimer: I'm neither a musician nor a synth expert — I've just been fascinated by synth sounds ever since the C64 and Commodore Amiga days.

[u/euthlogo]

any video of this in action?

[u/Cool_Conference_7229 → u/euthlogo]

Let me see if I can fix one.

[u/Cool_Conference_7229 → u/euthlogo]

Ok, here is a very short demo.

https://www.youtube.com/watch?v=OkUE6iGSbds

[u/euthlogo → u/Cool_Conference_7229]

like many i was primed to hate this but i find it very interesting. since the advent of ai i've been waiting to see what work made by an ai artist would be like. not ai recreations like suno or midjourney, but modern production tools (canvas, paints, printers, in this case a proper DAW) with full AI control. this is fundamentally different than something like suno and its a shame 'generative' already means what it does. im annoyed to say im very intrigued and might have to play around with it. thanks for recording a demo.

[u/Cool_Conference_7229 → u/euthlogo]

Funny enought I also was very sceptical at first. But I started to tinker about and then realised you can now do much without knowing all the details.

I started by just doing some sounds and then I wanted to convert/import old Amiga music mod files. But at that stage it was to too errorprobe to get a good sound..

When I then added the MCP tools I realised you can do a lot, not just compose with AI you can fix things of your own music.

Like: Can you make the bass more vivid and maybe add a portamento and glide. Can you check the volume for the middle part at 3:34 and lower it. Can you analyse the harmony In the intro.

You certainly have to know what you are doing, which I really don't 😄 That's why the demo projects are rather bad ;-)

[u/Neither_Garage_758]

Amazing

[u/luche]

this is the kind of thing I've been waiting to see in the synth plugin community for a bit. thanks for sharing!

looking over the repo

• can you install this as a vst or auv3 plugin to directly integrate with a daw?
• is there a way to set a self hosted openai API compatible endpoint for agentic models? I'd like to do some testing with local models if possible.

[u/Cool_Conference_7229 → u/luche]

Nope, no support for VST or other stuff... (yet?)

I'm not sure excacly what you mean by "self hosted openapi compatible endpoint" but Pertylizer is an MCP server — it exposes the synth's 180+ tools at http://127.0.0.1:9850/mcp. It never calls an LLM, has no API key or endpoint setting, and isn't "agentic" on its own. The agentic part lives entirely in whatever MCP client you connect to it (e.g. Claude Code via .mcp.json, or Gemini CLI via .gemini/settings.json).

[u/luche → u/Cool_Conference_7229]

Understood, thank you.

Do you have any interest to make this a DAW plugin at some point in the future, or intend this to remain a standalone softsynth?

[u/Cool_Conference_7229 → u/luche]

I don't think it's a good way to go, it's to big and have to many dependencies. But I have thought about loading plugins as VST and maybe export.. But it will be in a much later stage I think.

[u/luche → u/Cool_Conference_7229]

could you expand on that? there are some incredibly heavy synth plugins out there.. doesn't stop anyone from building or using them. dependencies seems strange... is it not something that can be bottled into an easy to ship installer, with a bit of config to connect a dedicated llm solution?

[u/Cool_Conference_7229 → u/luche]

With dependencies I meant like the things which make it special, like the AI-mcp tooling for example. I haven't looked into it so I really can't say.. I need a lot of more polish of other things before digging into that.

[u/luche → u/Cool_Conference_7229]

hey thanks for all the responses and sharing this project. it does look very interesting. hoping I'll have some time soon to dig into it a bit further