I'm using flutter and had to download xcode to build the app so I guess it's a necessary step.

I'm using flutter too, I've found a website called codemagic, so I can make builds for iOS. My laptop is windows. The bulk of development is building for Android.

It's nearly impossible to get away from Xcode entirely. You will always need the simulator and probably some entitlement / asset tools. What you can do is write the bulk of your app as an SPM package using your editor of choice, and then include that as a local or GitHub repo dependency in your Xcode shell app (which has no code, just assets, preview assets and plist files)

I feel like most Xcode free-swift setups still secretly depends on xcodebuild for anything real.

I had xcede[0] setup that worked well.

[0] https://codeberg.org/luxmentis/xcede

I use Xcode, but just to run the app with Command-R so I can see log messages or stack traces if I need to drop it into the agent for analysis. Everything else is done by Claude Code / Codex.

I even read the source using Zed not Xcode :)

tauri 2 + rust + webview give you a native feeling if you don't need any Apple frameworks

> .. webview ..

That would make it an web app tho. Not sure if that's what OP wants when making Swift apps.