Running Windows desktop tests from Linux with disposable VMs (qawolf.com)

78 pointsby jadenlemmon7 hours ago27 comments

trollbridge 7 hours ago

Not to dismiss this, but when I was working on a problem on Windows recently my LLM (6.1 Sol / oh-my-pi) ended up doing the same thing, although it used Qemu and snapshots to accomplish that.

Eventually it got to a point it could iterate very quickly and was really only constrained by total system RAM which meant a dozen at most copies could be running simultaneously.

mcoliver 6 hours ago

Same. It's quite remarkable. Two other things Claude and I have done. I have a spare workstation running proxmox in my garage. Claude can now create, snapshot, delete, extend, start, and restart vms. It's been great. I also have skills that allow it to spin up spot instances when I need to test things that rely on hardware I don't have like certain Nvidia gpus or 100gb networking. What a time to be alive.

trollbridge 6 hours ago

To be fair, you never needed LLMs to be able to do all that.

rsolva 6 hours ago

To be fair, often you have good ideas, but not enough time. I have had a lot of fun having Qwen3.8 27b running at home, having it explore and implement ideas that was way to ambitious for the time I had available to tinker with Home Assistant and Proxmox. It is an enabler and can also be a good tutor, when used as such. I have learned a lot.

alasano 6 hours ago

"be able" implying capability rather than time constraints and opportunity cost which is what most people on here refer to when they mention something cool they're doing with LLMs that they weren't doing before.

mcoliver 5 hours ago

True but it was a whole bunch of fragile scripts/functions with lots of ifs. I'll give you a real world example of what's different. Agent spins up a VM to run a build on windows. During the build it realizes it's out of disk space. Agent resizes disk on demand. Build finishes and it launches the app. Runs some tests and realizes it's cpu and memory constrained. Adjust the cpu and memory and runs new tests. tears it down and sends the perf logs back to me. Or it looks at system resources and decides what other vms it can pause while it runs the tests and then turns them back on after.

There are so many unknowns and the ability for the agents to dynamically adjust and make decisions allows for some really powerful autonomous workflows.

c-hendricks 3 hours ago

Not sure the lower bounds of CPU/memory/disk space that your project needs should be considered "so many unknowns".

mcoliver 2 hours ago

Man. HN is brutal. Just one example. And yes, when you're looping agents and builds to identify performance optimizations and build features sometimes you don't know what those bounds are going to be and it's much faster to let the agents work it out, update their learnings, and keep building.

joinjune 6 hours ago

We used several different LLM's and our own skills to build out some novel approaches to document creation with new workflows using QEMU in Linux after upgrading our desktops to Ubuntu from Windows 11.

Luker88 6 hours ago

what is the minimum license that I would need to do this legally, with just one or two VMs?

OneDeuxTriSeiGo 6 hours ago

It depends what you need to run. There exist a number of Windows images which can be run without any paid license requirements for test and validation IIRC.

https://learn.microsoft.com/en-us/windows-hardware/manufactu...

jadenlemmon 6 hours ago

it depends on where you are running it. If using a cloud provider, license is typically built into hourly price.

xlazom00 5 hours ago

This is super slow solution I did build similar solution based on dockur. And it was sooo slow. Just use windows docker containers on windows server. With some hacks you can run any gui application.

defied 6 hours ago

We’ve been providing a similar approach with disposable VMs and emulators/simulators since 2014 at TestingBot. We run our own single-use QEMU VMs with Windows and Linux.

madduci 6 hours ago

Same here, we used Proxmox as host to spin up ready images built with Packer and starting them with Vagrant to provision the app on the system under test, the whole triggered by the CI build

ranger_danger 6 hours ago

Why not use snapshots/save states so the VMs start back up right away? This is what I do with QEMU.

ProblemExplorer 6 hours ago

Doesn't Windows do some drama if its not activated and has been running without a license for more than 48h or similar?

I don't use windows but have to run windows 11 VM to access special devices. I don't want to bother with licenses, although I have them, and creating a new incus QEMU VM from scratch everytime from an ISO takes ~5m, freeing me up from worrying about all the activation and license drama.

If the activation and license setup was easy/automated in QEMU, I would imagine snapshots would be way easier. I would prefer not needing to connect to the internet.

c-hendricks 6 hours ago

> Doesn't Windows do some drama if its not activated and has been running without a license for more than 48h or similar

Your info might be like decades out of date, since windows 10 it's just been an overlay and locking out some appearance customizations (you can still set them via other means though).

ProblemExplorer 5 hours ago

Ok, so you're saying I can run windows 11 without a license or activation for months on end?

c-hendricks 3 hours ago

Yes, since the release of Windows 10 in 2014.

fhn 3 hours ago

thanks for the advertisement

itissid 6 hours ago

Why is the comments from the OP marked as flagged or dead at the bottom?

realusername 6 hours ago

Looks like AI written so it got auto-flagged by a bot probably

msla 4 hours ago

What's the licensing on Windows 11 these days? Can an individual still run an unregistered Windows 11 installation basically forever with only minor restrictions?

jadenlemmon 4 hours ago

For an individual, mostly yes. Unactivated will just keep running with a watermark & some minor restrictions.

blahgeek 6 hours ago

Nothing new here. VM + WebRTC + warm pool to absorb boot time

jadenlemmon 7 hours ago

I work at QA Wolf and lead the development of automated testing for Windows desktop apps. I wanted to share what we did and how we did it.

This post covers how we added Windows to a test infrastructure built around Linux runners. The central problem was giving each test a clean, logged-in Windows desktop without waiting 2–3 minutes for a VM to boot.

We extended the device scheduler we already use for mobile testing. It maintains warm pools of Windows VMs on GCE. Each test claims a ready VM, uses it once, and releases it. The VM is then destroyed and replaced, rather than cleaned up for the next test.

The test code stays in a Linux pod. Playwright connects to browsers on the Windows machine over CDP, while WebdriverIO and Appium handle native apps through WinAppDriver. A small Go agent captures the desktop with FFmpeg and streams it over WebRTC for live viewing and recordings.

That separation lets a single test move between a browser and a native Windows app without requiring a separate Windows runner pipeline.

The trade-off is that we keep idle Windows VMs ready and install the app under test on every run. In exchange, tests don’t inherit another run’s files, registry changes, or saved sessions.

I’d love to answer any questions!