Ah that's dumb. One of the best things about wslg is pretty pedestrian, but very helpful. You can make Windows shortcuts to launch applications in a very easy way now.
Within the "Target:" setting, for a shortcut, just "wslg.exe /some/program -args". No wrappers, no setting DISPLAY, no calling /bin/sh -c "whatever", etc.