Yeah I'm pretty sure my onboard 10G Marvell AQtion ethernet is the source of most of my stability woes. About half the time any of these machines boot up, Windows bluescreens within the first couple minutes, and I think it has something to do with the iSCSI service crashing. Never had trouble in the old house where the machines had 1G network -- but load times were painful.
Luckily if the machines don't crash in the first couple minutes, then they settle down and work fine...
Yeah I could get higher-quality 10G cards and put them in all the machines but they seem expensive...
If you make a decision on a 10G card (SFP or ethernet) I'd like to hear what you picked.
Bulk buying is probably hard, but ex-enterprise Intel 10G on eBay tends to be pretty inexpensive. Dual spf+ x520 cards are regularly available for $10. Dual 10g-base-t x540 cards run a bit more, with more variance, $15-$25. No 2.5/5Gb support, but my 10g network equipment can't do those speeds either, so no big deal. These are almost all x8 cards, so you need a slot that can accomidate them, but x4 electrical should be fine (I've seen reports that some enterprise gear has trouble working properly in x1/x4 slots beyond bandwidth restrictions which shouldn't be a problem; if a dual port card needs x8 and you only have x4 and only use a single port, that should be fine)
I think all of mine can pxeboot, but sometimes you have to fiddle with the eeprom tools, and they might be legacy only, no uefi pxe, but that's fine for me.
And you usually have to be ok with running them with no brackets, cause they usually come with low profile brackets only.
x520s with full-height brackets do exist (I have a box full of them), but you may pay like $3-5/ea more than the more common lo-pro bracket ones. If you're willing to pop the bracket off, you can also find full-height brackets standalone and install your own.
Also, in general: in my experience avoiding 10gbe rj45 is very worthwhile. More expensive, more power consumption, more heat generation. If you can stick a sfp+ card in something, do it. IMO 10gbe rj45 is only worthwhile when you've got a device that supports it but can't easily take a pcie nic, like some intel NUCs.
Another data point that it is indeed possible. I had a dual Xeon E5-2690 v2 setup with two RX 580 8GB cards passed through to separate VMs, and with memory and CPU pinning it was a surprisingly resilient setup. 150+ FPS in CSGO with decent 1% lows (like 120 if I remember correctly?) which was fine since I only had 60Hz monitors. I have a Threadripper workstation now, I should test out to see what kind of performance I can get out of that for VM gaming...
> Yeah I could get higher-quality 10G cards and put them in all the machines but they seem expensive...
I have had very good luck with Intel X540 cards. $20-40 on eBay, and there’s hundreds (if not thousands) available. They’re plug-and-play on any modern Linux, but need an Intel driver on windows if I remember correctly. I’ve never had one die and I’ve never experienced a crash or network dropout in the 9 years I’ve been running them. The Marvell chipset just seems terrible, unfortunately - I’ve had problems with it on multiple different cards and motherboards on every OS under the sun.
The good intel 10G cards were not expensive at all by the way, I bought them for later additions, and they were cheaper than the premium we paid for the money-gamer motherboards that included 10G cards that I saw you were unhappy about too.
I think my muni fiber install happening this week might have a 10G-baseT handoff, and I've got a port for that open on my switch in the garage. If that works out, that will be neat, but I'll need to upgrade some more stuff to make full use of that.
> onboard 10G Marvell AQtion ethernet
I had similar problems with an Aquantia 10GbE NIC (which AQtion appears to be the rebranded name for, post-acquisition by Marvell), and it turned out to be the network chip overheating because it was poorly thermally bonded to a VRM heatsink that defaulted to turning on at something like 90C. Adding a thicker thermal pad and setting the VRM fan to always be on at 30% solved my problems.
I think it probably isn't the same problem, though, because I only have stability issues at initial startup. If it boots and doesn't BSOD in the first five minutes then it's fine... even through heavy network and disk use (like installing updates).
10gbase-t ethernet is harder to pick, a lot of those cards run incredibly hot particularly the ones that expect server style cooling. Heard bad things about all of them.
Also heard that Windows has a hard time reaching 10G anyway.
No need to buy new for most computing equipment unless you're looking for the absolute latest and greatest.
I think another issue is the limited amount of PCI-E lanes now that HEDT is dead. I picked up a 5930k for my build at the time for its 40 PCI-E lanes. But now consumer CPUs basically max out at 20-24 lanes.
Also with the best CPUs for gaming nowadays being AMD's X3D series because of its additional L3 cache, I wonder about the performance hit with 2 different VMs fighting for cache. Maybe the rumored 9950X3D will have 2 3D caches and you'd be able to pin the VMs to each CPU cores/cache. The 7950X3D had 3D cache only on half of its cores, so games generally performed better pinned to only those cores.
So with only 2-3 VMs/PC, and you still needing a GPU for each VM which are the most expensive part anyway, I'd pay a bit more to do it without VMs. The only way I'd be interested in multiseat VM gaming again would be if I could utilize GPU virtualization: split up a single GPU into many VMs. But like you say in the article that's usually been limited to enterprise hardware. And even then it'd be interesting only for the flexibility, being able to run 1 high-end GPU for when I'm not having a party.
I bet one could put an unreasonable amount of effort into convincing an Nvidia Bluefield card to pretend to be a disk well enough to get Windows to mount it. I imagine that AWS is doing something along those lines too, but with more cheap chips and less Nvidia markup…
There has got to be a way to convince Windows to do an overlay block device that involves magic words like “thin provisioning”. But two seconds of searching didn’t find it. Every self-respecting OS (Linux, FreeBSD, etc) has had this capability for decades, of course. Amusingly, AFAICT, major clouds also mostly lack this capability — performance of the obvious solution in AWS (boot everything off an AMI) is notoriously poorly performing.
It really shouldn't. Microsoft invented or popularized Receive Side Scaling [1], which helps get things lined up for high throughput; but applications probably need to do a bit of work too.
[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/n...
Silicom PEG210 Silicom PE210G2BPI40-T-SD-BC7 Intel x540 based bypass NIC. In case you want to have the two ports connect together when the computer is off or something. Setup time is a bit more, but you can also configure them to act like normal NICs.
Usually show up around $15-25 like other x540 dual rj45 cards, but sometimes a bit less, cause they're weird.