I am trying to transfer data via USB at high speed without data corruption, silent resets and occasional device disconnects. Those are things that happen because the USB controllers on my motherboard made by AMD with some help from ASMedia do not function correctly at the speed they advertise. So given the problem the right solution is to get a firmware or hardware fix for these USB controllers, however that’s unlikely to happen. So I’m trying to find a workaround. I already have one (PCIe add-in card) but now I’m also testing running the bad controllers at half-speed which seems successful so far but I was wondering if there’s a way to do it in software. I’m currently bottlenecking the links by using 5Gb hubs between the controllers and the devices.
Unfortunately it won’t because the transfers are happening between ZFS and the hardware storing the data so I can’t control the data rate at the application level (there are many different applications) or even at the ZFS level. This is why in this particular case I’m stuck with a potential hardware-related workaround. I mean I could do something stupid like configuring a suboptimal recordsize in ZFS but there could still be spikes and I’d prefer to get the hardware to stop losing bits and hoping ZFS would catch that. Decreasing data rates is a generally acceptable strategy to deal with signalling issues, if the decreased rate is usable for the application at hand. In my case it is.
I’d recommend another top-level reply to your post, or a new post, describing precisely how your hardware and ZFS pools are set up, alongside a description of the firmware/stability issues you’re seeing, and solutions you’ve tried already. We’re a bit far down a comment chain at the moment, so you’ll probably get more engagement that way. Not trying to be an ass - this actually sounds like an interesting (and, I’m sure, obnoxious) problem. Putting all the cards on the table will help people give you a more complete answer more quickly.
The thing is that I’m already at the last couple of leaves in the investigation tree and I’m not willing to change anything upwards of the USB driver level. That’s why there isn’t much point in getting people to spin their wheels for solutions I can’t or won’t apply. If I was completely unable to get the data corruption and disconnects under control, I’d trash the system and replace it with Intel. Fortunately, a PCIe add-in USB controller seems to work well so I avoided the most costly solution. At this point I don’t actually need to get the motherboard ports to work well but I’m curious to follow down the signalling rabbit hole because I’m not the only one who’s having this problem and the problem doesn’t affect just this one use case. If I find a solution like an in-line 5Gb USB hub (reduces data rate), or just using USB-C ports instead of USB-A (reduces noise), or using this kind of cable instead of that kind, I could throw that as a cheaper workaround in this ZFS thread and elsewhere. The PCIe cards work but aren’t cheap.
That all makes sense - I guess at this point, I’m simply trying to offer some constructive criticism about how to present nuanced problems that involve both hardware and software gremlins in a way that you’ll get the most productive conversation and interaction from the user base here.
I am trying to transfer data via USB at high speed without data corruption, silent resets and occasional device disconnects. Those are things that happen because the USB controllers on my motherboard made by AMD with some help from ASMedia do not function correctly at the speed they advertise. So given the problem the right solution is to get a firmware or hardware fix for these USB controllers, however that’s unlikely to happen. So I’m trying to find a workaround. I already have one (PCIe add-in card) but now I’m also testing running the bad controllers at half-speed which seems successful so far but I was wondering if there’s a way to do it in software. I’m currently bottlenecking the links by using 5Gb hubs between the controllers and the devices.
A bit of searching brought me here - would this suit your purposes?
Edit: amusingly, one of the replies to the original question also points out that this is yet another classical example of the X-Y problem
Unfortunately it won’t because the transfers are happening between ZFS and the hardware storing the data so I can’t control the data rate at the application level (there are many different applications) or even at the ZFS level. This is why in this particular case I’m stuck with a potential hardware-related workaround. I mean I could do something stupid like configuring a suboptimal recordsize in ZFS but there could still be spikes and I’d prefer to get the hardware to stop losing bits and hoping ZFS would catch that. Decreasing data rates is a generally acceptable strategy to deal with signalling issues, if the decreased rate is usable for the application at hand. In my case it is.
Heh, we’re still on the X-Y problem to a degree.
I’d recommend another top-level reply to your post, or a new post, describing precisely how your hardware and ZFS pools are set up, alongside a description of the firmware/stability issues you’re seeing, and solutions you’ve tried already. We’re a bit far down a comment chain at the moment, so you’ll probably get more engagement that way. Not trying to be an ass - this actually sounds like an interesting (and, I’m sure, obnoxious) problem. Putting all the cards on the table will help people give you a more complete answer more quickly.
The thing is that I’m already at the last couple of leaves in the investigation tree and I’m not willing to change anything upwards of the USB driver level. That’s why there isn’t much point in getting people to spin their wheels for solutions I can’t or won’t apply. If I was completely unable to get the data corruption and disconnects under control, I’d trash the system and replace it with Intel. Fortunately, a PCIe add-in USB controller seems to work well so I avoided the most costly solution. At this point I don’t actually need to get the motherboard ports to work well but I’m curious to follow down the signalling rabbit hole because I’m not the only one who’s having this problem and the problem doesn’t affect just this one use case. If I find a solution like an in-line 5Gb USB hub (reduces data rate), or just using USB-C ports instead of USB-A (reduces noise), or using this kind of cable instead of that kind, I could throw that as a cheaper workaround in this ZFS thread and elsewhere. The PCIe cards work but aren’t cheap.
That all makes sense - I guess at this point, I’m simply trying to offer some constructive criticism about how to present nuanced problems that involve both hardware and software gremlins in a way that you’ll get the most productive conversation and interaction from the user base here.