Discussion:
[Libusb-devel] libusb and windows support?
Jef Driesen
2011-02-20 20:38:03 UTC
Permalink
Hi,

I'm writing a cross-platform library (windows, linux and macosx) where I would
like to use libusb for usb communication. But I wonder which version I should
use: 0.1 or 1.0? Probably the latest one, but I get the impression that for
windows support, the older 0.1 is still the recommended one (due to its
libusb0.sys driver support). Is that correct?

Jef
Pete Batard
2011-02-20 21:12:17 UTC
Permalink
Hi Jef,

Currently libsub-1.0 uses WinUSB rather than libusb0.sys to access USB
devices in a generic fashion (it also has HID support, but I'll assume
your device is not HID).
If you can live with the limitations of WinUSB detailled here:
http://www.libusb.org/wiki/windows_backend#KnownRestrictions as well
as the fact that Windows support from libusb 1.0 is still
experimental, and has not yet been included in an official release,
you should probably use libusb 1.0. For the record, 0.1 is being
discontinued for Linux and OS-X, so if your aim is to be
multiplatform, 1.0 is probably what you want (and I have knowledge of
companies successfully using libusb 1.0 to support their device on
Linux+OS-X+Windows). Libusb 1.0 is also more friendly to things like
asynchronous transfers.

Also, once we have sorted the release bottleneck, the Windows backend
will get libusb0.sys support in one form or another, and will be
advertised as the replacement for libusb-win32 afterwards.

Regards,

/Pete
Xiaofan Chen
2011-02-21 06:46:58 UTC
Permalink
Post by Pete Batard
Hi Jef,
Currently libsub-1.0 uses WinUSB rather than libusb0.sys to access USB
devices in a generic fashion (it also has HID support, but I'll assume
your device is not HID).
http://www.libusb.org/wiki/windows_backend#KnownRestrictions
The OS support may or may not be a limitation for you.
WinUSB does not support Windows 2000, Windows 2003
(32/64bit) and Windows XP 64bit.
Post by Pete Batard
as well as the fact that Windows support from libusb 1.0 is still
experimental, and has not yet been included in an official release,
you should probably use libusb 1.0. For the record, 0.1 is being
discontinued for Linux and OS-X,
"discontinue" might not be the right word. It is still there but
no longer being maintained. However, you can still use
libusb-compat-0.1 on top of libusb-1.0.
http://www.libusb.org/wiki/libusb-compat-0.1
Post by Pete Batard
so if your aim is to be
multiplatform, 1.0 is probably what you want (and I have knowledge of
companies successfully using libusb 1.0 to support their device on
Linux+OS-X+Windows). Libusb 1.0 is also more friendly to things like
asynchronous transfers.
Also, once we have sorted the release bottleneck, the Windows backend
will get libusb0.sys support in one form or another, and will be
advertised as the replacement for libusb-win32 afterwards.
I agree with Pete that you should look at libusb-1.0 if you accpet
the limitations and the fact that it has not been officially released
for the Windows platform.

If you think the status of libusb-1.0 on Windows is not acceptable,
then you can continue using libusb-0.1 or libusb-compat-0.1 under
Linux and Mac OS X and libusb-win32 under Windows.

libusb-win32 is still alive even though we would very much like
to move on to libusb-1.0 API as well. On the other hand,
even after moving to libusb-1.0 API, the kernel driver part of
the libusb-win32 project will still be alive.
--
Xiaofan
Peter Stuge
2011-02-21 22:08:04 UTC
Permalink
Hi Jef,
Post by Jef Driesen
I'm writing a cross-platform library (windows, linux and macosx)
where I would like to use libusb for usb communication.
Great!
Post by Jef Driesen
But I wonder which version I should use: 0.1 or 1.0?
You should try to use 1.0 without doubt.
Post by Jef Driesen
Probably the latest one, but I get the impression that for
windows support, the older 0.1 is still the recommended one
(due to its libusb0.sys driver support). Is that correct?
I wouldn't say so. Yes, libusb0.sys is a better fit for libusb and is
more widely compatible, but maybe WinUSB will work for you.

And even if WinUSB doesn't work for you, you could still test the
libusb0 branch of libusb-stuge.git which contains Graeme's version of
libusb which does include libusb0.sys support.


//Peter
Xiaofan Chen
2011-02-22 08:40:43 UTC
Permalink
Post by Peter Stuge
Post by Jef Driesen
Probably the latest one, but I get the impression that for
windows support, the older 0.1 is still the recommended one
(due to its libusb0.sys driver support). Is that correct?
I wouldn't say so. Yes, libusb0.sys is a better fit for libusb and is
more widely compatible, but maybe WinUSB will work for you.
And even if WinUSB doesn't work for you, you could still test the
libusb0 branch of libusb-stuge.git which contains Graeme's version of
libusb which does include libusb0.sys support.
Last time I tried the libusb0.sys git branch and it did not really
work well.

On the other hand, given the slow integration of the Windows
backend (let alone libusb0.sys backend), I am not sure in
the end it will matter any more. Travis is working on a new
generation of libusb-win32 kernel driver based on KMDF
and it will be much easier to be integrated into the current
libusb-1.0 Windows codes since it can emulate WinUSB API.
In fact, we have working prototypes of the driver and it works
well with the current libusb-1.0 WinUSB backend.
--
Xiaofan
Graeme Gill
2011-02-22 11:34:50 UTC
Permalink
Post by Xiaofan Chen
the end it will matter any more. Travis is working on a new
generation of libusb-win32 kernel driver based on KMDF
and it will be much easier to be integrated into the current
libusb-1.0 Windows codes since it can emulate WinUSB API.
Hmm. I'm not sure that is actually an advantage. The libusb0.sys
has a lot more orthogonal API, and provides some functions
missing from winusb that are essential with particuar USB devices.

Graeme Gill.
Xiaofan Chen
2011-02-22 12:26:52 UTC
Permalink
Post by Graeme Gill
Post by Xiaofan Chen
the end it will matter any more. Travis is working on a new
generation of libusb-win32 kernel driver based on KMDF
and it will be much easier to be integrated into the current
libusb-1.0 Windows codes since it can emulate WinUSB API.
Hmm. I'm not sure that is actually an advantage. The libusb0.sys
has a lot more orthogonal API, and provides some functions
missing from winusb that are essential with particuar USB devices.
No worry, the driver will support the libusb-win32 API
(backward compatible), libusb-1.0 API and WinUSB API
through different dlls.
--
Xiaofan
Travis
2011-02-22 15:36:43 UTC
Permalink
Post by Graeme Gill
Post by Xiaofan Chen
the end it will matter any more. Travis is working on a new
generation of libusb-win32 kernel driver based on KMDF
and it will be much easier to be integrated into the current
libusb-1.0 Windows codes since it can emulate WinUSB API.
Hmm. I'm not sure that is actually an advantage. The libusb0.sys
has a lot more orthogonal API, and provides some functions
missing from winusb that are essential with particuar USB devices.
Would it be possible for you to provide us with a quick list so we
exactly which functions your are talking about?

libusb0.dll is nearly fully supported. i.e. Your old 0.1.12 dll will
work with this driver, so I guess I'm curious as to what you mean by
orthogonal. You must be talking about WinUSB.dll compared to libusb0.dll?

Right now our new KMDF driver is missing support for multiple
configurations only; something I plan to look into deeper if necessary.

Regards,
Travis
Graeme Gill
2011-02-23 00:03:50 UTC
Permalink
Post by Travis
Would it be possible for you to provide us with a quick list so we
exactly which functions your are talking about?
Hi Travis,

I was comparing the i/f of winusb.dll to libusb0.sys really.

I'm thinking about things like IOCTL_ABORT_ENDPOINT and
IOCTL_RESET_DEVICE.
Post by Travis
libusb0.dll is nearly fully supported. i.e. Your old 0.1.12 dll will
work with this driver, so I guess I'm curious as to what you mean by
orthogonal. You must be talking about WinUSB.dll compared to libusb0.dll?
libusb0.sys seems to uniformly implement a set of USB operations that
correspond to the libusb API, while winusb appears to have only partial
coverage.

Graeme Gill.
Pete Batard
2011-02-23 13:31:18 UTC
Permalink
Post by Graeme Gill
I was comparing the i/f of winusb.dll to libusb0.sys really.
I'm thinking about things like IOCTL_ABORT_ENDPOINT and
IOCTL_RESET_DEVICE.
BOOL WINAPI WinUSB_DeviceIoControl(
__in WINUSB_INTERFACE_HANDLE InterfaceHandle,
__in DWORD dwIoControlCode,
__in_opt LPVOID lpInBuffer,
__in DWORD nInBufferSize,
__out_opt LPVOID lpOutBuffer,
__in DWORD nOutBufferSize,
__out_opt LPDWORD lpBytesReturned,
__inout_opt LPOVERLAPPED lpOverlapped
);

which we can easily add and then use in the Windows backend. That's
pretty much how I'm planning to add the few features we are missing from
WinUSB when using libusbK. Wouldn't actually use the WinUSB prefix
there, since it's only meant for LUsbK.

The only major annoyance I see with using an API that must be compatible
with WinUSB is the need to obtain a handle to the first interface before
the others can be accessed, but I could see an IOCTL or an alternate
LUsbK_initialize call to allow direct retrieval to interface handles, as
well as a new API call, simply taking the file handle and an interface
number to do just that.

All in all, it looks to me like we should be able to work with an
extended version of the WinUSB API to provide the unconstrained
functionality required for libusb-1.0. Of course, I'll have a better
idea when I try to do just that in the K branch.

Regards,

/Pete
Peter Stuge
2011-02-24 05:39:34 UTC
Permalink
it looks to me like we should be able to work with an extended
version of the WinUSB API to provide the unconstrained
functionality required for libusb-1.0.
Great if true! Important for me that no DLL is required besides
libusb-1.0.dll, regardless of which kernel driver is used, but I
think this is also what is intended.
Of course, I'll have a better idea when I try to do just that in
the K branch.
..which should again be based on libusb.git or at your option the
merge queue that is libusb-stuge.git.


//Peter
Xiaofan Chen
2011-02-24 07:45:19 UTC
Permalink
Post by Peter Stuge
it looks to me like we should be able to work with an extended
version of the WinUSB API to provide the unconstrained
functionality required for libusb-1.0.
Great if true! Important for me that no DLL is required besides
libusb-1.0.dll, regardless of which kernel driver is used, but I
think this is also what is intended.
I am not sure what you mean here. For the WinUSB backend,
then you need the winusb.dll. For the new libusbK backend,
you will need the libusbk.dll.
--
Xiaofan
Xiaofan Chen
2011-02-24 07:50:03 UTC
Permalink
Post by Xiaofan Chen
Post by Peter Stuge
it looks to me like we should be able to work with an extended
version of the WinUSB API to provide the unconstrained
functionality required for libusb-1.0.
Great if true! Important for me that no DLL is required besides
libusb-1.0.dll, regardless of which kernel driver is used, but I
think this is also what is intended.
I am not sure what you mean here. For the WinUSB backend,
then you need the winusb.dll. For the new libusbK backend,
you will need the libusbk.dll.
On the other hand, I do not think this is an issue. For the
winusb.dll, it is either already in the system or it will be
installed by the driver installation process. For the libusbk.dll,
it will be installed together with the driver (along with libusb0.dll).
Actually we can choose to include libusb-1.0.dll in the driver
installation as well. The driver installation will take care of
the 32bit/64bit issue. For the 64bit system, both 32bit
and 64bit dll will be installed.
--
Xiaofan
Peter Stuge
2011-02-24 08:23:00 UTC
Permalink
Post by Xiaofan Chen
Post by Peter Stuge
it looks to me like we should be able to work with an extended
version of the WinUSB API to provide the unconstrained
functionality required for libusb-1.0.
Great if true! Important for me that no DLL is required besides
libusb-1.0.dll, regardless of which kernel driver is used, but I
think this is also what is intended.
I am not sure what you mean here. For the WinUSB backend,
then you need the winusb.dll.
Are you sure? I didn't think so?


//Peter
Xiaofan Chen
2011-02-24 08:37:41 UTC
Permalink
Post by Peter Stuge
Post by Xiaofan Chen
Post by Peter Stuge
it looks to me like we should be able to work with an extended
version of the WinUSB API to provide the unconstrained
functionality required for libusb-1.0.
Great if true! Important for me that no DLL is required besides
libusb-1.0.dll, regardless of which kernel driver is used, but I
think this is also what is intended.
I am not sure what you mean here. For the WinUSB backend,
then you need the winusb.dll.
Are you sure? I didn't think so?
Yes I am sure. It has always been the case since the very
beginning. Read the code and search for "winusb.dll".
http://git.libusb.org/?p=libusb.git;a=blob;f=libusb/os/windows_usb.c

2417 /*
2418 * WinUSB API functions
2419 */
2420 static int winusb_init(struct libusb_context *ctx)
2421 {
2422 DLL_LOAD(winusb.dll, WinUsb_Initialize, TRUE);
2423 DLL_LOAD(winusb.dll, WinUsb_Free, TRUE);
2424 DLL_LOAD(winusb.dll, WinUsb_GetAssociatedInterface, TRUE);
2425 DLL_LOAD(winusb.dll, WinUsb_GetDescriptor, TRUE);
2426 DLL_LOAD(winusb.dll, WinUsb_QueryInterfaceSettings, TRUE);
2427 DLL_LOAD(winusb.dll, WinUsb_QueryDeviceInformation, TRUE);
2428 DLL_LOAD(winusb.dll, WinUsb_SetCurrentAlternateSetting, TRUE);
2429 DLL_LOAD(winusb.dll, WinUsb_GetCurrentAlternateSetting, TRUE);
2430 DLL_LOAD(winusb.dll, WinUsb_QueryPipe, TRUE);
2431 DLL_LOAD(winusb.dll, WinUsb_SetPipePolicy, TRUE);
2432 DLL_LOAD(winusb.dll, WinUsb_GetPipePolicy, TRUE);
2433 DLL_LOAD(winusb.dll, WinUsb_ReadPipe, TRUE);
2434 DLL_LOAD(winusb.dll, WinUsb_WritePipe, TRUE);
2435 DLL_LOAD(winusb.dll, WinUsb_ControlTransfer, TRUE);
2436 DLL_LOAD(winusb.dll, WinUsb_ResetPipe, TRUE);
2437 DLL_LOAD(winusb.dll, WinUsb_AbortPipe, TRUE);
2438 DLL_LOAD(winusb.dll, WinUsb_FlushPipe, TRUE);
2439
2440 api_winusb_available = true;
2441 return LIBUSB_SUCCESS;
2442 }

It is the same for HID backend, "hid.dll" is used.

It is actually quite common that you do not talk to the
kernel driver directly but rather through a dll.
--
Xiaofan
Pete Batard
2011-02-24 10:53:04 UTC
Permalink
Post by Xiaofan Chen
Post by Peter Stuge
Post by Xiaofan Chen
Post by Peter Stuge
Great if true! Important for me that no DLL is required besides
libusb-1.0.dll, regardless of which kernel driver is used, but I
think this is also what is intended.
I am not sure what you mean here. For the WinUSB backend,
then you need the winusb.dll.
Are you sure? I didn't think so?
Yes I am sure. It has always been the case since the very
beginning.
It is the same for HID backend, "hid.dll" is used.
Indeed.

Regards,

/Pete
Pete Batard
2011-02-24 10:42:03 UTC
Permalink
Post by Peter Stuge
Of course, I'll have a better idea when I try to do just that in
the K branch.
..which should again be based on libusb.git or at your option the
merge queue that is libusb-stuge.git.
Right. So that:

1. it's a major PITA to do anything since both mainline and -stuge are
missing important stuff, like, you know, the MSVC project files, or
toggable logging, which I need for any serious development. But sure, I
should waste my time duplicating files over and over, to compensate for
mainline's obvious deficiencies in getting anything addressed.

2. it's not integrated for at least another 6-12 months, whereas the
plan here, since mainline is unable to release anything with Windows, is
to integrate K into my branch as soon as it's stable enough, so that
people who've been waiting to use libusb0/k.sys on Windows actually have
a semi-official release that does so. Or am I supposed to release
another bunch of binaries for the K branch, and waste time that I
shouldn't have to waste, maintaining 2 (or more) different binary sets.
Talk about not confusing your users. In the absence of regular official
libusb releases that support Windows, I am currently seeing my branch as
compensating for what mainline is unable to do. Therefore, that's what
I'll keep using as my base.

Neither mainline or -stuge are ready for K. Without an effective
release, there are way too many contention points left in the air for
Windows, so it just doesn't make sense to use something that is still
far from having been stabilized (and which the libusb maintainers seem
to have no intention of stabilizing quickly) as one's foundation.

If you want anything on either libusb.git or -stuge.git, then get your
act together and sort out releases in a timely manner.
How's that reworked configure.ac in -stuge coming along by the way? From
what I'm seeing, you seem to have plenty of time acking patches for
openOCD, but surprisingly very little for a project where you are, in
effect, the _only_ maintainer...

Regards,

/Pete
Peter Stuge
2011-02-24 11:18:05 UTC
Permalink
Post by Pete Batard
Post by Pete Batard
the K branch.
..which should again be based on libusb.git or ..
1. it's a major PITA to do anything since both mainline and -stuge are
missing important stuff, like, you know, the MSVC project files, or
toggable logging, which I need for any serious development.
Would suggest to start by adding those in two separate commits before
anything else in K. Need only do that work once, since each commit in
K should apply easily also when the branch that K started from moves
forward.
Post by Pete Batard
But sure, I should waste my time duplicating files over and over,
No, but making the commits once and for all, that you would like to
propose, exactly to not have to duplicate the work later.
Post by Pete Batard
2. mainline .. my branch
No problem - you would easily include the changes in your
"collection" branch where you gather all important things.
Again should go smoothly, without conflicts.
Post by Pete Batard
am I supposed to release another bunch of binaries for the K branch
No, or at least I don't think that would be useful. The purpose of
the K branch (any branch really) is to make it as easy as possible
to add the code into libusb.git.
Post by Pete Batard
I am currently seeing my branch as compensating for what mainline
is unable to do. Therefore, that's what I'll keep using as my base.
Diverging further from libusb.git increases the required duplicated
effort.
Post by Pete Batard
Neither mainline or -stuge are ready for K.
WinUSB is in there, so I wouldn't expect the K commit to be very
different from how they would look on top of -pbatard master?
Post by Pete Batard
How's that reworked configure.ac in -stuge coming along by the way?
It's mostly removed, but still needs some more looking at. Am having
fun with patches for Darwin from Freenect right now.
Pete Batard
2011-02-24 12:18:56 UTC
Permalink
Post by Peter Stuge
Would suggest to start by adding those in two separate commits before
anything else in K.
Except, these are commits that maintainers explicitly expressed that
they didn't want in mainline, and I could add xusb as a prereq there too.
Daniel was adamant at one stage that he didn't want the MSVC project
files (but I'll assume we're over that now), and you don't seem to want
toggable logging or xusb.

Of course you're going to tell me that having the above or not shouldn't
impact the K patches for mainline, but there's other stuff that I might
be relying upon, like new enum, and considering that mainline seems to
be very adamant about trying to remove stuff that I do, I'm not gonna
take any risks.

In short, I do see mainline as a mess of shifty sand right now, as far
as Windows is concerned, which creates extra work for me that I wouldn't
have to do if integration issues were sorted out as they came along.
Besides, it's not like we're going to try to delay 1.0.9 to integrate K,
since K is still in early stages and incomplete (or are you suddenly
going to do RERO for K?). My view is that 1.0.9 should have Windows as
it is now, 1.0.10 should be the release that sorts MSVC/project files,
def generation, new enum, and do the overall catchup with my branch, and
1.0.11 should be the release looking into integrating K. And this is
from a RERO man. Anything else is just asking to multiply problems and
keep on not being able to produce libusb releases.
Post by Peter Stuge
Need only do that work once,
Provided I can rely on you guys not deciding, once again, to want to
remove stuff on a whim. Until there's a release, I consider that
anything I propose is up in the air, so I'm not gonna take that risk. As
things being kept in balance for more than a year have taught me, it's
just not worth the trouble gambling on dependencies that still aren't
part of an official release.
Post by Peter Stuge
Post by Pete Batard
But sure, I should waste my time duplicating files over and over,
No, but making the commits once and for all,
Well, the problem is the once and for all. You don't really want
toggable logging, or HID, or xusb, do you? So I'd be relying on "once
and for never" commits for my K branch. Might as well branch off my
forked repo then.

But seriously, just give us a stable mainline base where contention
points have been sorted, and then we'll discuss how contributors should
best branch from it. I've seriously had it with trying to base my work
on stuff that is going to be up in the air for months. If you want to
give advice about how mainline is the stable base to use for new
developments, then get your act together and actually stabilize mainline
through releases.
Post by Peter Stuge
Post by Pete Batard
am I supposed to release another bunch of binaries for the K branch
No, or at least I don't think that would be useful.
I do. I think people want libusb0/k.sys support ASAP, as you yourself
expressed that WinUSB and HID had too many limitations, and you also
expressed the idea that most Windows people would rely on binary
releases. So I'm just following your line of reasoning here.
Post by Peter Stuge
The purpose of
the K branch (any branch really) is to make it as easy as possible
to add the code into libusb.git.
Nope. The #1 purpose of any branch I create is feedback (through RERO),
else I'd just keep it private until I'm satisfied that it should be good
enough for integration. And since experience has shown me that topic
branches, such as hotplug, don't fare well in term of feedback (and I
don't see how branching off mainline or -pbatard would change anything
there), the sooner the K branch is dropped and integrated into -pbatard
the sooner we'll get effective testing, and, as a result, the better the
quality of K code.
Adding the code in libusb.git is secondary to getting good feedback so
that we have something of quality to integrate into mainline, when the
time comes. It's really not that hard to create a few patches for
integration from whatever branch.
Post by Peter Stuge
Diverging further from libusb.git increases the required duplicated
effort.
Don't worry, I'll be the one doing that effort. And I find that overall,
doing it this way reduces the duplicated effort I need to produce, by
maintaining myriads of branches.

I've become very effective in crafting patches for mainline out of my
big fat -pbatard branch. It's only if libusb can't bridge the gap with
Windows for another year that it's going to become really troublesome.
Until then, it's just more effective this way.
Post by Peter Stuge
Post by Pete Batard
Neither mainline or -stuge are ready for K.
WinUSB is in there, so I wouldn't expect the K commit to be very
different from how they would look on top of -pbatard master?
When I started, it was hard to tell whether K would depend on/modify
new-enum. It appears that it doesn't, but I assumed that it would. As
such, I'd have to produce a new-enum patch against mainline before I
could even consider going into K. Yeah, I'll have to produce a new-enum
patch for mainline eventually, but I'd rather do that when I have the my
own integration branch up, which depends on your configure.ac. All in
all, branching off mainline doesn't make sense. It's simply too far
behind, with too many unresolved contention points, to be a serious
contender as a branch point for anything Windows.
Post by Peter Stuge
Post by Pete Batard
How's that reworked configure.ac in -stuge coming along by the way?
It's mostly removed, but still needs some more looking at. Am having
fun with patches for Darwin from Freenect right now.
OK. Just remember that it's a major bottleneck for me, so the sooner you
get a finalized configure.ac, the better.
Pete Batard
2011-02-22 16:38:01 UTC
Permalink
Post by Xiaofan Chen
Travis is working on a new
generation of libusb-win32 kernel driver based on KMDF
and it will be much easier to be integrated into the current
libusb-1.0 Windows codes since it can emulate WinUSB API.
In fact, we have working prototypes of the driver and it works
well with the current libusb-1.0 WinUSB backend.
And this new driver is now supported (through the WinUSB emulation DLL)
in my new 'K' branch [1].

This has been added as a new sub-backend, meaning that you can use both
WinUSB and libusbwK (the new driver) at the same time.

A massive thanks to Travis for his single-handed work here, since the
WinUSB compatibility made the integration very straightforward, and
while this is something I kinda wished for for the evolution of
libusb0.sys, I didn't think it would ever happen.

I think providing a WinUSB compatible driver has advantages, since
WinUSB was intended to be a generic USB API in the first place, just
like libusb0.sys, and we should now be able to do something about its
limitations. Sure some of Microsoft's implementation choices are very
disputable, but a lot of it does provide the functionality that libusb
needs, so we might very well be able to go with an extended Microsoft
API (i.e. with calls added to bypass the WinUSB limitations wgere
needed), instead of implementing 2 separate APIs that essentially access
the same driver.

Anyway, will have to see how it turns out. If you're interested, please
test the 'K' branch. I'll be working on adding driver support in libwdi
as well.

Regards,

/Pete


[1]
http://git.libusb.org/?p=libusb-pbatard.git;a=shortlog;h=refs/heads/K;js=1
Loading...