LSP febr SDK - other problems found

[Viviana Vc]

Hi all,

I'm working on the LSP sample from the SDK from febr 2003, and I found
some issues:

1) when it is removed from a chain, it doesn't delete it's name from the
old chains. I.e. LSP1 over BP, LSP2 over LSP1 over BP. When removing
LSP1 everything works ok, but the name remains in the the LPS2 chain.

2) I've tried this sample along with probably the first LSP provided by
MS, that I found on:
http://www.microsoft.com/msj/0599/layeredservice/layeredservice.aspx
The problem is that everything works fine when the chain is like:
LSP-old over LSP-2003 over BP
but doesn't work when it is like:
LSP-2003 over LSP-old over BP

The WSPSocket fails, actually the
NextProviderSocket = Provider->NextProcTable.lpWSPSocket
returns INVALID_SOCKET.

I couldn't find out the reason, but I'm sure that on the market are some
products using this LSP so every product with the new LSP won't work
with those ones ... :(
So, if anybody has a suggestion maybe Anthony or Ken can check this, as
it's really annoying 2 LSPs provided by MS not to be compatible. Thx, in
advance for any help or suggestion.

3) I've tried to use ctime() function in my LSP and something weird
happened. If it's only one LSP in the system, everything seems to work
fine, but if there are 2 identical LSPs, if ctime is used, I get a crash
"The memory can not be written ....." when closing the application, for
instance when closing internet explorer. Has anybody any idea why this
happens? I would have liked to use ctime() for debugging the LSP, but it
seems that it's safer not to use it ... Why?


Hope that somebody can help me,
Thx a lot,
Viv

[Ken Wickes [MSFT]]

It's been a while since I looked at this. There certainly is code to remove
the LSP from other chains, at least for XP, in the sample. Is that code
getting executed for you?

Not sure about the WSPSocket problem, never had a problem with that.
--
Ken Wickes [MSFT]
This posting is provided "AS IS" with no warranties, and confers no rights.

Post by Viviana Vc
Hi all,
I'm working on the LSP sample from the SDK from febr 2003, and I found
1) when it is removed from a chain, it doesn't delete it's name from the
old chains. I.e. LSP1 over BP, LSP2 over LSP1 over BP. When removing
LSP1 everything works ok, but the name remains in the the LPS2 chain.
2) I've tried this sample along with probably the first LSP provided by
http://www.microsoft.com/msj/0599/layeredservice/layeredservice.aspx
LSP-old over LSP-2003 over BP
LSP-2003 over LSP-old over BP
The WSPSocket fails, actually the
NextProviderSocket = Provider->NextProcTable.lpWSPSocket
returns INVALID_SOCKET.
I couldn't find out the reason, but I'm sure that on the market are some
products using this LSP so every product with the new LSP won't work
with those ones ... :(
So, if anybody has a suggestion maybe Anthony or Ken can check this, as
it's really annoying 2 LSPs provided by MS not to be compatible. Thx, in
advance for any help or suggestion.
3) I've tried to use ctime() function in my LSP and something weird
happened. If it's only one LSP in the system, everything seems to work
fine, but if there are 2 identical LSPs, if ctime is used, I get a crash
"The memory can not be written ....." when closing the application, for
instance when closing internet explorer. Has anybody any idea why this
happens? I would have liked to use ctime() for debugging the LSP, but it
seems that it's safer not to use it ... Why?
Hope that somebody can help me,
Thx a lot,
Viv

[Viviana Vc]

On Wed, 10 Sep 2003 12:41:04 -0700, "Ken Wickes [MSFT]"

Post by Ken Wickes [MSFT]
It's been a while since I looked at this. There certainly is code to remove
the LSP from other chains, at least for XP, in the sample. Is that code
getting executed for you?

Yes, I'm working on an XP and is working fine, the problem is that the
name is not removed from the old chains. So, everything works fine,
but it doesn't look good.
It's like:
L2 over L1 over BP
L2 over BP
L1 over BP
BP

If you now remove the L1 everything will work ok, but looking with
sporder.exe at the chains, they will look like:
L2 over L1 over BP
BP

as L1 removed itself phisically from the chain, but it's name is still
there.

Post by Ken Wickes [MSFT]
Not sure about the WSPSocket problem, never had a problem with that.

But have you tried the new LSP with that old one? Do you have any idea
why would this happen?

What about the ctime problem? Any idea?

Thx,
Viv

[arkadyf]

about ctime() - it looks very strange because all still on user level , did
you tried other
time functions ?
Arkady

Post by Viviana Vc
On Wed, 10 Sep 2003 12:41:04 -0700, "Ken Wickes [MSFT]"

Post by Ken Wickes [MSFT]
It's been a while since I looked at this. There certainly is code to remove
the LSP from other chains, at least for XP, in the sample. Is that code
getting executed for you?

Yes, I'm working on an XP and is working fine, the problem is that the
name is not removed from the old chains. So, everything works fine,
but it doesn't look good.
L2 over L1 over BP
L2 over BP
L1 over BP
BP
If you now remove the L1 everything will work ok, but looking with
L2 over L1 over BP
BP
as L1 removed itself phisically from the chain, but it's name is still
there.

Post by Ken Wickes [MSFT]
Not sure about the WSPSocket problem, never had a problem with that.

But have you tried the new LSP with that old one? Do you have any idea
why would this happen?
What about the ctime problem? Any idea?
Thx,
Viv

[Viviana Vc]

Hmmm, not very sure if this is a ctime problem. I mean if I have it in
the code, the ctime call, and 2 of my LSPs installed I get the crash
when closing the IE. If I remove the ctime everything seems to be fine.

But I suspect that there actually is another problem, because even if I
remove the ctime call, and install the original version of LSP febr 2003
over the new.net, when I close IE I get the same crash.

So I suspect that the problem is somehwere else ... but not sure.

I assume that I'm not the first one trying to install the LSP over
new.net, so is there anyone knowing how this problem can be fixed.

As I already said, installing an older version of LSP sample (5 years
old), I don't get this problem, so I assume this is in the code of the
LSP sample febr 2003.

If anybody can help, please give me some hints.

Thx a lot,
Viv

Post by arkadyf
about ctime() - it looks very strange because all still on user level , did
you tried other
time functions ?
Arkady

Post by Viviana Vc
On Wed, 10 Sep 2003 12:41:04 -0700, "Ken Wickes [MSFT]"

Post by Ken Wickes [MSFT]
It's been a while since I looked at this. There certainly is code to

remove

Post by Viviana Vc

Post by Ken Wickes [MSFT]
the LSP from other chains, at least for XP, in the sample. Is that code
getting executed for you?

Yes, I'm working on an XP and is working fine, the problem is that the
name is not removed from the old chains. So, everything works fine,
but it doesn't look good.
L2 over L1 over BP
L2 over BP
L1 over BP
BP
If you now remove the L1 everything will work ok, but looking with
L2 over L1 over BP
BP
as L1 removed itself phisically from the chain, but it's name is still
there.

Post by Ken Wickes [MSFT]
Not sure about the WSPSocket problem, never had a problem with that.

But have you tried the new LSP with that old one? Do you have any idea
why would this happen?
What about the ctime problem? Any idea?
Thx,
Viv

[Viviana Vc]

I found the problem ... or let's say what is causing this ...

If you have 2 LSPs, that are using ctime and you build the sources
with /MT flag (or in Makefile $(cvarsmt)), you'll get a crash when
trying to close IE.

If you build the sources with /MD ($(cvarsdll)) then everything works
fine. This is unfortunately not an option for me, as I have some
templates stuff in my LSP that can be built only with VC 7.1, so if I
use /MD means that I have also to provide to the customers the
msvc*71.dlls which is ok, but being related with the LSP, they have to
be put in system dir, which is not recommended by MS, so I have to use
the /MT flag, which is just not "compatible" with ctime, so I have to
find another solution.

Viv

Post by arkadyf
about ctime() - it looks very strange because all still on user level , did
you tried other
time functions ?
Arkady

Post by Viviana Vc
On Wed, 10 Sep 2003 12:41:04 -0700, "Ken Wickes [MSFT]"

Post by Ken Wickes [MSFT]
It's been a while since I looked at this. There certainly is code to

remove

Post by Viviana Vc

Post by Ken Wickes [MSFT]
the LSP from other chains, at least for XP, in the sample. Is that code
getting executed for you?

Yes, I'm working on an XP and is working fine, the problem is that the
name is not removed from the old chains. So, everything works fine,
but it doesn't look good.
L2 over L1 over BP
L2 over BP
L1 over BP
BP
If you now remove the L1 everything will work ok, but looking with
L2 over L1 over BP
BP
as L1 removed itself phisically from the chain, but it's name is still
there.

Post by Ken Wickes [MSFT]
Not sure about the WSPSocket problem, never had a problem with that.

But have you tried the new LSP with that old one? Do you have any idea
why would this happen?
What about the ctime problem? Any idea?
Thx,
Viv

[Alun Jones [MS MVP]]

Post by arkadyf
about ctime() - it looks very strange because all still on user level , did
you tried other
time functions ?

Those time functions from the RTL are really not worth calling in
multithreaded code, or anything that smacks of system driver (like an LSP),
where you're essentially running in multithreaded / multiprocess operations.

For reason why, here's a quote from the documentation for ctime:

"A call to ctime modifies the single statically allocated buffer used by the
gmtime and localtime functions. Each call to one of these routines destroys
the result of the previous call. ctime shares a static buffer with the
asctime function. Thus, a call to ctime destroys the results of any previous
call to asctime, localtime, or gmtime."

So, basically, ctime is _not_ reentrant by any means, and shares that
non-reentrancy with several other time functions!

Most of the old C run-time library functions are designed for use in a
single process environment, prior to threads.

Alun.
~~~~

[Please don't email posters, if a Usenet response is appropriate.]

[Viviana Vc]

Yes, but I've put the ctime code into a critical section and I thought
that this will be ok ...

Anyhow, is there any Win API that I could use instead to print the time
in a readable format?

Thx,
Viv

Post by Alun Jones [MS MVP]

Post by arkadyf
about ctime() - it looks very strange because all still on user level , did
you tried other
time functions ?

Those time functions from the RTL are really not worth calling in
multithreaded code, or anything that smacks of system driver (like an LSP),
where you're essentially running in multithreaded / multiprocess operations.
"A call to ctime modifies the single statically allocated buffer used by the
gmtime and localtime functions. Each call to one of these routines destroys
the result of the previous call. ctime shares a static buffer with the
asctime function. Thus, a call to ctime destroys the results of any previous
call to asctime, localtime, or gmtime."
So, basically, ctime is _not_ reentrant by any means, and shares that
non-reentrancy with several other time functions!
Most of the old C run-time library functions are designed for use in a
single process environment, prior to threads.
Alun.
~~~~
[Please don't email posters, if a Usenet response is appropriate.]

[Alun Jones [MS MVP]]

Post by Viviana Vc
Yes, but I've put the ctime code into a critical section and I thought
that this will be ok ...

Great. Now all you need to do is find every Winsock application that runs
over your stack, and put every ctime() / localtime() / gmtime() call into
the same critical section.

I don't think that you can do that.

Post by Viviana Vc
Anyhow, is there any Win API that I could use instead to print the time
in a readable format?

You could format it yourself, you could accept that time logging inside what
is essentially a device driver is not something that's particularly
essential to be clearly readable, or you could use GetTimeFormat.

Alun.
~~~~

[Please don't email posters, if a Usenet response is appropriate.]

[Viviana Vc]

Post by Alun Jones [MS MVP]

Post by Viviana Vc
Yes, but I've put the ctime code into a critical section and I thought
that this will be ok ...

Great. Now all you need to do is find every Winsock application that runs
over your stack, and put every ctime() / localtime() / gmtime() call into
the same critical section.
I don't think that you can do that.

True ... I thought just from my LSP point of view ... :(

Post by Alun Jones [MS MVP]

Post by Viviana Vc
Anyhow, is there any Win API that I could use instead to print the time
in a readable format?

You could format it yourself, you could accept that time logging inside what
is essentially a device driver is not something that's particularly
essential to be clearly readable, or you could use GetTimeFormat.

Thx,
Viv

Post by Alun Jones [MS MVP]
Alun.
~~~~
[Please don't email posters, if a Usenet response is appropriate.]

[dim]

Hi!
I've been working on the same LSP sample (SDK 2003) and finally it works (on
both 9x and 2000/XP), but not so long ago it crashed a winsock catalog in
registry on two PCs, both having an New.net (www.new.net) installed (and,
Norton Antivirus - something makes me think that there's its fault too).
New.net uses LSP and NSP for resolving fake domains like .shopping , .home
etc , but it detects any chained LSP and refuses to install its LSP part
(installs just an NSP) showing a corresponding message in browser during
installation. I made an experiment and installed my provider after New.net
on win98 under VMWare, everything worked fine and uninstalled fine too. Then
I made the same thing on another PC (win98) without troubles.

PS: now my colleague reported a crash of my LSP on win98 with MS (Socks ?)
Proxy and Norton Antivirus. As far as I know MS Proxy uses its own LSP...

Could be there some bug in WCSWriteProviderOrder() from sporder.dll ?

If someone else knows anything else about such behavior of LSP, please reply
!
Thanks.

[Viviana Vc]

Post by dim
Hi!
I've been working on the same LSP sample (SDK 2003) and finally it works (on
both 9x and 2000/XP), but not so long ago it crashed a winsock catalog in
registry on two PCs, both having an New.net (www.new.net) installed (and,

I'm having the exactly same problem. When new.net is installed and then
I install my LSP, I.E. works fine, but when I try to close IE it crashes
with "The memory could not be written ..."

This can be a new.net problem , or the LSP sample problem. I suspect
being an LSP problem (febr 2003), because until 2 month ago I worked on
the old LSP version provided by MS (5 years ago), and that one installed
over new.net works perfectly.

If you had the same problem, can you please tell me how you fixed it? It
would be really useful. Or you are talking about some other crash?

Post by dim
Norton Antivirus - something makes me think that there's its fault too).
New.net uses LSP and NSP for resolving fake domains like .shopping , .home
etc , but it detects any chained LSP and refuses to install its LSP part
(installs just an NSP) showing a corresponding message in browser during
installation. I made an experiment and installed my provider after New.net
on win98 under VMWare, everything worked fine and uninstalled fine too. Then
I made the same thing on another PC (win98) without troubles.
PS: now my colleague reported a crash of my LSP on win98 with MS (Socks ?)
Proxy and Norton Antivirus. As far as I know MS Proxy uses its own LSP...
Could be there some bug in WCSWriteProviderOrder() from sporder.dll ?
If someone else knows anything else about such behavior of LSP, please reply
!
Thanks.

[dim]

Hi!

This is a very strange crash, imho it's not caused by new.net LSP, try using
an original sample from SDK instead to see who is guilty. As far as I know
this sample works fine, the only problem I had was the installation on
win9x, I thought I solved it, but it still corrupts a winsock catalog.

These are some hints for LSP developers:

1) When you install the LSP, you must layer over the entire address family:
for example you are providing some tcp services, so you must layer over tcp,
udp and raw (and, if there are any other protocols you must layer over them
too).
But beware, there's a bug in win2000 RSVP that causes explorer.exe to hang
on start, I lost a couple of days trying to solve this hang, and discovered
that it happens only with service packs 1 and 2, and it was fixed in SP 3 (I
was using sp2). So there's a hint: don't ever layer over RSVP, see my code
(instlsp.cpp):

// Get the catalog IDs for all existing providers
for(i=0; i < iTotalProtocols ;i++)
{
if (pProtocolInfo[i].ProtocolChain.ChainLen != LAYERED_PROTOCOL
&& pProtocolInfo[i].iAddressFamily == AF_INET // IP address family
&& pProtocolInfo[i].ProtocolChain.ChainLen == 1 // base provider
// don't layer over RSVP (due to a bug that hangs up some progs
Win2K, fixed in SP3)
&& (bIsWinNT ? !(pProtocolInfo[i].dwServiceFlags1 &
XP1_QOS_SUPPORTED) : true)
)
...
Note the XP1_QOS_SUPPORTED option, this way I avoid QOS providers (RSVP).

2) Install on 9x:
IMHO, SDK sample was not tested with 9x, because it doesn't install neither
can install properly. In instlsp.cpp it writes %SYSTEMROOT%\lspname.dll for
both win2k and 9x and it's wrong! 9x doesn't have or doesn't expand
%SYSTEMROOT%. So you must do it manually (remember to convert everything to
unicode):

bool bIsWinNT = (GetVersion() < 0x80000000); // OS type flag
memset(ProviderPath, 0, sizeof(ProviderPath));
if(bIsWinNT) {
GetSystemDirectoryW(ProviderPath, MAX_PATH + 1);
} else {
char szPath[MAX_PATH + 1];
UINT iPathLen = GetSystemDirectoryA(szPath, sizeof(szPath));
MultiByteToWideChar(CP_ACP, 0, szPath, -1 /* -1 means null terminated
*/, ProviderPath, sizeof(ProviderPath));
}
wcscat(ProviderPath, L"\\");
wcscat(ProviderPath, ProviderName);
...
(I used LSP sample from SDK 2003 RC2, but the newest one has the same bug)


3) Maybe would useful to make a backup of winsock catalog before
installation and monitor it later from inside the LSP or some sort of
service process. If somebody inserts its provider this monitor would detect
such intent and check the integrity and loadability of all underlying
providers and remove any ill behaved one. Anyway, it's just an idea :) , I
can't be sure that MS won't change the winsock catalog
format/location/naming.


PS: Do anybody know what does MS team saying about LSPs and where can we
found more info ?

---------------------------------------------
Dim.
NetV, Development dept.

[Viviana Vc]

At the 2nd issue is just enough if you provide as provider path
just the lsp name: lsp.dll without any path. It's working fine on all
systems in this way: 9x/2k/xp.

Viv

Post by dim
Hi!
This is a very strange crash, imho it's not caused by new.net LSP, try using
an original sample from SDK instead to see who is guilty. As far as I know
this sample works fine, the only problem I had was the installation on
win9x, I thought I solved it, but it still corrupts a winsock catalog.
for example you are providing some tcp services, so you must layer over tcp,
udp and raw (and, if there are any other protocols you must layer over them
too).
But beware, there's a bug in win2000 RSVP that causes explorer.exe to hang
on start, I lost a couple of days trying to solve this hang, and discovered
that it happens only with service packs 1 and 2, and it was fixed in SP 3 (I
was using sp2). So there's a hint: don't ever layer over RSVP, see my code
// Get the catalog IDs for all existing providers
for(i=0; i < iTotalProtocols ;i++)
{
if (pProtocolInfo[i].ProtocolChain.ChainLen != LAYERED_PROTOCOL
&& pProtocolInfo[i].iAddressFamily == AF_INET // IP address family
&& pProtocolInfo[i].ProtocolChain.ChainLen == 1 // base provider
// don't layer over RSVP (due to a bug that hangs up some progs
Win2K, fixed in SP3)
&& (bIsWinNT ? !(pProtocolInfo[i].dwServiceFlags1 &
XP1_QOS_SUPPORTED) : true)
)
...
Note the XP1_QOS_SUPPORTED option, this way I avoid QOS providers (RSVP).
IMHO, SDK sample was not tested with 9x, because it doesn't install neither
can install properly. In instlsp.cpp it writes %SYSTEMROOT%\lspname.dll for
both win2k and 9x and it's wrong! 9x doesn't have or doesn't expand
%SYSTEMROOT%. So you must do it manually (remember to convert everything to
bool bIsWinNT = (GetVersion() < 0x80000000); // OS type flag
memset(ProviderPath, 0, sizeof(ProviderPath));
if(bIsWinNT) {
GetSystemDirectoryW(ProviderPath, MAX_PATH + 1);
} else {
char szPath[MAX_PATH + 1];
UINT iPathLen = GetSystemDirectoryA(szPath, sizeof(szPath));
MultiByteToWideChar(CP_ACP, 0, szPath, -1 /* -1 means null terminated
*/, ProviderPath, sizeof(ProviderPath));
}
wcscat(ProviderPath, L"\\");
wcscat(ProviderPath, ProviderName);
...
(I used LSP sample from SDK 2003 RC2, but the newest one has the same bug)
3) Maybe would useful to make a backup of winsock catalog before
installation and monitor it later from inside the LSP or some sort of
service process. If somebody inserts its provider this monitor would detect
such intent and check the integrity and loadability of all underlying
providers and remove any ill behaved one. Anyway, it's just an idea :) , I
can't be sure that MS won't change the winsock catalog
format/location/naming.
PS: Do anybody know what does MS team saying about LSPs and where can we
found more info ?
---------------------------------------------
Dim.
NetV, Development dept.

[t***@yahoo.com]

I think you guys should check out http://www.opswat.com/ there are 2 or 3 products that may be a match, I think that OESIS Framework at http://www.opswat.com/products/oesis-framework provide a single interface to many antivirus packages, another option is I think Metascan at http://www.opswat.com/products/metascan which is more for ISV, I also found that many antivirus engines certified by OPSWAT at http://www.opswat.com/certified

I hope this helps.
Regards,

Tom