Discussion:
[fossil-users] Fossil on cygwin64
Lluís Batlle i Rossell
2013-07-22 19:21:57 UTC
Permalink
Hello,

today I built fossil on cygwin64, and it built but "it didn't work". Cloning, a
line in os_win.c complained about not having permission to create a file (a tmp
file with some kind of random string) in C:/windows.

I wonder... why is it running any os_win code? shouldn't cygwin look like unix, to
fossil?

I've been using happily fossil in cygwin 32-bit since years, and only today I
tried this cygwin64 (completely new for me). Has anyone tried it? Maybe I am
doing something very wrong.

I tried both the 'configure' makefile, and Makefile.classic.

Regards,
Lluís.
Stephan Beal
2013-07-22 19:38:45 UTC
Permalink
Post by Lluís Batlle i Rossell
I've been using happily fossil in cygwin 32-bit since years, and only today I
tried this cygwin64 (completely new for me). Has anyone tried it? Maybe I am
doing something very wrong.
Speculation: i _suspect_ that the C macros which determine "is this
cygwin/mingw" simply misfire for 64-bit environments.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Martin Gagnon
2013-07-22 20:13:30 UTC
Permalink
Post by Lluís Batlle i Rossell
Hello,
today I built fossil on cygwin64, and it built but "it didn't work". Cloning, a
line in os_win.c complained about not having permission to create a file (a tmp
file with some kind of random string) in C:/windows.
I wonder... why is it running any os_win code? shouldn't cygwin look like unix, to
fossil?
I've been using happily fossil in cygwin 32-bit since years, and only today I
tried this cygwin64 (completely new for me). Has anyone tried it? Maybe I am
doing something very wrong.
I tried both the 'configure' makefile, and Makefile.classic.
I don't know if it can help you, but I use mingw64 with bash shell from
msys and I don't have this problem. MinGW version should use a bit more
more from the os_win code since it's not emulating a fully posix
environement like cygwin.

I stop building fossil for cygwin long time ago and I found that cygwin
version of fossil was a lot slower (with big repo) and I had problem
when using the same checkout from cygwin and non-cygwin binary.
--
Martin G.
Jan Nijtmans
2013-07-24 08:33:38 UTC
Permalink
Post by Lluís Batlle i Rossell
Hello,
today I built fossil on cygwin64, and it built but "it didn't work". Cloning, a
line in os_win.c complained about not having permission to create a file (a tmp
file with some kind of random string) in C:/windows.
I can easily reproduce that (see below). The Cygwin folks made some local
modifications to sqlite, allowing cooperation between Cygwin and Windows
programs. Cygwin64 doesn't have those modifications yet.
Post by Lluís Batlle i Rossell
I stop building fossil for cygwin long time ago and I found that cygwin
version of fossil was a lot slower (with big repo) and I had problem
when using the same checkout from cygwin and non-cygwin binary.
That should be fixed now. For details, see:
<http://cygwin.com/ml/cygwin-announce/2013-06/msg00014.html>
<https://www.fossil-scm.org/index.html/info/340cee03ee>
You can try it on Cygwin32 using
./configure --disable-internal-sqlite
There it should work fine now, but feel free to report any additional
problems you find.

On Cygwin64 that doesn't work yet due to the above mentioned reason:
$ ./configure --disable-internal-sqlite
....
$ make
....
$ ./fossil update
Autosync: ....
SQLITE_CANTOPEN: os_win.c:34063: (3)
winOpen(/var/tmp/etilqs_8td6VBhZb6xMX3V\etilqs_QL9aCIutNgLB3ZP) - The
system cannot find the path specified.
SQLITE_CANTOPEN: os_win.c:34063: (3)
winOpen(/var/tmp/etilqs_FoaHNBQa56cVGrh\etilqs_PXfZEjH5dBl8Cm5) - The
system cannot find the path specified.
SQLITE_CANTOPEN: cannot open file at line 34071 of [cbea02d938]
SQLITE_CANTOPEN: statement aborts at 38: [CREATE TEMP TABLE
onremote(rid INTEGER PRIMARY KEY);] unable to open database file
./fossil: unable to open database file: {CREATE TEMP TABLE
onremote(rid INTEGER PRIMARY KEY);}

If you have recently updated your fossil executable, you might
need to run "fossil all rebuild" to bring the repository
schemas up to date.

Just wait on the Cygwin64 people to bring out a new Sqlite package with
the same fixes already done in Cygwin32.

Thanks!

Regards,
Jan Nijtmans
Lluís Batlle i Rossell
2013-07-24 08:39:15 UTC
Permalink
Great! Very informative. Thank you.

Do you happen to know if I can build and run programs for cygwin32, in a
cygwin64 installation? or I should run two cygwin setups in orthogonally, 32 and
64?
Post by Jan Nijtmans
Post by Lluís Batlle i Rossell
Hello,
today I built fossil on cygwin64, and it built but "it didn't work". Cloning, a
line in os_win.c complained about not having permission to create a file (a tmp
file with some kind of random string) in C:/windows.
I can easily reproduce that (see below). The Cygwin folks made some local
modifications to sqlite, allowing cooperation between Cygwin and Windows
programs. Cygwin64 doesn't have those modifications yet.
Post by Lluís Batlle i Rossell
I stop building fossil for cygwin long time ago and I found that cygwin
version of fossil was a lot slower (with big repo) and I had problem
when using the same checkout from cygwin and non-cygwin binary.
<http://cygwin.com/ml/cygwin-announce/2013-06/msg00014.html>
<https://www.fossil-scm.org/index.html/info/340cee03ee>
You can try it on Cygwin32 using
./configure --disable-internal-sqlite
There it should work fine now, but feel free to report any additional
problems you find.
$ ./configure --disable-internal-sqlite
....
$ make
....
$ ./fossil update
Autosync: ....
SQLITE_CANTOPEN: os_win.c:34063: (3)
winOpen(/var/tmp/etilqs_8td6VBhZb6xMX3V\etilqs_QL9aCIutNgLB3ZP) - The
system cannot find the path specified.
SQLITE_CANTOPEN: os_win.c:34063: (3)
winOpen(/var/tmp/etilqs_FoaHNBQa56cVGrh\etilqs_PXfZEjH5dBl8Cm5) - The
system cannot find the path specified.
SQLITE_CANTOPEN: cannot open file at line 34071 of [cbea02d938]
SQLITE_CANTOPEN: statement aborts at 38: [CREATE TEMP TABLE
onremote(rid INTEGER PRIMARY KEY);] unable to open database file
./fossil: unable to open database file: {CREATE TEMP TABLE
onremote(rid INTEGER PRIMARY KEY);}
If you have recently updated your fossil executable, you might
need to run "fossil all rebuild" to bring the repository
schemas up to date.
Just wait on the Cygwin64 people to bring out a new Sqlite package with
the same fixes already done in Cygwin32.
Thanks!
Regards,
Jan Nijtmans
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Jan Nijtmans
2013-07-24 09:16:19 UTC
Permalink
Post by Lluís Batlle i Rossell
Do you happen to know if I can build and run programs for cygwin32, in a
cygwin64 installation? or I should run two cygwin setups in orthogonally, 32 and
64?
I would recommend to compile fossil using:
make -f win/Makefile.mingw PREFIX=x86_64-w64-mingw32-
Then you get a WIN64 version of fossil.exe which
works in any environment (Cygwin32, Cygwin64, Msys, CMD),
and is super fast.

All Cygwin64 and Cygwin32 dll's have the same name, so you
cannot mix executables from those two environments.

Regards,
Jan Nijtmans
Lluís Batlle i Rossell
2013-07-24 09:36:39 UTC
Permalink
Post by Jan Nijtmans
Post by Lluís Batlle i Rossell
Do you happen to know if I can build and run programs for cygwin32, in a
cygwin64 installation? or I should run two cygwin setups in orthogonally, 32 and
64?
make -f win/Makefile.mingw PREFIX=x86_64-w64-mingw32-
Then you get a WIN64 version of fossil.exe which
works in any environment (Cygwin32, Cygwin64, Msys, CMD),
and is super fast.
I think our main usage for a cygwin fossil is that we develop using a cygwin
terminal with bash and vim. And we want fossil to spawn vim properly on commit.

iiuc, a native fossil can't spawn a proper vim editor in the cygwin bash
terminal. Is it right? Do you know any workaround?
Post by Jan Nijtmans
All Cygwin64 and Cygwin32 dll's have the same name, so you
cannot mix executables from those two environments.
Ah ok. Good to know!

Thank you,
Lluís.
Jan Nijtmans
2013-07-24 09:53:02 UTC
Permalink
Post by Lluís Batlle i Rossell
I think our main usage for a cygwin fossil is that we develop using a cygwin
terminal with bash and vim. And we want fossil to spawn vim properly on commit.
That should work, only you should do something like:
export EDITOR="C:/Cygwin64/bin/bash.exe -c vim"
I don't know vim enough to comment on that.

Windows Fossil nowadays uses Notepad.exe as default
commit editor, isn't that much easier?

Regards,
Jan Nijtmans
Lluís Batlle i Rossell
2013-07-24 10:04:44 UTC
Permalink
Post by Jan Nijtmans
Post by Lluís Batlle i Rossell
I think our main usage for a cygwin fossil is that we develop using a cygwin
terminal with bash and vim. And we want fossil to spawn vim properly on commit.
export EDITOR="C:/Cygwin64/bin/bash.exe -c vim"
I don't know vim enough to comment on that.
Windows Fossil nowadays uses Notepad.exe as default
commit editor, isn't that much easier?
notepad easier than vim? Not at all, for a brain trained on vim. :)
Lluís Batlle i Rossell
2013-07-24 10:06:12 UTC
Permalink
Post by Jan Nijtmans
Post by Lluís Batlle i Rossell
I think our main usage for a cygwin fossil is that we develop using a cygwin
terminal with bash and vim. And we want fossil to spawn vim properly on commit.
export EDITOR="C:/Cygwin64/bin/bash.exe -c vim"
I don't know vim enough to comment on that.
Ah, the point isn't bash; the point is at processes passing properly the fds
for the *terminal*. I guess that once you put a non-cygwin process in the family
tree, the children can't access the terminal ancestor properly.
Martin Gagnon
2013-07-24 12:01:47 UTC
Permalink
Post by Lluís Batlle i Rossell
Post by Jan Nijtmans
Post by Lluís Batlle i Rossell
I think our main usage for a cygwin fossil is that we develop using a cygwin
terminal with bash and vim. And we want fossil to spawn vim properly on commit.
export EDITOR="C:/Cygwin64/bin/bash.exe -c vim"
I don't know vim enough to comment on that.
Ah, the point isn't bash; the point is at processes passing properly the fds
for the *terminal*. I guess that once you put a non-cygwin process in the family
tree, the children can't access the terminal ancestor properly.
Have you try gvim -f ? (Native windows GUI version of vim) so your comment
will not end by ':wq' :-) like what could happens with notepad.
--
Martin G.
Lluís Batlle i Rossell
2013-07-24 12:48:08 UTC
Permalink
Post by Lluís Batlle i Rossell
Post by Lluís Batlle i Rossell
Post by Jan Nijtmans
Post by Lluís Batlle i Rossell
I think our main usage for a cygwin fossil is that we develop using a
cygwin
Post by Lluís Batlle i Rossell
Post by Jan Nijtmans
Post by Lluís Batlle i Rossell
terminal with bash and vim. And we want fossil to spawn vim properly
on commit.
Post by Lluís Batlle i Rossell
Post by Jan Nijtmans
export EDITOR="C:/Cygwin64/bin/bash.exe -c vim"
I don't know vim enough to comment on that.
Ah, the point isn't bash; the point is at processes passing properly the
fds
Post by Lluís Batlle i Rossell
for the *terminal*. I guess that once you put a non-cygwin process in the
family
Post by Lluís Batlle i Rossell
tree, the children can't access the terminal ancestor properly.
Have you try gvim -f ? (Native windows GUI version of vim) so your comment
will not end by ':wq' :-) like what could happens with notepad.
If I use windows with *all inside a cygwin terminal* is because I adapt very bad
to unusual environments. :)
Warren Young
2013-07-24 11:06:28 UTC
Permalink
Post by Jan Nijtmans
SQLITE_CANTOPEN: os_win.c:34063: (3)
winOpen(/var/tmp/etilqs_FoaHNBQa56cVGrh\etilqs_PXfZEjH5dBl8Cm5) - The
system cannot find the path specified.
I'm not sure whether this is a SQLite 3.7.17 thing or if it is due to
one of the build option changes in recent versions of Cygwin SQLite.
All I know is that it seems to be a new thing.

As you can see from the error, SQLite is generating a temp file name
with backslashes in it. This is purely wrong on Windows, whether
running under Cygwin or not.

I haven't bothered looking any deeper into it than that. I hope it just
gets fixed for me in .18. If not, I may have to hack around it in my
next release of Cygwin SQLite.

In the meantime, some people have had luck setting the new environment
variable CYGWIN_SQLITE_LOCKING=posix. This means Cygwin SQLite won't
interoperate well with Windows native SQLite, but if you don't need
that, I believe it will bypass this problematic tmp file name generation.
Post by Jan Nijtmans
Just wait on the Cygwin64 people to bring out a new Sqlite package with
the same fixes already done in Cygwin32.
Um, it's the same people. Me. :)

Both packages are generated from the same source, with the same build
options. If they behave differently, it's likely because the SQLite
code is doing something wrong with an ifdef, or because Cygwin itself
behaves differently.
Jan Nijtmans
2013-07-24 11:30:56 UTC
Permalink
Post by Jan Nijtmans
Just wait on the Cygwin64 people to bring out a new Sqlite package with
Post by Jan Nijtmans
the same fixes already done in Cygwin32.
Um, it's the same people. Me. :)
Both packages are generated from the same source, with the same build
options. If they behave differently, it's likely because the SQLite code
is doing something wrong with an ifdef, or because Cygwin itself behaves
differently.
It looks like SQLite is doing something wrong here:
<http://www.fossil-scm.org/index.html/info/b88edfefbb>
Of course, it was done for a good reason (cooperation
between Windows and Cygwin using the same sqlite
database), but now that that's fixed in Cygwin I think
that SQLite should be more UNIX-like on Cygwin.

Could you try that fix, please?

Regards,
Jan Nijtmans
Warren Young
2013-07-24 19:46:33 UTC
Permalink
Post by Jan Nijtmans
<http://www.fossil-scm.org/index.html/info/b88edfefbb>
That patch is fine on Cygwin as long as you build Fossil with the
external SQLite, rather than the bundled SQLite.

The bundled SQLite is almost certainly broken with that patch because it
probably isn't using the Cygwin 1.7.20+ F_LCK_MANDATORY feature, so it
won't cooperate properly with any program based on native Windows SQLite.

(And if stock Fossil does turn on mandatory locking on Cygwin via the
new Cygwin feature, they also need to add a way to turn it *off*, as
Cygwin SQLite does with CYGWIN_SQLITE_LOCKING=posix, else the program
won't be able to cooperate with other programs based on Cygwin SQLite.)
Post by Jan Nijtmans
Could you try that fix, please?
What exactly are you asking me to do here?

I've already indicated that I'm going to wait until 3.7.18 comes out to
see if they've fixed the backslash problem, and if they haven't, hack
around it in Cygwin SQLite. That still requires you to build Fossil
with the system SQLite to get my fixes.

I don't have checkin rights on either Fossil or SQLite core, so if you
want the stock distributed version changed in some way, you're better
off asking one of the core developers.
Jan Nijtmans
2013-07-25 09:50:07 UTC
Permalink
Post by Warren Young
The bundled SQLite is almost certainly broken with that patch because it
probably isn't using the Cygwin 1.7.20+ F_LCK_MANDATORY feature, so it won't
cooperate properly with any program based on native Windows SQLite.
Agreed. It was just an experiment. Closed now.
Post by Warren Young
I've already indicated that I'm going to wait until 3.7.18 comes out to see
if they've fixed the backslash problem, and if they haven't, hack around it
in Cygwin SQLite. That still requires you to build Fossil with the system
SQLite to get my fixes.
Next SQLite will be 3.8.0, which is in beta now, is already included in
Fossil. This means that the backslash problem is not fixed yet.

However, I think I found the cause of this problem! In os_win.c, the
function used to determine the temporary directory is
GetTempPath(). This function is documented to return:
1) The path specified by the TMP environment variable.
2) The path specified by the TEMP environment variable.
3) The path specified by the USERPROFILE environment variable.
4) The Windows directory.
We see that it apparently returns the Windows directory, it cannot
fiund any of the environment variables!
SQLITE_CANTOPEN: os_win.c:34887: (5)
winOpen(C:\Windows\etilqs_S8FDopj6QWC9mu4) - Access is denied.
$ ls -la /cygdrive/c
total 7265374
......
drwxrwx---+ 1 TrustedInstaller TrustedInstaller 0 May 23
09:10 Windows
......
We don't have sufficient rights in the C:\Windows directory! Cygwin clears
the Windows environment space, therefore none of the environment
variables can be found.

Proposed solution here (should move to sqlite if accepted):
<http://www.fossil-scm.org/index.html/info/fec3c7fe3a>

Regards,
Jan Nijtmans
Richard Hipp
2013-07-25 10:29:42 UTC
Permalink
Native, pure-blooded windows binaries run just fine on cygwin, right? So
why are we complicating the code with exceptions, special cases, and hacks
for cygwin?
--
D. Richard Hipp
***@sqlite.org
Jan Nijtmans
2013-07-25 10:43:53 UTC
Permalink
Post by Richard Hipp
Native, pure-blooded windows binaries run just fine on cygwin, right? So
why are we complicating the code with exceptions, special cases, and hacks
for cygwin?
There are three things that a windows fossil binary can never do
in the Cygwin environment:
1) handle Cygwin (Unix) links and mount points
2) setting the Windows file-system case-sensitive (use both "Makefile"
and "makefile")
3) use a Cygwin program as commit/stash editor

For me personally those 3 things are not important, but apparently
(see earlier messages in this thread) for other people it is. Unfortunately!

I'm trying to find out what the minimum patch is to get the Cygwin build
of fossil (both 32-bit and 64-bit) working again, so fossil can be
built out-of-the box on Cygwin again. Of course, any feedback
is welcome.

Regards,
Jan Nijtmans
Martin Gagnon
2013-07-25 11:44:16 UTC
Permalink
Post by Jan Nijtmans
Post by Richard Hipp
Native, pure-blooded windows binaries run just fine on cygwin, right? So
why are we complicating the code with exceptions, special cases, and hacks
for cygwin?
There are three things that a windows fossil binary can never do
1) handle Cygwin (Unix) links and mount points
2) setting the Windows file-system case-sensitive (use both "Makefile"
and "makefile")
3) use a Cygwin program as commit/stash editor
For me personally those 3 things are not important, but apparently
(see earlier messages in this thread) for other people it is.
Unfortunately!
Post by Jan Nijtmans
I'm trying to find out what the minimum patch is to get the Cygwin build
of fossil (both 32-bit and 64-bit) working again, so fossil can be
built out-of-the box on Cygwin again. Of course, any feedback
is welcome.
In Theory, fossil should build and work on fossil like on any other unix
like Operating system (like linux/*bsd etc..) That's what cygwin is for.

I have the feeling that in some part of the code, cygwin is treated as
windows and in other place it is treated as unix-like (posix). I guess
this is the problem.

I believe that when building for cygwin, it should never goes on any
#ifdef that are special for windows. So if cygwin really work as
expected, fossil/sqlite code should not need much exceptions in order to
work in Cygwin.

Per example, on native windows we cannot just do "./configure && make",
we need a special manually maintain Makefile. But on cygwin, it *should*
work.

May be I'm a bit naive ?
--
Martin G.
Lluís Batlle i Rossell
2013-07-25 11:53:07 UTC
Permalink
Post by Richard Hipp
Post by Jan Nijtmans
Post by Richard Hipp
Native, pure-blooded windows binaries run just fine on cygwin,
right? So
Post by Jan Nijtmans
Post by Richard Hipp
why are we complicating the code with exceptions, special cases,
and hacks
Post by Jan Nijtmans
Post by Richard Hipp
for cygwin?
There are three things that a windows fossil binary can never do
1) handle Cygwin (Unix) links and mount points
2) setting the Windows file-system case-sensitive (use both "Makefile"
and "makefile")
3) use a Cygwin program as commit/stash editor
For me personally those 3 things are not important, but apparently
(see earlier messages in this thread) for other people it is.
Unfortunately!
Post by Jan Nijtmans
I'm trying to find out what the minimum patch is to get the Cygwin build
of fossil (both 32-bit and 64-bit) working again, so fossil can be
built out-of-the box on Cygwin again. Of course, any feedback
is welcome.
In Theory, fossil should build and work on fossil like on any other unix
like Operating system (like linux/*bsd etc..) That's what cygwin is for.
I have the feeling that in some part of the code, cygwin is treated as
windows and in other place it is treated as unix-like (posix). I guess
this is the problem.
I believe that when building for cygwin, it should never goes on any
#ifdef that are special for windows. So if cygwin really work as
expected, fossil/sqlite code should not need much exceptions in order to
work in Cygwin.
Per example, on native windows we cannot just do "./configure && make",
we need a special manually maintain Makefile. But on cygwin, it *should*
work.
This is also my understanding.

And as for "why we need fossil-for-cygwin" from Richard... I think it is almost
the same reason of why we need cygwin. :)

I don't use windows because I like it's UI. When I use windows, I want it the
closest to unix. This means terminals, fds, unix sockets, fork, process groups,
etc. Cygwin is great at building all that over the weird windows OS.

Using a non-cygwin program in cygwin quite breaks the magic (terminal fds not
passing on fork, different filesystem views, paths, permissions, etc.).

Regards,
Lluís.
Richard Hipp
2013-07-25 11:59:38 UTC
Permalink
Post by Martin Gagnon
In Theory, fossil should build and work on fossil like on any other unix
like Operating system (like linux/*bsd etc..) That's what cygwin is for.
"grep" tells me that there are 33 instances of the __CYGWIN__ macro in
Fossil, in 8 different files. So if you use "sed" to change them all to
__CYGWIN_OFF_ (or something else harmless) and then do "./configure; make",
does it work? (I don't have cygwin installed so this is not something I
can easily test.)

If it does work, then I move for the immediate banishment of all __CYGWIN__
macros.

If it doesn't work, is that a bug in cygwin?
--
D. Richard Hipp
***@sqlite.org
Lluís Batlle i Rossell
2013-07-25 12:05:00 UTC
Permalink
Post by Richard Hipp
Post by Martin Gagnon
In Theory, fossil should build and work on fossil like on any other unix
like Operating system (like linux/*bsd etc..) That's what cygwin is for.
"grep" tells me that there are 33 instances of the __CYGWIN__ macro in
Fossil, in 8 different files. So if you use "sed" to change them all to
__CYGWIN_OFF_ (or something else harmless) and then do "./configure; make",
does it work? (I don't have cygwin installed so this is not something I
can easily test.)
If it does work, then I move for the immediate banishment of all __CYGWIN__
macros.
fossil clone in cygwin64 now works perfectly:
$ sed s/__CYGWIN__/__CYGWIN__OFF__/ -i src/*
$ ./configure
$ make -j4

Thank you! :)
Richard Hipp
2013-07-25 12:10:36 UTC
Permalink
Post by Martin Gagnon
Post by Richard Hipp
Post by Martin Gagnon
In Theory, fossil should build and work on fossil like on any other
unix
Post by Richard Hipp
Post by Martin Gagnon
like Operating system (like linux/*bsd etc..) That's what cygwin is
for.
Post by Richard Hipp
"grep" tells me that there are 33 instances of the __CYGWIN__ macro in
Fossil, in 8 different files. So if you use "sed" to change them all to
__CYGWIN_OFF_ (or something else harmless) and then do "./configure;
make",
Post by Richard Hipp
does it work? (I don't have cygwin installed so this is not something I
can easily test.)
If it does work, then I move for the immediate banishment of all
__CYGWIN__
Post by Richard Hipp
macros.
$ sed s/__CYGWIN__/__CYGWIN__OFF__/ -i src/*
$ ./configure
$ make -j4
Move to banish __CYGWIN__ from both Fossil and SQLite sources. Do I have a
second?
--
D. Richard Hipp
***@sqlite.org
Jan Danielsson
2013-07-25 12:26:30 UTC
Permalink
Post by Richard Hipp
Move to banish __CYGWIN__ from both Fossil and SQLite sources. Do I have a
second?
Seconded.

/Jan
Jan Danielsson
2013-07-25 12:24:39 UTC
Permalink
On 7/25/13 2:05 PM, Lluís Batlle i Rossell wrote:
[---]
Post by Richard Hipp
"grep" tells me that there are 33 instances of the __CYGWIN__ macro in
Fossil, in 8 different files. So if you use "sed" to change them all to
__CYGWIN_OFF_ (or something else harmless) and then do "./configure; make",
does it work? (I don't have cygwin installed so this is not something I
can easily test.)
If it does work, then I move for the immediate banishment of all __CYGWIN__
macros.
So .. we used the __CYGWIN__ macro to explicitly break fossil on
cygwin? That seems unnecessarily creative to me.

Out of curiosity (there must be one or two here who put them there):
What are/were their intended purposes [in the fossil code]?

/Jan
Warren Young
2013-07-25 20:42:23 UTC
Permalink
Post by Jan Danielsson
So .. we used the __CYGWIN__ macro to explicitly break fossil on
cygwin? That seems unnecessarily creative to me.
It is well known that the creators of Cygwin do this sort of thing
because They're Just Mean. Maybe Fossil's creators are the same way?
Post by Jan Danielsson
What are/were their intended purposes [in the fossil code]?
I'm up for some spelunking. Let's go:

compat/zlib/gzguts.h: Forces HAVE_VSNPRINTF to true. This sounds like a
configure script bug to me. If it's fixed now, the ifdef should go. If
vsnprintf() is still not being detected on Cygwin, that problem should
be corrected at its source, not hacked around like this.

compat/zlib/contrib/inflate86/inffast.S: ELF vs. COFF executable format
inference, to provide the leading underscore MS linkers want. Since the
resulting binary works, one of the following must be true:

a) Fossil is using the plain C version, not the optimized assembly; or
b) the Cygwin linker now copes with this difference; or
c) WIN32 is somehow getting defined by this point, so the __CYGWIN__
test is redundant.

compat/zlib/contrib/puff/pufftest.c, compat/zlib/{examples,test}/*:
Example code that came with zlib, so it doesn't matter if it doesn't
build and work correctly.

compat/zlib/zutil.h: This seems bogus. The way it's coded implies that
"WIN32" is defined when __CYGWIN__ is, which isn't currently true,
unless you pass -mwin32 to the compiler, which you shouldn't in this
case. However, if I'm wrong, and WIN32 *is* somehow being defined here,
the test is appropriate and shouldn't be removed: zlib blobs created
under Cygwin *should* be marked as having been created "on Unix" rather
than "on Win32".

src/sqlite3.c: __CYGWIN__ stuff to be removed through other channels. :)

src/th_tcl.c: Not sure if this one can go or not. It might depend on
whether your Fossil is linking to a Cygwin Tcl or the native Windows
Tcl. Lluís, if you're reading this, did you test TH1 under Cygwin?
Which Tcl is your fossil.exe linked to? (Ask ldd if you're not sure.)
Martin Gagnon
2013-07-25 20:57:55 UTC
Permalink
Post by Warren Young
Post by Jan Danielsson
So .. we used the __CYGWIN__ macro to explicitly break fossil on
cygwin? That seems unnecessarily creative to me.
It is well known that the creators of Cygwin do this sort of thing
because They're Just Mean. Maybe Fossil's creators are the same way?
Post by Jan Danielsson
What are/were their intended purposes [in the fossil code]?
compat/zlib/gzguts.h: Forces HAVE_VSNPRINTF to true. This sounds
like a configure script bug to me. If it's fixed now, the ifdef
should go. If vsnprintf() is still not being detected on Cygwin,
that problem should be corrected at its source, not hacked around
like this.
compat/zlib/contrib/inflate86/inffast.S: ELF vs. COFF executable
format inference, to provide the leading underscore MS linkers want.
a) Fossil is using the plain C version, not the optimized assembly; or
b) the Cygwin linker now copes with this difference; or
c) WIN32 is somehow getting defined by this point, so the
__CYGWIN__ test is redundant.
Example code that came with zlib, so it doesn't matter if it doesn't
build and work correctly.
compat/zlib/zutil.h: This seems bogus. The way it's coded implies
that "WIN32" is defined when __CYGWIN__ is, which isn't currently
true, unless you pass -mwin32 to the compiler, which you shouldn't
in this case. However, if I'm wrong, and WIN32 *is* somehow being
defined here, the test is appropriate and shouldn't be removed: zlib
blobs created under Cygwin *should* be marked as having been created
"on Unix" rather than "on Win32".
src/sqlite3.c: __CYGWIN__ stuff to be removed through other channels. :)
src/th_tcl.c: Not sure if this one can go or not. It might depend
on whether your Fossil is linking to a Cygwin Tcl or the native
Windows Tcl. Lluís, if you're reading this, did you test TH1 under
Cygwin? Which Tcl is your fossil.exe linked to? (Ask ldd if you're
not sure.)
I'm wondering, why someone would want to build cygwin version of fossil
and link with native TCL and vice-versa ? When you build cygwin
version, you can have everything you need in cygwin version under the
cygwin tree. Tyring to mixte native and cygwin stuff here is looking for
trouble.

If you use mingw toolchain inside cygwin to produce native Windows
binary, it's another story, (this is like cross compiling) and in that
case, __CYGWIN__ should not be defined at all and this should behave
exactly like building directly fossil on a mingw/msys setup using the
Makefile.mingw. Exactly like cross-compiling mingw apps under FreeBSD
or Linux.

(I don't know how the ./configure script deal with cross-compiling)
--
Martin G.
Joe Mistachkin
2013-07-25 22:03:54 UTC
Permalink
What about all the __CYGWIN__ blocks in the following files?

1. add.c
2. blob.c
3. checkin.c
4. db.c
5. file.c
6. utf8.c

Frankly, I'm not convinced of how many of these are actually necessary.

--
Joe Mistachkin
Warren Young
2013-07-25 22:49:47 UTC
Permalink
Post by Joe Mistachkin
What about all the __CYGWIN__ blocks in the following files?
I guess they already got taken out of the trunk. I did my spelunking in
a current pull of the tree.

That explains why Richard referred to "8 files" having CYGWIN blocks,
but I found only 7. He must have been referring to src/* only.

I assume that whoever removed these blocks did the same sort of analysis
I did for those outside the src subtree, and determined that they were
in fact pointless.
Joe Mistachkin
2013-07-25 22:52:53 UTC
Permalink
Post by Warren Young
I guess they already got taken out of the trunk. I did my spelunking in
a current pull of the tree.
I'm simply searching trunk for "__CYGWIN__".

--
Joe Mistachkin
Warren Young
2013-07-26 02:34:05 UTC
Permalink
Post by Joe Mistachkin
Post by Warren Young
I guess they already got taken out of the trunk. I did my spelunking in
a current pull of the tree.
I'm simply searching trunk for "__CYGWIN__".
I was stuck on a branch from February. <wince>

Now that I'm actually looking at the trunk, back into the cave we go:


add.c, line 25: You should be #including windows.h here instead of
providing these prototypes. That makes Cygwin's w32api package a build
dependency, but if fossil.exe is going to do registry API calls when
built with Cygwin, that seems a reasonable dependency.

add.c, filenames_are_case_sensitive(): Keep it. The Cygwin registry
setting *is* a better default here than the Unix case default.



blob.c, blob_write_to_file(): Why not let the OS catch this error? I'd
take this test out for native Windows, too.


blob.c, BOM and UTF-16 awareness: I think this code is well-intentioned
for Cygwin, but probably not quite right.

For Cygwin, you should probably assume UTF-8 by default, as for any
modern *ix. Text editors under Cygwin default to UTF-8, not UTF-16 as
do native Windows Unicode text editors. If there is a BOM, you should
probably do the conversion from UTF-16 to UTF-8 on all platforms, not
just Windows and Cygwin.



checkin.c, commit_warning(): Really? Why can't you use iconv(3)?



db.c, db_open(): Wrong. Fix or remove.

First, hard-coding a conversion to "/cygdrive/%c/%s" will break if the
user has changed the cygdrive mount point in /etc/fstab. This is more
common than you may realize. If you mount cygdrive at the root, you can
use shorter quasi-DOS paths like /c/Windows. I use this feature.

Second, if Fossil really must mung Windows drive paths to POSIX paths on
Cygwin instead of passing paths blindly to the OS like any other program
would, it should be done at a higher level than the DB layer.

If you really must mung paths, ask the Cygwin DLL to do it for you, via
cygwin_conv_path(). This fixes the cygdrive mount point problem, too.

Personally, I'd say remove the ifdef entirely.



db.c, db_open_config(), user DB location: This function has its heart
in the right place, but it's awfully brittle.

Consider just the native Windows case. If, the first time you run
Fossil, LOCALAPPDATA isn't defined, it will put the DB in %APPDATA%.
But if LOCALAPPDATA is then later defined, it doesn't continue searching
the fall-back hierarchy to find its earlier DB file. It will create a
new one in %LOCALAPPDATA% and ignore the one it created earlier, losing
any settings you had.

The following algorithm provides proper fall-back behavior for native
Windows and ensures that pure Cygwin users get the ~/.fossil file they
expected:

1. Search the three native Windows environment variable directories for
_fossil, using the first one found.

2. If none are found and this is Cygwin, try ~/.fossil, too.

3. If the DB still isn't found, create a new one in ~/.fossil if this is
Cygwin, or in %FOO%/_fossil on native Windows, where %FOO% is the
highest-precedence environment variable found.

Alternatively, add a feature to Fossil that lets you override the
hard-coded DB file location(s) with a high-precedence environment
variable. (That is, a second EV on Unix, and a fourth on Windows.)
Say, FOSSILDB. Then you can set it to c:\cygwin\home\foo to get
interoperability between native and Cygwin fossil.exe. There might be
Unix users who can find a use for this feature, too.



db.c, db_open_config(), '.' -> '_' smashing: Cargo cult detected.

Windows NT derivatives and programs written with NT-derived OSes in mind
treat files with a leading dot like any other file. The only programs
that don't are those that insist on having a file name extension --
which shouldn't be given .fossil files to begin with -- and programs
with DOS/Win9x heritages.

Cygwin dropped Win9x support in 2009. Microsoft dropped Win9x support
in 2006. DOS and Win16 compatibility got dropped for 64-bit systems in
Vista, in 2007. We no longer need to pussy-foot around leading dots on
Windows in 2013.

The comment claims "some window systems [have] problems" with file names
beginning with a dot. Are any of these systems still supported by
Microsoft? And, "many apps" have problems? Quantify "many". If
nonzero, how many of these require access to Fossil DB files?



db.c, cmd_open(): Same as above.

For that matter, I think Fossil 2.0 should use .fslckout on native
Windows, too, and avoid the need for any special-casing here at all.
(Why 2.0? It's a breaking change, so it shouldn't be done lightly.)



db.c, db_open_config(), file access test: This should be unconditional,
or be removed. Unix gets by without it, so there must be some higher
level code that checks whether the DB was opened for writing.

Surely "chmod 400 ~/.fossil" is a problem on Unix, too?



db.c, db_open_repository(): Another bogus /cygdrive hack. Remove or
replace with a cygwin_conv_path() call.



db.c, line 2115: Keep this. It's correct.



file.c, drive letter awareness: Cygwin will yell at you if you use paths
like "z:/path/to/whatever" but it does cope with them. Until that
changes, Fossil should follow the same rules if it's going to bother
caring about path name details in the first place. The code has to be
there for the _WIN32 case, so it doesn't cost anything.

Anyone thinking Cygwin is wrong to yell at you for using drive letters
should think on why file_without_drive_letter() in file.c doesn't have
an "ifdef __CYGWIN__" in it. Try patching your Fossil to make this code
run under Cygwin and see how far you get with that.

Those who know why this will fail should think on whether Fossil should
be trying to do its own drive letter stuff on Cygwin in the first place.



file.c, backslash hackery: Remove. Cygwin users depending on Fossil to
fix backslashes for them need a good smack with a clue-by-four.



file.c, case smashing: Broken, two ways.

First, Cygwin can be configured to be case-sensitive. It should be
looking at the Cygwin registry setting, as add.c does.

Second, it's ignoring the "case-sensitive" Fossil user setting. Never
mind Cygwin, shouldn't the native Windows build of Fossil obey "=on" if
you set it?



sqlite3.c, th_tcl.c: Already dealt with above.



utf8.c: Use iconv(3) in the Unix cases to address the "TODO"s, then get
rid of the __CYGWIN__ ifdefs so Cygwin uses this new code. Then you can
get rid of the Win32 API prototypes at the top of the file. (No need to
#include windows.h here, as with add.c, since the goal is to remove the
dependence on the Win32 API here entirely.)

There are also some file name conversions in here, which are adequately
addressed by commentary above.
Jan Nijtmans
2013-07-26 07:15:01 UTC
Permalink
Post by Warren Young
db.c, db_open(): Wrong. Fix or remove.
First, hard-coding a conversion to "/cygdrive/%c/%s" will break if the user
has changed the cygdrive mount point in /etc/fstab. This is more common
than you may realize. If you mount cygdrive at the root, you can use
shorter quasi-DOS paths like /c/Windows. I use this feature.
You are absolutely right. Fixed now in:
<http://www.fossil-scm.org/index.html/info/5af289e999>
Forgot that there already was a function doing that, this was an
ill-thought code duplication. Thanks!

For your other remarks, I suggest that you become a fossil
code committer. See:
<http://www.fossil-scm.org/index.html/doc/trunk/www/contribute.wiki>
That makes it easier to discuss any suggestions you have, so they
eventually can find their way into trunk. As you are the Cygwin SQLite
package maintainer that would be a logical step.

Regards,
Jan Nijtmans
Warren Young
2013-07-26 22:11:00 UTC
Permalink
Post by Jan Nijtmans
For your other remarks, I suggest that you become a fossil
code committer.
I've yet to step beyond evaluating Fossil for use here. I like its
features and how it works, but I think I'm going to be forced to use git
for interoperability reasons. (We're currently using svn, and need to
step into the DVCS world soon.)

Even if we did choose Fossil, we'd be using it on Linux, for the vast
majority of checkins. Checkins from the Cygwin side would be under 1%
of all our use.

Bottom line, I'd have little personal stake in these patches.
Half-hearted patches make for bad code.

All I'm doing here in this thread is evaluating whether these existing
code blocks make sense for Cygwin, and suggesting alternatives where
they don't make sense. Someone who actually cares how and whether
Fossil works on Cygwin should be doing the patches.

Jan Nijtmans
2013-07-25 13:46:21 UTC
Permalink
Post by Richard Hipp
If it does work, then I move for the immediate banishment of all __CYGWIN__
macros.
Doing that will break four things:
- Accessing a check-out repository on Cygwin, while the previous check-out
was done in win32.
- Allow usage of win32 paths (possibly coming from WIN32 variables) in fossil.
- Cygwin doesn't allow '\' in paths like UNIX does, or the behavior of
UNC paths.
- Cooperation of Cygwin with windows programs using sqlite (like TortoiseSVN)
Post by Richard Hipp
If it doesn't work, is that a bug in cygwin?
More likely, it's a limitation of Cygwin running on Windows.

Not seconded.

In SQLite various __CYGWIN__ #ifdef's can be removed if another locking
mechanism is put in place. Attached is the current patch the Cygwin
people use for their build of SQLite accomplishing this. But this only
works on Cygwin > 1.7.20, and it doesn't follow the SQLite coding
style, so it will need some more work.

Regards,
Jan Nijtmans
Lluís Batlle i Rossell
2013-07-25 13:59:30 UTC
Permalink
Post by Jan Nijtmans
Post by Richard Hipp
If it does work, then I move for the immediate banishment of all __CYGWIN__
macros.
- Accessing a check-out repository on Cygwin, while the previous check-out
was done in win32.
- Allow usage of win32 paths (possibly coming from WIN32 variables) in fossil.
- Cygwin doesn't allow '\' in paths like UNIX does, or the behavior of
UNC paths.
- Cooperation of Cygwin with windows programs using sqlite (like TortoiseSVN)
As a cygwin user, this breakage is what I expect by any cygwin version of a
program, be it fossil, mercurial, git, vim, ...

I'd never use a cygwin-git checkout with non-cygwin-git, for example. I never
tried and i would not expect it to work.
Post by Jan Nijtmans
Post by Richard Hipp
If it doesn't work, is that a bug in cygwin?
More likely, it's a limitation of Cygwin running on Windows.
Not seconded.
In SQLite various __CYGWIN__ #ifdef's can be removed if another locking
mechanism is put in place. Attached is the current patch the Cygwin
people use for their build of SQLite accomplishing this. But this only
works on Cygwin > 1.7.20, and it doesn't follow the SQLite coding
style, so it will need some more work.
I have no idea how locking has to be handled in cygwin, but if the issue is at
using a sqlite db with both cygwin and non-cygwin programs, this is a tricky
thing.

As for me, I don't expect cygwin programs' data to cope well with windows
programs' data.

Regards,
Lluís.
Warren Young
2013-07-25 21:59:39 UTC
Permalink
Post by Jan Nijtmans
Post by Richard Hipp
If it does work, then I move for the immediate banishment of all __CYGWIN__
macros.
- Accessing a check-out repository on Cygwin, while the previous check-out
was done in win32.
Are you merely predicting this, or are you reporting that you've tried
it and found that it does break when you do this?

I don't recall seeing any __CYGWIN__ protected code blocks on my
spelunking trip -- reported to the list earlier -- that would cause such
an incompatibility.
Post by Jan Nijtmans
- Allow usage of win32 paths (possibly coming from WIN32 variables) in fossil.
What are you talking about? Fossil's __CYGWIN__ protected code blocks
don't affect how the Cygwin DLL interprets file paths.
Post by Jan Nijtmans
- Cygwin doesn't allow '\' in paths like UNIX does, or the behavior of
UNC paths.
Ditto.
Post by Jan Nijtmans
- Cooperation of Cygwin with windows programs using sqlite (like TortoiseSVN)
What problem are you actually envisioning here? There is no
TortoiseFossil to cause problems in this case.

If someone builds a pure Cygwin fossil.exe without the __CYGWIN__ blocks
enabled, what native Windows program runs to break the Cygwin fossil.exe?

If there is such a problem, the fix is simple:

./configure --disable-internal-sqlite

Then it builds against my Cygwin SQLite library, which *does* cope with
the problems you're thinking of.

If you build Fossil with all the __CYGWIN__ blocks disabled and use the
internal SQLite, it should behave as if linking to Cygwin SQLite
3.7.17-3+ with the POSIX advisory locking mode enabled. That's
perfectly fine if you don't need the SQLite DB file to be accessible
from a native Windows SQLite based program at the same time.
Post by Jan Nijtmans
this only works on Cygwin > 1.7.20
That isn't a problem in the Cygwin SQLite package, since you have to go
out of your way to update the SQLite package only, without updating the
Cygwin DLL, too.

I can understand if the SQLite developers want to hold off adopting this
patch into the shipped version of SQLite. They may have Cygwin based
customers who don't keep their Cygwin installations up to date. Plus,
I'd like to see this mandatory locking feature get more than a month of
use before all the __CYGWIN__ blocks get ripped out of upstream SQLite.

Once we're sure the new Cygwin DLL based locking works just as well as
the stuff we've replaced, I'd be happy to see upstream adopt this patch.
Post by Jan Nijtmans
and it doesn't follow the SQLite coding
style, so it will need some more work.
I tried to write the patch to blend into the surrounding code.

I've just noticed the spaces-vs-tabs difference, which I've fixed in my
local copy.

Is there anything more substantial that needs changing?

Perhaps you don't like my enum naming scheme, or the fact that I used an
enum for my constants instead of preprocessor macros.
Warren Young
2013-07-26 03:48:03 UTC
Permalink
Some of what I wrote was based on wrong assumptions due to being stuck
here on a February branch of Fossil's repo. Now that I've looked at the
__CYGWIN__ blocks in an up-to-date Fossil trunk, I understand your post
better, Jan. Updated commentary inline below.
Post by Jan Nijtmans
Post by Jan Nijtmans
Post by Richard Hipp
If it does work, then I move for the immediate banishment of all __CYGWIN__
macros.
- Accessing a check-out repository on Cygwin, while the previous check-out
was done in win32.
I proposed two fixes for this in my second spelunking report. (My
improved algorithm for finding _fossil, and the FOSSILDB feature.)
Post by Jan Nijtmans
Post by Jan Nijtmans
- Allow usage of win32 paths (possibly coming from WIN32 variables) in fossil.
For the most part, Fossil shouldn't be caring as much about path details
as it currently does. Most programs just blindly throw paths at the OS,
and let it deal with it.

To the extent that Fossil *does* have to care, it shouldn't try to fix
backslashes or drive letters for Cygwin users. The Cygwin DLL does this
already, albeit with a warning. Duplicating the functionality in Fossil
is wrongheaded, for the same reason you don't duplicate functionality in
any other program.
Post by Jan Nijtmans
./configure --disable-internal-sqlite
Then it builds against my Cygwin SQLite library, which *does* cope with
the problems you're thinking of.
I see that this doesn't work right now, because I hadn't yet shipped
3.7.17-3 for Cygwin64. I'm working on fixing that now.
Post by Jan Nijtmans
I've just noticed the spaces-vs-tabs difference, which I've fixed in my
local copy.
This will appear in my 64-bit packages.
Warren Young
2013-07-25 22:43:06 UTC
Permalink
Post by Richard Hipp
Native, pure-blooded windows binaries run just fine on cygwin, right?
Mostly, yes.

There are exceptions. The Windows console infrastructure isn't as
general and as easy to hook into a the Unix TTY equivalent, so there are
programs that only work properly in a native Windows console. Programs
in this class typically fail when run in mintty, or under screen, or via
Cygwin ssh.
Post by Richard Hipp
So why are we complicating the code with exceptions, special cases, and
hacks for cygwin?
Ideally, Cygwin should be treated as just another Unix-like OS.

Unfortunately, it runs on top of a Windows kernel, which simply doesn't
do some things in a way that allows it to provide proper POSIX/Linux
semantics. Over time, Cygwin has gotten better at patching over these
differences, but some are intractable enough that they remain. Part of
the problem is a need to continue working with the native Windows side
of things.

The mandatory vs advisory file locking stuff is a good example. I
assume you're aware -- at least peripherally -- of the changes that
happened to Cygwin SQLite recently. It's all an attempt to paper over
yet another of these differences between Windows and POSIX semantics.
Warren Young
2013-07-26 03:08:54 UTC
Permalink
Post by Warren Young
Post by Jan Nijtmans
Just wait on the Cygwin64 people to bring out a new Sqlite package with
the same fixes already done in Cygwin32.
Um, it's the same people. Me. :)
Oh, I see what you mean. I forgot that I didn't release 3.7.17-3 for
64-bit as well. You're the first person to ask, and it was so obliquely
that I didn't realize it at first. :)

I'll be RFU'ing 3.7.17-3 for Cyginw64 soon.
Loading...