cygwin-setup & rebaseall

[Satish Balay]

I don't see any FAQ regarding rebaseall - hence this post.

It appears that rebaseall is required ocassionally to get cygwin [esp
python] working. [having been bitten by this a few times,] Each time I
do any cygwin updates - I make sure I run 'rebaseall' before doing
anything else.

Q: Does cygwin setup run rebaseall either during initial install or
during subsequent updates? [guessing no]

Q: Is it safe to say: 'if cygwin is installed from scratch - rebaseall
is not required'

[for our tools] cygwin/python is a pre-requisite - and our users
appear to stumble across this issue regularly. And asking users to
always run rebaseall is an extra hurdle]

Thanks,
Satish

[Brian Dessent]

Post by Satish Balay
Q: Does cygwin setup run rebaseall either during initial install or
during subsequent updates? [guessing no]

No, setup has no clue about rebaseall.

Post by Satish Balay
Q: Is it safe to say: 'if cygwin is installed from scratch - rebaseall
is not required'

That's not quite correct. Whether a rebase is required depends on
several factors - which packages you've selected, and whether those
packages have been compiled with --enable-auto-image-base. As you have
noted, the rebase is only required for certain packages, particularly
those that use dynamically loaded modules such as python or perl. Many
people use Cygwin all the time and never run into the problem because
most programs don't suffer from this.

Additionally, maintainers of these packages have (I believe) recently
started compiling them with --enable-auto-image-base which should
eliminate (or reduce) the need to rebase.

Post by Satish Balay
[for our tools] cygwin/python is a pre-requisite - and our users
appear to stumble across this issue regularly. And asking users to
always run rebaseall is an extra hurdle]

Always running rebase after any setup operation seems like overkill. It
should only be necessary if you upgrade python or a dynamic library that
python might load at runtime, and then only if python wasn't compiled
with the above flag -- I don't know the current status of which
maintainers are doing that.

So, it seems to me like it would be a lot easier just to say "Run rebase
if you get an "Unable to remap" error" instead of "Run rebase
religiously after any setup operation." Or, if you know that python
triggers this problem then perhaps it should be, "Always run rebaseall
after a clean install, and then don't worry about it again unless you
get the "unable to remap" error."

Brian

[Satish Balay]

Post by Brian Dessent
Additionally, maintainers of these packages have (I believe) recently
started compiling them with --enable-auto-image-base which should
eliminate (or reduce) the need to rebase.

Ok so the problem is likely to go away for python users.

Post by Brian Dessent

Post by Satish Balay
[for our tools] cygwin/python is a pre-requisite - and our users
appear to stumble across this issue regularly. And asking users to
always run rebaseall is an extra hurdle]

Always running rebase after any setup operation seems like overkill. It
should only be necessary if you upgrade python or a dynamic library that
python might load at runtime, and then only if python wasn't compiled
with the above flag -- I don't know the current status of which
maintainers are doing that.
So, it seems to me like it would be a lot easier just to say "Run rebase
if you get an "Unable to remap" error" instead of "Run rebase
religiously after any setup operation." Or, if you know that python
triggers this problem then perhaps it should be, "Always run rebaseall
after a clean install, and then don't worry about it again unless you
get the "unable to remap" error."

Currently the burden is on [cygwin] users to know/keep track of this
info. And not all instructions/documetation get read by users. So each
time there is such a breakage, cygwin get a bad repo [why doesn't it
just work?]

So I hope cygwin can improve on this. Users seeing "Unable to remap"
errors [however infrequently] should be avoided - esp if it can be
automated.

[maybe the fix is: for any package that is likely to break - add a
flag - which triggers setup to run rebaseall - after
install/upgrade. So no overhead for pacakges that don't break - but
always overhead for packages that break.]

Thanks,
Satish

[Brian Dessent]

Post by Satish Balay
Ok so the problem is likely to go away for python users.

I don't know if Jason has started compiling Python with
--enable-auto-image-base or not. It will also depend on the maintainers
of any libraries that are dynamically loaded by a python module using
the flag as well.

Post by Satish Balay
Currently the burden is on [cygwin] users to know/keep track of this
info. And not all instructions/documetation get read by users. So each
time there is such a breakage, cygwin get a bad repo [why doesn't it
just work?]
So I hope cygwin can improve on this. Users seeing "Unable to remap"
errors [however infrequently] should be avoided - esp if it can be
automated.

This is true. It is a rather bad situation. It is mitigated by the
fact that it only affects certain packages used in certain
circumstances. Cygwin is also somewhat aimed at users that are familiar
with reading documentation, and this problem is documented in
/usr/share/doc/cygwin/python-*.README.

Post by Satish Balay
[maybe the fix is: for any package that is likely to break - add a
flag - which triggers setup to run rebaseall - after
install/upgrade. So no overhead for pacakges that don't break - but
always overhead for packages that break.]

It's not that simple. The --enable-auto-image-base feature is, in
theory, the correct way to fix this problem. This option has been
traditionally been disabled by default, for reasons I'm not familiar
with. At some time in the past I think it had some problems. And I
seem to recall that it can still cause conflicts in certain
circumstances because Cygwin uses a small amount of memory past the end
of the DLL for record-keeping purposes, which the rebaseall script
accomodates by adding padding.

Automatically running rebaseall from setup.exe has issues too. For one
thing, it would run into problems if the user had programs or services
running. The rebaseall script bails if it cannot write to a DLL, so
unless the user was very careful to close all running programs, it would
fail in almost all cases. Rebaseall would have to be modified to rebase
in-use DLLs, but this would require the user to reboot. And, somehow
this would have to be communicated to setup.exe so it could notify the
user. Plus, consider if setup.exe installed a DLL that was in use
(writing it to name.dll.new, and scheduling that for replacement at next
boot) and then rebaseall was run. The rebaseall script would have to
know to rebase name.dll.new and not name.dll. It just gets more and
more complicated. The only workable solution would be to incorporate
the entire rebaseall functionality internally into setup.exe, which is
not an insignificant undertaking, and one which no one is eager to
undertake.

On top of that, there is a problem with running out of address space. I
don't think this happens (yet!) with the official packages but someone
has reported that it can happen if you suppliment an installation with
some of the Cygwin ports like KDE. So this solution of rebasing every
single DLL is clearly not scaleable into the future, unless the Cygwin
DLL's memory layout is rearranged, or unless DLLs are rebased into
another memory space.

Brian

[Jason Tishler]

Post by Brian Dessent

Post by Satish Balay
Ok so the problem is likely to go away for python users.

I don't know if Jason has started compiling Python with
--enable-auto-image-base or not.

I have been using --enable-auto-image-base since the very first Cygwin
Python release -- about 5 years.

Post by Brian Dessent
[snip]
The only workable solution would be to incorporate the entire
rebaseall functionality internally into setup.exe, which is not an
insignificant undertaking, and one which no one is eager to undertake.

FWIW, I submitted patches to Cygwin setup.exe to add rebase
functionality:

http://www.cygwin.com/ml/cygwin-apps/2002-02/msg00010.html

Unfortunately, they were rejected by a previous setup.exe maintainer.

Jason

[Igor Pechtchanski]

Post by Brian Dessent

Post by Satish Balay
[maybe the fix is: for any package that is likely to break - add a
flag - which triggers setup to run rebaseall - after
install/upgrade. So no overhead for pacakges that don't break - but
always overhead for packages that break.]

Automatically running rebaseall from setup.exe has issues too. For one
thing, it would run into problems if the user had programs or services
running. The rebaseall script bails if it cannot write to a DLL, so
unless the user was very careful to close all running programs, it would
fail in almost all cases. Rebaseall would have to be modified to rebase
in-use DLLs, but this would require the user to reboot. And, somehow
this would have to be communicated to setup.exe so it could notify the
user. Plus, consider if setup.exe installed a DLL that was in use
(writing it to name.dll.new, and scheduling that for replacement at next
boot) and then rebaseall was run. The rebaseall script would have to
know to rebase name.dll.new and not name.dll. It just gets more and
more complicated. The only workable solution would be to incorporate
the entire rebaseall functionality internally into setup.exe, which is
not an insignificant undertaking, and one which no one is eager to
undertake.

Plus, as Jason mentions, rebasing corrupts some DLLs. So, until a robust
solution for that is found, automatically rebasing in setup.exe isn't such
a hot idea.
Igor

[Satish Balay]

Post by Igor Pechtchanski
Plus, as Jason mentions, rebasing corrupts some DLLs. So, until a robust
solution for that is found, automatically rebasing in setup.exe isn't such
a hot idea.

So the curent assesment is :

- python - and similar tools can break - is so use rebase to fix it.
- rebase is broken - is lets not use it all the time.

[so it appears cygwin can always be in broken state - this is not good]

And Jason's post was in Feb-2002 - and things have been in this bad
state since a long time.

Mabe those broken packages now work now? - so this objection can be
removed?

Thanks,
Satish

[Christopher Faylor]

Post by Satish Balay

Post by Igor Pechtchanski
Plus, as Jason mentions, rebasing corrupts some DLLs. So, until a robust
solution for that is found, automatically rebasing in setup.exe isn't such
a hot idea.

- python - and similar tools can break - is so use rebase to fix it.
- rebase is broken - is lets not use it all the time.
[so it appears cygwin can always be in broken state - this is not good]
And Jason's post was in Feb-2002 - and things have been in this bad
state since a long time.

And, yet, somehow, we survive.

cgf

[Satish Balay]

Post by Christopher Faylor

Post by Satish Balay
And Jason's post was in Feb-2002 - and things have been in this bad
state since a long time.

And, yet, somehow, we survive.

But at a cost of regular user frustation - which I was hoing could be
avoided.

Post by Christopher Faylor
From the replies so far - it appears things are unlikely to

change.. Oh well..

Satish

[Brian Dessent]

Post by Satish Balay
- python - and similar tools can break - is so use rebase to fix it.
- rebase is broken - is lets not use it all the time.

As far as I know almost all of the issues of corrupted DLLs have been
fixed, but there still remains at least one known problem: emacs will
hang and refuse to start after running rebaseall, unless you revert
cygncurses-7.dll to its non-rebased version. However other ncurses
programs don't seem to have an issue with the rebased DLL.

And as I said, the problem of in-use DLLs is a serious hinderance to
automating the rebaseall process. Even if the ncurses/emacs mystery was
solved, this would still make integrating rebaseall into setup somewhat
challenging. I did not have a chance to review Jason's setup patches
from 2002 but from my quick look it seems that it did not address this
issue.

Nobody is intentionally trying to make Cygwin hard to use, it's just
that fixing it so that no user intervention is ever required is a
problem that requires more effort than anyone has ever desired to
spend. I don't know why Jason's efforts in 2002 to rebase from within
setup stalled, as I was not following the development of setup at the
time. I can only assume there were too many objections and he did not
have time or desire to address them.

Brian

[Satish Balay]

Post by Brian Dessent
As far as I know almost all of the issues of corrupted DLLs have been
fixed, but there still remains at least one known problem: emacs will
hang and refuse to start after running rebaseall, unless you revert
cygncurses-7.dll to its non-rebased version. However other ncurses
programs don't seem to have an issue with the rebased DLL.

I've had issues with emacs when using it over ssh. Maybe this is the
reason. I should try reverting this dll and try again.

Satish

[Satish Balay]

Post by Satish Balay

Post by Brian Dessent
As far as I know almost all of the issues of corrupted DLLs have been
fixed, but there still remains at least one known problem: emacs will
hang and refuse to start after running rebaseall, unless you revert
cygncurses-7.dll to its non-rebased version. However other ncurses
programs don't seem to have an issue with the rebased DLL.

I've had issues with emacs when using it over ssh. Maybe this is the
reason. I should try reverting this dll and try again.

Ok - I tried a complete 'reinstall' [from setup menu] - and now emacs
works fine [both locally - and via ssh]

Satish

[Satish Balay]

Post by Brian Dessent
Automatically running rebaseall from setup.exe has issues too. For one
thing, it would run into problems if the user had programs or services
running. The rebaseall script bails if it cannot write to a DLL, so
unless the user was very careful to close all running programs, it would
fail in almost all cases. Rebaseall would have to be modified to rebase
in-use DLLs, but this would require the user to reboot. And, somehow
this would have to be communicated to setup.exe so it could notify the
user. Plus, consider if setup.exe installed a DLL that was in use
(writing it to name.dll.new, and scheduling that for replacement at next
boot) and then rebaseall was run. The rebaseall script would have to
know to rebase name.dll.new and not name.dll. It just gets more and
more complicated. The only workable solution would be to incorporate
the entire rebaseall functionality internally into setup.exe, which is
not an insignificant undertaking, and one which no one is eager to
undertake.

Doesn't setup have similar problem with updating 'packages' for
running apps? I think it prompts for a reboot to complete. So I'm
guessing the same approach can be used when it needs a rebase.

Post by Brian Dessent
On top of that, there is a problem with running out of address space. I
don't think this happens (yet!) with the official packages but someone
has reported that it can happen if you suppliment an installation with
some of the Cygwin ports like KDE. So this solution of rebasing every
single DLL is clearly not scaleable into the future, unless the Cygwin
DLL's memory layout is rearranged, or unless DLLs are rebased into
another memory space.

Ok - here you are claiming there are fundamental problems with cygwin
which rebase might not be able to fix.

Satish

[Dave Korn]

Post by Satish Balay
Doesn't setup have similar problem with updating 'packages' for
running apps? I think it prompts for a reboot to complete. So I'm
guessing the same approach can be used when it needs a rebase.

It's not as straightforward as all that. The facility to update a file on
reboot is built into the windows OS, setup.exe just makes use of it. There
is no facility to rebase a file on reboot built into windows!


cheers,
DaveK

[Satish Balay]

Post by Dave Korn

Post by Satish Balay
Doesn't setup have similar problem with updating 'packages' for
running apps? I think it prompts for a reboot to complete. So I'm
guessing the same approach can be used when it needs a rebase.

It's not as straightforward as all that. The facility to update a file on
reboot is built into the windows OS, setup.exe just makes use of it. There
is no facility to rebase a file on reboot built into windows!

Not ideal - but it could do either fo the following: [and both of
these I think are better options then current mode - which silently
leaves packages in potentially broken state]

- open a dialog box with a message:
"some packages that require 'rebaseall' are updated. This operation
cannot be done at this time due to active cygwin processes. Please
reboot, shutdown cygwin services, and run 'rebaseall' from 'ash'."

- open a dialog box with a message: "some packages that require
'rebaseall' are updated - hence setup is incomplete. Please reboot,
shutdown cygwin services and rerun setup - so that it can continue
and fix this."

Satish

[Brian Dessent]

Post by Satish Balay
"some packages that require 'rebaseall' are updated. This operation
cannot be done at this time due to active cygwin processes. Please
reboot, shutdown cygwin services, and run 'rebaseall' from 'ash'."

The thing is, you can't just do a "partial" rebaseall. If you update
just one DLL, you can't just pick a new base address for that one DLL,
you have to start over and reassign all new base addresses to every
DLL. This is because the current algorithm for choosing the base
addresses just packs them together back to back, so adding/modifying one
could potentially cause all those below to have to be moved. You can
get around this by setting up some kind of registry of assigned base
addresses, or treating the problem like a heap, but that get
extraordinarily more complex because you have to maintain state.

This is why the whole rebasing method is not scaleable and why we should
really be aiming to throw away rebase and use --enable-auto-image-base
for every package.

So, the reason I mention all that is that it means that the above dialog
box would be displayed *any time* that setup was run with *any cygwin
program or service active*, no matter how small or irrelevent. We would
get flooded with complaints about this extra annoying dialog box, and
"why can't setup replace in use files like it used to", and "I don't
care about rebasing, all my apps work fine, how can I disable this?"

Post by Satish Balay
- open a dialog box with a message: "some packages that require
'rebaseall' are updated - hence setup is incomplete. Please reboot,
shutdown cygwin services and rerun setup - so that it can continue
and fix this."

You wouldn't want to bail partway through a rebase and leave things in a
partial state. What setup would have to do is perform the full rebase
of all DLLs, scheduling any that were in use for replacement with their
rebased copy on the next reboot, as it already does for upgrading DLLs.
But as I've been trying to say this requires a great deal of coding in
setup because you have to internalize all the rebasing logic, it can't
just punt to the rebaseall script.

Brian

[Satish Balay]

Post by Brian Dessent

Post by Satish Balay
"some packages that require 'rebaseall' are updated. This operation
cannot be done at this time due to active cygwin processes. Please
reboot, shutdown cygwin services, and run 'rebaseall' from 'ash'."

The thing is, you can't just do a "partial" rebaseall. If you update
just one DLL, you can't just pick a new base address for that one DLL,
you have to start over and reassign all new base addresses to every
DLL. This is because the current algorithm for choosing the base
addresses just packs them together back to back, so adding/modifying one
could potentially cause all those below to have to be moved. You can
get around this by setting up some kind of registry of assigned base
addresses, or treating the problem like a heap, but that get
extraordinarily more complex because you have to maintain state.

Here I was sugesting 'rebaseall' which I gues is not a "partial" rebase

Post by Brian Dessent
This is why the whole rebasing method is not scaleable and why we should
really be aiming to throw away rebase and use --enable-auto-image-base
for every package.

According to Jason's post, --enable-auto-image-base is been in use for
a while. [Since python doesn't always work without rebaseall] - is
there some other issue with this approach? [mabye some dependent
package should be built with --enable-auto-image-base - but it isn't?]

Post by Brian Dessent
So, the reason I mention all that is that it means that the above dialog
box would be displayed *any time* that setup was run with *any cygwin
program or service active*, no matter how small or irrelevent. We would
get flooded with complaints about this extra annoying dialog box, and
"why can't setup replace in use files like it used to", and "I don't
care about rebasing, all my apps work fine, how can I disable this?"

So if rebase is done - any package with a dll would have to trigger a
rebase? I agree - if every update would trigger this - its not good.

Post by Brian Dessent

Post by Satish Balay
- open a dialog box with a message: "some packages that require
'rebaseall' are updated - hence setup is incomplete. Please reboot,
shutdown cygwin services and rerun setup - so that it can continue
and fix this."

You wouldn't want to bail partway through a rebase and leave things in a
partial state. What setup would have to do is perform the full rebase
of all DLLs, scheduling any that were in use for replacement with their
rebased copy on the next reboot, as it already does for upgrading DLLs.
But as I've been trying to say this requires a great deal of coding in
setup because you have to internalize all the rebasing logic, it can't
just punt to the rebaseall script.

Here I was not implying partial rebase. Currently rebaseall doesn't
start doing anything if it sees any running cygwin process... [so the
same thing could be done]

Satish

[Brian Dessent]

Post by Satish Balay
According to Jason's post, --enable-auto-image-base is been in use for
a while. [Since python doesn't always work without rebaseall] - is
there some other issue with this approach? [mabye some dependent
package should be built with --enable-auto-image-base - but it isn't?]

It is used for python itself, but it's not used for DLLs in other
packages that python modules might load. That is why you still have to
rebaseall for python to work. In order to completely retire rebaseall,
*every* package that contains a DLL must be compiled with this option.

Of course in reality, not every DLL necessarily *has* to be rebased,
only those that might be dynamically loaded at runtime, and even then
it's not always necessary. But it's hard to determine which particular
cases those might be, and so in the general case you have to assign a
unique base address to every DLL.

Post by Satish Balay
So if rebase is done - any package with a dll would have to trigger a
rebase? I agree - if every update would trigger this - its not good.

Given the current method that rebaseall uses, yes, in theory it is
necessary to rerun rebaseall any time any DLL is modified. But in
practice of course this is not true - as evidenced by the fact that many
(most?) cygwin users never even have to deal with rebasing. This is why
it's so much easier to say "run rebaseall if/when you get this error"
than to definitively and automatically solve the problem so that it has
no possibility of ever occuring.

Brian

[Satish Balay]

Post by Brian Dessent

Post by Satish Balay
According to Jason's post, --enable-auto-image-base is been in use for
a while. [Since python doesn't always work without rebaseall] - is
there some other issue with this approach? [mabye some dependent
package should be built with --enable-auto-image-base - but it isn't?]

It is used for python itself, but it's not used for DLLs in other
packages that python modules might load. That is why you still have to
rebaseall for python to work. In order to completely retire rebaseall,
*every* package that contains a DLL must be compiled with this option.
Of course in reality, not every DLL necessarily *has* to be rebased,
only those that might be dynamically loaded at runtime, and even then
it's not always necessary. But it's hard to determine which particular
cases those might be, and so in the general case you have to assign a
unique base address to every DLL.

I've tried using python after doing a complete 'reinstall' and so far
It works fine. I'll keep track of any future breakages.

It will be great if rebase is not required at all...

Satish

[Jason Tishler]

Satish,

Post by Satish Balay
I've tried using python after doing a complete 'reinstall' and so far
It works fine. I'll keep track of any future breakages.
It will be great if rebase is not required at all...

The following is a good test:

$ python /usr/lib/python2.4/test/regrtest.py

Do you get any "unable to remap" errors?

Jason

[Satish Balay]

Post by Jason Tishler
Satish,

Post by Satish Balay
I've tried using python after doing a complete 'reinstall' and so far
It works fine. I'll keep track of any future breakages.
It will be great if rebase is not required at all...

$ python /usr/lib/python2.4/test/regrtest.py
Do you get any "unable to remap" errors?

I don't see any remap errors, but the first time I tried - it gave
thread errros. The second time - it completed fine.

BTW: we disable threads on cygwin+python-2.4. Is this problem likely
to get fixed? [I'm guessing its upstream issue - where I see sem_xx
errors on AIX as well]

thanks,
Satish

-------------

test_slice
test_socket
sem_init: Resource temporarily unavailable
Unhandled exception in thread started by <bound method SmallBufferedFileObjectCl
assTestCase.clientRun of <test.test_socket.SmallBufferedFileObjectClassTestCase
testMethod=testFullRead>>
Traceback (most recent call last):
File "/tmp/python.572/usr/lib/python2.4/test/test_socket.py", line 121, in cli
entRun
File "/usr/lib/python2.4/threading.py", line 348, in wait
self.__cond.wait(timeout)
File "/usr/lib/python2.4/threading.py", line 197, in wait

[Jason Tishler]

Satish,

Post by Satish Balay

Post by Jason Tishler

Post by Satish Balay
I've tried using python after doing a complete 'reinstall' and so
far It works fine. I'll keep track of any future breakages.
It will be great if rebase is not required at all...

$ python /usr/lib/python2.4/test/regrtest.py
Do you get any "unable to remap" errors?

I don't see any remap errors,

Good.

Post by Satish Balay
but the first time I tried - it gave thread errros. The second time -
it completed fine.
sem_init: Resource temporarily unavailable
Unhandled exception in thread started by <bound method SmallBufferedFileObjectCl
assTestCase.clientRun of <test.test_socket.SmallBufferedFileObjectClassTestCase
testMethod=testFullRead>>
File "/tmp/python.572/usr/lib/python2.4/test/test_socket.py", line 121, in cli
entRun
File "/usr/lib/python2.4/threading.py", line 348, in wait
self.__cond.wait(timeout)
File "/usr/lib/python2.4/threading.py", line 197, in wait

The above is a known problem:

http://www.cygwin.com/ml/cygwin/2005-07/msg01378.html

Post by Satish Balay
BTW: we disable threads on cygwin+python-2.4. Is this problem likely
to get fixed?

I can only say that it is on my list. If this is important to you, then
why not help debug the problem?

Post by Satish Balay
[I'm guessing its upstream issue - where I see sem_xx errors on AIX as
well]

Actually, Python configure.in has the following:

# Bug 662787: Using semaphores causes unexplicable hangs on Solaris 8.
case $ac_sys_system/$ac_sys_release in
SunOS/5.6) AC_DEFINE(HAVE_PTHREAD_DESTRUCTOR, 1,
Defined for Solaris 2.6 bug in pthread header.)
;;
SunOS/5.8) AC_DEFINE(HAVE_BROKEN_POSIX_SEMAPHORES, 1,
Define if the Posix semaphores do not work on your system) ;;
esac

So, you can workaround this problem, by the following:

$ configure
$ sed 's=/\* #undef HAVE_BROKEN_POSIX_SEMAPHORES \*/=#define HAVE_BROKEN_POSIX_SEMAPHORES 1=' pyconfig.h >foo
$ mv foo pyconfig.h
$ make

FWIW, I have tried this and it solves the problem reported in the post
above.

Jason

[Brian Dessent]

Post by Satish Balay
Doesn't setup have similar problem with updating 'packages' for
running apps? I think it prompts for a reboot to complete. So I'm
guessing the same approach can be used when it needs a rebase.

Yes, of course setup already has that logic. That's what I was
referring to above about name.dll.new. My point was that you can't just
have setup call the rebaseall script as a final step because the "in use
handling" of the two has to be in sync. It has to be tightly
integrated.

Brian