Telnet / SSH connection timeout on LAN

[Spinfusion]

My goal:
To set up rsync over SSH, to sync my org-mode GTD files, activated every
time my laptop or pc goes idle, as described by Gina Trapani.

Network hardware architecture:
Windows 7 laptop and desktop, connected directly by cat 5e ethernet cable.
Also both connected via wifi to the same router.

Network functionality:
Windows firewalls currently disabled
Windows shared folders works for directory browsing and editing
Synctoy worked to sync drives, although it spewed a lot of errors and took a
long time and many retries to sync ~250 gb. Did it over the ethernet cable
because wifi was too slow.

Cygwin functionality:
ping works.
telnet error: connection timed out
ssh error: connection timed out
ssh connection to localhost - works

Background and guesses:

I've read and tried many fixes with no luck, to the best of my limited
understanding.

I doubt network names are configured correctly; I'm out of my depth there.
Pinging works with fixed IP addresses assigned for the ethernet connection.

I don't know what I'm doing and would appreciate some guidance. Cheers.



--
View this message in context: http://cygwin.1069669.n5.nabble.com/Telnet-SSH-connection-timeout-on-LAN-tp119480.html
Sent from the Cygwin list mailing list archive at Nabble.com.

--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Marco Atzeri]

Post by Spinfusion
To set up rsync over SSH, to sync my org-mode GTD files, activated every
time my laptop or pc goes idle, as described by Gina Trapani.
Windows 7 laptop and desktop, connected directly by cat 5e ethernet cable.
Also both connected via wifi to the same router.
Windows firewalls currently disabled
Windows shared folders works for directory browsing and editing
Synctoy worked to sync drives, although it spewed a lot of errors and took a
long time and many retries to sync ~250 gb. Did it over the ethernet cable
because wifi was too slow.

Are you sure it was over the cable ?
A lot of errors means a broken cable or a weak wifi.

Post by Spinfusion
ping works.
telnet error: connection timed out
ssh error: connection timed out
ssh connection to localhost - works

It seems a routing table problem, not a cygwin issue.

see windows command
route /help

Post by Spinfusion
I've read and tried many fixes with no luck, to the best of my limited
understanding.
I doubt network names are configured correctly; I'm out of my depth there.
Pinging works with fixed IP addresses assigned for the ethernet connection.
I don't know what I'm doing and would appreciate some guidance. Cheers.

This can give you some understanding
http://www.sevenforums.com/tutorials/320978-static-routes-create-remove.html

Regards
Marco


--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Andrey Repin]

Greetings, Spinfusion!

Post by Spinfusion
To set up rsync over SSH, to sync my org-mode GTD files, activated every
time my laptop or pc goes idle, as described by Gina Trapani.
Windows 7 laptop and desktop, connected directly by cat 5e ethernet cable.
Also both connected via wifi to the same router.

What are assigned IP addresses on both links?
Your further hints indicate that you have name resolution conflict due to
overlapping network address space.
Disconnect from WiFi on both hosts and try only cable.

Post by Spinfusion
Windows firewalls currently disabled
Windows shared folders works for directory browsing and editing
Synctoy worked to sync drives, although it spewed a lot of errors and took a
long time and many retries to sync ~250 gb. Did it over the ethernet cable
because wifi was too slow.
ping works.
telnet error: connection timed out
ssh error: connection timed out
ssh connection to localhost - works
I've read and tried many fixes with no luck, to the best of my limited
understanding.
I doubt network names are configured correctly; I'm out of my depth there.
Pinging works with fixed IP addresses assigned for the ethernet connection.
I don't know what I'm doing and would appreciate some guidance. Cheers.

--
With best regards,
Andrey Repin
Monday, July 6, 2015 18:24:38

Sorry for my terrible english...


--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Joseph B]

Hi Andrew,

Desktop

LAN Adapter
IP: 192.168.0.2 (static)
Subnet Mask: 255.255.255.0
no DNS server

Wifi Adapter
192.168.254.18 (dynamic)
255.255.255.0
gateway: 192.168.254.254

Laptop

LAN
192.168.0.1 (static)
255.255.255.0
no DNS

Wifi
192.168.254.19 (dynamic)
255.255.255.0
192.168.254.254

I don't think these namespaces overlap?

Note that my ethernet cable is standard 5e, not crossover. I am
directly wiring the computers without a switch or hub. This is not
supposed to work, and indeed it doesn't work very well. I am content
at this point to use wifi's slower transfer speed, since the big 250
GB transfer is done. So I would rather stop using the ethernet cable,
as it is a likely failure point.

Testing with ethernet cable only, wifi disabled on both:

I re-attached the ethernet cable. After manually changing the network
type from "public" to "private (work)" on both computers, I was able
to access Windows shared folders, as before. Network is still labeled
"unidentified" on both machines, as before.

Synctoy runs about the same. It finishes the 250 GB sync with about
1000 errors. Not sure whether it's faster; maybe.

Cygwin ping works from laptop to desktop. It sends 4 packets, time 0 ms.
However, from desktop to laptop, it sends 13 successful pings
averaging .5 ms and then freezes. It never presents the statistical
summary. Ctrl-c does not break the loop. Had to restart the Cygwin
window.
I did not observe this behavior before. Ping from desktop windows CLI
succeeds normally.

Cygwin telnet fails to cross the cable both ways, with error "unable
to connect to remote host: Connection timed out". Telnetting localhost
gives "connection refused". I don't know how to set up a telnet
server.

Cygwin ssh to localhost / ip address succeeds. E.g. "ssh localhost"
Connection to other machine fails with error "connect to host
192.168.0.x port 22: Connection timed out". Command was e.g. "ssh
192.168.0.1"

Thanks for the advice. Please let me know what to do next. Adieu.

Post by Andrey Repin
Greetings, Spinfusion!

Post by Spinfusion
To set up rsync over SSH, to sync my org-mode GTD files, activated every
time my laptop or pc goes idle, as described by Gina Trapani.
Windows 7 laptop and desktop, connected directly by cat 5e ethernet cable.
Also both connected via wifi to the same router.

What are assigned IP addresses on both links?
Your further hints indicate that you have name resolution conflict due to
overlapping network address space.
Disconnect from WiFi on both hosts and try only cable.

Post by Spinfusion
Windows firewalls currently disabled
Windows shared folders works for directory browsing and editing
Synctoy worked to sync drives, although it spewed a lot of errors and took a
long time and many retries to sync ~250 gb. Did it over the ethernet cable
because wifi was too slow.
ping works.
telnet error: connection timed out
ssh error: connection timed out
ssh connection to localhost - works
I've read and tried many fixes with no luck, to the best of my limited
understanding.
I doubt network names are configured correctly; I'm out of my depth there.
Pinging works with fixed IP addresses assigned for the ethernet connection.
I don't know what I'm doing and would appreciate some guidance. Cheers.

--
With best regards,
Andrey Repin
Monday, July 6, 2015 18:24:38
Sorry for my terrible english...

--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Warren Young]

Post by Joseph B
LAN Adapter
IP: 192.168.0.2 (static)
Subnet Mask: 255.255.255.0
no DNS server
Wifi Adapter
192.168.254.18 (dynamic)
255.255.255.0
gateway: 192.168.254.254

Mask 255.255.255.0 means the first three octets are the network number, and the last is the station number. Thus, your two machines are on *completely different networks*, at a logical level. Your OS is sending the packets from one to your home router, and your router doubtless has no idea that it is supposed to be routing between two different 192.168 networks.

Solution: Put the static machine on 192.168.254.x.

Look at your DHCP server’s configuration to find a safe value for x.

And since you said you have no idea what you’re doing, I will tell you that your LAN’s DHCP server is almost certainly a feature on your home router. DHCP configuration is different for every router.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Warren Young]

Post by Spinfusion
To set up rsync over SSH, to sync my org-mode GTD files, activated every
time my laptop or pc goes idle, as described by Gina Trapani.

If you’re referring to the following article, it was written about a month before Dropbox went live.

http://lifehacker.com/196122/

Since then, Google, Apple, Microsoft and others have jumped onto the free cloud storage bandwagon. All of those services address every one of the advantages Gina gives for her solution, except open source, and for that, you’ve got the likes of tarsnap:

http://www.tarsnap.com/

Why are you reinventing these perfectly good wheels, poorly?

Yes, poorly. rsync requires a pretty heavy network transaction to figure out if files have changed. These other services monitor local files and initiate the delta transfer only when they see a local change. That cost is best paid as rarely as possible; “on every idle” is too often.

Anyway, how is futzing around with DIY backup schemes Getting Things Done? :)

Post by Spinfusion
ssh error: connection timed out

The remote machine either:

a) isn’t running ssh; verify with “netstat -na | grep 22.*LISTEN"

b) is running it, but it’s firewalled; just because you think the firewall is down doesn’t mean it is; there could be two or more firewalls, for example

c) it’s on another port; check the /etc/sshd_config file, unless you know for a fact that you never touched it

Failing all that, what does ssh -vvv say?

Post by Spinfusion
I don't know what I'm doing and would appreciate some guidance.

Did you set Cygwin sshd up using ssh-host-config? That should be all the guidance you require. Maybe run it again and pay more attention to the things it tells you this time? :)

--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Andrey Repin]

Greetings, Warren Young!

Post by Warren Young
Why are you reinventing these perfectly good wheels, poorly?
Yes, poorly. rsync requires a pretty heavy network transaction to figure
out if files have changed.

O.o
I'm rsync'ing about 15 gigabytes of my home directory with just a few megs of
network exchange.
What I'm doing wrong?

Either way, the issue has been resolved and was a simple user error.
--
With best regards,
Andrey Repin
Thursday, July 9, 2015 09:02:57

Sorry for my terrible english...


--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Warren Young]

Post by Andrey Repin

Post by Warren Young
rsync requires a pretty heavy network transaction to figure
out if files have changed.

I'm rsync'ing about 15 gigabytes of my home directory with just a few megs of
network exchange.

“Just?”

That was my definition of “heavy”.

Consider all the disk I/O required. In its default mode, rsync must do a full directory tree scan on the directory to be transferred, on *both* ends. For each file with a different mtime or size, it must then recompute all the hashes in that file, again on both sides.

Can you really handwave away megs of network I/O and potentially gigs of disk I/O? Do you never use locate(1) instead of find(1)? Same issue.

On top of that, the OP wants to do this every time the machine becomes idle. Even if it was idle a few seconds ago, did some work, and is idle again, the OP wants all this work to be done all over again.

Horribly wasteful.

I believe Dropbox and its major competitors avoid the need for this tree scan by hooking into the OS’s filesystem change notification API, so that they don’t do any network or disk I/O until one of the files it is responsible for changes.

That’s the right way.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Stephen John Smoogen]

Post by Warren Young

Post by Andrey Repin

Post by Warren Young
rsync requires a pretty heavy network transaction to figure
out if files have changed.

I'm rsync'ing about 15 gigabytes of my home directory with just a few megs of
network exchange.

“Just?”
That was my definition of “heavy”.
Consider all the disk I/O required. In its default mode, rsync must do a full directory tree scan on the directory to be transferred, on *both* ends. For each file with a different mtime or size, it must then recompute all the hashes in that file, again on both sides.
Can you really handwave away megs of network I/O and potentially gigs of disk I/O? Do you never use locate(1) instead of find(1)? Same issue.
On top of that, the OP wants to do this every time the machine becomes idle. Even if it was idle a few seconds ago, did some work, and is idle again, the OP wants all this work to be done all over again.
Horribly wasteful.
I believe Dropbox and its major competitors avoid the need for this tree scan by hooking into the OS’s filesystem change notification API, so that they don’t do any network or disk I/O until one of the files it is responsible for changes.
That’s the right way.

Dropbox etc work well if you have a fast and low latency upstream
channel. Trying to backup even small changes over a 1.5 MB DSL (still
the most common in the United States) can seriously affect any and all
network activity. Heck even with a 20 MB uplink speed you are looking
at serious delays to do backups of just general changes to a system.
In situations where bandwidth is limited then the local rsync is the
way to go. [If you can tie in a dump like behaviour via the OS
filesystem change api then even better but local services would need
to be what is looked at for a majority of places (at least in the
US).]

Post by Warren Young
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Andrey Repin]

Greetings, Warren Young!

Post by Warren Young

Post by Andrey Repin

Post by Warren Young
rsync requires a pretty heavy network transaction to figure
out if files have changed.

I'm rsync'ing about 15 gigabytes of my home directory with just a few megs of
network exchange.

“Just?”
That was my definition of “heavy”.

Sorry, but you can't sync data without sending data.
And these few megs is the actual data I generate daily.

Post by Warren Young
Consider all the disk I/O required. In its default mode, rsync must do a
full directory tree scan on the directory to be transferred, on *both* ends.
For each file with a different mtime or size, it must then recompute all the
hashes in that file, again on both sides.

Wrong. In default mode, rsync only care about timestamp and size.
It will not go on hashing crusade unless explicitly told to.

Post by Warren Young
Can you really handwave away megs of network I/O and potentially gigs of
disk I/O? Do you never use locate(1) instead of find(1)? Same issue.

I normally know where my stuff is, so I don't need to `find` or waste space on
`locate` hash tables.

Post by Warren Young
On top of that, the OP wants to do this every time the machine becomes
idle. Even if it was idle a few seconds ago, did some work, and is idle
again, the OP wants all this work to be done all over again.
Horribly wasteful.
I believe Dropbox and its major competitors avoid the need for this tree
scan by hooking into the OS’s filesystem change notification API, so that
they don’t do any network or disk I/O until one of the files it is responsible for changes.
That’s the right way.

VSS would be the right way, but I still did not find time to research its
extents.


--
With best regards,
Andrey Repin
Saturday, July 11, 2015 05:35:21

[Warren Young]

Post by Andrey Repin

Post by Warren Young
Consider all the disk I/O required. In its default mode, rsync must do a
full directory tree scan on the directory to be transferred, on *both* ends.
For each file with a different mtime or size, it must then recompute all the
hashes in that file, again on both sides.

Wrong. In default mode, rsync only care about timestamp and size.
It will not go on hashing crusade unless explicitly told to.

You misunderstood what I wrote. Given a changed mtime or file size, rsync then must re-read the file on both sides in order to generate a series of checksums on chunks of the file in order to determine which parts of the file have changed, so that it transfers only the changes. See:

https://en.wikipedia.org/wiki/Rsync#Determining_which_parts_of_a_file_have_changed

If you skip this step, you must transfer the entire file contents to the other side, since a difference only in the mtime and/or file size doesn’t tell you which parts of the file have changed.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple

[Andrey Repin]

Greetings, Warren Young!

Post by Warren Young

Post by Andrey Repin

Post by Warren Young
Consider all the disk I/O required. In its default mode, rsync must do a
full directory tree scan on the directory to be transferred, on *both* ends.
For each file with a different mtime or size, it must then recompute all the
hashes in that file, again on both sides.

Wrong. In default mode, rsync only care about timestamp and size.
It will not go on hashing crusade unless explicitly told to.

You misunderstood what I wrote. Given a changed mtime or file size, rsync
then must re-read the file on both sides in order to generate a series of
checksums on chunks of the file in order to determine which parts of the
https://en.wikipedia.org/wiki/Rsync#Determining_which_parts_of_a_file_have_changed
If you skip this step, you must transfer the entire file contents to the
other side, since a difference only in the mtime and/or file size doesn’t
tell you which parts of the file have changed.

In my environment, a small touch to the original file cause changes throughout
the entirety of its stored image. ('cause storage format is actually an
archive, and a small change here and there in the source file cause massive
shifts in the resulting image.)
You see, it is a matter of application and environment.


--
With best regards,
Andrey Repin
Monday, July 13, 2015 19:31:58

Sorry for my terrible englis

[Warren Young]

Post by Andrey Repin
In my environment, a small touch to the original file cause changes throughout
the entirety of its stored image. ('cause storage format is actually an
archive, and a small change here and there in the source file cause massive
shifts in the resulting image.)

Unless those files are written using either whole-archive compression or whole-archive encryption, rsync should still be able to find substantial savings in the transfer with its rolling checksums. rsync won’t be confused by simple changes like a new byte added to the middle of a file, shifting all subsequent bytes down by one.

Some “archive” formats do use compression, but in a piecewise fashion, so that changing one byte of one piece of the archive may cause that entire chunk to change, but it might not affect any of the others. An example of this is the Fossil database format.

You can figure out if your archive files work this way by adding -v to your rsync command. It reports a ratio of the on-disk data size to the transfer size as “speedup is N”, where N > 1.0 means it is not re-sending the entire file. The output of --stats gives similar info, more verbosely.

The point I made in the original post, however, is that all this work to save network bandwidth comes at a disk I/O and CPU cost in the case of rsync, because it doesn’t have a daemon that can sit around watching for filesystem change events. The larger the files are with respect to the change sizes, the greater the waste.

Always-running software like Dropbox avoids much of this cost because it can watch for those events, and thus only do work when the OS tells it that a particular file has changed.

I have also left out another disadvantage of rsync: it’s basically a one-way operation. If you ever need two-way (or N-way) syncing, you’re better off moving to one of the many alternatives that know how to do this correctly. Multilateral syncing is surprisingly hard to get right.

I don’t mean to advertise for Dropbox, just to give it as an example that everyone can relate to.

An alternative that’s open source, more secure, and definitely does pay attention to the OS’s filesystem event API is SpiderOak. You can see from their Github contents that they’ve got OS-specific file change notifiers:

https://github.com/SpiderOak

Now contrast Syncthing, which has many of the same virtues, but currently doesn’t have file change notification built in, causing some third party to write a helper for Syncthing to fill the gap:

https://syncthing.net/
https://github.com/syncthing/syncthing-inotify/

These tables may be helpful:

https://en.wikipedia.org/wiki/Comparison_of_file_synchronization_software
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple