Non-US keyboard (PT) bash problem

[Lino Miguel Martins Tinoco]

Hi!

I can't have special characters displayed correctly on bash (ç, Ç and accents). They are displayed as ? when I do a ls but they get displayed correctly if I pipe the results or send them to a file. On command line they are displayed as the respective octal value (ç is displayed as \347). Before I try the set meta-flag solution, bash was interpreting some of these symbols (hitting ç twice gave me an ls).

Also tried codepage:oem, but didn't worked. On a DOS box this characters are displayed ok. They are also displayed ok on any editor (vi, emacs, ...). This happens on console window and xterm or rxvt.


Thanks,

--Tinoco

[Igor Pechtchanski]

Post by Lino Miguel Martins Tinoco
Hi!
I can't have special characters displayed correctly on bash (ç, Ç and
accents). They are displayed as ? when I do a ls but they get displayed
correctly if I pipe the results or send them to a file. On command line
they are displayed as the respective octal value (ç is displayed as
\347). Before I try the set meta-flag solution, bash was interpreting
some of these symbols (hitting ç twice gave me an ls).

For 'ls' you may wish to use the --show-control-chars flag. For bash (or,
rather, readline), you will need all four of

set meta-flag on
set convert-meta off
set input-meta on
set output-meta on

(see <http://www.cygwin.com/faq/faq_3.html#SEC46>), and/or the respective
manpages.

Post by Lino Miguel Martins Tinoco
Also tried codepage:oem, but didn't worked. On a DOS box this characters
are displayed ok. They are also displayed ok on any editor (vi, emacs,
...). This happens on console window and xterm or rxvt.

When you say "DOS box", I presume you mean the actual Windows Command
Prompt window, so looks like your problem is with bash. Since you seem to
have the same behavior in console, xterm, and rxvt, this doesn't look like
a font issue. If the above doesn't help, please post the exact steps to
reproduce the problem.
Igor

[Lino Miguel Martins Tinoco]

The --show-control-chars flag worked fine and I've created a new alias for ls. The set solution doesn´t work, I've already tried it before.

Also tried to export this variables (have them on my ~/.profile):

LANG=***@euro
LC_ALL=***@euro
LANGUAGE=***@euro
LC_TYPE=***@euro
LC_CTYPE=***@euro
OUTPUT_CHARSET=***@euro
export LANG LC_ALL LANGUAGE LC_TYPE LC_CTYPE OUTPUT_CHARSET


***@euro is defined in /usr/share/locale/locale.alias as:

portuguese pt_PT.ISO-8859-1
pt_PT pt_PT.ISO-8859-1
***@euro pt_PT.ISO-8859-15
***@EURO pt_PT.ISO-8859-15


And in /usr/X11R6/lib/X11/locale/locale.alias I have:

pt pt_PT.ISO8859-1
pt_PT pt_PT.ISO8859-1
pt_PT.88591 pt_PT.ISO8859-1
pt_PT.88591.en pt_PT.ISO8859-1
pt_PT.iso88591 pt_PT.ISO8859-1
pt_PT.ISO-8859-1 pt_PT.ISO8859-1
pt_PT.ISO_8859-1 pt_PT.ISO8859-1
pt_PT.iso885915 pt_PT.ISO8859-15
pt_PT.iso-885915 pt_PT.ISO8859-15
pt_PT.ISO-8859-15 pt_PT.ISO8859-15
pt_PT.ISO8859-15 pt_PT.ISO8859-15
***@euro pt_PT.ISO8859-15
***@EURO pt_PT.ISO8859-15
pt_PT.utfO pt_PT.UTF-8
pt_PT.UTF-***@euro pt_PT.UTF-8


On /usr/X11R6/lib/X11/locale/locale.dir I also have these 2 lines:

iso8859-1/XLC_LOCALE: pt_PT.ISO8859-1
iso8859-15/XLC_LOCALE: pt_PT.ISO8859-15


Still those special characters get displayed as an octal value. This is na exaple of what I'm saying:

***@pc701523 ~/My Documents# cd Programa\347\343o/
***@pc701523 ~/My Documents/Programação#


Thanks,

--Tinoco

[Igor Pechtchanski]

Post by Lino Miguel Martins Tinoco
The --show-control-chars flag worked fine and I've created a new alias
for ls. The set solution doesnŽt work, I've already tried it before.

Exactly what have you tried? Please post the output of 'bind -V' in your
bash shell. FWIW, I could reproduce your behavior when "output-meta" is
set to "off".

Post by Lino Miguel Martins Tinoco
[a whole bunch of locale stuff snipped]

This doesn't affect readline (which is what's giving you the problem).

Post by Lino Miguel Martins Tinoco
Still those special characters get displayed as an octal value. This is
~/My Documents/Programação#

As I said before, you should have *all four* of

set meta-flag on
set convert-meta off
set input-meta on
set output-meta on

in your ~/.inputrc. Note: these *aren't* bash commands, these are
readline options; if you want to have the same effect from the bash
command prompt, you have to use "bind 'set output-meta on'" instead.
HTH,
Igor

[Lino Miguel Martins Tinoco]

My ~/.inputrc has all the set you mentioned:

***@pc701523 ~# cat .inputrc
set meta-flag on
set convert-meta off # to show it as character, not the octal representation
set input-meta on
set output-meta on # to show 8-bit characters

# Mappings para teclado PT
#"\M-g": "ç"
#"\M-G": "Ç"
***@pc701523 ~#


And bind -V gives:

blink-matching-paren is set to `on'
byte-oriented is set to `on'
completion-ignore-case is set to `off'
convert-meta is set to `off'
disable-completion is set to `off'
enable-keypad is set to `off'
expand-tilde is set to `off'
history-preserve-point is set to `off'
horizontal-scroll-mode is set to `off'
input-meta is set to `on'
mark-directories is set to `on'
mark-modified-lines is set to `off'
mark-symlinked-directories is set to `off'
match-hidden-files is set to `on'
meta-flag is set to `on'
output-meta is set to `off'
page-completions is set to `on'
prefer-visible-bell is set to `on'
print-completions-horizontally is set to `off'
show-all-if-ambiguous is set to `off'
visible-stats is set to `off'
bell-style is set to `audible'
comment-begin is set to `#'
completion-query-items is set to `100'
editing-mode is set to `emacs'
keymap is set to `emacs'


I tried with set meta-flag on and off, the results are the same.

Thanks,

--Tinoco

[Igor Pechtchanski]

Post by Igor Pechtchanski
set meta-flag on
set convert-meta off # to show it as character, not the octal representation

Nope, convert-meta is used to strip the 8th bit from each 8-bit character
and convert it to the meta key (ESC) followed by the lower 7 bits.

Post by Igor Pechtchanski
set input-meta on
set output-meta on # to show 8-bit characters

^^^^^^^^^^^^^^^^^^
The octal representation vs. the actual character is controlled by
output-meta. See "man bash" for more details.

Post by Igor Pechtchanski
# Mappings para teclado PT
#"\M-g": "ç"
#"\M-G": "Ç"
[snipped]
convert-meta is set to `off'
input-meta is set to `on'
meta-flag is set to `on'
output-meta is set to `off'

^^^^^^^^^^^^^^^^^^^^^^^^^^^
Ah. There's your problem. For some reason, the "output-meta" setting in
your .inputrc didn't get propagated to your shell.

First, try typing "bind 'set output-meta on'" at the shell prompt, and
observe that it fixes your problem. Then, try to find out why the setting
didn't get propagated -- did you restart the shell after editing your
~/.inputrc? If you did, one of your startup files may be resetting this
-- try running a "bash --login -i -v" from a Windows command prompt, and
search the output for "bind 'set output-meta off'".

Post by Igor Pechtchanski
I tried with set meta-flag on and off, the results are the same.

Incidentally, as I just realized while reading the above, meta-flag is a
synonym for input-meta, so using both is redundant.
Igor

[Lino Miguel Martins Tinoco]

Looking closely to the output of bind -V I found that output-meta is
off. I tried a few things and the problem was the comment after the 'set
output-meta on'. After clearing the comment, it went as expected.

Thanks for the tips and help, and sorry for the trouble (it was all my
fault).

--Tinoco