[Clfs-commits] [CLFS Trac] #182: UTF-8

[CLFS Trac]

#182: UTF-8
--------------------+-------------------------------------------------------
  Reporter:  ken    |       Owner:  clfs-commits at lists.cross-lfs.org.
      Type:  task   |      Status:  new                              
  Priority:  minor  |   Milestone:  CLFS Standard 1.2.0              
 Component:  BOOK   |     Version:  CLFS Standard SVN                
Resolution:         |    Keywords:                                   
--------------------+-------------------------------------------------------
Comment (by ken):

 I had 14 failures in glibc/localedata tests.  Possibly related to the sed
 for UTF-8, don't know.  Haven't really looked at the other testsuites yet,
 beyond a quick glance at gcc (failures in mudflap, i think - similar to
 4.3.2) and binutils (more failures than before, probably because this is
 x86_64-64).  Maybe, if I can find the enthusiasm to rebuild the final
 system without that sed, I might do so to see what changes in the tests.

 The variables in the i18n script were mostly new to me - I've never
 encountered people adding extra partial maps, I thought normal maps
 already included things like the euro - but not a problem. I do have a
 problem with defaulting to the 'windowkeys', but that isn't major and
 easily altered by the user. I'll also note that my Compose key in my local
 keymap doesn't seem to be working on this build - the dead keys for
 latin1-compatible accents are still working, forgot to test the things
 I've added to AltGr.

 On to the main test of man pages:
 Perhaps I'm missing something, e.g. maybe groff now needs some sort of
 override or post-install configuration to use the new functionality ? For
 the moment, my test results in everything except en_GB.UTF-8 are labelled
 as "unusable".

 I wanted to test the following areas:

 possible unrendered text in the error message for 'man foo' because
 man-1.6f provides localized messages in legacy encodings. I have some
 (long) iconv invocations that I normally use to fix this.

 possible unrendered text for 'man apropos' and the other pages supplied
 with man-1.6f in legacy encodings. Again, I have some iconv invocations to
 fix this.

 man-pages for vim might need to be moved to be found (fr,it,pl,ru).

 man-pages from shadow (typically, tested with passwd (5)) are supplied as
 UTF-8. The scripts (ja, ko, zh_*) can't be rendered in the console, and in
 the event I didn't think it was worthwhile to build xorg because of the
 errors I found.

 Results (testing with LC_ALL=whatever man something)
 'gibberish' means the text rendered, typically as a mixture of accented
 vowels and occasional
 icelandic letters, but was clearly non-sensical for the language.
 'invalid UTF-8' means one or more inverse '?' symbols, caused by invalid
 UTF-8 characters arriving at the pager.
 '<xx>' means the pager displayed these hex values in inverse video

  These were all tested using my (UTF-8) sigma-general 'font', which can
 handle the main latin and cyrillic languages as well as monotonic greek.

 bg_BG.UTF-8 man foo invalid UTF-8, man 1 apropos gibberish

 cs_CZ.UTF-8 man foo invalid UTF-8, man 1 apropos invalid UTF-8, man 5
 passwd mix of invalid UTF-8 and gibberish

 da_DK.UTF-8 man foo ok (all ascii), man 1 apropos ok

 de_DE.UTF-8 man foo ok, man 1 apropos ok, man 5 passwd gibberish and
 invalid UTF-8

 el_GR.UTF-8 man foo invalid UTF-8, man 1 apropos gibberish

 es_ES.UTF-8 man foo invalid UTF-8, man 1 apropos and man 5 passwd <C3><93>
 etc

 fi_FI.UTF-8 similar to es_ES.UTF-8

 fr_FR.UTF-8 man foo ok (ascii), man 1 apropos, man 1 vim, man 5 passwd
 <C3><A9> etc
 interestingly, if I point man to fr_FR.ISO8859-1/man1/vim.1 it renders
 correctly.

 At this point I gave up. I guessI can fix the 'man foo' issue, but the
 rest is so broken I don't know where to begin looking.

-- 
Ticket URL: <http://trac.cross-lfs.org/ticket/182#comment:16>
CLFS Trac <http://trac.cross-lfs.org>
The Cross Linux From Scratch Project.