20687: 4.2.2 news

20688: Unicode/UTF-8 FAQ notes

ChangeLog

@@ -1,3 +1,10 @@
+2005-01-11  Peter Stephenson  <pws@csr.com>
+
+	* 20688: Etc/FAQ.yo: add notes on Unicode/UTF-8 adapted from
+	Clint.
+
+	* 20687: NEWS: news for 4.2.2.
+
 2005-01-11  Bart Schaefer  <schaefer@brasslantern.com>
 
 	* 20666 (tweaked): Src/exec.c: In closemn(), discard the multio

Etc/FAQ.yo

@@ -43,14 +43,14 @@ whenlatex(report(ARG1)(ARG2)(ARG3))\
 whenman(report(ARG1)(ARG2)(ARG3))\
 whenms(report(ARG1)(ARG2)(ARG3))\
 whensgml(report(ARG1)(ARG2)(ARG3)))
-myreport(Z-Shell Frequently-Asked Questions)(Peter Stephenson)(2004/08/13)
+myreport(Z-Shell Frequently-Asked Questions)(Peter Stephenson)(2005/01/11)
 COMMENT(-- the following are for Usenet and must appear first)\
 description(\
 mydit(Archive-Name:) unix-faq/shell/zsh
-mydit(Last-Modified:) 2001/08/13
+mydit(Last-Modified:) 2005/01/11
 mydit(Submitted-By:) email(pws@pwstephenson.fsnet.co.uk (Peter Stephenson))
 mydit(Posting-Frequency:) Monthly
-mydit(Copyright:) (C) P.W. Stephenson, 1995--2004 (see end of document)
+mydit(Copyright:) (C) P.W. Stephenson, 1995--2005 (see end of document)
 )
 
 This document contains a list of frequently-asked (or otherwise
@@ -88,6 +88,7 @@ Chapter 2:  How does zsh differ from...?
 2.4. tcsh?
 2.5. bash?
 2.6. Shouldn't zsh be more/less like ksh/(t)csh?
+2.7. What is zsh's support Unicode/UTF-8?
 
 Chapter 3:  How to get various things to work
 3.1. Why does `$var' where `var="foo bar"' not do what I expect?
@@ -935,6 +936,46 @@ sect(Shouldn't zsh be more/less like ksh/(t)csh?)
   help.
 
 
+sect(What is zsh's support for Unicode/UTF-8?)
+
+  `Unicode', or UCS for Universal Character Set, is the modern way
+  of specifying character sets.  It replaces a large number of ad hoc
+  ways of supporting character sets beyond ASCII.  `UTF-8' is an
+  encoding of Unicode that is particularly natural on Unix-like systems.
+
+  Q: Does zsh support UTF-8?
+
+  A: zsh's built-in printf command supports "\u" and "\U" escapes
+  to output arbitrary Unicode characters.  ZLE (the Zsh Line Editor) has
+  no concept of character encodings, and is confused by multi-octet
+  encodings.
+
+  Q: Why doesn't zsh have proper UTF-8 support?
+
+  A: The code has not been written yet.
+
+  Q: What makes UTF-8 support difficult to implement?
+
+  A: In order to handle arbitrary encodings the correct way, significant
+  and intrusive changes must be made to the shell.
+
+  Q: Why can't zsh just use readline?
+
+  A: ZLE is not encapsulated from the rest of the shell.  Isolating it
+  such that it could be replaced by readline would be a significant
+  effort.  Furthermore, using readline would effect a significant loss of
+  features.
+
+  Q: What changes are planned?
+
+  A: Introduction of Unicode support will be gradual, so if you are
+  interested in being involved you should join the zsh-workers mailing
+  list.  As a first step ZLE will be rewritten to use wide characters
+  internally.  Character based widgets can then operate on a single wide
+  character instead of a single byte, and the proper display width can be
+  calculated with wcswidth().
+
+
 chapter(How to get various things to work)
 
 sect(Why does mytt($var) where mytt(var="foo bar") not do what I expect?)

NEWS

@@ -64,6 +64,11 @@ Changes since zsh version 4.2.0
   compatibility mode the option is turned off by default and the option
   letter -T turns it on, for compatibility with FreeBSD sh.
 
+- New WIDGETSTYLE, WIDGETFUNC parameters in ZLE.
+
+- Glob qualifiers (+func) is shorthand for (e:func:) (calls command,
+  typically shell function, with no argument).
+
 New features between zsh versions 4.0 and 4.2
 ---------------------------------------------