Detailed Requirements and Choices for Your Bi-Directional Word Processor

There are two extremes for the paradigm for the bidirectional word processor (BDWP):

1. The user has full control over directionality of characters and the BDWP obeys as with Berry et al's bidirectional troff (that was used to typeset the Brief Bi-Directional Text Reading Lesson ).
2. The user lets the BDWP determine the directionality of characters as described by the Unicode Bidirectional Algorithm (UBDA).

You and your customer have to decide what kind of user your BDWP will support. Then you and your customer have to pick one paradigm that is appropriate for this user. Then, the BDWP you specify follows this and only this paradigm.

Here are more details on the two paradigms:

In EITHER case, the characters are in Unicode and the file operated on by the BDWP follows the Unicode standard. That is, if you view the file that your BDWP has produced through any other fully Unicode compliant application, that application's visual order view is the same as your BDWP's visual order view. This assumption means that whatever paradigm you choose, your BDWP makes Unicode compliant files.

1. User in Full Control:
   1. There are ONLY strong characters.

      Each character's direction is determined by the default direction of the alphabet of the keyboard through which the character is entered. That is, if the user is using a Hebrew keyboard, then typing a space results in an RTL space being put into the file. If the user is using an English keyboard, then typing a space results in an LTR space being put into the file. The same holds for all characters, e.g., digits and punctuation, which are classified as weak and neutral in Unicode; they are strong in this paradigm.

   2. There is no implicit embedding, but there is explicit embedding.

      Any sequence of characters, selected with the pointer (mouse), can be made a subdocument with either direction, LTR or RTL. Thus, if the user is making a LTR document that contains a RTL address that contains LTR digits in the middle of which are LTR characters, then the user will have to make the RTL address an RTL subdocument in order to get the LTR letters inside the digits not to be moved to the wrong place (See the bidirectional text reading lesson.).

   3. Any character can be forced to have either direction, even the opposite of its default.

      Any sequence of characters, selected with the pointer, can be given either direction, LTR or RTL. Thus, the user is able to specify a sequence of RTL characters to be LTR or a sequence of LTR characters to be RTL, e,g, in order to write a manual for a BDWP.

2. BDWP chooses character directions as specified by the UBDA:
   1. There are strong, weak, and neutral characters, as specified by the UBDA. That is, no matter which keyboard a user uses to enter a space, the one Unicode space goes into the file. This space is neutral and has no default direction. Its direction is determined by its context according to the UBDA.
   2. There are control characters, i.e., LRM, RLM, LRE, RLE, LRO, RLO, PDF, with which a user can force character directions and embeddings that differ from the default specified by the UBDA.

The tradeoff, FROM THE USER'S VIEWPOINT, are:

1. User in Full Control:
   • Con: The user must be fully aware of character directionality and must convey her intent to the BDWP by careful choice of the keyboard with which she enters a character, including the space or punctuation. The user must be fully aware of subdocuments and their directions and must convey her intent to the BDWP by careful specification of subdocuments.
   • Pro: The methods of conveying intents are simple and are therefore easy to understand and remember.
2. BDWP chooses character directions as specified by the UBDA:
   • Pro: For most of what the ordinary user does, the defaults assumed by the UBDA are correct and the user gets the visual order view that she wants. Control characters have to be inserted into the file for only the occasional non-standard situation in which the defaults assumed by the UBDA are not correct.
   • Con: The UBDA is complicated. Therefore, it is difficult for a user to understand when and why the visual view she sees is not what she wants. There are multiple ways, involving inserting different sets of control characters, for a user to force the desired visual order view, and it is easy for the user to make mistakes using these control characters.

If you choose Paradigm 1, User in Full Control, then your BDWP will create the fiction that the user is under full control and that each character entered is strong, while, in fact, the file is stored according to the Unicode standard.

One possible way to achieve this fiction is for the BDWP to keep track of character directions and subdocuments and to drop control characters into the file in the correct places to cause the UBDA to interpret the file so that each character has the direction the user thinks it has and that the file's embeddings match the document--subdocument structure that the user thinks the file has.

If you choose Paradigm 2, the BDWP chooses character directions as specified by the UBDA. Even so, you have to choose a modus operandi from among a spectrum of modi operandi. The variation along this spectrum is according to the degree of well-formedness checking or enforcing that the BDWP does as the user enters control characters:

1. Modus Operandi 1 (No checking or enforcing):

   There is a command to enter each control character at the place the cursor points. For each such command, the BDWP checks nothing and simply inserts the specified character at the place in the file pointed to by the cursor. However, the resulting visual order view will be unpredictable if the file is ill-formed, e.g., an LRE, RLE, LRO, or RLO is not paired with a matching PDF.

2. Modus Operandi 2 (Checking):

   There is a command to enter each control character at the place the cursor points. For each such command, the BDWP checks that the placement of the control characters follows the constraints for well-formedness, e.g., that embedding and overriding commands are paired properly with PDFs. You will have to decide how to treat the situation in which an embedding or overriding has been started, but its corresponding PDF has not been given yet.

3. Modus Operandi 3 (Enforcing):

   There is a command to enter each of LRM and RLM at the place the cursor points. There is a command to enter each of a LRE-PDF, a RLE-PDF, a LRO-PDF, and a RLO-PDF pair at the beginning and end of the region selected by the current selection. With such commands, it is guaranteed that the entered control characters follow the constraints for well-formedness.

Your user's manual must describe somewhere in the beginning the chosen paradigm and the chosen modus operandi if it is relevant for your chosen paradigm.

The above has described the paradigms and modi operandi in terms of what the user DOES and can DO, but there is more. No matter what paradigm and modus operandi you choose, you need to provide commands that allow the user to VIEW what he has input and to see how the software has interpreted what is in the file. This ability to view is particularly important when the software is going to be making default assumptions that may differ from the user's intent. Even when the user is in full control, he can make mistakes (REALLY???), and therefore, the user needs to be able to see how the software has interpreted what he has entered.

So you will need commands to show:

• the file in visual order,
• the file in logical order,
• control characters,
• each character's strength (if the notion exists in your BDWP),
• each character's direction,
• the document-subdocument (embedding) structure with each's direction, and
• the overriding structure with each's direction.

Note that in a showing of a file in logical order,

• each Arabic, Persian, and Urdu letter is shown in its stand-alone form, which is the form that has as its code the code for the letter; in particular, no connecting forms are shown, and
• no ligatures are formed; one sees the stand-alone form for each letter that would make up a ligature in the visually ordered view.

Note that one could achieve these views by

1. having a command for each
2. having attributes that can be set or unset that apply to the current view

Finally, no matter your paradigm and modus operandi don't forget to deal with:

• fixing all the failings of your selected word processor (MS Word, MS Wordpad, MS Notepad, or Mac TextEdit),
• providing a non-confusing way to select with the pointer characters that are contiguous in the logical order even when the selection straddles a directional change,
• highlighting a selection properly, even if the selection crosses a direction change boundary and the resulting sudden change in the direction of the growth of the highlighted region causes cognitive dissonance,
• dynamic shaping and ligaturing of Arab characters as they are displayed, while the file stores only the stand-alone form of each individual letter,
• includes a way to break the connection between any two characters in an Arabic, Persian, or Urdu word without splitting the word into two words (effectively putting a zero-width non joiner between the two characters),
• providing a way to disable ligaturing of any sequence of Arabic, Persian, and Urdu characters that normally make a ligature, e.g., the lam and the alif, to not form the usual ligature (by putting a zero-width joiner between the characters),
• providing a way to select the current keyboards to use to input characters of various alphabets (This capability may be provided by the underlying operating system.).

The problem with a selection that crosses a direction change boundary, which does not have to be the beginning of an embedding or an overriding, is the cognitive dissonance caused by the fact that as the selection crosses the direction change boundary, the highlighted region starts to grow in the direction opposite that of the hand movement. So, the user is tempted to start moving the hand the other way. But, moving the other way should cause undoing of the selection, not growing of the selection.

There are at least two ways to deal with this cognitive dissonance:

1. Find a whole new way to implement the highlighting that avoids the problem.
2. Accept that cognitive dissonance will happen when a selection is done across a direction change boundary in the visually ordered view, but provide a logically ordered view in which the selection can be done without cognitive dissonance.