Continued from Patch for variable arguments bug in Tinyfugue (screen corruption)

I was looking at Tinyfugue's source code in order to see what exactly is currently being used from the ncurses library.  I was curious about whether Tinyfugue was currently using any of the main ncurses code for display purposes, or if ncurses was just being used for the termcap information.  That's when I noticed that the ./configure output from the RHEL 3 machine was different from my Ubuntu machine.

RHEL 3:
checking for library containing tgetent... -lncurses

Ubuntu:
checking for library containing tgetent... no
configure: WARNING: Hardcoding terminal codes.

Quite simply, I didn't have the ncurses development libraries installed.  Not a huge deal since getting them from the apt repositories is pretty easy.

The surprise came when I ran a new configure on the original Tinyfugue source (without the va_list patch).  Once I ran Tinyfugue, there was no screen corruption.  I was pretty shocked.  I thought maybe I was running on an unclean directory.  So I started from scratch and after compiling, Tinyfugue was running with no screen corruption.  I went ahead and uninstalled the ncurses development library, cleaned and compiled Tinyfugue, and once again there was screen corruption.

My conclusion is that something with ncurses (either ncurses itself or with how Tinyfugue builds the code in the first place) that prevents the issue with the va_list I experienced.  I'm not going to look to0 much in to this because with my va_list patch, the issue is resolved in both ncurses and non-ncurses builds.

Most of the modifications that I have made to Tinyfugue's source code have been through a shell account on an old machine running 32-bit RHEL 3.  I've never had a problem compiling and using Tinyfugue unless the changes that I've made have broken the build.  Just today I decided to move everything over to my own computer.  I'm currently running a 64bit build of Ubuntu.  As I was switching over, I decided to run builds of the various branches that I work on (dotnotation, C style block comments, Top status area, Telnet option negotiation, etc).  Everything compiled smoothly, with some additional warnings that weren't present on the older compiler available in RHEL 3.  The big surprise was when I attempted to run the latest build of Tinyfugue that includes my telnet option negotiation patch.  I was shocked that the screen was garbled with input, status, and output areas all overlapping.  Even after exiting Tinyfuge, my Konsole session was not working correctly.

Of course, the first that I did was fall back to the unaltered tf-50b8 source code and ran a build.  When the compilation finished, I ran Tinyfugue and saw that the same screen errors were occuring.  I then started debugging Tinyfugue to find out the exact cause of the issue.  After too much time, I finally narrowed it down to the vSprintf function in tfio.c file.  The vSprintf function is passed a va_list.  The va_list is correctly intialized using va_start and torn down using va_end in the function calling vSprintf, so the basic code logic is sound.  The issue is that vSprintf tries to use the va_list after it's been passed to the vsprintf function.  According to the C99 documentation ([ISO/IEC 9899:1999], Section 7.15), the va_list actually is actually in an indeterminate state after returning from the vsprintf function.  So once Tinyfugue tries to use the va_list a second time, it fails (but relatively gracefully).  The failure in this case is just enough to throw the screen functions off, but not enough to crash Tinyfugue.  The solution is to modify vSprintf to use the va_copy macro to make a copy of the va_list, then pass the copy off to the vsprintf function and keep the original copy for internal use.  When work is done on the copied va_list, va_end is called on it.

After getting that patch in place, Tinyfugue now works on my 64 bit Ubuntu machine.  I need to patch all of my branches, but I'm also including the patch here seperately so that anyone who wants to can use it.  I'm unsure if this is an issue with new GCC builds or if it is because of the switch from 32bit to 64bit linux.  In either case, the updated code compiles correctly and shows no signs of screen corruption on either system.

Continued at Patch for variable arguments bug in Tinyfugue (screen-corruption) (Cont)

fix64bit_va.patch (1.43 kb)

Tinyfugue was last updated officially on January 14th 2007.  Since that time several "out of band" communication protocols, such as ATCP, GMCP/ATCP2, and AARD (102), have been created to facilitate data transfer between MUD clients and MUD servers.  These protocols use telnet option "subnegotiation".  Tinyfugue needs to be updated to be able to negotiate these newer protocols, and any other future protocols, that use telnet option subnegotiation.

Before telnet option subnegotiation can occur, telnet option "negotiation" must occur.  Telnet option negotiation is used to decide which protocols the client and server support.  Once a specific telnet option has been agreed upon, they can exchange information using the appropriate telnet option subnegotiation for that telnet option.

Tinyfugue has several basic telnet options hardcoded in its source code.  Tinyfugue sends back a negative response when it does not recognize a telnet option.   Telnet option negotiation must occur before telnet option subnegotiation can exchange data.  Since Tinyfugue responds negatively to all unknown options, it is not possible to begin exchanging data through telnet subnegotiation.  Updating the source code to recognize GMCP, ATCP, and AARD (102) telnet options is easy.  However, updating the source code for these specific options does not provide a generic method for dealing with other protocols that be created in the future.

One method for solving this issue is to provide Tinyfugue hook events (like the ICONFAIL and SEND hooks) for the several stages of telnet negotiation.  With those telnet option hooks in place, the end user would have the ability to accept or deny arbitrary telnet options from servers.  Having control over arbitrary telnet options allows the end user to respond to options that are currently in use, and also support options that may come in to use later.

Tinyfugue has a framework for telnet option subnegotiation that is currently used for MCCP, NAWS, and terminal type negotiation.  Telnet option subnegotiation has a very generic format.  All telnet option subnegotiation messages begin with "IAC SB XX" (255 250 XX) where XX is the telnet option identifier (GMCP uses 201, ATCP uses 200, AARD uses 102, etc).  After that byte sequence, the data is sent in whatever format is specified by the protocol documentation (for example GMCP uses JSON formatted messages).  Finally, "IAC SE" (255 240) is sent signifying the end of the data.  This generic format for telnet subnegotiation means that when Tinyfugue receives a subnegotiation message from the server it is easy to pass the data to the end user for manipulation.  The telnet option ID and subnegotiation data could be provided through the use of a Tinyfugue hook.

Providing hooks for telnet option negotiation and subnegotiation only solves half of the problem.  The end user also needs to be able to respond to the telnet option negotiation request.  It does not help the end user to know that the server is requesting the GMCP telnet option when the end user cannot reply back to the server.  The same issue applies to telnet option subnegotiation.  In particular, newer telnet subnegotiation options (such as GMCP) allow the end user to request information about the MUD server through client initiated subnegotiation messages.  The solution to this half of the problem is to provide a set of commands that can send telnet commands back to the server.

A small set of specialized Tinyfugue functions could be created in order to allow the end user to send telnet commands.  There would be a minor issue with using these specialized functions because most telnet commands use byte arrays. End users need the ability to provide some information (such as the telnet negotiation option ID) in the form of numbers or bytes, but other information as normal text (such as the JSON data for GMCP).  Also, the functions must be generic enough to allow the input of data for any protocol that might be implemented in the future.

One way to solve the issue of specialized functions with generic or unknown requirements would be to modify Tinyfugue's Send function to accept raw telnet data using a new command switch (such as -r for raw).  If that command switch was available, then it might be possible to create Tinyfugue script libraries that are able to correctly respond to current telnet options.  When new telnet options become available, new script libraries could be created without the need to edit Tinyfugue's source code.

Thus Tinyfugue currently needs a set of hooks for telnet option negotiation and telnet option subnegotiation and a command or commands used to respond (and initiate) those same items.  Once all of those pieces are in place, libraries for telnet options like GMCP, ATCP, and AARD (102) can be created through Tinyfugue scripts.

I'm currently working on a top status bar for Tinyfugue.  This is mostly done inside of file output.c.  So far the hardest part has been deciding on how to integrate the top status bar without breaking any currently working scripts.  Adding macros such as /status_add_top and /status_rm_top is too cumbersome.  Using the -r row option of all current /status_* macros with negative numbers would allow current scripts to remain unchanged, and require only the row specifier be changed in order to move items currently on the bottom status bar to the top status bar.

Tinyfugue already has the -r option to the /status* macros.  This option specifies the row on which to place the status field, and is 0 based.  When using the -r option, -r0 would put the field on the first status line and -r2 would put the field on line 3.

It is possible to extend the use of the -r option to provide a top status bar by allowing the -r option to take negative numbers.  Currently this is prevented in the source code by making all row values less than 0 equal to 0.  After removing this check, it is possible to use negative numbers to designate rows in the top status bar.  The main issue with using those negative numbers is that now the way in which top and bottom status rows are accessed is different.  -r0 is the first row of the bottom status bar, but -r-1 is the first row of top status bar.  This breaks the symmetry of 1 and -1 since 1 would be bottom row 2 and -1 would be top row 1.

One method to achieve this dual use of the -r option is to mirror the current bottom status bar system.  Right now the bottom status area consists of a group of arrays where the row number from the -r option is used as the array index.  The top status bar could have its own set of arrays that are created and initialized in the same manner as the bottom status arrays.  Then in the code, a check could be made for a negative number and a conversion be done on that number to get the current array index into the top status arrays.  If the row was negative, the index would be set to (-row - 1).  This negates the row number and then subtracts 1.  Using this logic, -1 turns in to 0 (-1 to 1 then subtract 1 to get 0), -2 turns in to 1, etc.  Using this method, most of the current code can be left unmodified.  The main work is switching which set of arrays the code is working on from the bottom status arrays to the top status arrays.  The main drawbacks to the mirrored set of arrays is the continual conversion of the row in to an index for the top status arrays and the increased complexity of having 2 sets of arrays and correctly switching between the two.

An alternate method would be to use the negative row number from the -r option as the actual index into the current arrays.  Normally this would not work since a negative index in to an array is generally out of bounds.  However, if an array had a size equal to the combined size of the top and bottom status arrays, then a fake index could be used to represent 0.

For an array with 10 total status lines (5 for the top status bar and 5 for the bottom status bar), a fake index could be created at 5 and defined as the 0 for the array.  Then the row number from the -r option would be added to the fake index in order to get the real index in to the array.  To find the real index, add the fake index and row number together.  In other words, "real index = (fake index + row number)". There is a example below that demonstrates how this works.  This method also benefits from being able to use most of the current code without modification.  The main drawbacks to this method are keeping the top and bottom indexes in sync, ensuring that all array lookups are done from the fake index instead of 0, and that iterations over the whole array cover all items.

Example:

[0] - [1] - [2] - [3] - [4] - [5] - [6] - [7] - [8] - [9]
 ^         Top           ^     ^        Bottom         ^
 `-----------------------'     +-----------------------'
                               ^
Fake index at 5 ---------------'

If using -r2 then row = 2 and the real index is (5 + 2) = 7

If using -r-2 then row = -2 and the real index is (5 + -2) = 3

When naming variables or macros (that are to be used as functions), Tinyfugue only allows the use of AlphaNumeric characters and underscores.  This patch enables the dot, ".", as a valid character in variable and function names.  Even though Tinyfugue doesn't recognize scripts as classes, enabling "dot notation" allows for a style of coding that reflects an object oriented approach to script organization.

Instructions for compiling

1. Download the Tinyfugue source (http://tinyfugue.sourceforge.net/#source)
2. Download the dotnotation.patch file into the newly created tf-50b8/src directory.  A directory listing of this directory should show multiple files ending in .h and .c
3. Open a command prompt and "cd" into the tf-50b8/src directory.
4. Run "patch < dotnotation.patch" without the quotes.
5. Run "cd .." to go up to the parent folder
6. Run "./configure" to run the configuration program
7. Run "make" to compile the program
8. Assuming you are an admin (either by group membership or through sudo), run "make install" to install the newly created "tf" executable

Instructions for testing

Once the executable has been compiled, the following script should test the "dot notation" functionality.

/def dotnotation.test=/return Testing dot notation: %*

/eval /echo -aCred - $[dotnotation.test("Hello World")]

If the patch is working, then this should print "Testing dot notation: Hello World" out to the screen in red.  If the patch isn't working, then you'll get a message stating "% EVAL: expression syntax error: expected operator or ']', found '.'."

Caveats

With this patch, Tinyfugue doesn't see any difference between the dot and any other valid characters.  This means that it is possible to use dots in such a way that they are valid in a Tinyfugue sript, but would be invalid in most object oriented languages.

These are all perfectly legal: like  is perfectly legal.  Other items that are perfectly legal are:

• /def ...=/return "All dots for a function name and would be called as ...()"
• /def a.b.=/return "function name ends in a dot and would be called as a.b.()"
• /def .a.b=/return "function name begins with a dot and would be called as .a.b()"
• /def a..b=/return "function name has two consecutive dots and would be called as a..b()"
• /set ..a.b=1
• /set ...=1
• /set ..=1
• /set a..=1

The addition of the dot as a valid charact for variables also makes the behavior of some current scripts change.

For example, "/set myname=David%;/eval /echo My name is %myname." would not work as expected and would instead print out "My name is".

From Tinyfugue's (new) view, the variable starts at % and continues until it hits an invalid character for a variable name.  In this case, the dot is a valid character, so Tinyfugue sees the variable name as "myname.", looks it up, doesn't find it and puts a blank in its space.  This is actually already covered by "/help substitution" with the text "The brackets are recommended for clarity, but may be omitted if there is no default and the text following it can not be misinterpreted as part of the selector.".

In this case, the dot could be misinterpreted as part of the variable name and thus brackets should be used.  The fix for this is to use proper variable enclosures.  A working copy of the above example would be "/eval /echo My name is %{myname}."  This unambiguously tells Tinyfugue that the variable name is "myname" and not "myname.".

Comparison to currently available naming options

As this patch does not provide any enforcement of commonly used dot notation restrictions, the patch is only useful in helping make scripts more readable.  It provides for a less ambiguous method of providing object oriented-like naming schemes in Tinyfugue.

The only non-AlphaNumeric character currently available is the underscore.  The underscore can be used as a way to organize scripts, but its meaning can be ambiguous as the underscore is also commonly used to specify a space character.  It is not always easy to tell whether an underscore's logical meaning is to specify a space in a proper name or if its meaning is to specify a member value or function.

By providing two special characters, the dot and the underscore, function and variable names can be formulated in such a way that the logical meaning of the name is clearer.  For example, with /def a_b.c_d it is easy to that this function represents a logical member of the a_b object called c_d whereas with /def a_b_c_d that same logical meaning is not readily apparent.

dotnotation.patch (2.77 kb)