MXP enhancements, was PR #4200

freshman
Posts: 39
Joined: Mon Oct 05, 2020 9:56 pm

MXP enhancements, was PR #4200

Post by freshman »

Hi people,

I've been asked if I wanna revive the 4200 PR, but do it in much smaller chunks, and well, it was done 2 years ago. So, a few tiny things need to be done to it s.t. it fits into the now change development branch.

My main goal is, to get all the fixes of 4200 in first (because then mudlet will become usable for me) than maybe add other things regarding mxp, fix issues other people have, maybe do some further extension (read below).

To achieve that, I've come up with following roadmap for PRs:

1. Make <version> support a style argument. This is a very, very tiny patch. Why do it first? Well, as a 'test balloon or probe' to see how the integration collaboration works out, it is mandatory in the MXP v1.0 definition which Mudlet negotiates and it (kind of) screws setup of my mud.

I already did this as a new branch of development, I'll send a PR right after this post.

2. Make the <bold>, <underline>, <italics>, <strikeout> tags actually work. Again, this is relatively simple, and I'd assume these tags are not so often used since there are good vt100 alternatives. So why do it? Well, they are already parsed and handled by the code (somewhat) but don't have any effect, so I think this should be fixed. Within that patch/pr I'll also add the short aliases <b>, <u>, <s>... of these... because, well these are shorter and kindof the recommended way to used in MXP. <color> BTW, does already work (not sure about setting fonts, but this is not used so often, so no priority for me).

Then the patches will get more complex:

3. <!EN> or entities don't work proper, if I remember right, there are issues with setting and evaluating them. So here I'll put in what I did to fix this. At this point, I think, I should also fix/extent the mxp test code snippets.

4. <!EL> element definitions also had some issues. I need to doublecheck what was exactly broken, but I think there were issues with unnamed parameters (just defined by position) and default values of omitted parameter.

-- if we reach this point, the mxp code will become usable, because normally with MXP, when connecting to the mud, you get a 'style sheet' uploaded with a bunch of <!EL> definitions like how to display an item, or a weapon and what context menu to display with them which will then be used throughout the game, for example with <W>sword</W> and which might make use of some entities or parameters to change details.

5. There are some tiny issues with multiline send menus, for example, they don't display a special 'hover' link right (if one is explicitly specified by the mud)

6. Hints of menus are always displayed all caps. I think, Mudlet should not do that. If the mud wants to have them send in CAPS it should set them in CAPS, I think mudlet should not force everything to caps. Now, this is a style issue. it can be argued, and I'm prepared for people insisting mudlet must enforce all caps.. But well, I think that is wrong.

--- so this is the original content of PR#4200 . For some things, esp. the EN / EL changes, there is some test code too, that I would do when I work on those parts.


After that, other things I noticed and that came to my mind for the future (only ideas, I don't promise anything!) :

a) if you disconnect and just reconnect (not restart mudlet). The MXP state is not reset, not sure if this holds for other terminal states too. But it is bad, of course. For example, if by a bug (maybe wrong MXP code send by the mud), the terminal is screwed up (like black on black colour), disconnecting and reconnecting does not help. You need to close and reopen mudlet... Not good (oh, and this actually already effects the currently released version... remember... <color> already works.)

b) Now this is a really hard thing: MXP defines different modes (open modes, secure modes... etc.). I saw in the discussions here that there are quite different opinions by this, but for one these would define an MXP reset sequence that allows the mud to reset all mxp states at login or maybe in response to something like "stty sane" to counteract runaway tags like in a). Second, they allow to enable automatic closing of tags at the end of line in common situations which removes the issue with runaway tags for most common cases. But... this is (IMHO) a bit more difficult to do. Definitely it would need to add interpretation of more esc sequences and some trigger to the mxp code when a \n is seen. (and for each element we need to define if it is privileged or not)

c) provide some lua access to mxp variables and esp. current room descr and exits which will already be specially tagged by an MXP compliant mud. I think the latter might be very useful for the/a automapper.

d) maybe some way to debug the mxp status might be helpful. Like seeing current entity or elements settings. Not sure yet how to do this (like a special mudlet command, or some button in a menu), but it could be helpful.

e) maybe see if the sound/music tags work and if not make them working.

So, this is my current roadmap. I've to add I'm quite busy at the moment.. So I might/will need my time.

User avatar
SlySven
Posts: 1023
Joined: Mon Mar 04, 2013 3:40 pm
Location: Deepest Wiltshire, UK
Discord: SlySven#2703

Re: MXP enhancements, was PR #4200

Post by SlySven »

Just a couple of notes:
freshman wrote:1. Make <version> support a style argument...
I'm not sure I follow you, according to the available reference material, well this the MUD server can send a STYLE attribute in the pseudo-XML that specifies a style-sheet "version number". It isn't clear to me how Mudlet is supposed to handle that - as it's styling is based upon Qt's own styling language which I very much is NOT going to be the same as whatever it is that C/ZMud might use.
freshman wrote: ... does already work (not sure about setting fonts, but this is not used so often, so no priority for me).
That is not going to work currently as Mudlet only supports a single font in any particular console. As it happens I do have longer term plans to try and support more than one font (by making use of the <SGR> 11 m to <SGR> 19 m codes to switch away from the default (which would be chosen by a later <SGR> 10 m ANSI Escape sequence). This is something that could be encoded in the TChar class - but would be limited by the rendering system so that the fonts must have a sizing so that they have the same horizontal advance (effective width)...

freshman
Posts: 39
Joined: Mon Oct 05, 2020 9:56 pm

Re: MXP enhancements, was PR #4200

Post by freshman »

Hmm, I think the term 'style' from the mxp definition might be a bit misleading here. At the very first, its just some text string that has no effect at all, except that it is returned when the mud asks just <version> and well, <version some-text> does not generate some answer being send to the mud - it just remembers "some-text" as the style (the name or id of the style, not the style itself).

The style here is not something like a CSS style, or what you may consider as Mudlets style. Style in the MXP sense is a collection of entity and element definitions that the mud makes. Without going into the details of the actual definions, it would allow the mud to make definitions for elements WI AI WW AW and IT that would then allow the mud to send <WI>sword</WI> or <AI>chainmail</AI> in the ongoing text or inventory printout which may use special colouring for those text, but also add a context menu for examine, wield, drop for WI and examine, wear, drop for AI. If these are wielded or worn already, the inventory command can use WW or AW instead, which would display context menus "examine, unwield, drop" or "examine, remove, drop" resp. Similarly, the shortcut IT, when used in a room descript could provide a context menu with "examine, take". Of course, a real implementation
of AI would allow for <AI id="chainmail">a rusty chainmail</AI> or <AI "chainmail">a rusty chainmail</AI>

The collection of these 'shortcuts' or elements is what you would name an MXP style sheet. Of course, you can also do this with GMCP to download lists of items from the mud and display them in separate windows under control of some lua code and mudlet stylesheet. It's just a different way to do it, and it will only work with mudlet.

Now, in theory, multiple players online in the game could have different stylesheets and this macro definitions - if the setup has just been changed while they have been online. In theory, the mud can check the actual version of the style that has been used to setup the player with <version> and maybe act accordingly. Admittedly, this is not very likely to happen. The major with the current implementation of <version> in the real world is that <version> will send a reply to the mud, but <version 12345> should not (but if it does.. well, the unexpected msg confuse the mud)

User avatar
SlySven
Posts: 1023
Joined: Mon Mar 04, 2013 3:40 pm
Location: Deepest Wiltshire, UK
Discord: SlySven#2703

Re: MXP enhancements, was PR #4200

Post by SlySven »

TBH That has gone a little over my head. So I will just read through what you come up with and see if that makes things clearer. ;)

User avatar
Vadi
Posts: 5045
Joined: Sat Mar 14, 2009 3:13 pm

Re: MXP enhancements, was PR #4200

Post by Vadi »

Hey, the roadmap sounds quite good! One remark Lua access to MXP - that's already and thing, try 'lua mxp' to have a look at it.

freshman
Posts: 39
Joined: Mon Oct 05, 2020 9:56 pm

Re: MXP enhancements, was PR #4200

Post by freshman »

While I go through the original PR#4200 stuff - I just made PR#6802 to make simple MXP formats like 'BOLD' etc. work - I realize that this PR might actually be the largest one:

When I had it done - and remembering that a colour tag like <C magenta> not closed due to a bug in the mud code (or by a malevolent fellow player using it in a message to you) will change your colour permanently until you exit and reopen mudlet - I thought:

Dammit: What if those mxp commands have been disabled intentionally because there are some muds outside that don't close them properly? Hence I claimed: I cannot enable those tags until they are properly closed at an end of line as documented in the standard. Searching for a place in the code to hook this in, I noticed the hooks for MXP to do this are already in the buffer function, the only thing missing was to actually reset the colour and text decorations. Though simple, that enlarged the patch a bit. But without this, I fear the risk that some poorly written mud screws the terminal are too high.

Also, the support for the different MXP modes from my roadmap and which I claimed will be difficult to add is already mostly there. So, certainly the hooks are already in the code and there'll be only tiny things to add.. if at all.

I now think, the remaining MXP issues on my list are much easier fixed and even more importantly, can be broken down into much smaller, independent pieces/PRs.

freshman
Posts: 39
Joined: Mon Oct 05, 2020 9:56 pm

Re: MXP enhancements, was PR #4200

Post by freshman »

Well, 1st things first, I'll be on vacation now for two weeks. I'll be able to react to msgs and are also able
to work on code, build mudlet, work on github; but only with very, very little time as my SO will beat me if I sit
at the computer for too much time.

I just released a bunch of pull requests. I probably should double-check a few more times, but they should contain all parts of the
original #4200 MXP enhancement PR except for two more cosmetic issues.

If they'll make it into the main tree, all/most MXP render issues I encountered should be gone.

'Cosmetic issues' are:

1) You cannot specify a mouse over hint for MXP menus: aka (from the protocol definition)
<send "command1|command2|command3" hint="click to see menu|Item 1|Item 2|Item 2">this is a menu link>/SEND>

Now, while this a mandatory element, some clients (AFAIK cmud) don't support it too.. so, maybe not many muds use it and/or will have a
way to circumvent the issue. It's a pity though, as Mudlets default behaviour to just display the whole, 'unclickable' menu at mouse-over
is IMHO quite irritating and sub-optimal.

freshman
Posts: 39
Joined: Mon Oct 05, 2020 9:56 pm

Re: MXP enhancements, was PR #4200

Post by freshman »

2) Mudlet will force SEND hints to be all uppercase. But not consistently. Without checking every detail, it seems normal
<SEND>look</SEND> have a lower case tooltip, but <SEND>say PROMPT</SEND> have it all upper. But if the game explicitly specifies HINTS,
they are forced to all upper. You can see the effects again by logging in as a guest on aldebaran-mud.de 2000 in the default prompt where
i, look and room exits are lower case, but the tooltips for say and examine are all UPPER (they are SEND PROMPT tags) as are the menus.

I don't really care what the default tooltip is - it is a default, but IMHO it should at least be consistent. However, if the mud explicitly provides a hint, IMHO it should be used as is. If the game developer wants it all upper case.. well, just set it so. Do not expect Mudlet to deliberately do it for you.

However, someone might whine / complain if his send menu hints out of a sudden start to become all lowercase.

Please someone advise which way to go here.

Maybe it should be a configuration/preference option (maybe on by default) like: Force MXP Tooltips to all UPPERCASE. But then a tiny hint
where in the code to add this to the preferences menu would be great and avoid having me search for it for ages.

freshman
Posts: 39
Joined: Mon Oct 05, 2020 9:56 pm

Re: MXP enhancements, was PR #4200

Post by freshman »

Finally: in my original road map I mentioned lack of support for the various OPEN/SECURE modes, I now realized it is already here, but somewhat
hidden behind the scenes and virtually w/o effects (as many MXP things are in Mudlet). I'd guess, to make it fully work will require only tiny patches here and there but when it now deals with the simple formatting tags already, this no longer seems to be a high priority.

Instead, I now realized / re-encountered the fact that entities do not really resolve outside of MXP elements. User defined ones will
just resolve to ';' and (most painfully) HTML-style entities like &frac12; resolve to a garbage character. This is IMHO very annoying.

The root cause is well understood and boils down to TBuffer.cpp, TBuffer::translateToPlainText line 656:

TMxpProcessingResult const result = mpHost->mMxpProcessor.processMxpInput(ch);

which basically just allows a single 8bit latin1 char returned at the reference &ch , in addition to the enum TMxpProcessingResult.
I need to get a QString out of processMxpInput, together with some more info (aka: more enum values) if there is one, and how to deal with it.
  1. Now, more often using classical languages, normally I'd just pass another reference like:

    QString entityValue;
    TMxpProcessingResult const result = mpHost->mMxpProcessor.processMxpInput(ch, &entityValue);
  2. But I guess you might prefer to blow up TMxpProcessingResult to be a real class containing an enum and a QString (even though it
    then cannot be passed/returned by reference)
  3. Finally some weird mechanism can be setup where processMxpInput(ch) actually returns the entity value one char per call. But that seems
    really weird and bloated. Also, localbuffer into which ch points, is a std::string with 8bit chars in various encodings,
    where entities are QStrings with 16bit UTF chars. It will be quite complicated to rebuild the different encodings from the Qstring.

    So, IMHO, better return the QString from processMxpInput and add&run 8bit chars through the 'normal' localbuffer loop and handle 16bit chars separately.
In all cases, a counter measure is necessary to avoid endless loops by <!EN foo &bar;><!EN bar &foo;>&foo; probably recursive interpolation
should not work, as is the case within MXP elements already.

Well, I think c) is out of the question, but I'd like to have some feedback on a) and b).

Regards,
Michael.

User avatar
Vadi
Posts: 5045
Joined: Sat Mar 14, 2009 3:13 pm

Re: MXP enhancements, was PR #4200

Post by Vadi »

I will be on leave for the next two weeks as well, so won't be able to check PRs in the meantime starting tomorrow. Will think on what you've posted!

Post Reply