Bugs in MXP implementation, is this work in progress?

freshman
Posts: 37
Joined: Mon Oct 05, 2020 9:56 pm

Bugs in MXP implementation, is this work in progress?

Post by freshman »

Hello Developers,

I'm wondering if the MXP module is still being worked on. Comparing with what I remember when I last tried, it seems to have improved, since I can now login to my mud and there is 'some' kind of (MXP) functionality. Still, there are two major bugs that render the MXP functions quite useless to us:

1) There is no (working) support for entities (<ENTITY ...> or <!EN ...>
(re)defining 'entities' &customname; )

2) When defining elements, parameters for embedded commands are apparently transformed to uppercase, example:

<!EL ITI '<SEND "examine &ID;|drop &ID;" HINT="examine|drop">' ATT='ID'>

(btw entities as element args seem to work..)
When used, the hint menu shows EXAMINE and DROP. Oddly enough, w/o an element, directly using <SEND>, it works.

with those two issues - esp. 1) - fixed, MXP could start to be useful. There are additional issues too:

3) It is very irritating if the whole command menu pops up as a hint when hovering over a multiline menu like ITI above, but you can not click it,
as it is a hint only. It should give some hint you need to right click or there is more. Actually, giving n+1 hint texts, mud should be able to
specify the 'menu' hint text anyway.

4) &text; doesn work in element definitions. Of course, I assume this is
due to the total lack of entity support (see 1)).

5) No support for 'EXPIRE'

6) Sending unclosed strings from the mud (try, no don't): <!el red '<color red>> crashes the client. Same holds for not closing elements with </name>. Now for !el this is difficult, as it can be used in locked secure mode and can theoretically span over hundreds of lines... still some watchdog etc. to bail out and reset would be helpful. If you lockup mudlet with it, you need to close the app and reopen. Just reconnecting to reset the terminal won't help... Oh.. how about responding to the muds \e[3z ... to reset the mxp interpreter.. hmm?0

Now, you can say, !el is a secure tag, so you can trust the mud not to misuse it. But not closing open tags does the same, though open mode should reset at each \n by MXP standards.. (cite: Also, when in OPEN mode, any unclosed OPEN tags are automatically closed when a newline is received from the MUD.) Well, it seems to me, there is no support for the MXP line tags at all.

7) When using <VERSION> it seems a '\n' is also send to the screen or main window..
Not too bad, but it disturbs my loading/init screen...

I didn't dig any deeper for now. All this above is mandatory MXP, btw. There are no optional parts in this. Honestly, none of the clients out now is fully v1.0 compatible. But claiming MXP=1.0 in <VERSION> even w/o <!EN> is... well... bold.

Now, I'd like to know if someone is really working on this, or if I'd have to code this myself and convince you to include it, s.t. mudlet is another option for our plys . I had recoded the whole MXP thing of mudportal to be 1.0 compliant (admittedly w/o user(!) defined line tags and <var>s (since none
are displayed in the gui anyway)), so I don't really feel like doing all this again (and just adding our mud to the default list given in mudlet might not suffice as a motivation). Also I've little interest to code in W10, though that's the version most needed. Now.. after a quick glance over your instructions a W10 rebuild seems a doable task, though I'm yet unsure if its C++, C#, Java or else (probably it doesn't matter anyway).

P.S. Talking about stability, sending something like \e]2;Cool Title\a (which sets the window title in xterm) does also lockup mudlet. Again, even reconnect does not help. Mudlet must be closed.

Now, you didn't claim xterm support (nor vt100 with line graphs, double sized characters and printer support..), still.. some random esc sequence should not lockup the client, IMHO. Some garbage chars and then has to recover..

Regards,
Freshman.

freshman
Posts: 37
Joined: Mon Oct 05, 2020 9:56 pm

Re: Bugs in MXP implementation, is this work in progress?

Post by freshman »

Well, well. as we say in Germany: No answer is also an answer.

In the meantime (among other unrelated things), I managed to make my own W10 build of mudlet. I also had a (quick) look at the source.
To my surprise the MXP modules looks pretty well organized, it even actually does some quite elaborate handling of entities and variables, and stuff.
Alas, there is not the slightest attempt to use the entities so well organized in memory.

I really don't understand this. It seems untypical to me to have a very elaborate code to scan even the complexed options of entities but not use them
in the rest of the code - even for testing. Anyway, I'll now try to dig in deeper and solve issues 1 & 2). I'm yet unsure, but w/o 4) our mud might have one issue or the other. I'm not sure if I can address 3. For the rest.. probably noone except me will care.. and my priorities are different.. so.. well, we'll see.

Everything is nicely encapsulated in countless objects, hence it will need quite some time to understand the global structure of operations and get things going.

Stay tuned for updates, but I'll have to take my time.

breakone9r
Posts: 24
Joined: Sun Jul 26, 2020 1:39 am

Re: Bugs in MXP implementation, is this work in progress?

Post by breakone9r »

I know the devs would be really happy to have another person familiar with MXP working on it. I dont think it's been touched in ages except in response to a couple of issues I had, namely the addition of supporting MSP thru MXP, and working around what appears to be a bug in the MXP code of the mud I frequent.

User avatar
SlySven
Posts: 1019
Joined: Mon Mar 04, 2013 3:40 pm
Location: Deepest Wiltshire, UK
Discord: SlySven#2703

Re: Bugs in MXP implementation, is this work in progress?

Post by SlySven »

TBH Very few persons fully understand MXP - I don't (and I sometimes wonder about Zugg!) - the Mudlet MXP code did get a major refactoring recently - if you can use Git you can look back through the history and see that @gcms on GitHub has made some substantial changes a few months back: https://github.com/Mudlet/Mudlet/pull/4016 . From what I do know about our handling I'd have put it at only Version 0.3 or 0.4 but that was before that person really cleaned up the MXP stuff. For the record Mudlet is a largely C++ application (though there is some C code as well). It also supports a Lua user-scripting system and some Lua is used at runtime (and during compilation so you will want a Lua installation in your build environment as well). It is multi-OS and uses Qt libraries which aides with getting the code to run on all those OSes.

At present we offer prebuilt 64-bIt MacOs and GNU/Linux binaries and a 32-Bit Windows 8.0-10 one - we've officially dropped supporting Win 7 as that is now EOL - I do not know if it is possible to still compile a compatible application with the current generation of tools/development environments as at least some parts do not work now with Win7.

I compile and run Mudlet on Linux and FreeBSD and I am currently developing in a 64-Bit Windows 10 environment and I am trying to construct a build system that will compile both 32 AND 64 Bit Windows versions - since Qt do not provide a 64Bit SDK for Windows in their On-line installers I have installed, and am suggesting others switch to using, a MSYS2/Mingw-w64 environment instead of using the direct offering of Qt - I am largely the author of Compiling on Windows 7+ (MSYS2 Alternative).

If you wish to chat about this (or any other aspect) of Mudlet you may find you get a quicker respond in our Discord Guild this link should get you there: https://discord.com/invite/kuYvMQ9 .

User avatar
Vadi
Posts: 5035
Joined: Sat Mar 14, 2009 3:13 pm

Re: Bugs in MXP implementation, is this work in progress?

Post by Vadi »

The MXP implementation for the most part satisfies most games' usage of it and we're happy with that level. It did get rewritten recently as pointed out above, gaining a few features in the process. Thanks for checking this out!

freshman
Posts: 37
Joined: Mon Oct 05, 2020 9:56 pm

Re: Bugs in MXP implementation, is this work in progress?

Post by freshman »

Well, I won't call myself an MXP expert as I read above, but I digged a bit into it from the mud view some ten years ago.

With some wild or maybe educated guessing, I already made some progress - more than I expected. I hope C++ covers all con/destruction and memory management issues well, because I don't have a clue yet. - Seems to work though. I also tend to have a more archaic coding style tending to use overloaded operators ( + [] implicit casts and stuff) which might not be to everyones taste. Mostly since I don't know all the existing member functions by heart, but I know which operator just does it.

I was wrong: Actually MXP entities are resolved, the !EN tag handler - though looking nice - was still broken. It also seems &text; works, I dunno why it failed during my first tests (but see below on resolving to last char only).

I could also resolve the all CAPS mxp send menu hints. They were deliberately forced all CAPS when used in SEND. Why? The muds send an explicit hint text to use and it is always forced all CAPS? I guess someone loves it this way and will burn me... but if the Mud requests an explicit text it should be used, or? I also managed to support setting the menu hint itself (the mouse hover tooltip of a multi-choice menu) through MXP, independent of the individual command hints. I don't know where the link menus are all used in mudlet, it also seems there is a lua function. While even the lua function might benefit from the changed hint handling, I aimed to change ONLY the behaviour of the MXP send menus.

Ah, I forgot to mention: Using entities within the send cmd/hint definition didn't work right when the entity contained '|'

From that point on the majority of our muds mxp code started to work.

I also had to hack !EN to support setting entries to empty strings. I've the slight feeling positional args are generally not handled well when there is an empty string arg "" among them, but I have no current use case except setting an entity to empty string.

Right now I noticed default arguments like (example from original MXP specs):
<!ELEMENT boldtext '<COLOR &col;><B>' ATT='col=red'>
<boldtext>This is bold red</boldtext>
<boldtext col=blue>This is bold blue text</boldtext>
<boldtext blue>This is also bold blue text</boldtext>

don't work as specified.

[Yeah, there is no working 'B', try <!ELEMENT boldtext '<SEND><COLOR &col;>' ATT='col=red'> for demo instead]. Apparently mudlet defines
a parameter tag named "col=red" instead. You must use ATT='col' but then there is no red default setting, I think (need to doublecheck) &col; might even resolve to &col; or col; then.

That's what I'm looking at right now.

BTW, I use Cmud and Mushclient in parallel to compare behaviour and to deduce what a 'common sense' interpretation of MXP is. I don't count in my enhanced mudconnector version, since that does it how I think it should be done... hence is strongly biased)

-

What I stumbled upon and is very annoying though not MXP related: You cannot just hit return, empty lines are not send to the mud. Very annoying if you page something with "more" or even want to use the ed inline editor (need to check <tab> some time). Again, I'm sure not everyone will like to have it changed... What do you think?

It also seems MXP entities used outside of MXP elements are not handled right, they just resolve to the last character... This is only a first impression, I need to check further. Talking about entities, std entities, say &frac12; seem to resolve to UTF-8 proper - but the default charset cannot display them right (on W10). Any idea/hint how to make it work?

Finally, I'm a bit scared on trying to implement the different MXP line modes and reset escape sequences. Though all works in a permanent secure mode, our players can send any tags in says and tells in open mode (as the MXP standard suggests), and if the mud client doesn't stop it by enforcing open tags only, they can screw up other peoples clients - not good. They can even send a non closed open tag and screw mudlet. So... well.. we'll see.

You are probably very curious on what I'm coding there, but I need much, much more testing.. Each flaw I fix directly leads to the next issue (say default args) above..

Regarding discord, I don't use it.. I'm more into async communication, and have to.

User avatar
SlySven
Posts: 1019
Joined: Mon Mar 04, 2013 3:40 pm
Location: Deepest Wiltshire, UK
Discord: SlySven#2703

Re: Bugs in MXP implementation, is this work in progress?

Post by SlySven »

Finally, I'm a bit scared on trying to implement the different MXP line modes and reset escape sequences. Though all works in a permanent secure mode, our players can send any tags in says and tells in open mode (as the MXP standard suggests), and if the mud client doesn't stop it by enforcing open tags only, they can screw up other peoples clients - not good. They can even send a non closed open tag and screw mudlet. So... well.. we'll see.
The primary issue you are going to have with so-called "secure" tags is that that only really makes sense in a closed source client - anyone can take our code (or indeed any other open source MUD client) and hack it to inject rogue sequences - the only sure fire means to prevent harmful behaviour is in the Server... at least in my humble opinion.

(Similarly for that stupid "REGISTERED" attribute in the <VERSION> tag - it is pointless for a non-proprietary client - though I have only just noticed it was optional in http://www.zuggsoft.com/zmud/mxp.htm!)

freshman
Posts: 37
Joined: Mon Oct 05, 2020 9:56 pm

Re: Bugs in MXP implementation, is this work in progress?

Post by freshman »

Hmm, SlySven, I think there is a misunderstanding about the meaning of secure (and open) mode in MXP.
Of course, anyone can create its own client and mud and make them do whatever bad you can imagine.

Secure mode with in MXP basically means, the client may trust the MXP code was created by the game driver in kind of a controlled, sensible fashion.
Open mode means, the MXP tags can be from an arbitrary player and must not be trusted. Thus only 'harmless' formatting tags work in open mode and at end of line all are closed on automatic.

This is so, s.t. a player can "say I <c red>love</c> you." and people see him talk in red. If this player forgets the </c> (by accident or not) it does not
matter, at the end of the line <c> is autoclosed anyway and the screen formatting is under full control of the game and the mudlib again.

Of course, the mud could parse the arguments of say and ensure nothing bad can happen, but this is quite some coding effort (and then there is tell,
and you might allow people engrave a ring or write a coloured note etc. and in all cases you need to check it too... and then.. the mxp client might
feature someover fancy formatting command not part of the standard, and players may want to use it...

Open mode just overcomes all this: It is a way for the mud server to tell the mud client: Look: These commands may come from a careless or troll player, so please ensure that nothing bad happens, everything resets proper at the end of the line and that nothing bad happens to the the mud session of our beloved user.

If <!en> or <!el> or <send> or <a> were open, any player may redefine some other players system (well: mud) elements or entities (our mud has about 70 of them, for inventories, shop listings, all kind of basic stuff) and screw another players session. With a well placed <send> or <a> tag, he might lure another player to open a browser page to install some malware or to do a command in the other players session (like: delete character.. or imagine the target player is an archwiz: the rogue player could make him delete files or shutdown the server...).

Yes, if you connect to a server with a highly capable client it might screw your system. But secure mode ensures that MXP commands are coming from the admins of the game you are connecting to and not some arbitrary guest player troll that seeks to harm all people online: (shout click <SEND "delete profile" HINT="here">here</SEND> for pictures from our last mud party). If you don't trust the admins.. well, then kust find another game

Now, if your mud does not allow players to generate mxp tags gor other players at all, you are fine. You can just go into locked secure mode at login, and that's it.

I hope that clarifies the meaning behind open and secure mode.

-

Regarding the optional registered flag of z/cmud, well yes. I agree. z/cmud is not freeware. Probably their intention was that mud admins would
generate a warning: Please register your client. Or maybe an unregistered client would work only in level 1 or a newbie area, but not beyond.
A nice idea, but I'd assume no mud admin cared if people bought zmud or not.

Freshman.

User avatar
SlySven
Posts: 1019
Joined: Mon Mar 04, 2013 3:40 pm
Location: Deepest Wiltshire, UK
Discord: SlySven#2703

Re: Bugs in MXP implementation, is this work in progress?

Post by SlySven »

Now, if your mud does not allow players to generate mxp tags gor other players at all, you are fine. You can just go into locked secure mode at login, and that's it.
That the thing though - with a client such as Mudlet the user could generate so-called "secure" tags and use them in messages to the Server - and it would have no way of telling that it came from the user and not the Client - so the Server should never trust anything coming FROM the Client, unless, perhaps it is a proprietary one - but I do wonder if that is what Zugg was thinking when he first thought of the idea of these tags...

freshman
Posts: 37
Joined: Mon Oct 05, 2020 9:56 pm

Re: Bugs in MXP implementation, is this work in progress?

Post by freshman »

SlySven wrote:
Sat Oct 24, 2020 3:15 am
That the thing though - with a client such as Mudlet the user could generate so-called "secure" tags and use them in messages to the Server - and it would have no way of telling that it came from the user and not the Client - so the Server should never trust anything coming FROM the Client, unless, perhaps it is a proprietary one - but I do wonder if that is what Zugg was thinking when he first thought of the idea of these tags...
You are looking at it from a totally wrong direction: Normally the client is not sending any MXP tags to the server at all (ok, there are the answers to <SUPPORT> and <VERSION>). This is all about MXP Tags sent to you from the server. The idea is that tags created by the game logic, admins and server are send secure, but any tags send upon request of ANOTHER player (for example say <B>I'm bold</B>) are open, that is your client knows, that these tags have not been generated by the mud itself, but an arbitrary, other player.

And just to clarify that, if I say you send 'say <B>I'm bold</B>', yes, this is mxp-encoded, but the client and mud don't do anything special with it. It's arbitrary ascii text entered and can be entered even w/o a MXP capable client. The mud just relays it as arbitrary ascii text. Some mud-clients of other connected people can interprete it, some not. But for MXP capable clients the mud ensures that it is relayed in open mode, so the target client knows that this MXP code was just relayed from a source that cannot be trusted.

The idea is that you trust the mud admins more than another player that might just attempt to screw your terminal to pk you.

And yes, the client generated answers to <support> and <version> are also prefixed with a 'secure mode'-code. But this is just to avoid accidently entering these as a player as part of a command like 'say do you know how the <version> tag works?'. If someone wants to really screw this in his open source client and sent wrong answers... go ahead.. only thing is: it will screw his connection to the mud alone, but not the connections of others..

Post Reply