Mudlet & screenreaders

LexTheSame
Posts: 5
Joined: Fri Jan 11, 2013 11:25 am

Re: Mudlet & screenreaders

Post by LexTheSame »

Hi,

I would like to raise back this topic.

I am myself a blind mud player and software developer, using both windows and mac platforms to do my job. I was very pleased to see mudlet authors being interested in making this product accessible to screen readers. However, the qt-baset nature together with long-decaying state of QT accessibility support are our major obstacles on this way.

I think that making a special console interface especially for screen reader users, and thus excluding them from mainstream experience is not quite right. This creates maintenance and support headaches. On the other hand, the harsh truth is that qt is not yet ready to provide perfect out-of-the-box accessibility for mainstream UI.

I was sorry to discover that official mudlet 2.1 windows distribution seems to not include QT Accessibility plugin, because the mudlet window is completely inaccessible with NVDA, eventhough other qt programs are (semi) accessible.

I see two possible dirrections to move to:
1. the easiest. enable qt accessibility on platforms where it is present (mac, windows). test and see what might be improved by utilising QAccessibleWidget and other qt accessibility interfaces. Make settings & options dialog accessible. Think about making output window accessible (perhaps by utilizing custom screenreader APIs).
2. consider changing GUI toolkit :-)

p.s. Developers are encouraged to use NVDA <http://www.nvda-project.org> - a free and open source screen reader for windows to perform accessibility testing on windows, and built-in voiceover utility on mac (accessible with command+f5).


Thanks,
Lex

User avatar
kevutian
Posts: 217
Joined: Fri Aug 20, 2010 8:18 pm
Location: United Kingdom
Contact:

Re: Mudlet & screenreaders

Post by kevutian »

Hi, Lex and thanks for posting.

To be perfectly honest, the main issue for furthering development on this front has been a sheer lack of support from blind players who are in a position to both test and advise these capabilities of the client. There has been a lot of time and work expended on this front, but owing to the above, this has been halted somewhat.

What we really right now is feedback on how the client performs in this area. i.e. How it currently performs, what is lacking and what work can be done to improve the existing implementation.

If you might be in a position to offer your assistance here, that would be absolutely awesome.

I am personally very eager to see improvement in this area. Whilst I myself am a sighted player, I am of the mind that expanding current capabilities of Mudlet is both prudent and common sense.

Please feel free to comment further and if possible, encourage other blind players to do the same. This will most certainly help massively.

Thanks again and hope to hear back soon.

User avatar
Heiko
Site Admin
Posts: 1548
Joined: Wed Mar 11, 2009 6:26 pm

Re: Mudlet & screenreaders

Post by Heiko »

I have put the development of Mudlet's Screen Reader Edition on halt because I did not have enough quality participation from the blind community in order to make a quality product. I had about 8 blind players who took part in the testing - and not a single one of them had any useful programming background, in fact they couldn't even make simple triggers or trivial Lua scripts to test the interface on their own. I got very contradictory feedback. One player was able to use everything just fine, while 3 others couldn't even connect to their game server and gave up quickly (probably without having properly read the help files.

Qt isn't used for the user interface in the screen reader edition because Qt's accessibility support was nowhere near good enough. Qt5 claims to have better support, but documentation was still missing the last time I had a look. So what is being used: I use the windows C API to get better screen reader support and I use windows SAPI directly because this is the only way to get full control over what is being spoken. The user has a reading cursor in the text that he can move freely, skip lines, words etc.. The interface consists of 2 console windows (game console and command console plus a popup file editor to write your scripts.). The current state of the project is close to alpha stage. The screen reader is just a front end to the current Mudlet version, so blind players have the full capabilits of the client including package installation, auto walking support from the mapper etc..

The voice quality of NVDA is very poor compared to SAPI and I don't know of any blind user that actually uses NVDA. In Germany people seem to use JAWS - most haven't even tried NVDA. This non-willingness to try out new software in fear of destroying their current setup is also a problem when trying to get people to test. Most blind users seem to be dependent on seeing people for setting up their computers.

The interface of the screen reader version is a mix of native windows GUI and console application. More GUI elements could be added in the future if there is more interest from the blind users. Until then the development will not continue. If a few active blind testers want to help this project - or if you want to participate yourself as a blind software developer, I will continue development again.

LexTheSame
Posts: 5
Joined: Fri Jan 11, 2013 11:25 am

Re: Mudlet & screenreaders

Post by LexTheSame »

Heiko wrote: Qt isn't used for the user interface in the screen reader edition because Qt's accessibility support was nowhere near good enough. Qt5 claims to have better support, but documentation was still missing the last time I had a look. So what is being used: I use the windows C API to get better screen reader support and I use windows SAPI directly because this is the only way to get full control over what is being spoken. The user has a reading cursor in the text that he can move freely, skip lines, words etc.. The interface consists of 2 console windows (game console and command console plus a popup file editor to write your scripts.). The current state of the project is close to alpha stage. The screen reader is just a front end to the current Mudlet version, so blind players have the full capabilits of the client including package installation, auto walking support from the mapper etc..
Where I can download the latest mudlet screen reader edition?
Talking about direct control over spoken feetback, I am not sure it is what you really need. Screen reader developers put a lot of time and efford to make user experience effective and customizable, and replicating that efford is imo an overkill. If we utilize standart native controls, most screen readers will pickup the interface easily, allowing user to customize the output in whatever way he/she wants. For exammple, you get braille display support for free.
Heiko wrote:The voice quality of NVDA is very poor compared to SAPI and I don't know of any blind user that actually uses NVDA.
Here you confuse screen reader with speech synthesizer. Screen reader is a program which collects information about events in the system and uses different means of communicating this information to blind person (mostly speech and braille). Speech synthesizer is a piece of software which takes text on input and computes/plays an actual voice tone. You can use whatever synthesizer with every screen reader. e.g. you can easily use sapi voices with NVDA (and jaws, and any other windows screen reader).
Espeak, the synthesizer NVDA ships with by default, is in no way the most natural sounding voice. But do not forget that blind users often prefer low latency and ease of recognition on high speech rates to human sounding.

As for popularity, NVDA counts as being third most used SR in the windows world (http://webaim.org/projects/screenreadersurvey4/#primary), while being second in some local markets. I recomended to use it because 1) it is free, 2) it supports all modern accessibility interfaces. In particular, its qt support is superior to jaws.
Heiko wrote:The interface of the screen reader version is a mix of native windows GUI and console application. More GUI elements could be added in the future if there is more interest from the blind users. Until then the development will not continue. If a few active blind testers want to help this project - or if you want to participate yourself as a blind software developer, I will continue development again.
I will definitely evaulate the current SR version and get back with my feedback.
However currently there is some obstacles preventing me from switching to mudlet from my current client and hacking it for my needs.
1. I respect lua, but Python takes mutch stronger position in my heart as for scripting languages. I have significant ammount of mud client scripts in python.
2. I want to have the same (or close) experience as my sighted peers. For example, I often provide support to friends. If we use different interface, this means we have difficulties communicating about problems etc.
I explained my attitude about isolated vs mainstream experience in the previous posting.
3. While I am trying to slowly move all my activities to mac, I discovered mudlet as a brilliant cross-platform mud client. But SR version seems to be available only for windows.

Thanks,
Lex

User avatar
Heiko
Site Admin
Posts: 1548
Joined: Wed Mar 11, 2009 6:26 pm

Re: Mudlet & screenreaders

Post by Heiko »

1. As noted above I do use windows standard controls which are well supported by screen readers.

2. There is no standard control for text streams. Consequently, I can't rely an screen reader support and have to make my own as far as the text stream console is concerned.

3. Most blind users that I know use windows telnet to play. Newly arriving text is being read out loud. You have no text buffer to speak of and no freely moveable text cursor as you can only read what is printed on the screen and newly arriving text messes everything up. The second most used alternative is the old console client TF. You do have a scrollable buffer, but no freely moveable reading cursor as you can only read what is displayed on the screen. The third most used client by blind players is Mushclient. Mushclient has no built in support for blind users but offers a little plugin that uses the speak function of sapi, jaws and nvda to speak any newly arriving text. This often leads to duplications of what is being spoken. You do not have a freely moving reading cursor, but you can read what is being displayed on the screen similar to TF. This is very ineffective and uncomfortable. As noted earlier I haven't met a single blind user who was able to write simple triggers and scripts in Mushclient although the interface is old windows 95 style and uses windows standard controls all over the place. In other words, this client is well supported by screen readers but still not really accessible. I'm trying to find out how the interface can be made as nicely accessible as possible so that blind players can actually use the full feature sets of a modern top client and enjoy the games more, but this requires more participation from active blind players.

4. I'm not against NVDA. I just haven't met anybody who uses it. NVDA users are welcome to help with development.

5. Which client are you using?

User avatar
KaVir
Posts: 86
Joined: Mon Feb 08, 2010 8:38 pm
Contact:

Re: Mudlet & screenreaders

Post by KaVir »

You may find this of interest:

Of the 435 active characters on God Wars II, 98 use a screen reader, and of those only 23 last connected from a unique IP address. Of those 23, 1 refused to identify their client. The remaining 22 characters are broken down by client as follows:

MUSHCLIENT 14 (63.6%)
DUMB 7 (31.8%)
ANSI,VT100,DEC-VT100 1 (4.5%)


The "ANSI,VT100,DEC-VT100" is Monkey Term. I'd speculate that DUMB is VIPmud, but I'm not sure.

LexTheSame
Posts: 5
Joined: Fri Jan 11, 2013 11:25 am

Re: Mudlet & screenreaders

Post by LexTheSame »

Heiko wrote: 2. There is no standard control for text streams. Consequently, I can't rely an screen reader support and have to make my own as far as the text stream console is concerned.
You're right. However most screen readers have an option to automatically read the newly arrived text on the screen. That's how we used to use most mud clients.

Another more "correct" option is to use live regions (on windows desktop applications can do it by utilizing IAccessible2 interface). This is how Instantbird implemented automatic reading of income IM messages.
Heiko wrote: As noted earlier I haven't met a single blind user who was able to write simple triggers and scripts in Mushclient although the interface is old windows 95 style and uses windows standard controls all over the place. In other words, this client is well supported by screen readers but still not really accessible.
What kind of problems do blind users you met have with writing mush client scripts?
Heiko wrote: I'm trying to find out how the interface can be made as nicely accessible as possible so that blind players can actually use the full feature sets of a modern top client and enjoy the games more, but this requires more participation from active blind players.
Have you tried to compile mudlet windows version with QT Accessibility plugin enabled? This might make most of option/settings dialogs accessible.

For example, virtualbox is also written in qt and because they ship qt accessibility plugin, the program, despite some quirks, is usable.
Heiko wrote: 5. Which client are you using?
Jaba MUD Client - http://sourceforge.net/projects/jmc/

it uses custom control to output text, so I written an app module for NVDA to automatically read incoming text captured from it. I use NVDA review cursor to navigate through the text (numpad7-9 - move by line, numpad4-6 move by word, 1-3 - move by character etc). Arrow keys allow me to move through the input history and my current input line.

I can easily operate with JMC interface to add/edit/delete aliases/triggers/hotkeys, however most of the time I prefer to use tintin-style commands to do the same from the input line itself.

User avatar
Heiko
Site Admin
Posts: 1548
Joined: Wed Mar 11, 2009 6:26 pm

Re: Mudlet & screenreaders

Post by Heiko »

LexTheSame wrote:most screen readers have an option to automatically read the newly arrived text on the screen. That's how we used to use most mud clients.

Another more "correct" option is to use live regions (on windows desktop applications can do it by utilizing IAccessible2 interface). This is how Instantbird implemented automatic reading of income IM messages.
1. If you use the screenreader's say or speak function for newly arrived text you can't fast forward in the new text because you have no idea where the voice is at any given moment in time. This is why I use SAPI directly to enable forward jumps, line skips etc..

2. Using live regions on the screen does not work because if you recieve a chunk of, say 50 lines, and your screen height is 20 lines, Mudlet will only print the last 20 lines of those 50 newly recieved lines on the screen for performance reasons as you couldn't read the other 30 anyways. These 30 lines would be lost and wouldn't be read aloud by the screen reader. In my approach, however, the screen isn't used and the reading cursor just operates on the invisible text buffer giving you full control of what is being read, fast forwards etc..

Have you tried to compile mudlet windows version with QT Accessibility plugin enabled? This might make most of option/settings dialogs accessible.

For example, virtualbox is also written in qt and because they ship qt accessibility plugin, the program, despite some quirks, is usable.
Yes. You have access to settings but no access to the more important parts of the client's GUI. Mudlet's trigger editor is optimized for seeing people. It uses a lot of tool buttons, tree widgets, table widgets, combo boxes and check box groups that are not properly supported by Qt. You could navigate the various tree widgets, but that's about it pretty much. A special GUI for blind users is much more effective.
I use Jaba MUD Client - http://sourceforge.net/projects/jmc/

it uses custom control to output text, so I written an app module for NVDA to automatically read incoming text captured from it. I use NVDA review cursor to navigate through the text (numpad7-9 - move by line, numpad4-6 move by word, 1-3 - move by character etc). Arrow keys allow me to move through the input history and my current input line.

I can easily operate with JMC interface to add/edit/delete aliases/triggers/hotkeys, however most of the time I prefer to use tintin-style commands to do the same from the input line itself.
This client seems to be Russian only. The English website seems to be in Russian also.

Given my above 50 lines of new text chunk example, can you fast forward in newly arrived unread text?

tspivey
Posts: 3
Joined: Fri Aug 03, 2012 7:26 am

Re: Mudlet & screenreaders

Post by tspivey »

I'm also a blind mudder. I use MUSHClient, which outputs to NVDA directly with MushReader, which is a plugin I wrote to support several screen readers.
The system cursor is always in the text box, so I can edit with the arrows, while the NVDA navigator is on the output. I then can easily read the current screen worth of mud text with my navigation keys. New text gets read automaticly, even if it's long and scrolls off; my plugin sends it directly to NVDA.

If I want to read something long like a shop list, I'll capture it to a notepad with another plugin. I can then read it like an edit box. The ctrl+u recall is another sometimes useful feature for capturing exact lines while creating triggers.

User avatar
Heiko
Site Admin
Posts: 1548
Joined: Wed Mar 11, 2009 6:26 pm

Re: Mudlet & screenreaders

Post by Heiko »

I'm glad that you take part in this discussion. In fact, it was your Mushclient plugin that I talked about above when referring to Muschlient as Mushclient natively only supports SAPI speak, but in the end it seems very similar to me as both just add the newly arrived lines from the MUD to the screen reader's lines to be spoken buffer which means that we have no further control over them i. e. you can listen to them all or skip them all, but there's no way to skip a few boring lines or reread a line because we have no fine grained control over the reading cursor if we are using the screen reader to speak new text. The other problem of Mushclient is that it duplicates lots of lines in certain situations which can be very irritating. This is because Mushclient does not have real support for this sort of thing. All that is done is pushing newly arring lines to the screen reader's speak function.

In my first couple of Mudlet screenreader versions I replicated your plugin behaviour, but I got mixed feedback. One of my testers who was using Mushclient was fine with this and had zero problems whatsoever. However, the rest of my testers with telnet or vipmud background couldn't deal with it - and the vipmud users complained that they couldn't fast forward the reading cursor in newly arriving text, which is clearly a big drawback of Mushclient screenreader support also because skipping known text is vital in high action PK MUDs.
Because of this I switched to SAPI and did a screen reader emulation so to speak. Using low level windows keyboard hooks, I keep the screenreaders out of business when dealing with the main console screens and I print nothing on the screen, and emulate the screen reader reading cursor behaviour directly on the text buffer. Consequently, the user can do anything he wants with the reading cursor i. e. skip words, lines and do any kind of jumps within the buffer at any given time. I guess that most of the problems that people still have is that they think that it's a standard text control and might expect different behaviour or the screen reader cursor emulation may not be similar enough. I'm just guessing here and I'd need blind users to tell me why they have issues and what needs to be improved.

Post Reply