Archive for the 'Tools' Category

Fangs extension moving to Mozilla hosting

I discovered that my first post about the Fangs Screen reader emulator add-on was posted on November 22 in 2004. That is more than five years ago. At that time Mozilla hosting for add-ons was pretty rough and I couldn’t work out how to release updates. Alas, I hosted the add-on and updates on my own.

Yesterday I discovered that Mozilla add-on hosting has improved a lot so in the future you will find Fangs there. It should make updating a lot simpler. Version 1.06 adds compatibility with Firefox 3.6.

See Fangs screen reader emulator add-on over at Mozilla.org.

Pitfalls of Web Accessibility Evaluation Tools

Summary: Automated web accessibility evaluation tools are hard to trust, understand and only provides feedback on a small amount of factors that influence accessibility. Also, a unified web evaluation methodology should be adopted to provide consistent results across tools.

Introduction

When you start working with web accessibility as a site owner you will typically be exposed to online accessibility evaluation tools recommended by your supplier. These tools typically let you enter a link to a web page after some automated checks are made you get a report of all the errors that were found.

While these tools may be a good way to convince your organization to increase funding for accessibility work you should be careful how you interpret their results. Your website may be good enough already and if you try to fix all reported errors you may be spending money in the wrong thing.

As an example of how difficult these tools may be to trust and understand I have selected some of the more popular ones from the Web Accessibility Initiative list of tools and performed some tests.

Letting accessibility evaluation tools test themselves

Here, each of the tools were pointed at their own starting page. When possible WCAG 1.0 triple A settings were selected.

Self test result
Tool Errors Warnings
Deque Worldspace 6 6
WAVE 0 0
Functional Accessibility Evaluator 0 0
AChecker 0 115
Eval Access 0 124
Cynthia says 1 0
TAW 3 0

Very few errors as expected. After all, these tools are built by professionals and I would expect them to have checked their own service.

Letting them test each other

So, what do they say about each other? Only one way to find out.

Cross test result (Application that performed the test in the header row)
Tool World-
space
WAVE FAE AChecker Eval Access Cynthia says TAW Sum
Worldspace 6 0 3 4 14 3 23 53
WAVE 43 0 19 11 9 2 7 91
FAE 4 0 0 3 2 1 1 11
AChecker 52 0 0 0 14 4 7 77
Eval Access 13 0 3 6 0 1 2 25
Cynthia says 10 0 11 16 13 1 13 64
TAW 8 0 0 10 1 1 3 23

It is understandable that people find it hard to make use of web accessibility evaluation tools. How are you supposed to interpret these results? None of the tools are in agreement on any of the tested pages. Similar results would be returned for most pages you evaluate.

Observations

  • WAVE didn’t find any accessibility issues in any of the pages. Also, WAVE would display a fun error message if you try to make it check itself by URL (I had to copy and paste the source instead).
  • The output from many of the tools are really hard to interpret, especially if you are new in the field of web accessibility. TheĀ  TAW tool, for example, displays tiny icons all over the page and you have to hover them to see what they mean.
  • Worldspace uses nested tables for layout (something that WAVE didn’t complain about).

What would be your advice for a site owner that wants to increase accessibility on his/her website? How can they check if their supplier did the right thing when creating the markup?

(Please leave a comment or send me an email if you find any errors).

Fangs for Firefox 3 available

Sorry for the delay. Here is an updated version (1.0.4) of Fangs for Firefox 3. Your previous version may not update automatically in which case you need to uninstall it, restart Firefox, and then download/install it from the Fangs project page.

A big thank you to Stuart Middleton who showed me the necessary steps to get rid of the annoying security warning that Firefox 3 displays for unsigned extensions.

Bringing Accessibility into the Development Process

Web accessibility is, in my experience, often considered late in the development process. Typically, accessibility evaluation is conducted by outside experts after the application is delivered and content is produced. This leads to issues being reported to developers late in the project, at a time when changes cost more.

In order to make accessibility development efforts more efficient I believe that accessibility has to be integrated into all stages of a project with as much automation as possible. Here are some ideas on how this can be done for the developer role.
Read more about Bringing Accessibility into the Development Process

Evaluation of WYSIWYG editors (2007)

It has been almost a year since I tested accessibility features in some of the more popular WYSIWYG editors commonly found in open source content management systems (see Evaluation of WYSIWYG editors). During this time, most of these editors have been further developed. Let’s have a look at how they fare a year on.

Test method

The test method is the same as last time (to be able to see if scores changed compared to the previous test). In short, I tried to create a sample document in each of the editors. The sample document contains markup commonly found on the web. All editors were tested on the same date and the online demo version was used when available. If there was an option to enable more features, all were enabled. Please note that I am primarily testing the output of the editor and not the accessibility of the editor itself.

Tested editors

Most of the editors from the previous test are included this time. Based on feedback from the comments I have also added a couple of new contestants.

From the previous test:

  1. EditOnPro by Realobjects. Commercial license.
  2. XStandard by Belus Technology. Commercial license. Lite version free (also see special license for open source CMS projects)
  3. FCKeditor by Frederico Caldeira Knabben. Open source (LGPL), commercial license available.
  4. CuteEditor by Cute Soft. Commercial license.
  5. TinyMCE by Moxiecode. Open source (LGPL).
  6. New: Xinha. Open source BSD-style license.
  7. New: WYMeditor. Open source (MIT and GPL). For an introduction to WYMeditor see Visually Editing Semantics – What You See Is What You Mean.
  8. New: Loki by Carleton College. Open source (GPL).

Missing from this round is Kupu as it has not come out with any new release since the previous test. Also, eWebEditPro could not be tested. The online demo gives an error: “Error loading mycontent1″.

Test result

The total score for each editor is listed below. For full test result details see here.

Editor Score (out of 19)
XStandard 19
EditOnPro 16
TinyMCE 15
FCK-editor 14
WYMeditor 14
Loki 13
Xinha 13
Cute Editor 10

If you find error in the results, please leave a comment or send an e-mail to peter krantz at gmail.com.

Observations

  • Many of the tested editors have implemented more accessibility features since the previous test. This is good.
  • The bold and italics icons are still used when creating emphasis.
  • TinyMCE is one of the few editors that doesn’t nest list items properly. According to the developer comment to my previous article there is an option to corrent this behaviour but it isn’t enabled by default for some reason. TinyMCE now has support for acronyms and abbreviations (with icons that are easy to understand). TinyMCE would have had a score of 18 if the nesting was fixed.
  • XStandard is the only editor that pass all the tests.
  • WYMeditor is a very interesting contender. Although in a very early stage of development (only two versions released) it managed to score 14 points. It is also the only editor that clearly displays to the user what type of markup is being edited.
  • Loki is one of the few editors that has a proper icon for block quotes (it looks like a quote character).
  • Only XStandard and TinyMCE provide functionality to create acronyms.

There are now more editors that provide functionality to do proper semantic markup. This is great! My guess is that it would be easy to add many of the remaining features (acronyms, abbreviations and inline quotes) to many of the editors.

Which one will you be selecting for your next project and why?

Peter Krantz, peter.krantz@giraffe.gmail.com (remove giraffe).