Summary

ssax and Unicode entities

Metadata

Description

The SSAX library includes a file, html-entity-codes.scm, that defines a table of Unicode entity codes. This table could be used to replace ssax:predefined-parsed-entities, so that XHTML files that include Unicode entity codes can be parsed with SSAX. However, the Chicken sxml-transforms library does not export these definitions.

The code that constructs the table of Unicode entity codes assumes Unicode-aware integer->char, so it should not be included in sxml-transforms by default. Perhaps we need to a separate egg, something like utf8-sxml-transforms that utilizes the utf8 library?

Changes and comments

[2010-08-10 17:35:25 UTC] zbigniew removed milestone 4.6.0

[2010-08-10 17:35:25 UTC] zbigniew wrote:

Ivan,

integer->char is already unicode aware.

 $ csi -e '(write (integer->char #x3bb))'
 \u3bb

What is not unicode aware is converting this character to a string using (string c) or (make-string 1 c). However, you can use the system procedure ##sys#char->utf8-string:

 csi -p '(##sys#char->utf8-string (integer->char #x3bb))'
 λ

It is trivial to modify html-entity-unicode-chars to call this instead of core make-string, which will not require the utf8 egg. In fact I did a similar thing in the 'ssax' egg so that numeric entities are parsed into utf8 sequences without requiring the utf8 egg.

Speaking of which, shouldn't this be filed against the ssax egg and not against sxml-transforms?

[2010-08-10 19:35:12 UTC] sjamaan changed summary

[2010-08-10 19:35:12 UTC] sjamaan wrote:

I think that's right. ssax is about parsing, sxml-transforms is about transformation of already-parsed sxml.

I'll change the title to match. I don't know who maintains ssax, but I don't.

[2010-08-10 20:35:27 UTC] zbigniew changed keywords from sxml-transforms unicode entity to ssax unicode entity

[2010-08-10 20:35:27 UTC] zbigniew changed status from new to assigned

[2010-08-10 20:35:27 UTC] zbigniew changed owner from sjamaan to zbigniew

[2010-08-10 20:35:27 UTC] zbigniew wrote:

No one really maintains it, but I can make this change.

Ivan, do you just want me to export html-entity-unicode-chars from ssax, after fixing it to handle utf8? I don't know to use the lowlevel SSAX parser so if you could give me a short test case that would be appreciated.

[2010-08-11 07:23:35 UTC] iraikov wrote:

Yes, that would be fine. I also have not used the low level parser, but I will look at it and see if I can make it work. Any XML file that includes a Unicode entity (e.g. hellip) would be a good test case.

Replying to zbigniew: > No one really maintains it, but I can make this change. > > Ivan, do you just want me to export html-entity-unicode-chars from ssax, after fixing it to handle utf8? I don't know to use the lowlevel SSAX parser so if you could give me a short test case that would be appreciated.

[2011-06-04 13:24:29 UTC] sjamaan wrote:

Isn't this fixed now? Can this ticket be closed?

[2011-09-08 23:23:56 UTC] zbigniew changed status from assigned to closed

[2011-09-08 23:23:56 UTC] zbigniew set resolution to fixed

[2011-09-08 23:23:56 UTC] zbigniew wrote:

Added html-entity-unicode-chars in version 5.0.5. I have absolutely no idea how to use it --- you can't pass it to ssax:xml->sxml --- but at least it has the right value!