Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#64720 closed defect (fixed)

dblatex: incorrect UTF-8 handling and escaped characters in listing environment

Reported by: oksmith77 Owned by: neverpanic (Clemens Lang)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc:
Port: dblatex

Description

Hi,

I am using dblatex 0.3.12 built with MacPorts.

The handling of Unicode characters does not seem correct to me.

The dblatex option

latex.encoding=utf8

should result in a UTF-8 .tex file with Unicode characters as is except within a listing environment. In that case, the Unicode characters need to be properly escaped. As far as I am concerned, they need no additional translation/mapping.

TEST #1 =======

The following produces a .tex file as expected

% dblatex --type=tex -o mytest.tex mytest.xml

The following does *NOT* produce a .tex file as expected.

% dblatex -P latex.encoding=utf8 --type=tex -o mytest.tex mytest.xml

In the above, the escape characters in the listing are incorrect.

TEST #2 =======

The following does *NOT* produce a .tex file as *FAILS* expected

% dblatex --type=tex -o mytest2.tex mytest2.xml

I recommend a more meaningful error message that includes the suggestion to use UTF-8 and xelatex.

The following should produce the expected .tex file but does *NOT*.

% dblatex -P latex.encoding=utf8 --type=tex -o mytest2.tex mytest2.xml

Again, the escape characters in the listing are incorrect.

TEST #3 =======

The following does *NOT* produce a .tex file as *FAILS* expected.

% dblatex --type=tex -o mytest3.tex mytest3.xml

The missing characters could be optionally added to the mapping table and then the test would work.

The following does work correctly.

% dblatex -P latex.encoding=utf8 --type=tex -o mytest3.tex mytest3.xml

Please let me know if you need additional information.

Thanks.

Attachments (1)

mybug.tar.gz (2.9 KB) - added by oksmith77 3 years ago.
Test files

Download all attachments as: .zip

Change History (5)

Changed 3 years ago by oksmith77

Attachment: mybug.tar.gz added

Test files

comment:1 Changed 3 years ago by jmroot (Joshua Root)

Owner: set to neverpanic
Status: newassigned
Summary: UTF-8 handling and escaped characters in listing environmentdblatex: incorrect UTF-8 handling and escaped characters in listing environment

comment:2 Changed 3 years ago by neverpanic (Clemens Lang)

I believe this needs a patch. This problem does not seem to be specific to macports, though, so this should be addressed upstream.

  • lib/dbtexmf/dblatex/rawverb.py

    old new  
    3636        n = tex_handler_counter[self._errors]
    3737        for c in ntext:
    3838            if ord(c) > 255:
    39                 c = str(self.pre) + c + str(self.post)
     39                c = self.pre.decode(self.output_encoding) + c + self.post.decode(self.output_encoding)
    4040                n += 1
    4141            text += c
    4242        tex_handler_counter[self._errors] = n

comment:3 Changed 3 years ago by neverpanic (Clemens Lang)

Resolution: fixed
Status: assignedclosed

In 1e94a17651ea6d0c9b45e7e7b09859c469698c2f/macports-ports (master):

dblatex: Fix python3 encoding problem

Closes: #64720

comment:4 Changed 3 years ago by oksmith77

I submitted a bug report to the project Sourceforge page. You may want to upload your patch there.

Thanks of taking care of this so quickly!

Note: See TracTickets for help on using tickets.