Five months ago I released TPerlRegEx for Delphi 2009 which enables PCRE‘s UTF-8 support when compiled with Delphi 2009, so you can use TPerlRegEx with Unicode strings. TPerlRegEx still supports Delphi 2007 and earlier using Ansi strings.

Unfortunatety, until today’s new release, TPerlRegEx for Delphi 2009 had a rather embarrasing bug: it didn’t actually enable the UTF-8 support in PCRE if you did not set the Options property to something different than the default. That means SetOptions doesn’t get called, which was the only spot where TPerlRegEx set the PCRE_UTF8 flag.

The new release of TPerlRegEx fixes this by adding these five lines to the constructor:

{$IFDEF UNICODE}
  pcreOptions := PCRE_UTF8 or PCRE_NEWLINE_ANY;
{$ELSE}
  pcreOptions := PCRE_NEWLINE_ANY;
{$ENDIF}

This makes sure TPerlRegEx tells PCRE to use UTF8 when TPerlRegex uses UTF8String.

When using the buggy version of TPerlRegEx with Delphi 2009, this subject:

PerlRegEx1.Subject := '€';

will match ^.{3}$ but not ^.$.

When using the corrected version of TPerlRegEx, ^.{3}$ fails while ^.$ matches.

The reason is that in UTF-8, the euro symbol is encoded as three bytes. When PCRE operates in UTF-8 mode, the dot matches one Unicode code point, regardless of how many bytes that code point is encoded with in UTF-8. (All code points take between 1 and 4 bytes.) But when the PCRE_UTF8 flag is not set, PCRE operates in 8-bit mode, and the dot always matches one byte.

In Delphi 2007 and prior, ^.$ also mathes the euro symbol, because it takes up one byte in the Windows (Ansi) code pages.

If you download TPerlRegEx again, it’ll properly handle Unicode in Delphi 2009.