Hands-off by design
Pick USTRING in the DCT, USE it on a control, PUT it to SQL.
No conversion buffers, no “wide mode” flag, no per-screen ceremony.
It just works the way Clarion always did.
Wide text end to end: declare it, type it, store it, browse it.
No new programming model. No special “Unicode mode.” If you already
know CSTRING, you already know USTRING —
and your existing ANSI apps stay byte-identical.
Pick USTRING in the DCT, USE it on a control, PUT it to SQL.
No conversion buffers, no “wide mode” flag, no per-screen ceremony.
It just works the way Clarion always did.
MSSQL, ODBC, PostgreSQL, and SQLite: USTRING(n) ↔ NVARCHAR(n-1)
(logical length round-trips). Keys, GET, SET/NEXT, and bound WHERE values
stay wide — never narrowed literals.
Classic TPS is untouched. New driver: DRIVER('TopSpeedW'),
.tpsw files, USTRING fields + UNICODE blobs, keys that work,
and clean version rejects instead of silent mojibake.
On SQL there is no MEMO(n) — long text is the blob.
TEXT,USE(blob) displays and edits;
IMAGE,USE(blob) displays. Window binding ships today;
Report is the same mental model (next surface).
Add UNICODE to a REPORT and wide text prints end to end —
preview, print-after-preview, direct print. A report left un-opted
posts a loud error 546 on wide data — never silent mojibake.
Source (UTF-8 / UTF-16), Window/Report designers, DCT messages/tips, IME, emoji panel, clipboard. Window titles, menus, POPUP(), LIST columns and headings, captions and free-form ENTRY/TEXT/COMBO carry exact UTF-16 on any system codepage.
No USTRING? No wide designer text? Rebuild and ship — same binary
behavior as before. Opt-in tokens (e.g. PostgreSQL
UNICODECONNECT=1) never change default connects.
Import Tables: nvarchar → USTRING, ntext /
nvarchar(max) → UNICODE memo/blob, binary max → BINARY blob —
attributes already ticked. Data Browser edits wide without narrowing.
Emoji and non-BMP text edit as atomic characters on wide ENTRY/COMBO. Color emoji via DirectWrite where available; ordinary edits never leave half a pair behind.
Garbled text from an encoding mismatch — e.g. UTF-8 bytes read as Windows-1252.
Classic example: café instead of café.
Converting wide (UTF-16) text down to ANSI/8-bit. Characters with no mapping
in the target code page are lost or replaced (often ?).
Lossless round-trips exactly. Lossy trades exactness for compatibility. Keep data in USTRING end to end when it must survive.
If you tested the previous build, these are the changes to look at first. Most items trace back to a beta field report or a request on the beta forum — that is exactly what the beta is for; keep them coming.
ReportPreviewClass. Text search across every page with the hits highlighted on the page (red for the current one, orange for the rest; Ctrl+F, F3/Shift+F3, Aa for match case, a clear button), a sidebar of page thumbnails (click one to go there; hidden on a one-page report), page marks (Ctrl+M, Mark Pages With Hits, Clear) with Print Marked Pages and Save Marked Pages As, Fit Page, Ctrl+plus/minus zoom, "100% (actual size)", the wheel scrolls a zoomed page and the highlights follow, Shift+drag over a page copies the text under the band to the clipboard (the status bar says "Text copied to Clipboard"), and the zoom and sidebar state come back next time through the INI. The text index is built on its own thread as soon as the preview opens, so a 1,000-page report opens at once and a search typed early runs when the index arrives. On a wide report (REPORT,UNICODE) the search and the copied text are Unicode — Ω, an emoji, any script; on a classic report it is ANSI text, as the page was written. The Report procedure's Preview Options tab adds Show the Find bar, Show the page thumbnails sidebar, Remember the zoom and the sidebar, and Fit Page.PrintPreviewClass (the shipped previewer of every earlier release) generates ReportPreviewClass; a derived or third-party class named there is left alone. "Keep the classic PrintPreviewClass previewer" (Global Properties ▸ Classes ▸ Process & Reports, off by default) puts the old one back. The Clarion (legacy) template chain has the same checks (Global Properties ▸ Classes ▸ Reports) and maps PrintPreviewClass to the modern previewer on its ABC Class preview mode.Report{PROP:Unicode} = True right after its report opens, so a report that never carried the UNICODE attribute prints its USTRING fields, wide captions and emoji as typed on .emf pages, error 546 stops, the PDF/HTML/TXT targets register their wide face, and the previewer's search is Unicode — 150 reports across 20 apps migrate with one regenerate each. A Report procedure's "Unicode pages" prompt (General tab) overrides it for the one report that feeds a third-party WMF-only tool. What to watch for: a third-party page tool that reads .wmf files sees .emf files now; error 546 no longer fires where it used to.report_unicode define. The same check writes the compiler's define into the project (report_unicode=>on under Project Properties ▸ Compiling ▸ Defines), so every REPORT structure in the application — hand-coded modules included — carries the UNICODE attribute when it compiles. A hand-coded program sets the define itself, or writes PRAGMA('define(report_unicode=>on)') at the top of a module; a WINDOW is never affected, and Report{PROP:Unicode} = False before the first PRINT opts one report out.PROP:Unicode on a REPORT. The run-time face of the UNICODE report attribute is now documented: Report{PROP:Unicode} = TRUE after OPEN and before the first PRINT makes a report declared without the attribute print as a UNICODE report (EMF pages); reading it returns TRUE for any report that prints Unicode, by attribute or by property. A write after the first PRINT is ignored (latched). The undocumented 7A82h number keeps working as its hidden twin.ABPREVIEW.INC/.CLW beside a GUI-free PageTextIndexClass (the per-page text index that third-party viewers can use as-is; every decision point is a VIRTUAL). The ReportManager drives it through a new IReportPreviewer interface (its previewer parameter is typed PrintPreviewClass, so a sibling class needs the interface arm — the template emits it). Three new stock icons ride along: ICON:PageUp, ICON:PageDown, ICON:Clear (the Window Designer's Icon list has them). The EMF Transition Note has the details.ABPRPDF.CLW compiles without warnings, and long Unicode PDF text is whole. Three Windows API prototypes in the PDF generator lacked PROC, and one local was declared twice under one label (LOC:Units / LOC:UnitS — labels are case-insensitive). The duplicate was a real defect: the compiler used the STRING(2) declaration for the unit count, so a wide report text longer than 99 units came out in the PDF with a two-digit count (150 units drew as 15). Fixed in the LibSrc class; no application change.DATETIME is a new data type: a date and a time of day in one value, precise to 100 nanoseconds. The value is the number of seconds since midnight, December 28, 1800 (the Clarion Standard Date zero) with seven decimal places, so it carries the full precision of SQL Server's datetime2(7) and of a .NET DateTime; the whole DATE range, 1801 through 9999, is exact. Zero is the blank date-time, as for DATE and TIME. Two values compare and sort in time order, a KEY on a DATETIME field orders the file by it, CLEAR blanks it, and arithmetic works in seconds (+ 3600 is one hour, + 86400 a day; the difference of two values is the seconds between them). Declare it in source as Modified DATETIME,NAME('modified') (the NAME is optional: the column name is the label); &DATETIME references and *DATETIME parameters work, by value is refused. Under the hood it is a DECIMAL(19,7) (PDECIMAL on Oracle) whose external name the compiler tags | DATETIME — nothing to write by hand; a hand-tagged DECIMAL from the earlier build behaves identically.CREATE() declares it: MSSQL datetime2(7) (also reads and writes datetime2(0-6), datetime and smalldatetime; the fraction is truncated to the column's scale, never rounded — the Microsoft ODBC driver refuses an over-scale fraction), ODBC/PostgreSQL timestamp(6), SQLite DATETIME stored as the ISO text 'YYYY-MM-DD HH:MM:SS.fffffff', Oracle TIMESTAMP(7) (Oracle rounds). A blank (zero) value is written as NULL and a NULL reads back as zero. A comparison against a DATETIME field in a VIEW's PROP:Filter is sent to the backend as a timestamp literal and evaluated server-side (BIND the record, as for any VIEW filter). Known limitation: a legacy SQL Server datetime column is a 1/300-second grid, so WATCH/PUT on such a column matches only grid-exact fractions — new columns should be datetime2.@DT picture formats and enters a DATETIME: @DT shows the whole seconds (2026-09-09 13:05:07, 19 characters), @DT1 through @DT7 add decimal places of a second (@DT7 = every digit, 27 characters; digits beyond n truncated). Always the ISO order with a hyphen, a space and colons, so the text sorts and every SQL backend understands it; a zero value displays blank. On an ENTRY control and in DEFORMAT the picture accepts the displayed form, the T form (2026-09-09T13:05:07.5), fewer decimals, or the date alone. STRING(@DT7) declares a 27-byte string. For the date or time part alone in any @D/@T picture, take the parts with DateTimeUtil.DateTimeUtil (INCLUDE('DATETIME.INC'), LibSrc) converts between a DATETIME and the DATE and TIME parts: Set(dt, date, time), SetEx(dt, date, secOfDay, ticks), SetNow(dt) (the precise system clock — 100 ns, not the hundredths of CLOCK()), GetDate, GetTime, SecOfDay, Ticks, AddSeconds, Diff(a, b) in seconds, Text(dt) and Parse(dt, text); every method has a *PDECIMAL twin for the Oracle form of the carrier.@DT7), the generator lowers it (DECIMAL(19,7) or PDECIMAL(19,7) on an Oracle table, with the tag), DDL from the dictionary emits the backend type per driver, DCTX/TXD round-trip it, and the synchronizer compares DATETIME against DATETIME. The SQL import wizards get a checkbox, "Import date-time columns as DATETIME", default OFF: off keeps today's four-field GROUP shape, so existing dictionaries and re-imports see no change; on creates one DATETIME field per timestamp column. The checkbox is on the connect page as well as the options page, because the Import path skips the options page. The IDE Data Browser shows a DATETIME column as a date-time (@DT7), not as a raw decimal, and the source editor colours and completes the keyword.DYNAMICFILE) reports the tag through WHO(), which DynFile.GetField now allows for.STRING(8) with a DATE/TIME group over it) is created as datetime2(7); the driver only knew the 2005 profile and emitted the group's byte length as the scale on every later server, "Specified scale 8 is invalid". Not new to this beta, but found by it.RUN() takes a USTRING command line. A program (or document) whose path holds characters outside the Windows ANSI code page is located and started exactly, quoted or not, with or without .exe, and the parameters travel as typed; RUNCODE() and ERRORCODE() behave as for a narrow command line. A USTRING whose characters all fit the ANSI code page takes the old path byte-for-byte, so nothing changes for existing command lines. This was the last narrow member of the wide file-name family: EXISTS, COPY/RENAME/ REMOVE, DIRECTORY(FILE:QueueW), FILEDIALOG, the PATH family and RUN are all wide now.COMMAND() for a program that lives in a folder outside the ANSI code page. Found by the gate for the item above: such a program saw its own path in COMMAND(1) and every real parameter shifted by one, because the runtime could not recognise the first command-line token as the program (the ANSI name best-fits to ?, an invalid file-name character). The program name is now matched wide first. If you keep test projects in folders named with a non-ANSI character (the ₿ folders from the forum), this affects any exe you start from there with parameters.CLIB.CLW stays narrow by design — four wide twins are exported. The 1995 low-level MAP over the TopSpeed C library is *CSTRING throughout and gets no W twin of the file: for USTRING text use the wide builtins (LEN, INSTRING, UPPER/LOWER, VAL/UVAL, EXISTS, COPY/RENAME/REMOVE, DIRECTORY(FILE:QueueW), the PATH family, RUN). The four file-name helpers that Clarion sources declare inline do get exported twins in ClaRUN, taking USTRING buffers of the same declared sizes: FnSplitW(*USTRING,*USTRING,*USTRING,*USTRING,*USTRING),SIGNED,RAW,NAME('_fnsplitW'), FnMergeW(...),RAW,NAME('_fnmergeW'), MkDirW(*USTRING),SIGNED,RAW,NAME('_mkdirW'), AccessW(*USTRING,SIGNED),SIGNED,RAW,NAME('_accessW'). Same flags and quirks as the originals (gated for parity). The header of CLIB.CLW carries this note and the SIZE(v)/SIZE(v[1]) buffer-length rule.ALL() with a negative count returns '' again. A padding count that goes below zero — ALL(' ', column - LEN(text)) with the column short of the text — reached the runtime as a huge unsigned number and the program died with "Clarion RTL internal exception" parameter 0B000000. C11 and build 14000 return an empty string, the same as a zero count; that contract is restored. LEFT, RIGHT, CENTER and SUB with negative counts were measured unchanged against 14000. Not a USTRING defect — the USTRING prototypes in the report were a coincidence.IMAGE,USE() accepts since the "a blob is a field" round: BLOB fields and MEMOs marked BINARY (not UNICODE ones). Dropping a BLOB field from the dictionary onto a window now creates an IMAGE bound to it, and the dictionary's control list offers IMAGE for a BLOB. The Report Designer's IMAGE keeps its plain text box: the compiler does not accept IMAGE,USE(blob) inside a REPORT.LIKE(blob) is illegal in every Clarion version; a MEMO column is fine), so the list-box formatter's field picker no longer lists BLOB fields. An app that already carries one now stops generation with a plain sentence naming the field instead of the compiler's "Illegal parameter for LIKE".UNI now finds Unicode like uni did; the old rule switched to a case-sensitive match as soon as the typed text held an upper-case letter. One shared pad, so Window Designer, Report Designer and the other property pads all change together.UNICODE is coloured as an attribute in the source editor. The keyword was missing from the editor's attribute list and showed black.DATETIME, the Date-Time Pictures (@DT), ReportPreviewClass, PageTextIndexClass and the Report template's Preview Options; PROP:Unicode and the report_unicode define on the REPORT pages; the "Unicode reports" and "Unicode pages" prompts; and the three stock icon equates.
USTRING declares a string of wide characters — 16-bit
UTF-16 code units, the native text form of Windows. A USTRING value is terminated
by a null wide character; the value is self-describing (no space padding).
Name USTRING(24) ! room for 24 wide characters
Alias &USTRING ! reference to a USTRING
USTRING(n) reserves n wide characters including the terminator
— so it holds up to n-1 characters of data (the CSTRING rule).
SIZE() returns bytes (n * 2);
LEN() returns the number of wide characters before the terminator.
USTRING can be initialized and assigned from both Unicode and ANSI sources. Any string or numeric expression may appear on the right side of an assignment to a USTRING — exactly the same as with STRING / CSTRING / PSTRING.
String positions, lengths, and limits for USTRING count UTF-16 code units — the same model Windows and the C wide-character library use. A code unit is not always a whole character on screen:
| On screen | UTF-16 units / LEN() |
|---|---|
A, α, ☃ | 1 |
😀 (emoji) | 2 (surrogate pair) |
👨👩👧 (joined family) | 8 |
🇺🇸 (flag) | 4 |
This is the same contract ANSI LEN always had (bytes under DBCS, not “visible
characters”). Practical consequences:
LEN(U) after typing one emoji can be 2 (or 8). Size fields and columns in units.U[i] indexes one unit. Indexing the middle of a surrogate pair yields half a character.0D800h–0DBFFh).
Unicode assigns every character a number called a code point
(A is U+0041); a UTF-16 code unit is one 16-bit
storage element. The two differ only outside the BMP (Basic Multilingual
Plane) — Unicode's original and most commonly used 65,536-code-point range:
U+0000–U+FFFF) always take one code unit.U+10000–U+10FFFF) take two code units — a
surrogate pair (lead unit 0D800h–0DBFFh, trail unit 0DC00h–0DFFFh;
those ranges are reserved inside the BMP for exactly this job and never appear as characters on their own).| Character | Code point | BMP? | UTF-16 storage |
|---|---|---|---|
A | U+0041 | Yes | 1 unit / 2 bytes |
é | U+00E9 | Yes | 1 unit / 2 bytes |
中 | U+4E2D | Yes | 1 unit / 2 bytes |
€ | U+20AC | Yes | 1 unit / 2 bytes |
☃ | U+2603 | Yes | 1 unit / 2 bytes |
😀 | U+1F600 | No | 2 units / 4 bytes |
𐐀 (Deseret) | U+10400 | No | 2 units / 4 bytes |
𠀀 (CJK Ext B) | U+20000 | No | 2 units / 4 bytes |
Text: A😀B
Unicode code points: U+0041 U+1F600 U+0042
UTF-16 code units: 0041 D83D DE00 0042
Three characters, four code units — LEN() returns 4.
Do not think of BMP as “normal characters” and non-BMP as “emoji”: some emoji are BMP and
take a single unit (☃, ☺), and many non-BMP characters are not emoji
at all (historic scripts, mathematical alphabets, supplementary CJK ideographs). Everyday
Latin text, punctuation, currency symbols and the common CJK range are all BMP. And
separately from surrogate pairs, one glyph on screen can be built from
several code points — the joined family in the table above is five code
points (man, joiner, woman, joiner, girl) totaling eight units; a flag is two code
points / four units.
The rule to test by: never assume one 16-bit element is one character. Code that indexes, truncates, copies, or measures UTF-16 text must keep surrogate pairs together, and a round trip must preserve both halves of every pair — no replacement characters, no truncation, no split pairs.
?). This is the only lossy direction.IF U).U'...', and wide characters via UCHR(code) (new in this refresh — the single-parameter form) or the equivalent CHR(code, 1) (e.g. UCHR(9731) = ☃). Both take the full Unicode range: an astral code point (10000h–10FFFFh) returns the surrogate pair — two UTF-16 units — so UCHR(1F4A9h) is 💩 directly, no manual D800h/DC00h math. 0–FFFFh returns exactly one unit verbatim (lone surrogates pass through, so pair assembly from two calls stays legal); outside 0–10FFFFh returns U+FFFD (�). Constant code points cost nothing at runtime (compile-time fold).U'...' (extended in this refresh): values are UTF-16 units in decimal — U'<937>' = Ω — and now also in hexadecimal with the trailing h and a leading decimal digit, like any Clarion numeric constant: U'<0D83Dh,0DCA9h>' is a surrogate pair written as its two units. A value from 10000h through 10FFFFh (decimal or hex) is read as a full code point and inserts its surrogate pair by itself — U'<1F4A9h>' is 💩 in one number — and a repeat count repeats the whole pair (U'<1F4A9h{3}>' is three of them). A value beyond 10FFFFh, or hex digits without the h, is a loud compile error — a Unicode constant is never silently truncated.<n> values are bytes — they always were — and a value above 255 silently keeps only its low byte, so '<55357>' is not a surrogate: it's the single byte 3Dh (=). Surrogates and wide units cannot be written in a plain literal at all — reach for U'...' or UCHR()._USTRING_ (=>on, beta refresh after 08-08-2026) beside the classic _C120_/_WIDTH32_ flags — code shared with older Clarion versions can gate its wide paths with COMPILE('**u**', _USTRING_) / OMIT('**u**', _USTRING_). Name matching is case-insensitive like the rest.&USTRING references, NEW/DISPOSE, THREAD, DIM'd arrays, GROUP fields, EQUATE initializers and OVER all work as for other string types (OVER exposes the raw UTF-16 byte image).MyProc(USTRING u)) are writable private copies, exactly like value STRING parameters — assign to them freely, changes never reach the caller. (Earlier beta builds made them read-only — error 241 on assignment. Note the value form is a USTRING capability: a value CSTRING parameter remains illegal.) And &USTRING reference assignment (Alias &= U) is fixed — in earlier beta builds it corrupted the stack frame and crashed at procedure exit; &= NEW USTRING(n), DISPOSE and passing the reference on now all behave.U[i] works as rvalue and lvalue; range slice U[a : b] is rvalue only — rejected as lvalue at compile time.CASE with wide OF literals and ranges works.Recommended: PROP:Codepage + PROP:Locale — three equivalent ways:
SYSTEM{PROP:Codepage} = 1253 ! preferred; also readable back
SYSTEM{PROP:Locale} = 1032
LOCALE('CLACODEPAGE','1253')
LOCALE('CLALCID','1032')
! or in the auto-loaded <exe>.ENV file:
! CLACODEPAGE=1253
! CLALCID=1032
Named values work: 'GREEK', 'WESTERN', 'UTF8' for codepage;
'EN-US', 'DE-DE' for LCID. Empty value resets to system defaults.
A flip is process-wide and immediate for new conversions/comparisons. Values converted
earlier keep their bytes. Legacy CLACASE/CLACOLSEQ/CLADIGRAPH still work — last writer wins.
A = TOANSI(expr [, cp]) ! result is ANSI, encoded at cp
U = TOUNICODE(expr [, cp]) ! result is wide; cp says how to READ ANSI input
Omitted cp = current PROP:Codepage. UTF-8 works as an explicit codepage
(65001 / 0FDE9h):
U8 = TOANSI(U, 0FDE9h) ! USTRING -> UTF-8 byte string
U = TOUNICODE(U8, 0FDE9h) ! UTF-8 byte string -> USTRING
The compiler reads .clw/.inc in mainstream Unicode formats — write U'Κλόουζ', mixed-script literals, or emoji directly:
!UTF8 as the first line, or project pragma module(encoding=>utf8).U'...' literals carry exact Unicode. Plain literals remain ANSI carriers; non-ANSI text needs U'...'. The same split governs angle-bracket codes: in a U'...' literal they are UTF-16 units / code points; in a plain literal they are bytes, and values above 255 silently truncate (see the plain-literal trap in §3). Exception: plain caption literals inside WINDOW/REPORT keep exact Unicode end to end.Exactly two encodings (Tools ▸ Options): Clarion ANSI (default — existing files round-trip byte-identical) and Clarion Unicode (UTF-8 with BOM). The option is the default for new files only; the status bar shows the current file’s encoding. Type a non-codepage character into an ANSI file and the editor offers to switch to UTF-8 at save. The editor never writes a file the compiler cannot read.
Type your language into captions, titles, messages, tooltips and report strings — nothing to enable.
Save, reopen, generate: exact round-trip and correct draw on any system codepage. Internally,
non-ANSI designer strings travel as readable ASCII U'...' metachar form
(e.g. BUTTON(U'B<233,20013,937>') for Bé中Ω). Apps without wide designer
text stay byte-identical. Older IDE builds cannot open an .app that uses wide designer text.
Every user-text field property (Message, ToolTip, Prompt, Column Heading, Description,
Initial Value, Choices/Values, True/False) accepts Unicode and round-trips into the running app.
The .dct format is unchanged. ANSI-only dictionaries are byte-identical.
Generated code from the native template engine is always ANSI. Non-ANSI text typed into an
embed cannot survive as raw characters — write U'...' with unit
metachars, or put the code in a UTF-8 INCLUDE file.
A TXA is ANSI text in the system codepage of the machine that wrote it — it never carried an encoding marker, so regional characters (é, Ω, Cyrillic…) were ambiguous on any other machine, and editing one in a UTF-8 editor was a mojibake trap. Two changes, both in the app generator only:
é for é) is read as UTF-8 and comes back repaired./u8 modifier before /ax —
ClarionCL /u8 /ax app.app app.txa; for the AutoExport on save lane
(Tools ▸ Options ▸ Import/Export ▸ Apps) tick Export as UTF-8 (with BOM). The file comes out as
the same content in UTF-8 with BOM; every default export is byte-identical to before. (The interim
CLATXAUTF8 environment variable from the previous refresh is gone.)
The boundary: the .app stores text in the system codepage, so an imported TXA may only contain
characters that codepage can represent. If an edit introduces something outside it, the import
stops with an error naming the line rather than silently mangling text. Real Unicode in string
literals and designer captions still travels as the U'...' metachar form described above — this is
an encoding lane for the file, not a change to what the TXA contains.
Fixed alongside: a prompt value containing a literal < followed by digits (e.g.
x<10>y) used to export unescaped, and the next import turned <10> into a
control character. It now re-escapes ('x<<10>y') and export/import cycles are stable.
Pick USTRING from the field-type list (driver-admitted — only drivers that support wide columns),
set Characters. USTRING(n) = n-1 data characters, 2n bytes on the record.
Generation emits it verbatim. The DCT explorer tree shows the size (USTRING(21)).
| Server column | Imports as |
|---|---|
nvarchar(m) / nchar(m) | USTRING(m+1) |
varchar(m) | CSTRING(m+1) |
char(m) | STRING(m) |
nvarchar(max) / ntext | MEMO/BLOB + UNICODE |
text | MEMO/BLOB (narrow, by design) |
image / varbinary(max) / PG bytea | MEMO/BLOB + BINARY |
binary(m) / varbinary(m) | STRING(m) (fixed-width) |
bigint / PG int8 | DECIMAL(19,0) (lossless) |
uniqueidentifier | CSTRING(37) (+READONLY when server-filled) |
STRING(n) is deliberately fixed-width (space-fill on read; full n with pad as data on write). Use max/BLOB when lengths vary.UNICODECONNECT=1 to the Datasource so wide types import as USTRING / UNICODE memo. Clarion-owned token — stripped before the driver; connects without it are byte-identical. MSSQL never needs this.bigint/int8 imports as DECIMAL(19,0) and the full 19-digit value round-trips on every driver lane — earlier builds imported LONG, which silently truncates past 2³¹.@s<n-1>, the CSTRING rule). (Early-beta builds defaulted every USTRING to @s20 — fixed.)
DDL: MSSQL script generator emits NVARCHAR(n-1) for USTRING(n),
NVARCHAR(MAX) past the 4000-unit cap (you run the script yourself).
Round trips: USTRING survives DCTX, TXD and TXA export/import (the TXA optionally in UTF-8 — see §5); app-pool variables accept it.
FORMAT/DEFORMAT and picture tokens accept USTRING values and wide runtime picture strings.
USTRING(@pic) sizes as picture width plus terminator and formats on store.
One deliberate difference: a pictured USTRING QUEUE field sorts by raw stored text,
while a pictured ANSI field sorts by the deformatted value — sort wide date fields on a LONG key for chronological order.
Pictures on controls carry Unicode too (beta refresh after 08-23-2026):
a picture whose literal text contains characters outside the codepage — a currency symbol like
@n~₿~-15.2, a wide date separator — can be set on a LIST column
({PROPLIST:Picture}) or on a pictured display STRING control and formats every value
with the real symbol; reading the picture back returns it intact. Previously the picture was stored
narrow before the (already wide) formatter ever saw it, so anything outside the codepage became
?. Pure-ANSI pictures take exactly the old path. The pictured-ENTRY
editor is the one surface that still presents ANSI (see §8, "known limitations"). (Beta refresh after 08-27-2026): the same holds for a
picture written in the LIST's own FORMAT() string (FORMAT('…@n~₿~-15.2@…')) —
before, only a picture set at runtime kept its symbol.
@Sn or no picture) + USTRING USE: IME, emoji panel, clipboard — exact UTF-16. Limits in units. {PROP:ScreenText} is wide. (Beta refresh after 08-26-2026: keystrokes that carry UTF-16 — KEYEVENTF_UNICODE/VK_PACKET injectors, the emoji panel on Windows builds that type rather than insert, IME-composed CJK — also arrive exact: the message pump is Unicode. Earlier builds best-fit those keystrokes to O??? even into a USTRING entry. ANSI-bound controls receive the bytes they always did; KEYCHAR() keeps its ANSI value.)|) and column headings now draw real glyphs, not best-fit. (FROM(ustring) as LIST source is not accepted — use a wide QUEUE.) Residual: a heading baked into the FORMAT() literal can still narrow — set it at runtime for the wide path; queued. A runtime-set wide heading also now survives a later reformat of the LIST (e.g. after a column-picture change) — in earlier beta builds the reformat wrecked it (beta refresh after 08-23-2026). Column pictures with non-codepage symbols work too (see §7).{PROP:Text}) and MENU/ITEM text carry exact UTF-16, any codepage, themed or classic; reading a caption back is wide. Runtime writes to the status bar ({PROP:StatusText}), message zone and tips are wide too. (These were previously listed as ANSI limitations — this refresh changed that.){PROP:Use} now switches the control onto the wide class on the spot (and back for a STRING) — earlier builds widened only for a compile-time-declared wide USE. This is what makes TopScan's in-cell editor go wide with no tool changes.CONTENTS() / ScreenText / Line,n are wide.TEXT,...,RTF + USTRING USE (beta refresh after 08-09-2026): displays, edits and saves back wide. The variable follows the content: plain text stays plain (exact UTF-16); rich content saves as RTF source text — 7-bit, non-ANSI riding standard RTF escapes (\uN / charset byte escapes), lossless round trip and portable to Word/WordPad, emoji included. Emoji are stored exactly (pairs never tear) but the hosted RichEdit draws them monochrome — the plain-TEXT OS floor. RTF(TEXT:FILE): file content carries Unicode via escapes; a non-ANSI filename does not resolve (ANSI lane — known limitation); reading the control back through {PROP:Text}, {PROP:Line} or the RTF templates' GetText returns best-fit ANSI text — the variable itself keeps the units (known limitation). Pre-refresh, this binding silently showed empty and discarded edits; stored values were never damaged.{PROP:Text} = <wide value> on any of these families keeps exact units — previously a runtime caption rewrite could narrow (it was on the known-limitation list).|, {} sub-menus, ~, +/-, [...]) is decided on the wide text itself, so a full-width | (U+FF5C) inside an item no longer splits the menu the way its best-fit | once did. Return values, separators, grayed/checked items and sub-menus are unchanged; an all-ANSI POPUP takes exactly the old code path.US). (Beta refresh after 08-09-2026: a color emoji at the right edge of an ENTRY was clipped in earlier beta builds — fixed.)Pictured ENTRY (@N/@D/@T...), SPIN, report-hosted TEXT: the wide value is never corrupted by display — presentation may show ? for ☃, but the variable keeps exact text. If the user edits, the stored result is the narrowed edited text (they edited the ANSI presentation).
This list keeps shrinking — this refresh moved runtime caption rewrites, POPUP menus and control pictures to the wide surfaces (the previous one moved window titles, menu text, the status bar, LIST split columns/headings and runtime-bound USE). What remains: the pictured-ENTRY editor — an ENTRY(@pic) edits through the ANSI presentation, including a picture whose literals carry wide symbols (ENTRY(@n~₿~...) shows ? while editing); the in-place editor owns caret movement, digit slots and literal skipping per character, so its wide form is a deliberate separate step — queued (free-form ENTRY(@s..) is fully wide); setting a wide picture at runtime on ENTRY/SPIN/COMBO via {PROP:Text} can still narrow (the display-control and LIST-column picture lanes are the wide ones — §7), and a wide caption write before OPEN(window) best-fits — queued; a heading baked into a FORMAT() literal and and lookup-key / name arguments (an icon name, a non-ANSI filename in the remaining ANSI file-name lanes — DIRECTORY() into a FILE:Queue, RUN) which stay ANSI by nature. Values always survive.
(REPORT printing is no longer on this list — see Reports below. MESSAGE() is no longer on this list either — beta refresh after 08-08-2026: the text, caption and button captions display wide, including custom |-list button captions and the copyable CANCOPY form; astral emoji render via surrogate pairs. The icon name argument is a lookup key and stays ANSI — a non-ANSI icon filename does not resolve. Plain ANSI MESSAGE calls are unchanged.)
BSTRING is COM marshaling, not an entry type — stage input in USTRING and convert at the COM boundary.
UNICODE attribute
Add UNICODE to a REPORT statement and the report prints wide text end to end —
USTRING fields, wide captions, TEXT, LIST cells, string tallies, page numbers — through
preview, print-after-preview, and direct printing:
Rpt REPORT,AT(500,500,7000,4000),FONT('Arial',10),PREVIEW(PgQ),THOUS,UNICODE
Things to know and test:
.emf (not .wmf). The built-in previewer and printing handle them unchanged; a third-party preview tool that expects WMF files will not read them — which is why the attribute is explicit opt-in, never automatic.IReportGeneratorW — six methods, opt-in): EMF Page Files — Transition Note for Report Tool Authors (shipped alongside this guide).ERRORCODE() after PRINT.charset=utf-8 (including the per-page files of a multi-file export, which previously carried no declaration at all) and carries your characters as UTF-8; XML declares encoding="UTF-8" — and the label is now true — with UTF-8 values. Nothing to configure: the report-output templates register those targets wide exactly when the report carries the UNICODE attribute; regenerate the app and export. A report without the attribute — and every existing app — exports byte-identically as before.,UNICODE also survives the designer round-trip and APP generation. (Beta refresh after 08-14-2026: reopening a saved ,UNICODE report in the designer previously failed with "Unexpected attribute 'UNICODE'" — the structure validator did not know the attribute. Fixed.).emf pages could overwrite the first's, and closing one could delete the other's (on the WMF lane pages were silently dropped). Names are now unique across the whole process. Worth a look if you preview two reports concurrently.PROPPRINT:frompage / topage + PROP:FlushPreview) now prints the selected pages — a long-standing defect (silently printing nothing) was fixed in this build. Worth a quick test with your own preview code if you use ranges.TRN strings included. Worth testing with your own printer driver. In HTML exports the wide lane carries emoji as real UTF-8 characters in the string's real color; a narrow-lane export shows ?? placeholders. Without DirectWrite the page prints the previous form: monochrome outline emoji, exact values preserved.TRN string could show beside an emoji on a colored fill.
On a SQL backend there is no MEMO(n): long text and binary content are both the BLOB surface,
and a FILE BLOB is a first-class USE variable:
FT FILE,DRIVER('MSSQL','...'),OWNER('...'),NAME('dbo.Notes'),PRE(NT)
PK KEY(NT:Id),PRIMARY
Body BLOB,NAME('Body'),UNICODE ! nvarchar(max) - wide long text
Photo BLOB,NAME('Photo'),BINARY ! varbinary(max) - image bytes
Rec RECORD
Id LONG,NAME('Id')
END
END
W WINDOW('Note'),AT(,,300,200)
TEXT,AT(4,4,200,120),USE(NT:Body),HVSCROLL ! displays AND edits
IMAGE,AT(210,4,80,60),USE(NT:Photo) ! displays, read-only
END
BLOB,UNICODE → true Unicode TEXT end to end. Writeback is modified-gated and whole-content; size follows content. Store at ordinary USE points; PUT() persists.TEXT,USE(blob,BINARY), IMAGE,USE(blob,UNICODE), BLOB,UNICODE,BINARY, slices as USE variables.BLOB,UNICODE, SIZE(), {PROP:Size} and slice bounds count UTF-16 units; plain/BINARY count bytes. Record fields keep byte SIZE().NT:Photo{PROP:Size} = LEN(bmpBytes) ! size FIRST
NT:Photo[0 : LEN(bmpBytes) - 1] = bmpBytes
PUT(FT)
Performance tip: avoid AUTO on windows bound to large blobs (AUTO compares USE variables every event pass).
Window formatter / AppGen don't offer blobs in USE pickers yet (type USE by hand).
IMAGE save-back does not exist (write-in is assignment). Report-hosted controls do not
bind blobs. MEMO(n),UNICODE on a control is a compile error (the ISAM memo
display floor — on SQL, use the blob).
The two have always been different animals, and the Unicode work follows that grain rather than fighting it:
MEMO(n) is a bounded text field. It has a
declared cap, is fetched with the record, and is a first-class string operand in the
language — you can concatenate it, MATCH against it, assign it in
expressions. It is an ISAM-format concept: the TopSpeed memo store is where it
naturally lives — in fact the SQL drivers have never accepted a sized
MEMO(n) declaration at all (error 47 at open, "Driver does not
support MEMOs"; long content on SQL was always the blob). That language
citizenship is why a wide MEMO would be a large compiler project — and it is
deliberately not planned: MEMO(n),UNICODE is a loud
error by design, never a silent narrowing.BLOB is an unbounded entity. No declared cap,
never part of the record buffer, and deliberately outside the expression language —
everything goes through slices, {PROP:Size} and USE binding. That opacity
is why wide BLOB support is already complete: storage, transport,
display (this section), reports/exports and the Data Browser all handle
BLOB,UNICODE today.MEMO(n) has no
natural SQL shape (a fixed-cap text sidecar maps to nothing a server offers), so long
text is BLOB,UNICODE against nvarchar(max)/NCLOB
and binary is BLOB,BINARY against varbinary(max) — the blob
is the long-content surface, and the import wizard maps those columns to
blobs for you.BLOB,UNICODE works today on the
TopSpeedW driver; MEMO(n),UNICODE rejects loudly at open — by
design, permanently. Capped wide text that fits the record belongs in a
USTRING(n) field; anything bigger is the blob. Existing narrow
MEMO(n) fields are untouched either way.Rule of thumb: new work takes the blob; MEMO(n) remains what it has always
been — the ISAM-native narrow capped text field of the existing
application base. It is not being dragged into the wide future; the blob (and the
USTRING(n) record field) already are the wide future.
USTRING(n) fully supported. SQL Server: nvarchar(n-1) (UTF-16 at rest). PostgreSQL: varchar(n-1) (UTF-8 at rest, converted at the driver). SQLite: NVARCHAR(n-1) (UTF-8 at rest, engine-converted). Logical length round-trips — write 4 characters, read 4 characters. Code points exact.UNICODECONNECT automatically./ENCRYPT and /TRUSTSERVERCERTIFICATE=TRUE (checkboxes in the Driver String Builder). On an interactive connect against a self-signed server without the flag, the driver now asks “Trust the server certificate and reconnect?” — Yes connects on the modern driver; No preserves the old flow. Unattended configs never see a dialog.'' is not a valid switch error), and switches placed after a two-value /LOGFILE=file,n no longer get swallowed into the log filename.
UTF-8 at rest; driver transports UTF-16; engine converts. Wide columns and UNICODE memos are ordinary
UTF-8 TEXT — readable by any SQLite tool. Narrow STRING/CSTRING columns store your ANSI bytes verbatim
(byte-faithful round trip, but not valid UTF-8 to other tools — prefer wide columns when data must
interoperate). Import maps NVARCHAR/NCHAR → USTRING, NCLOB/NTEXT → UNICODE memos, BLOB → BINARY.
Sorting is code-point order; NOCASE folds ASCII only. Blob shrink stores exactly; standalone TIME
fields store HH:MM:SS[.hh] text and round-trip.
Beta refresh after 08-10-2026: imported keys now carry NOCASE when the underlying
SQLite index is declared with it (primary keys stay case-sensitive); a {PROP:SQL} SELECT that
returns no rows now describes its columns instead of failing with error 33; and 64-bit INTEGER columns
round-trip exactly through DECIMAL(19,0) — earlier builds carried them through a double,
rounding values past 15–16 digits and altering the stored type on write.
/UTF=8 or /UTF=16LE (Driver String Builder has a UTF group); default UTF-8 with BOM — self-describing, never silently narrowed. A BOM on an existing file wins. Mixing narrow and wide fields rejects with a reported error. Fields are fixed-width columns of n-1 units.42,"Bé中Ω"). USTRING next to a narrow string field rejects. UTF-8 only (UTF-16 is not a CSV encoding here). CSV quoting conventions hold for wide values.2n bytes of raw UTF-16LE verbatim; mixing is fully defined. OEM attribute rejects on USTRING records.
Unicode reaches TPS through a separate driver: DRIVER('TopSpeedW'), default extension
.tpsw. Classic TopSpeed, existing .tps files, and ANSI apps are completely untouched.
é sorts after z.BLOB,UNICODE — sizes/slices in units. MEMO(n),UNICODE is not supported — by design, permanently — and rejects loudly at open (see "MEMO versus BLOB" in the blob section for the rationale)..tps files as they are (see the big-file section below for upgrading them in place). The stamp is decided at CREATE and never changes behind your back. (Early-beta note: .tpsw files from builds before the big-format drop use older stamps — re-create them; both drivers refuse them loudly, nothing opens silently.)*.tpsw — fields arrive as USTRING with sizes, keys with NOCASE/descending, wide blobs already marked UNICODE), Data Browser, project references by name..tpsw files (beta refresh after 08-12-2026): TopSpeedW is in TopScan's driver list, and the Open dialog picks the right driver from the file you select. USTRING fields display and edit — the value is stored wide and round-trips bit-identically at rest; the browse list and edit box themselves still render through the ANSI codepage (characters outside it show as lookalikes on screen — a display floor only, never a data change). TopScan also moves to a Segoe UI default font (your own saved font choice is kept).
The classic .tps format tops out at 2 GB, and Clarion now guards that cliff instead of corrupting
the file: an ADD/APPEND/PUT that would grow past the limit is refused with err 90 / FILEERRORCODE 8281–8283,
"TopSpeed file size limit (2GB) reached". A full file is not a corrupt file — reads keep working, DELETE and
shrinking PUTs stay legal in the same session (free space and ADD works again), and CLOSE commits normally.
TopSpeedW-created files use the new big format and reach 512 GB (same guard family at that boundary, with the 512GB message). Everything else is unchanged: same record / key / memo / BLOB limits, same tools story.
To lift an existing small file (typically a classic .tps you are adopting through TopSpeedW — a driver
swap alone never changes a file's format):
OPEN(MyFile, 12h) ! exclusive: deny-all is required
Ans = SEND(MyFile, 'BIGFILE') ! query -> 'OFF' (small) / 'ON' (big)
Ans = SEND(MyFile, 'BIGFILE=ON') ! in-place upgrade, all tables, atomic
What to test: grow a file past 2 GB under TopSpeedW (big format — should just work); hit the 2 GB guard on a
classic-driver file and confirm the message + that deletes still work; run the BIGFILE upgrade on a classic-created .tps
(keys and old rows must survive, BUILD must work after); confirm an upgraded or big file gives the clean version error in an
old-driver app rather than anything silent.
This build also fixes two long-standing crash-recovery defects — in classic TopSpeed as well as TopSpeedW. A TPS file interrupted mid-commit (power loss, kill, crash) recovers from its on-file journal at the next open; previously a session that only read the crashed file could truncate that journal at close (silently changing which side of the interrupted transaction survives), and a write before the recovery was made permanent could leave the whole file unreadable (err 90 / fec 1477 on every access). Now: readers of a crashed file always see the correctly recovered data, see the same data on every open, and leave the file byte-for-byte untouched; the first write completes the recovery permanently. What to test: kill an app mid-update (Task Manager during a heavy batch is fine), then reopen and read several times — same rows every time; then write once, close, reopen — everything intact, no corruption errors.
The in-memory driver (DRIVER('Memory')) is now fully wide: USTRING fields, keys over them, and
BLOB,UNICODE all work, with sizes and slice bounds in units as everywhere else. Key ordering is
linguistic collation under the Windows locale — the same order a keyed QUEUE gives a USTRING
component, so a Memory-driver table and a keyed queue holding the same rows agree. NOCASE keys use the
case-insensitive linguistic fold. Note this deliberately differs from TopSpeedW's code-unit key order: each
driver keeps its native ordering personality (Memory's narrow keys have always been collation-ordered).
One known limitation: MEMO(n),UNICODE on the Memory driver is still accepted silently over narrow
bytes — do not use it; the loud reject (as TopSpeedW already has) is queued. Narrow memos are untouched.
Runtime-defined files were already wide: the field-type list includes USTRING and sizes are in units.
Beta refresh after 08-12-2026 for the DynFile wrapper class: SetFieldValueW
is the wide write twin (the classic SetFieldValue takes a STRING value and narrows — Clarion prototypes
cannot distinguish value STRING from value USTRING, so the wide form takes the W name);
GetField / SetFieldValue now accept both bare and prefixed field labels; and
CacheFile onto a Memory-driver target works (the cached copy comes back open, ready to read).
The client/server IP driver (DRIVER('IPDRV')) now carries USTRING fields and
UNICODE memo/blob end to end — client application, requester service, data server,
to the real backing table — round-tripping bit-identically at rest. Wide keys sort in the backing
driver's order (the data server owns the key, so IP inherits it — e.g. TopSpeedW's code-unit order);
nothing IP-specific to configure. Declare the fields USTRING / the memos and blobs ,UNICODE in the
shared dictionary and rebuild the data DLL; default all-ANSI tables are byte-identical to before. Non-ANSI
external file/table names and filter/order literals are separate planned steps,
not in this build.
The IP driver's SSL/TLS runtime refresh (OpenSSL 3.5) and the IP Data Server changes ship with the IP driver release, separately from this build.
Btrieve, xBase, Clarion .DAT and classic TopSpeed reject USTRING at CREATE/OPEN (err 47) — never a silently narrowed file.
USTRING columns display wide content; UNICODE memos/blobs render and edit wide. MSSQL browse uses the newest installed Microsoft driver and honors trust-certificate consent.
? consumes exactly one unit (emoji needs ?? or *).CHOOSE() accepts USTRING values (beta refresh after 08-23-2026): all-string values — USTRING, STRING or mixed — give a string result; assigned to a USTRING it keeps every unit, assigned to a STRING it narrows like any wide→narrow assignment. String mixed with numeric values stays numeric, as always. (Previously a USTRING value made the whole CHOOSE numeric, returning '0' even for a plain-string alternative.) USTRING in the index/condition position always worked.ALL(u, n) repeats wide text; n counts UTF-16 units like every other USTRING length — an astral emoji is 2 units, so ALL(U'💩 😀', 4) ends half-way into the second emoji (the dangling half-pair renders as a box). Build lengths from LEN() of the piece you repeat.NUMERIC() tolerates trailing spaces again (beta refresh after 08-22-2026): a previous refresh rejected any trailing space, so NUMERIC() of an ordinary space-padded STRING returned 0 — fixed. Related: an ENTRY(@s) bound to a USTRING keeps user-typed trailing spaces exactly as a CSTRING always has (the @s picture never clipped; a STRING only hid them in its padding) — equality still ignores trailing spaces; CLIP() where the exact value matters.
MATCH:Regular and STRPOS narrow to ANSI today (byte-based regex).
Match:Soundex narrows by design (ASCII phonetic algorithm).
GETINI / PUTINI (and the GETINIGROUP / PUTINIGROUP
twins) accept and return USTRING exactly. One Windows rule to know: the profile APIs preserve a UTF-16
.ini but never create one — so a wide write to a missing file now creates it
UTF-16 LE with BOM, while a wide write into an existing ANSI file still best-fits to the codepage.
To take an existing .ini wide, re-save it as "UTF-16 LE with BOM" in an editor (or delete it and let
the app recreate it). Reads handle both formats. All-ANSI usage is byte-identical to before.GETREG / PUTREG / DELETEREG / GETREGSUBKEYS /
GETREGVALUES are Unicode end to end — the registry has always stored names and string values
as UTF-16, so there is nothing to create or convert: key names, value names and REG_SZ /
REG_EXPAND_SZ values written from a USTRING land exactly (RegEdit shows the real characters), values
typed in RegEdit read back exactly into a USTRING, and GETREGSUBKEYS / GETREGVALUES into a
USTRING queue field list the real names. STRING receivers best-fit as before; all-ANSI usage leaves bytes at rest
unchanged. Two recorded edges: REG_MULTI_SZ reads deliver the ANSI form (a USTRING cannot carry the
NUL-separated list; a USTRING value writes a one-element list), and REG_BINARY /
REG_DWORD / REG_QWORD take a USTRING value through its ANSI form.&USTRING shows the pointer.! Hold exact typed text (any language, any emoji)
Name USTRING(64)
...
WINDOW ... ENTRY(@s60),USE(Name) ... ! free-form -> Unicode entry
! Detect a high surrogate before slicing/truncating — keep the pair together
IF i <= LEN(Name) AND VAL(Name[i]) >= 0D800h AND VAL(Name[i]) <= 0DBFFh
! Name[i] and Name[i+1] are ONE character (UTF-16 surrogate pair)
END
! Deliberate narrowing at an ANSI boundary
AnsiOut CSTRING(65)
AnsiOut = Name ! chars outside the codepage become '?'
! Lossless COM handoff
B BSTRING
B = Name ! wide -> BSTR, no codepage
Name = B ! BSTR -> wide
! UTF-8 interop
U8 = TOANSI(U, 0FDE9h) ! USTRING -> UTF-8 bytes
U = TOUNICODE(U8, 0FDE9h) ! UTF-8 bytes -> USTRING
! Long text / images on SQL: the blob IS the field
! Notes BLOB,NAME('Notes'),UNICODE <-> nvarchar(max)
! Photo BLOB,NAME('Photo'),BINARY <-> varbinary(max)
WINDOW ... TEXT,AT(...),USE(Pre:Notes),HVSCROLL
IMAGE,AT(...),USE(Pre:Photo) ...
! Store bytes into a blob: SIZE FIRST
Pre:Photo{PROP:Size} = LEN(bytes)
Pre:Photo[0 : LEN(bytes) - 1] = bytes
PUT(File)
! A wide TopSpeed file
WF FILE,DRIVER('TopSpeedW'),CREATE,PRE(W1) ! lands as .tpsw
UK KEY(W1:UName),DUP,NOCASE
Rec RECORD
Id LONG
UName USTRING(20)
END
END
,UNICODE added: preview, print-after-preview, direct print, band splits, LISTs. The same report un-opted should post error 546 on wide data and still complete; an all-ANSI report with the attribute should look pixel-identical to its classic twin. Export the UNICODE report to TEXT/HTML/XML and verify your language survives at rest (TEXT opens as UTF-8 with a BOM); a PDF export of the same report should show exactly one Report Export Notice. If you use a third-party preview/report tool, check how it reacts to a UNICODE report's .emf page files (expected: it will not read them — that is why the attribute is opt-in).
Most useful evidence: the exact text (as a U'...' literal or code points),
the declaration (field / control / driver string), what you expected, what happened —
and whether the same steps work with plain ANSI text.