Hands-off by design
Pick USTRING in the DCT, USE it on a control, PUT it to SQL.
No conversion buffers, no “wide mode” flag, no per-screen ceremony.
It just works the way Clarion always did.
Wide text end to end: declare it, type it, store it, browse it.
No new programming model. No special “Unicode mode.” If you already
know CSTRING, you already know USTRING —
and your existing ANSI apps stay byte-identical.
Pick USTRING in the DCT, USE it on a control, PUT it to SQL.
No conversion buffers, no “wide mode” flag, no per-screen ceremony.
It just works the way Clarion always did.
MSSQL, ODBC, PostgreSQL, and SQLite: USTRING(n) ↔ NVARCHAR(n-1)
(logical length round-trips). Keys, GET, SET/NEXT, and bound WHERE values
stay wide — never narrowed literals.
Classic TPS is untouched. New driver: DRIVER('TopSpeedW'),
.tpsw files, USTRING fields + UNICODE blobs, keys that work,
and clean version rejects instead of silent mojibake.
On SQL there is no MEMO(n) — long text is the blob.
TEXT,USE(blob) displays and edits;
IMAGE,USE(blob) displays. Window binding ships today;
Report is the same mental model (next surface).
Add UNICODE to a REPORT and wide text prints end to end —
preview, print-after-preview, direct print. A report left un-opted
posts a loud error 546 on wide data — never silent mojibake.
Source (UTF-8 / UTF-16), Window/Report designers, DCT messages/tips, IME, emoji panel, clipboard. Window titles, menus, POPUP(), LIST columns and headings, captions and free-form ENTRY/TEXT/COMBO carry exact UTF-16 on any system codepage.
No USTRING? No wide designer text? Rebuild and ship — same binary
behavior as before. Opt-in tokens (e.g. PostgreSQL
UNICODECONNECT=1) never change default connects.
Import Tables: nvarchar → USTRING, ntext /
nvarchar(max) → UNICODE memo/blob, binary max → BINARY blob —
attributes already ticked. Data Browser edits wide without narrowing.
Emoji and non-BMP text edit as atomic characters on wide ENTRY/COMBO. Color emoji via DirectWrite where available; ordinary edits never leave half a pair behind.
Garbled text from an encoding mismatch — e.g. UTF-8 bytes read as Windows-1252.
Classic example: café instead of café.
Converting wide (UTF-16) text down to ANSI/8-bit. Characters with no mapping
in the target code page are lost or replaced (often ?).
Lossless round-trips exactly. Lossy trades exactness for compatibility. Keep data in USTRING end to end when it must survive.
If you tested the previous build, these are the changes to look at first. The headline is the language: the @C picture composes any date picture with any time picture, and the Embed Editor keeps the text you type in any script. Most items trace back to a beta field report or a request on the beta forum — keep them coming.
@C. One letter of its own, beside @D and @T: @C shows the whole seconds, @C7 every digit, @C3D010-T04~T~ the composed ISO form. Everything else about the picture is unchanged, and the Dictionary Editor, the SQL import wizards, the QBE filter, the Data Browser and the Window Previewer all write and read @C. Earlier beta builds spelled this picture @DT; that spelling is now an invalid picture token (the compiler names the line and column). A dictionary from an earlier refresh heals itself in the Dictionary Editor: the first time you open a table, every DATETIME field's screen picture and stored control lines take the @C letter with the same decimals, the Dictionary Changes window lists what moved, the dictionary shows as modified and saves. Open each table that has a DATETIME field once (a table you never open keeps its stored text, and a window generated from it would carry the old letter until you do). Windows and embeds in an .app that spell @DT are re-typed by hand (the compiler points at each one), or in one pass through a TXA export, a replace of @DT by @C, and an import. Proposed on the beta forum: a one-letter picture keeps @D as the date family's own prefix, so every tool that classifies a picture by its second character stays right.@C composes a date picture and a time picture of your choice. The full form is @C[n][D…][T…][~separator~][B]: n keeps its meaning (0–7 fraction digits), the two parts are any numbered @D and @T picture with all of their own options (fill character, separator, epoch), written without their own @ sign (D06 for @D06) so that a composed picture sits inside a LIST FORMAT string like any other picture: FORMAT('120L(2)@CD06T3@'), and the text between the two parts is written last between tildes, as a currency is in @N: ~T~ for ISO 8601, ~_~ for an underscore, ~ at ~ for a word with its blanks, ~~ for no separator, omitted = a space. Omitted parts keep today's shape, so @C, @C3 and @C7 are unchanged. Some examples: @C3D010-T04~T~ gives 2026-09-09T13:05:07.123, @CD12T05~~ gives 20260909130507, @CD12T05~_~ gives 20260909_130507, @CD06T3 gives 09/09/2026 1:05PM, @CD8T4 gives 9 SEP 2026 13:05:07. FORMAT, DEFORMAT, STRING and ENTRY controls and the wide (USTRING) lane all take the composed form; a date-time shown with a chosen picture is typed back through that same picture. The fraction follows the seconds, so n above 0 needs a time part that shows seconds (@T4, @T5, @T6). The Window and Report Designers accept the composed form in the Picture property and draw it on the canvas, and the Source Editor colours the whole picture. The Help topic "Date-Time Pictures" has the full table, with a how-to-read line under the syntax.0{PROP:Text} = U'...') or declare it as a U'' title: the title bar, the frame's Window list (STD:WindowList) and the tab bar (PROP:TabBarVisible) show the same text in any script. Windows with plain captions draw exactly as before.3D D8 51 DC). Save in either tool stores exactly what you see. A BLOB,UNICODE column reads the same way in the Hex Viewer. STRING and every other column type look and save as before.BLOB,UNICODE column the right pane is headed UTF-16 and shows one cell per character unit, read live from the bytes: plain text reads as text, an emoji sits in one cell. Click a character and the cursor lands on its first byte in the hex pane, ready for a byte edit; text is typed in the Text Viewer. A STRING column keeps its ASCII pane.@C (any number of decimals) is a date-time to the filter: every operator (Equal, Between, IN, the comparisons) takes what the picture displays, a date alone included, and the generated filter uses the column's own picture. On a SQL backend the filter compares the column with the ISO literal ('2026-09-25 13:05:07.1234567'), and an Oracle table compares its TO_CHAR text. Existing date, time, number and string columns generate exactly what they did before, with one exception you will like: a SQL date IN list typed with a space after the comma now carries both dates. Two more rides along: the Clarion-lane date and time "Not Less or Equal" operators now generate their proper filter (their table rows had a spelling the operator list never matched), and a date-time entry accepts the time to the minute (2026-09-26 10:30) everywhere @C deformats: the QBE value, an ENTRY(@C7), DEFORMAT().@C before @D (the QBE control template and the ADO browse column typing). If your own code classifies a picture by its second character, give it a @C arm ahead of the @D one when you add DATETIME support.INIClass.InitW), creates the Company\Product sub-folders wide (the new SpecialFolder.CreateKnownDirIn) and sets SYSTEM{PROP:DataPath} from the USTRING, so a user whose profile folder carries characters outside the ANSI code page gets the exact path. Both template chains. An application that uses a CSIDL value generates exactly what it did before; nothing changes until you choose a Known Folder.SYSTEM{PROP:DataPath} takes a USTRING. Set it from a wide path and it is kept exact: a file with a USTRING NAME relative to it opens under that folder, and reading the property back gives the wide value. A file with a plain STRING name under a data path the ANSI code page cannot spell gets error 45, the same floor as a lossy wide NAME, never a best-fit folder.PROP:Details) and a Unicode bind value on the SQL trace line (BindData=1 in the driver-tracing INI) show their Greek, CJK, Arabic or emoji characters as they are, on SQL Server, PostgreSQL and the other ODBC backends, Oracle and SQLite alike; an accented letter in a plain STRING reads as text too. A new trace file starts with the UTF-8 byte-order mark, so Notepad, the IDE and VS Code open it correctly; delete a trace file written by an earlier build before you start a new one, so the two encodings do not mix. Only control characters are still escaped as <xxH>. From a beta report of ??? in the trace for Arabic text in a USTRING field: fixed now. The DEBUG: trace target shows the same characters in the debugger's output window.'...' literal in any embed (the Embed Editor or the Embeditor) and save: the literal is stored in the .app as its U'...' form, the same thing the Window and Report Designers do for a caption (U'<1055,1088,1080,1074,1077,1090>', the portable form a TXA carries too), and when you open the embed again the editor shows your letters: U'Привет'. The running window shows the text. ANSI-only embeds are saved byte-identical. Only string literals are converted; a comment still goes through the code page. From a beta report: fixed now.USE(Var,,?Var2) keeps its equate through open and save. An explicit field equate that is the variable name plus a digit suffix (?PagesAcross2, ?Cnt10, also with the number parameter present: USE(Var,3,?Var3)) came back as USE(Var) once the designer had opened the window, and the compile then reported duplicate equates. The designer keeps it now. USE(Var,,?Var) still comes back as USE(Var), which is the same equate. From a beta report: fixed now.FROM(U'...') with one U in front of the whole string constant, U on every piece of a constant continued with &, and {PROP:From} with a USTRING at run time; index entries under FROM, Unicode, USTRING and “drop list”. The “Which screen controls carry Unicode?” how-do-I says the same (a variable is never a declared FROM source, in any type). From a question on the beta forum.'...' literal is still an ANSI string, so a character your Windows code page cannot hold becomes _ when the line is compiled, and TOUNICODE(..., 65001) cannot bring it back. Write the text as a U'...' literal (the raw characters are fine in a UTF-8 file) and the USTRING holds it exactly. See "Typing your language into source" below.TableMapper.MapRsToGroup assigns each column through an ANY bound with WHAT() to the group field, and that assignment, an ANY taking a VARIANT, is the statement every ADO browse and form runs per column. It holds for every VARIANT type the recordset returns: integers, booleans, doubles, strings. A double into an integer field keeps its whole part, as build 14000 did and as a direct LONG = VARIANT does. From a beta report (build 14373): fixed now.If you tested the previous build, these are the changes to look at first. The headline is SQL: a lost server connection re-establishes itself and the application continues, and a SQL browse fetches its page in far fewer round trips. Most items trace back to a beta field report or a request on the beta forum — keep them coming.
PROJECT and never into the queue (a queue cannot LIKE() a BLOB). A TEXT or IMAGE bound to the field then follows the highlight bar. Both template chains.varchar(max) imports as a BLOB, so the whole column travels. It joins text, ntext, nvarchar(max) (BLOB with Unicode) and varbinary(max) (BLOB with Binary) under one rule: SQL Server long types are BLOBs, with no length cap. A re-import or synchronize of an existing dictionary proposes the BLOB as a type difference you accept or decline; nothing changes on its own (the earlier CSTRING(65535) import is closed).DateTimeUtil builds without adding DATETIME.CLW to the project. The class now carries its LINK/DLL attributes (equates _DateTimeLinkMode_ / _DateTimeDllMode_, 1 / 0 by default, so a data DLL can export it the ABC way). The methods that only read a value take CONST, and a new Pict(dt, picture) formats a DATETIME through any picture: Pict(dt, '@D4') shows the date part, Pict(dt, '@T4') the time part, @C pictures the whole value. A two-picture form joins the parts: Pict(dt, '@D10-', 'T', '@T04') & 'Z' is an ISO 8601 stamp, Pict(dt, '@D12', '_', '@T5') a file-name stamp, and a part that formats blank drops the separator. The help now states the @C rule: 0 through 7 fraction digits, @C0 = @C.LIST,MARK() without a FORMAT declares its mark field on its own line, the declarations and the sample priming sit inside !Region / !EndRegion pairs so the editor folds them, and a control or column with a @C picture is primed with a date-time sample (a real 2005-02-18 17:05:45) on a DECIMAL(19,7) variable. The List Formatter's picture editor also accepts @C0.sz (every non-RAW form) and z (RAW). A USTRING prototype now has a link name no other type can produce — USTRINGPASS@FszRszOszPszRAsz for value, *, <>, <*> and array parameters, a lone z for RAW, the same pattern as RAW CSTRING's c. What to do on the refreshed build: regenerate and rebuild every module of a multi-DLL solution together (template-generated .exp files regenerate on their own); a hand-written .exp with zu/su lines needs its lines changed to sz; Pro2Exp / Exp2Map users update the letter table.FILEERRORCODE() = 08S01, or 08S02 with /MULTIPLEACTIVERESULTSETS=TRUE), and the next operation reconnects by itself. In a generated application (ABC and Clarion chains) the user sees one error box and the usual retry prompt – Yes saves the record. Hand code simply retries the failed statement. PROP:Disconnect is unchanged. This includes a table the program opens for the first time after the drop (the ABC chain opens related tables the moment a save needs them): the open itself reconnects, so the first save after an outage goes through instead of ending the program. A browse that was paging when the link went simply keeps paging: the generated browse (ABC and Clarion chains) re-runs its page fill once from the last row it showed, on a fresh connection, and the user sees nothing at all. Hand code that walks a VIEW gets error 90 once and continues after a RESET to its last position.BRWn.PageSize = n (its ViewManager already fetched 20 rows per trip; 0 switches the page fetch off), the Clarion chain emits BUFFER(view, n, 2, 0, 60) after each OPEN(view). A Clarion-chain SQL browse gains the most: it moves from one row per round trip to n. Hand code: BUFFER(view, n); view{PROP:FetchSize} reads back the rows per trip. Measured on SQL Server: a plain OPEN = 1, the ABC default = 20, BUFFER(view, 35, 2, 0, 60) = 35.NEXT(view) or REGET(view, pos) the first read of a projected BLOB fetches that row's whole blob, so a TEXT or IMAGE bound to a browse hot-field BLOB shows the highlighted row's content the moment the bar lands on it./MULTIPLEACTIVERESULTSETS=TRUE streams the rows. With it the MSSQL driver reads a file opened for update as one streamed result set; without it the rows come through a server cursor, one round trip per fetch block. On a local network the difference is small; over a VPN or to a cloud server it is large. It is guidance, not a new default.BLOB,BINARY (PostgreSQL bytea, SQL Server varbinary(max)); the dictionary import declares them that way for you. On PostgreSQL keep the psqlODBC option bytea as LO switched on.PROP:BlobFetch). A VIEW that projects a BLOB or MEMO now keeps its page fetch (BUFFER(view, n) / PROP:FetchSize): the driver positions on your row and reads the page in one round trip. A 20-row page fill on SQL Server measured 110 us/row before and 11.5 after, with the right bytes on every shape (forward, PREVIOUS, an ORDER BY without an index, REGET). On PostgreSQL the gain is small: psqlODBC delivers the whole result set, BLOBs included, at execute. The default is never slower than before: with /MULTIPLEACTIVERESULTSETS=TRUE on SQL Server the driver keeps the one-row fetch (there the row arrives with its BLOB, and a report or export loop that reads every row's BLOB is faster that way); set MyView{PROP:BlobFetch} = 1 before BUFFER on a browse or form that reads the BLOB for the highlighted row only, = 0 forces one row at a time. It takes effect at the next SET(view). The browse templates carry it as 'BLOB fetch (SQL)' right under 'Page fetch size (SQL)' (BrowseBox > Actions > Default Behavior, both chains): Driver default / In rowset / One row at a time. The default changes nothing in an existing app. The setting decides how the blob is fetched, not whether it follows the highlight bar: for a TEXT or IMAGE to show the selected row's BLOB the field must be a browse hot field (see "AppGen: browse hot fields" above), under every setting. A BLOB that is not a hot field is not in the browse's VIEW at all.UNICODECONNECT form works DSN-less on the PostgreSQL Unicode driver and on SQL Server: name the driver DLL and the settings as keywords in the OWNER and nothing has to be configured on the user's machine. The F1 help gains the topic "Customizing a SQL Connection", which lays out the three layers – the OWNER string, the driver string switches, the ODBC driver's own settings – with a copy-ready example for each (§10 has the same).PROP:Details the record dump prints a USTRING field's text like any other field, and the SQL bind lines (BindData=1 in the driver-tracing INI) show the whole wide value under the C type WCHAR; a character outside the ANSI code page prints as ?, never as a look-alike. Before, a USTRING field dumped as an empty line and a bound Unicode value showed its first byte only, which made a healthy save look like a lost value. Fixed now.BindData=1 in the driver-tracing INI, the SQL trace now prints value = <data at execution> for a BLOB parameter, which is what a BLOB bind is. Before, the trace writer treated that bind as text and the first save of a non-empty BLOB ended in a "Record Insert Error ... (Access Violation)" box, a driver fault that looked like a failed save. Fixed now. On build 14313 the workaround is BindData=0.<0E9H> on the bind line and in the record dump alike, a value longer than the dump's 254 bytes ends in ... so you can tell the cap from truncated data, a USTRING field is declared with its size, an ODBC 3 type is named rather than numbered, and Oracle and SQLite traces show a Unicode bind value the same way the SQL Server trace does. A trace is a diagnostic: it never takes the program down.SpecialFolder finds the Downloads folder, and every other Known Folder. The CSIDL list stopped at Vista; the class now has a Known Folders lane beside it: GetKnownDir(KF:Downloads) returns the folder as a USTRING (exact for a folder outside the ANSI code page), GetKnownDir(KF:Documents, 'MyApp') appends a sub-directory, GetKnownError() is the HRESULT of the last call. KNOWNFOLDER.EQU declares the common KNOWNFOLDERIDs as KF: equates (Downloads, Documents, Desktop, Pictures, Music, Videos, LocalAppData, RoamingAppData, ProgramData, Profile, Fonts, ProgramFiles, ProgramFilesX86, Startup, the Public ones), and another one from KnownFolders.h is one more equate written the same way. The CSIDL methods are unchanged.RmDirW joins them. FnSplitW, FnMergeW, MkDirW, AccessW and the new RmDirW are real MAP entries in CLIB.CLW ("Low-Level File Manipulation (wide)"), so a program that includes CLIB.CLW calls them by name with nothing declared on its side; each takes USTRING buffers of the declared sizes and keeps the C-library contract (RmDirW removes an empty directory, -1 otherwise). They are RAW: pass a variable, never a literal.CHECK('x'),USE(U),VALUE(U'..',U'..') and an OPTION's RADIO('r'),VALUE(U'..') store the value's exact units in a USTRING USE variable, {PROP:Value} = U'..' at run time does the same ({PROP:TrueValue} / {PROP:FalseValue} on a CHECK, the third value of a 3-state), and assigning the variable selects the right radio or check state. {PROP:Value} of a PROMPT, STRING, BUTTON, OPTION, GROUP, TAB or a RADIO without a VALUE returns the caption's exact units without the accelerator ampersand, and a SHEET's USE variable receives the selected TAB's caption the same way. A SPIN with FROM(U'..'), declared or set at run time through {PROP:From}, steps through the list's exact units. A CHECK's {PROP:Value} is its TRUE value, as documented (PROP:TrueValue is the same property).CONTENTS(?Combo) returns the text typed into a COMBO that has no USE variable, as it does for an ENTRY or SPIN without one.U in front of the whole string: FROM(U'ALL|#ALL|None|#NA|Ελληνικά|#EL|中文|#ZH') keeps every entry and alternate value exact, into a USTRING USE variable or a STRING one. A constant continued over several lines with & needs the U on every piece (a plain '..' piece passes through the ANSI codepage before the join). At run time ?List{PROP:From} = ListText with a USTRING variable, or a U'..' literal, does the same. The help topic FROM carries the note.?List{PROP:FromQueue} returns the queue a LIST or COMBO is fed FROM. A whole queue or a single queue field, declared or set at run time through PROP:From, comes back as the value the run-time library identifies the queue by (what a *QUEUE parameter receives; not ADDRESS(), which is the record buffer). It returns 0 when the control is fed from a string, a GROUP or a class, so a program can test it first. The help topic shows the NAME() twin prototype that hands the value to a *QUEUE procedure. The internal property number some programs used for this stays as it was.<13,10> as a line break, exactly like the compiled window. Decimal metachars in a control's text (BUTTON('Line1<13,10>Line2'), a <9> tab, a <200> accent) and {n} repeats now show in the designer as they run; the hex spelling <0Dh,0Ah> already did. A FORMAT ~Header~ with a <13,10> in it too.PROP:BlobFetch and PROP:FetchSize, and the @C rule on the DateTime pages.Everything the beta added to the language, the runtime and the templates, in one list, with the section that explains it. Existing names that merely became Unicode-aware (the string functions, the INI/registry/file-name family) are in §11 and are not repeated here.
USTRING(n) — the wide string type; n = UTF-16 code units (§1–§3).U'…' literals with hex, code-point and surrogate escapes (§5).DATETIME type and the @C[n] pictures, 0–7 fraction digits (What's new build-14313 › DATETIME).@C[n][D…][T…][~separator~][B] — the composed date-time picture: any @D / @T picture for the two parts, the separator text last between tildes (~~ none) (What's new build-14407 › Pictures).REPORT,UNICODE attribute and the report_unicode=>on project define (§8 Reports; What's new build-14313).BLOB,UNICODE — the wide blob (§9). A wide MEMO(n) is not offered, by design.FILE:QueueW — the wide DIRECTORY() queue (What's new build-14313; §11).DRIVER('TopSpeedW'), the .tpsw extension and SEND('BIGFILE') (§10).PROP:BlobFetch on a VIEW — blank = the driver decides, 1 = the blob is read inside the fetched page, 0 = one row at a time; effective at the next SET(view) (What's new build-14373).PROP:FetchSize on a VIEW — not new; reads back the rows per round trip that BUFFER(view, n) asked for (What's new build-14373).PROP:Unicode on a REPORT — the run-time face of REPORT,UNICODE (§8 Reports).PROP:ClipEntry on SYSTEM — opt-in clip of trailing spaces on ENTRY accept (§8, Fully Unicode surfaces).PROP:DataPath on SYSTEM — takes and returns a USTRING; a narrow name under a data path the ANSI code page cannot spell gets error 45 (What's new build-14407 › Known Folders in the templates).UCHR(code) and the two-argument CHR(code, 1) — a code point to a wide character, astral range included (§5).UVAL(string [, pos]) — the code unit or code point at a position (§11).TOANSI() / TOUNICODE() — explicit conversion with a code page (§4).GETREGSUBKEYS() / GETREGVALUES() — new registry enumerators (§11).FnSplitW, FnMergeW, MkDirW, RmDirW, AccessW — the wide CLIB twins, declared in CLIB.CLW (What's new build-14373 › Wide file names).DateTimeUtil in DATETIME.INC (LibSrc), including Pict(dt, picture) and the two-picture Pict(dt, picture1, separator, picture2) (What's new build-14313 › DATETIME and build-14373).SpecialFolder.GetKnownDir(folderId [, subDirectory]) and GetKnownError() with the KF: equates of KNOWNFOLDER.EQU (What's new build-14373 › Known Folders).SpecialFolder.CreateKnownDirIn(folderId, subDirectory) — the wide CreateDirIn twin for a Known Folder; INIClass.InitW(filename [, nvType [, extraData]]) + FileNameW — the INI file at a USTRING path (What's new build-14407 › Known Folders in the templates).PROP:FromQueue — the queue a LIST or COMBO is fed FROM, 0 when it is not a queue; read only (What's new build-14373 › PROP:FromQueue).ReportPreviewClass, PageTextIndexClass, IReportPreviewer (ABPREVIEW.INC/.CLW) and the IReportGeneratorW interface for report generators (§8 Reports; What's new build-14313)._USTRING_ — predefined by this compiler for shared-code detection (§5).report_unicode=>on — every REPORT compiles with the UNICODE attribute (What's new build-14313).KF: values) beside the five CSIDL values (What's new build-14407 › Known Folders in the templates).sz / z — rebuild every module of a multi-DLL solution together (What's new build-14373 › Compiler).
USTRING declares a string of wide characters — 16-bit
UTF-16 code units, the native text form of Windows. A USTRING value is terminated
by a null wide character; the value is self-describing (no space padding).
Name USTRING(24) ! room for 24 wide characters
Alias &USTRING ! reference to a USTRING
USTRING(n) reserves n wide characters including the terminator
— so it holds up to n-1 characters of data (the CSTRING rule).
SIZE() returns bytes (n * 2);
LEN() returns the number of wide characters before the terminator.
USTRING can be initialized and assigned from both Unicode and ANSI sources. Any string or numeric expression may appear on the right side of an assignment to a USTRING — exactly the same as with STRING / CSTRING / PSTRING.
String positions, lengths, and limits for USTRING count UTF-16 code units — the same model Windows and the C wide-character library use. A code unit is not always a whole character on screen:
| On screen | UTF-16 units / LEN() |
|---|---|
A, α, ☃ | 1 |
😀 (emoji) | 2 (surrogate pair) |
👨👩👧 (joined family) | 8 |
🇺🇸 (flag) | 4 |
This is the same contract ANSI LEN always had (bytes under DBCS, not “visible
characters”). Practical consequences:
LEN(U) after typing one emoji can be 2 (or 8). Size fields and columns in units.U[i] indexes one unit. Indexing the middle of a surrogate pair yields half a character.0D800h–0DBFFh).
Unicode assigns every character a number called a code point
(A is U+0041); a UTF-16 code unit is one 16-bit
storage element. The two differ only outside the BMP (Basic Multilingual
Plane) — Unicode's original and most commonly used 65,536-code-point range:
U+0000–U+FFFF) always take one code unit.U+10000–U+10FFFF) take two code units — a
surrogate pair (lead unit 0D800h–0DBFFh, trail unit 0DC00h–0DFFFh;
those ranges are reserved inside the BMP for exactly this job and never appear as characters on their own).| Character | Code point | BMP? | UTF-16 storage |
|---|---|---|---|
A | U+0041 | Yes | 1 unit / 2 bytes |
é | U+00E9 | Yes | 1 unit / 2 bytes |
中 | U+4E2D | Yes | 1 unit / 2 bytes |
€ | U+20AC | Yes | 1 unit / 2 bytes |
☃ | U+2603 | Yes | 1 unit / 2 bytes |
😀 | U+1F600 | No | 2 units / 4 bytes |
𐐀 (Deseret) | U+10400 | No | 2 units / 4 bytes |
𠀀 (CJK Ext B) | U+20000 | No | 2 units / 4 bytes |
Text: A😀B
Unicode code points: U+0041 U+1F600 U+0042
UTF-16 code units: 0041 D83D DE00 0042
Three characters, four code units — LEN() returns 4.
Do not think of BMP as “normal characters” and non-BMP as “emoji”: some emoji are BMP and
take a single unit (☃, ☺), and many non-BMP characters are not emoji
at all (historic scripts, mathematical alphabets, supplementary CJK ideographs). Everyday
Latin text, punctuation, currency symbols and the common CJK range are all BMP. And
separately from surrogate pairs, one glyph on screen can be built from
several code points — the joined family in the table above is five code
points (man, joiner, woman, joiner, girl) totaling eight units; a flag is two code
points / four units.
The rule to test by: never assume one 16-bit element is one character. Code that indexes, truncates, copies, or measures UTF-16 text must keep surrogate pairs together, and a round trip must preserve both halves of every pair — no replacement characters, no truncation, no split pairs.
?). This is the only lossy direction.IF U).U'...', and wide characters via UCHR(code) (new in this refresh — the single-parameter form) or the equivalent CHR(code, 1) (e.g. UCHR(9731) = ☃). Both take the full Unicode range: an astral code point (10000h–10FFFFh) returns the surrogate pair — two UTF-16 units — so UCHR(1F4A9h) is 💩 directly, no manual D800h/DC00h math. 0–FFFFh returns exactly one unit verbatim (lone surrogates pass through, so pair assembly from two calls stays legal); outside 0–10FFFFh returns U+FFFD (�). Constant code points cost nothing at runtime (compile-time fold).U'...' (extended in this refresh): values are UTF-16 units in decimal — U'<937>' = Ω — and now also in hexadecimal with the trailing h and a leading decimal digit, like any Clarion numeric constant: U'<0D83Dh,0DCA9h>' is a surrogate pair written as its two units. A value from 10000h through 10FFFFh (decimal or hex) is read as a full code point and inserts its surrogate pair by itself — U'<1F4A9h>' is 💩 in one number — and a repeat count repeats the whole pair (U'<1F4A9h{3}>' is three of them). A value beyond 10FFFFh, or hex digits without the h, is a loud compile error — a Unicode constant is never silently truncated.<n> values are bytes — they always were — and a value above 255 silently keeps only its low byte, so '<55357>' is not a surrogate: it's the single byte 3Dh (=). Surrogates and wide units cannot be written in a plain literal at all — reach for U'...' or UCHR()._USTRING_ (=>on, beta refresh after 08-08-2026) beside the classic _C120_/_WIDTH32_ flags — code shared with older Clarion versions can gate its wide paths with COMPILE('**u**', _USTRING_) / OMIT('**u**', _USTRING_). Name matching is case-insensitive like the rest.&USTRING references, NEW/DISPOSE, THREAD, DIM'd arrays, GROUP fields, EQUATE initializers and OVER all work as for other string types (OVER exposes the raw UTF-16 byte image).MyProc(USTRING u)) are writable private copies, exactly like value STRING parameters — assign to them freely, changes never reach the caller. (Earlier beta builds made them read-only — error 241 on assignment. Note the value form is a USTRING capability: a value CSTRING parameter remains illegal.) And &USTRING reference assignment (Alias &= U) is fixed — in earlier beta builds it corrupted the stack frame and crashed at procedure exit; &= NEW USTRING(n), DISPOSE and passing the reference on now all behave.U[i] works as rvalue and lvalue; range slice U[a : b] is rvalue only — rejected as lvalue at compile time.CASE with wide OF literals and ranges works.Recommended: PROP:Codepage + PROP:Locale — three equivalent ways:
SYSTEM{PROP:Codepage} = 1253 ! preferred; also readable back
SYSTEM{PROP:Locale} = 1032
LOCALE('CLACODEPAGE','1253')
LOCALE('CLALCID','1032')
! or in the auto-loaded <exe>.ENV file:
! CLACODEPAGE=1253
! CLALCID=1032
Named values work: 'GREEK', 'WESTERN', 'UTF8' for codepage;
'EN-US', 'DE-DE' for LCID. Empty value resets to system defaults.
A flip is process-wide and immediate for new conversions/comparisons. Values converted
earlier keep their bytes. Legacy CLACASE/CLACOLSEQ/CLADIGRAPH still work — last writer wins.
A = TOANSI(expr [, cp]) ! result is ANSI, encoded at cp
U = TOUNICODE(expr [, cp]) ! result is wide; cp says how to READ ANSI input
Omitted cp = current PROP:Codepage. UTF-8 works as an explicit codepage
(65001 / 0FDE9h):
U8 = TOANSI(U, 0FDE9h) ! USTRING -> UTF-8 byte string
U = TOUNICODE(U8, 0FDE9h) ! UTF-8 byte string -> USTRING
The compiler reads .clw/.inc in mainstream Unicode formats — write U'Κλόουζ', mixed-script literals, or emoji directly:
!UTF8 as the first line, or project pragma module(encoding=>utf8).U'...' literals carry exact Unicode. Plain literals remain ANSI carriers (best-fit narrowed at compile time; a character the code page cannot hold becomes _, and no run-time conversion such as TOUNICODE(..., 65001) can recover it); non-ANSI text needs U'...'. The same split governs angle-bracket codes: in a U'...' literal they are UTF-16 units / code points; in a plain literal they are bytes, and values above 255 silently truncate (see the plain-literal trap in §3). Exception: plain caption literals inside WINDOW/REPORT keep exact Unicode end to end.Exactly two encodings (Tools ▸ Options): Clarion ANSI (default — existing files round-trip byte-identical) and Clarion Unicode (UTF-8 with BOM). The option is the default for new files only; the status bar shows the current file’s encoding. Type a non-codepage character into an ANSI file and the editor offers to switch to UTF-8 at save. The editor never writes a file the compiler cannot read.
Type your language into captions, titles, messages, tooltips and report strings — nothing to enable.
Save, reopen, generate: exact round-trip and correct draw on any system codepage. Internally,
non-ANSI designer strings travel as readable ASCII U'...' metachar form
(e.g. BUTTON(U'B<233,20013,937>') for Bé中Ω). Apps without wide designer
text stay byte-identical. Older IDE builds cannot open an .app that uses wide designer text.
Every user-text field property (Message, ToolTip, Prompt, Column Heading, Description,
Initial Value, Choices/Values, True/False) accepts Unicode and round-trips into the running app.
The .dct format is unchanged. ANSI-only dictionaries are byte-identical.
Generated code from the native template engine is always ANSI, and the .app stores
embed text in the system codepage. Starting with build 14407 the Embed Editor does
what the designers already do: a string literal your codepage cannot hold is saved as its
U'...' form in the .app
(U'<1055,1088,1080,1074,1077,1090>'), the editor shows the
letters again when you reopen the embed (U'Привет'), and the running window
shows the text. The exact rules, so there is no doubt:
'...' in the embed is checked on its own against the Windows ANSI code page of your system (not the Text Editor encoding option).é/ü on a Western European system, Cyrillic on a Russian system, Greek on a Greek system: byte-identical to before. Nothing changes for existing code.U'...'. Every character outside printable ASCII in that literal is stored as its UTF-16 code point in a <...> group, printable ASCII stays as typed: 'café Ω' becomes U'caf<233> <937>', an emoji its two-unit pair. This form is portable to any code page and survives a TXA.? for what it cannot hold).U'café Ω', with the U prefix visible because that is what the compiler compiles. A <...> group with a control code (<13,10>) stays a group; a hexadecimal group you typed yourself (U'<3A9h>') shows as U'Ω' and is stored in decimal on the next save, same value.The How-do-I topic "How do I put Greek, Russian, Chinese or other Unicode text into an embed?" leads with "type it", has the same rules, and keeps the two alternatives for the cases that want them: the text in an INCLUDE file saved as UTF-8, or the code points spelled by hand.
A TXA is ANSI text in the system codepage of the machine that wrote it — it never carried an encoding marker, so regional characters (é, Ω, Cyrillic…) were ambiguous on any other machine, and editing one in a UTF-8 editor was a mojibake trap. Two changes, both in the app generator only:
é for é) is read as UTF-8 and comes back repaired./u8 modifier before /ax —
ClarionCL /u8 /ax app.app app.txa; for the AutoExport on save lane
(Tools ▸ Options ▸ Import/Export ▸ Apps) tick Export as UTF-8 (with BOM). The file comes out as
the same content in UTF-8 with BOM; every default export is byte-identical to before. (The interim
CLATXAUTF8 environment variable from the previous refresh is gone.)
The boundary: the .app stores text in the system codepage, so an imported TXA may only contain
characters that codepage can represent. If an edit introduces something outside it, the import
stops with an error naming the line rather than silently mangling text. Real Unicode in string
literals and designer captions still travels as the U'...' metachar form described above — this is
an encoding lane for the file, not a change to what the TXA contains.
Also in this refresh: a prompt value containing a literal < followed by digits (e.g.
x<10>y) exports re-escaped ('x<<10>y'), so export/import cycles are stable.
Pick USTRING from the field-type list (driver-admitted — only drivers that support wide columns),
set Characters. USTRING(n) = n-1 data characters, 2n bytes on the record.
Generation emits it verbatim. The DCT explorer tree shows the size (USTRING(21)).
| Server column | Imports as |
|---|---|
nvarchar(m) / nchar(m) | USTRING(m+1) |
varchar(m) | CSTRING(m+1) |
char(m) | STRING(m) |
nvarchar(max) / ntext | MEMO/BLOB + UNICODE |
text | MEMO/BLOB (narrow, by design) |
image / varbinary(max) / PG bytea | MEMO/BLOB + BINARY |
binary(m) / varbinary(m) | STRING(m) (fixed-width) |
bigint / PG int8 | DECIMAL(19,0) (lossless) |
uniqueidentifier | CSTRING(37) (+READONLY when server-filled) |
STRING(n) is deliberately fixed-width (space-fill on read; full n with pad as data on write). Use max/BLOB when lengths vary.UNICODECONNECT=1 to the Datasource so wide types import as USTRING / UNICODE memo. Clarion-owned token — stripped before the driver; connects without it are byte-identical. MSSQL never needs this.bigint/int8 imports as DECIMAL(19,0) and the full 19-digit value round-trips on every driver lane — earlier builds imported LONG, which silently truncates past 2³¹.@s<n-1>, the CSTRING rule). (Early-beta builds defaulted every USTRING to @s20 — fixed.)
DDL: MSSQL script generator emits NVARCHAR(n-1) for USTRING(n),
NVARCHAR(MAX) past the 4000-unit cap (you run the script yourself).
Round trips: USTRING survives DCTX, TXD and TXA export/import (the TXA optionally in UTF-8 — see §5); app-pool variables accept it.
FORMAT/DEFORMAT and picture tokens accept USTRING values and wide runtime picture strings.
USTRING(@pic) sizes as picture width plus terminator and formats on store.
One deliberate difference: a pictured USTRING QUEUE field sorts by raw stored text,
while a pictured ANSI field sorts by the deformatted value — sort wide date fields on a LONG key for chronological order.
Pictures on controls carry Unicode too (beta refresh after 08-23-2026):
a picture whose literal text contains characters outside the codepage — a currency symbol like
@n~₿~-15.2, a wide date separator — can be set on a LIST column
({PROPLIST:Picture}) or on a pictured display STRING control and formats every value
with the real symbol; reading the picture back returns it intact. Previously the picture was stored
narrow before the (already wide) formatter ever saw it, so anything outside the codepage became
?. Pure-ANSI pictures take exactly the old path. A pictured ENTRY or SPIN
edits wide too (beta refresh after 08-24-2026): the in-place editor and the formatter share the picture, so a
USTRING USE under @N/@D/@T/@K/@P edits in any script and a
picture symbol outside the code page shows while editing, not only after. (Beta refresh after 08-27-2026): the same holds for a
picture written in the LIST's own FORMAT() string (FORMAT('…@n~₿~-15.2@…')) —
before, only a picture set at runtime kept its symbol.
@Sn or no picture) + USTRING USE: IME, emoji panel, clipboard — exact UTF-16. Limits in units. {PROP:ScreenText} is wide. (Beta refresh after 08-26-2026: keystrokes that carry UTF-16 — KEYEVENTF_UNICODE/VK_PACKET injectors, the emoji panel on Windows builds that type rather than insert, IME-composed CJK — also arrive exact: the message pump is Unicode. Earlier builds best-fit those keystrokes to O??? even into a USTRING entry. ANSI-bound controls receive the bytes they always did; KEYCHAR() keeps its ANSI value.)SYSTEM{PROP:ClipEntry} = TRUE clips them on accept for every free-form ENTRY(@S) in the program, narrow or wide. A STRING pads anyway and is unaffected either way. Default off = identical to before.|) and column headings now draw real glyphs, not best-fit. (FROM(ustring) as LIST source is not accepted — use a wide QUEUE.) Residual: a heading baked into the FORMAT() literal can still narrow — set it at runtime for the wide path; queued. A runtime-set wide heading also now survives a later reformat of the LIST (e.g. after a column-picture change) — in earlier beta builds the reformat wrecked it (beta refresh after 08-23-2026). Column pictures with non-codepage symbols work too (see §7).{PROP:Text}) and MENU/ITEM text carry exact UTF-16, any codepage, themed or classic; reading a caption back is wide. Runtime writes to the status bar ({PROP:StatusText}), message zone and tips are wide too. (These were previously listed as ANSI limitations — this refresh changed that.){PROP:Use} now switches the control onto the wide class on the spot (and back for a STRING) — earlier builds widened only for a compile-time-declared wide USE. This is what makes TopScan's in-cell editor go wide with no tool changes.CONTENTS() / ScreenText / Line,n are wide.TEXT,...,RTF + USTRING USE (beta refresh after 08-09-2026): displays, edits and saves back wide. The variable follows the content: plain text stays plain (exact UTF-16); rich content saves as RTF source text — 7-bit, non-ANSI riding standard RTF escapes (\uN / charset byte escapes), lossless round trip and portable to Word/WordPad, emoji included. Emoji are stored exactly (pairs never tear) but the hosted RichEdit draws them monochrome — the plain-TEXT OS floor. RTF(TEXT:FILE): file content carries Unicode via escapes; a non-ANSI filename does not resolve (ANSI lane — known limitation); reading the control back through {PROP:Text}, {PROP:Line} or the RTF templates' GetText returns best-fit ANSI text — the variable itself keeps the units (known limitation). Pre-refresh, this binding silently showed empty and discarded edits; stored values were never damaged.{PROP:Text} = <wide value> on any of these families keeps exact units — previously a runtime caption rewrite could narrow (it was on the known-limitation list).|, {} sub-menus, ~, +/-, [...]) is decided on the wide text itself, so a full-width | (U+FF5C) inside an item no longer splits the menu the way its best-fit | once did. Return values, separators, grayed/checked items and sub-menus are unchanged; an all-ANSI POPUP takes exactly the old code path.US). (Beta refresh after 08-09-2026: a color emoji at the right edge of an ENTRY was clipped in earlier beta builds — fixed.)A TEXT hosted on a report without the UNICODE attribute: the wide value is never corrupted by display — the printed presentation may show ? for ☃, but the variable keeps exact text. (Pictured ENTRY and SPIN left this list with the beta refresh after 08-24-2026 — see §7.)
This list keeps shrinking — the August refreshes moved window titles, menu text, the status bar, LIST headings and pictures (runtime and declared in FORMAT()), the pictured ENTRY/SPIN editor, POPUP menus, tooltips, RUN() and the file-name builtins to the wide surfaces. What remains: a wide caption write before OPEN(window) best-fits — queued; and lookup-key / name arguments (an icon name, a non-ANSI filename read into the narrow FILE:Queue by DIRECTORY() — declare a FILE:QueueW for the wide lane) which stay ANSI by nature. Values always survive.
(REPORT printing is no longer on this list — see Reports below. MESSAGE() is no longer on this list either — beta refresh after 08-08-2026: the text, caption and button captions display wide, including custom |-list button captions and the copyable CANCOPY form; astral emoji render via surrogate pairs. The icon name argument is a lookup key and stays ANSI — a non-ANSI icon filename does not resolve. Plain ANSI MESSAGE calls are unchanged.)
BSTRING is COM marshaling, not an entry type — stage input in USTRING and convert at the COM boundary.
UNICODE attribute
Add UNICODE to a REPORT statement and the report prints wide text end to end —
USTRING fields, wide captions, TEXT, LIST cells, string tallies, page numbers — through
preview, print-after-preview, and direct printing:
Rpt REPORT,AT(500,500,7000,4000),FONT('Arial',10),PREVIEW(PgQ),THOUS,UNICODE
Things to know and test:
.emf (not .wmf). The built-in previewer and printing handle them unchanged; a third-party preview tool that expects WMF files will not read them. Since build 14313 the templates switch every report to UNICODE by default, so an application that previews through a third-party previewer (Icetips Report Previewer and the like) needs two Global Properties switches on the Classes tab: clear “Unicode reports (EMF pages, wide text end to end)” and tick “Keep the classic PrintPreviewClass previewer”. With either one missing the preview opens nothing. Both ways forward are supported: keep the previewer version you have with the two switches set (reports print as before, no Unicode text on them), or move to the vendor's version built for this release, which reads the EMF pages and the wide interface, and put the switches back to their defaults. From a beta report: the help says so now.IReportGeneratorW — six methods, opt-in): EMF Page Files — Transition Note for Report Tool Authors (shipped alongside this guide).ERRORCODE() after PRINT.charset=utf-8 (including the per-page files of a multi-file export, which previously carried no declaration at all) and carries your characters as UTF-8; XML declares encoding="UTF-8" — and the label is now true — with UTF-8 values. Nothing to configure: the report-output templates register those targets wide exactly when the report carries the UNICODE attribute; regenerate the app and export. A report without the attribute — and every existing app — exports byte-identically as before.,UNICODE also survives the designer round-trip and APP generation. (Beta refresh after 08-14-2026: reopening a saved ,UNICODE report in the designer previously failed with "Unexpected attribute 'UNICODE'" — the structure validator did not know the attribute. Fixed.).emf (or .wmf) pages. Worth a look if you preview two reports concurrently.PROPPRINT:frompage / topage + PROP:FlushPreview) prints exactly the selected pages. Worth a quick test with your own preview code if you use ranges.TRN strings included. Worth testing with your own printer driver. In HTML exports the wide lane carries emoji as real UTF-8 characters in the string's real color; a narrow-lane export shows ?? placeholders. Without DirectWrite the page prints the previous form: monochrome outline emoji, exact values preserved.TRN string could show beside an emoji on a colored fill.
On a SQL backend there is no MEMO(n): long text and binary content are both the BLOB surface,
and a FILE BLOB is a first-class USE variable:
FT FILE,DRIVER('MSSQL','...'),OWNER('...'),NAME('dbo.Notes'),PRE(NT)
PK KEY(NT:Id),PRIMARY
Body BLOB,NAME('Body'),UNICODE ! nvarchar(max) - wide long text
Photo BLOB,NAME('Photo'),BINARY ! varbinary(max) - image bytes
Rec RECORD
Id LONG,NAME('Id')
END
END
W WINDOW('Note'),AT(,,300,200)
TEXT,AT(4,4,200,120),USE(NT:Body),HVSCROLL ! displays AND edits
IMAGE,AT(210,4,80,60),USE(NT:Photo) ! displays, read-only
END
BLOB,UNICODE → true Unicode TEXT end to end. Writeback is modified-gated and whole-content; size follows content. Store at ordinary USE points; PUT() persists.TEXT,USE(blob,BINARY), IMAGE,USE(blob,UNICODE), BLOB,UNICODE,BINARY, slices as USE variables.BLOB,UNICODE, SIZE(), {PROP:Size} and slice bounds count UTF-16 units; plain/BINARY count bytes. Record fields keep byte SIZE().NT:Photo{PROP:Size} = LEN(bmpBytes) ! size FIRST
NT:Photo[0 : LEN(bmpBytes) - 1] = bmpBytes
PUT(FT)
Performance tip: avoid AUTO on windows bound to large blobs (AUTO compares USE variables every event pass).
Window formatter / AppGen don't offer blobs in USE pickers yet (type USE by hand).
IMAGE save-back does not exist (write-in is assignment). Report-hosted controls do not
bind blobs. MEMO(n),UNICODE on a control is a compile error (the ISAM memo
display floor — on SQL, use the blob).
The two have always been different animals, and the Unicode work follows that grain rather than fighting it:
MEMO(n) is a bounded text field. It has a
declared cap, is fetched with the record, and is a first-class string operand in the
language — you can concatenate it, MATCH against it, assign it in
expressions. It is an ISAM-format concept: the TopSpeed memo store is where it
naturally lives — in fact the SQL drivers have never accepted a sized
MEMO(n) declaration at all (error 47 at open, "Driver does not
support MEMOs"; long content on SQL was always the blob). That language
citizenship is why a wide MEMO would be a large compiler project — and it is
deliberately not planned: MEMO(n),UNICODE is a loud
error by design, never a silent narrowing.BLOB is an unbounded entity. No declared cap,
never part of the record buffer, and deliberately outside the expression language —
everything goes through slices, {PROP:Size} and USE binding. That opacity
is why wide BLOB support is already complete: storage, transport,
display (this section), reports/exports and the Data Browser all handle
BLOB,UNICODE today.MEMO(n) has no
natural SQL shape (a fixed-cap text sidecar maps to nothing a server offers), so long
text is BLOB,UNICODE against nvarchar(max)/NCLOB
and binary is BLOB,BINARY against varbinary(max) — the blob
is the long-content surface, and the import wizard maps those columns to
blobs for you.BLOB,UNICODE works today on the
TopSpeedW driver; MEMO(n),UNICODE rejects loudly at open — by
design, permanently. Capped wide text that fits the record belongs in a
USTRING(n) field; anything bigger is the blob. Existing narrow
MEMO(n) fields are untouched either way.Rule of thumb: new work takes the blob; MEMO(n) remains what it has always
been — the ISAM-native narrow capped text field of the existing
application base. It is not being dragged into the wide future; the blob (and the
USTRING(n) record field) already are the wide future.
USTRING(n) fully supported. SQL Server: nvarchar(n-1) (UTF-16 at rest). PostgreSQL: varchar(n-1) (UTF-8 at rest, converted at the driver). SQLite: NVARCHAR(n-1) (UTF-8 at rest, engine-converted). Logical length round-trips — write 4 characters, read 4 characters. Code points exact.UNICODECONNECT automatically./ENCRYPT and /TRUSTSERVERCERTIFICATE=TRUE (checkboxes in the Driver String Builder). On an interactive connect against a self-signed server without the flag, the driver now asks “Trust the server certificate and reconnect?” — Yes connects on the modern driver; No preserves the old flow. Unattended configs never see a dialog.'' is not a valid switch error), and switches placed after a two-value /LOGFILE=file,n no longer get swallowed into the log filename.A SQL table's connection is set up in three layers. Each has its own scope and lifetime, and each is edited in a different place:
| Layer | Where you set it | Scope | What goes there |
|---|---|---|---|
| Driver string switches | the table's Options… button (the Driver String Builder), or DRIVER('ODBC','/SWITCH=value …'); at run time SEND(file,'/SWITCH=value') or file{PROP:DriverString} | one FILE, parsed at OPEN/CREATE; never sent to the server | /BUSYHANDLING, /ISOLATIONLEVEL, /LOGFILE, /LOGONSCREEN, /MULTIPLEACTIVERESULTSETS, /TURBOSQL, /ZEROISNULL; on MSSQL /ENCRYPT, /TRUSTSERVERCERTIFICATE |
| OWNER string | the table's Owner Name, or OWNER('…'); a variable works: OWNER(MyOwner) assigned before the first OPEN | one connection, shared by every file with identical OWNER text; spans threads (/BUSYHANDLING=2 = one per thread) | where the database is, who connects, every keyword the backend or its ODBC driver understands, plus Clarion's own tokens (UNICODECONNECT=1) |
| The ODBC driver's own settings | a DSN in the 32-bit ODBC Data Source Administrator (%WINDIR%\SysWOW64\odbcad32.exe; Clarion apps and the IDE are 32-bit), or the same keywords typed into a DSN-less OWNER | the machine (DSN) or the connection (keywords) | which driver DLL, SSL mode, binary column handling, fetch sizes |
The OWNER string has two forms. The comma form (datasource,user,password for ODBC, server,database,user,password for MSSQL) is assembled by the driver. The raw form is a complete connect string: when the text contains DSN= or DRIVER= (any case) it goes to the ODBC driver exactly as typed, minus Clarion's own tokens. file{PROP:ConnectString} returns what was sent.
PostgreSQL with Unicode columns, worked example. psqlODBC installs two 32-bit drivers, PostgreSQL Unicode(x86) and PostgreSQL ANSI(x86). USTRING columns need two things together: the Unicode driver and UNICODECONNECT=1 in the OWNER. The ANSI driver reports every column narrow whatever the application asks; without the token the Clarion driver connects as an ANSI application, so even the Unicode driver reports narrow and the Import Tables wizard produces CSTRING/STRING. With both, varchar/char import as USTRING and text as a UNICODE memo, and the fields read and write exactly. For bytea columns keep the driver option bytea as LO on and declare the field BLOB,BINARY.
! With a DSN: a 32-bit DSN 'sales' on the PostgreSQL Unicode(x86) driver
! (server, port, database, user, password, 'bytea as LO' ticked)
Customer FILE,DRIVER('ODBC'),OWNER('sales;UNICODECONNECT=1'),NAME('customer'),PRE(Cus)
! Without a DSN: the raw form names the driver and carries the settings as
! keywords (ByteaAsLongVarBinary=1 is psqlODBC's keyword for 'bytea as LO');
! a variable OWNER keeps the password out of the source
PgOwner CSTRING(512)
Customer FILE,DRIVER('ODBC'),OWNER(PgOwner),NAME('customer'),PRE(Cus)
CODE
PgOwner = 'DRIVER={PostgreSQL Unicode(x86)};Server=dbhost;Port=5432;Database=sales;' & |
'Uid=app;Pwd=' & CLIP(Password) & ';ByteaAsLongVarBinary=1;UNICODECONNECT=1'
OPEN(Customer)
Both forms take the Unicode connect (measured on this build: the DSN-less form, in either letter case, reaches the server through the wide connect; the same string without the token takes the classic connect). The Import Tables wizard accepts the same text in its Datasource field. On SQL Server none of this is needed: the MSSQL driver reports wide columns on every connection, and its encryption and certificate settings are driver string switches. The F1 help carries this as "Customizing a SQL Connection" under SQL Drivers.
UTF-8 at rest; driver transports UTF-16; engine converts. Wide columns and UNICODE memos are ordinary
UTF-8 TEXT — readable by any SQLite tool. Narrow STRING/CSTRING columns store your ANSI bytes verbatim
(byte-faithful round trip, but not valid UTF-8 to other tools — prefer wide columns when data must
interoperate). Import maps NVARCHAR/NCHAR → USTRING, NCLOB/NTEXT → UNICODE memos, BLOB → BINARY.
Sorting is code-point order; NOCASE folds ASCII only. Blob shrink stores exactly; standalone TIME
fields store HH:MM:SS[.hh] text and round-trip.
Beta refresh after 08-10-2026: imported keys now carry NOCASE when the underlying
SQLite index is declared with it (primary keys stay case-sensitive); a {PROP:SQL} SELECT that
returns no rows now describes its columns instead of failing with error 33; and 64-bit INTEGER columns
round-trip exactly through DECIMAL(19,0) — earlier builds carried them through a double,
rounding values past 15–16 digits and altering the stored type on write.
/UTF=8 or /UTF=16LE (Driver String Builder has a UTF group); default UTF-8 with BOM — self-describing, never silently narrowed. A BOM on an existing file wins. Mixing narrow and wide fields rejects with a reported error. Fields are fixed-width columns of n-1 units.42,"Bé中Ω"). USTRING next to a narrow string field rejects. UTF-8 only (UTF-16 is not a CSV encoding here). CSV quoting conventions hold for wide values.2n bytes of raw UTF-16LE verbatim; mixing is fully defined. OEM attribute rejects on USTRING records.
Unicode reaches TPS through a separate driver: DRIVER('TopSpeedW'), default extension
.tpsw. Classic TopSpeed, existing .tps files, and ANSI apps are completely untouched.
é sorts after z.BLOB,UNICODE — sizes/slices in units. MEMO(n),UNICODE is not supported — by design, permanently — and rejects loudly at open (see "MEMO versus BLOB" in the blob section for the rationale)..tps files as they are (see the big-file section below for upgrading them in place). The stamp is decided at CREATE and never changes behind your back. (Early-beta note: .tpsw files from builds before the big-format drop use older stamps — re-create them; both drivers refuse them loudly, nothing opens silently.)*.tpsw — fields arrive as USTRING with sizes, keys with NOCASE/descending, wide blobs already marked UNICODE), Data Browser, project references by name..tpsw files (beta refresh after 08-12-2026): TopSpeedW is in TopScan's driver list, and the Open dialog picks the right driver from the file you select. USTRING fields display and edit in any script: the browse list, the in-cell editor, the Text Viewer and the Hex Viewer carry the exact UTF-16 value, stored wide and round-tripping bit-identically at rest. TopScan also moves to a Segoe UI default font (your own saved font choice is kept).
Clarion now guards the classic .tps format's 2 GB limit: an ADD/APPEND/PUT that would grow past
it is refused with err 90 / FILEERRORCODE 8281–8283, "TopSpeed file size limit (2GB) reached" and the
file stays intact. A full file is not a corrupt file — reads keep working, DELETE and
shrinking PUTs stay legal in the same session (free space and ADD works again), and CLOSE commits normally.
TopSpeedW-created files use the new big format and reach 512 GB (same guard family at that boundary, with the 512GB message). Everything else is unchanged: same record / key / memo / BLOB limits, same tools story.
To lift an existing small file (typically a classic .tps you are adopting through TopSpeedW — a driver
swap alone never changes a file's format):
OPEN(MyFile, 12h) ! exclusive: deny-all is required
Ans = SEND(MyFile, 'BIGFILE') ! query -> 'OFF' (small) / 'ON' (big)
Ans = SEND(MyFile, 'BIGFILE=ON') ! in-place upgrade, all tables, atomic
What to test: grow a file past 2 GB under TopSpeedW (big format — should just work); hit the 2 GB guard on a
classic-driver file and confirm the message + that deletes still work; run the BIGFILE upgrade on a classic-created .tps
(keys and old rows must survive, BUILD must work after); confirm an upgraded or big file gives the clean version error in an
old-driver app rather than anything silent.
A TPS file interrupted mid-commit (power loss, kill, crash) recovers from its on-file journal at the next open — in classic TopSpeed as well as TopSpeedW. Readers of a crashed file always see the correctly recovered data, see the same data on every open, and leave the file byte-for-byte untouched; the first write completes the recovery permanently. What to test: kill an app mid-update (Task Manager during a heavy batch is fine), then reopen and read several times — same rows every time; then write once, close, reopen — everything intact, no corruption errors.
The in-memory driver (DRIVER('Memory')) is now fully wide: USTRING fields, keys over them, and
BLOB,UNICODE all work, with sizes and slice bounds in units as everywhere else. Key ordering is
linguistic collation under the Windows locale — the same order a keyed QUEUE gives a USTRING
component, so a Memory-driver table and a keyed queue holding the same rows agree. NOCASE keys use the
case-insensitive linguistic fold. Note this deliberately differs from TopSpeedW's code-unit key order: each
driver keeps its native ordering personality (Memory's narrow keys have always been collation-ordered).
One known limitation: MEMO(n),UNICODE on the Memory driver is still accepted silently over narrow
bytes — do not use it; the loud reject (as TopSpeedW already has) is queued. Narrow memos are untouched.
Runtime-defined files were already wide: the field-type list includes USTRING and sizes are in units.
Beta refresh after 08-12-2026 for the DynFile wrapper class: SetFieldValueW
is the wide write twin (the classic SetFieldValue takes a STRING value and narrows — Clarion prototypes
cannot distinguish value STRING from value USTRING, so the wide form takes the W name);
GetField / SetFieldValue now accept both bare and prefixed field labels; and
CacheFile onto a Memory-driver target works (the cached copy comes back open, ready to read).
The client/server IP driver (DRIVER('IPDRV')) now carries USTRING fields and
UNICODE memo/blob end to end — client application, requester service, data server,
to the real backing table — round-tripping bit-identically at rest. Wide keys sort in the backing
driver's order (the data server owns the key, so IP inherits it — e.g. TopSpeedW's code-unit order);
nothing IP-specific to configure. Declare the fields USTRING / the memos and blobs ,UNICODE in the
shared dictionary and rebuild the data DLL; default all-ANSI tables are byte-identical to before. Non-ANSI
external file/table names and filter/order literals are separate planned steps,
not in this build.
The IP driver's SSL/TLS runtime refresh (OpenSSL 3.5) and the IP Data Server changes ship with the IP driver release, separately from this build.
Btrieve, xBase, Clarion .DAT and classic TopSpeed reject USTRING at CREATE/OPEN (err 47) — never a silently narrowed file.
USTRING columns display wide content; UNICODE memos/blobs render and edit wide. MSSQL browse uses the newest installed Microsoft driver and honors trust-certificate consent.
? consumes exactly one unit (emoji needs ?? or *).CHOOSE() accepts USTRING values (beta refresh after 08-23-2026): all-string values — USTRING, STRING or mixed — give a string result; assigned to a USTRING it keeps every unit, assigned to a STRING it narrows like any wide→narrow assignment. String mixed with numeric values stays numeric, as always. (Previously a USTRING value made the whole CHOOSE numeric, returning '0' even for a plain-string alternative.) USTRING in the index/condition position always worked.ALL(u, n) repeats wide text; n counts UTF-16 units like every other USTRING length — an astral emoji is 2 units, so ALL(U'💩 😀', 4) ends half-way into the second emoji (the dangling half-pair renders as a box). Build lengths from LEN() of the piece you repeat.NUMERIC() accepts trailing spaces (beta refresh after 08-22-2026): NUMERIC() of an ordinary space-padded STRING is true, as it always was in C11. Related: an ENTRY(@s) bound to a USTRING keeps user-typed trailing spaces exactly as a CSTRING always has (the @s picture never clipped; a STRING only hid them in its padding) — equality still ignores trailing spaces; CLIP() where the exact value matters.
MATCH:Regular and STRPOS narrow to ANSI today (byte-based regex).
Match:Soundex narrows by design (ASCII phonetic algorithm).
GETINI / PUTINI (and the GETINIGROUP / PUTINIGROUP
twins) accept and return USTRING exactly. One Windows rule to know: the profile APIs preserve a UTF-16
.ini but never create one — so a wide write to a missing file now creates it
UTF-16 LE with BOM, while a wide write into an existing ANSI file still best-fits to the codepage.
To take an existing .ini wide, re-save it as "UTF-16 LE with BOM" in an editor (or delete it and let
the app recreate it). Reads handle both formats. All-ANSI usage is byte-identical to before.GETREG / PUTREG / DELETEREG / GETREGSUBKEYS /
GETREGVALUES are Unicode end to end — the registry has always stored names and string values
as UTF-16, so there is nothing to create or convert: key names, value names and REG_SZ /
REG_EXPAND_SZ values written from a USTRING land exactly (RegEdit shows the real characters), values
typed in RegEdit read back exactly into a USTRING, and GETREGSUBKEYS / GETREGVALUES into a
USTRING queue field list the real names. STRING receivers best-fit as before; all-ANSI usage leaves bytes at rest
unchanged. Two recorded edges: REG_MULTI_SZ reads deliver the ANSI form (a USTRING cannot carry the
NUL-separated list; a USTRING value writes a one-element list), and REG_BINARY /
REG_DWORD / REG_QWORD take a USTRING value through its ANSI form.&USTRING shows the pointer.! Hold exact typed text (any language, any emoji)
Name USTRING(64)
...
WINDOW ... ENTRY(@s60),USE(Name) ... ! free-form -> Unicode entry
! Detect a high surrogate before slicing/truncating — keep the pair together
IF i <= LEN(Name) AND VAL(Name[i]) >= 0D800h AND VAL(Name[i]) <= 0DBFFh
! Name[i] and Name[i+1] are ONE character (UTF-16 surrogate pair)
END
! Deliberate narrowing at an ANSI boundary
AnsiOut CSTRING(65)
AnsiOut = Name ! chars outside the codepage become '?'
! Lossless COM handoff
B BSTRING
B = Name ! wide -> BSTR, no codepage
Name = B ! BSTR -> wide
! UTF-8 interop
U8 = TOANSI(U, 0FDE9h) ! USTRING -> UTF-8 bytes
U = TOUNICODE(U8, 0FDE9h) ! UTF-8 bytes -> USTRING
! Long text / images on SQL: the blob IS the field
! Notes BLOB,NAME('Notes'),UNICODE <-> nvarchar(max)
! Photo BLOB,NAME('Photo'),BINARY <-> varbinary(max)
WINDOW ... TEXT,AT(...),USE(Pre:Notes),HVSCROLL
IMAGE,AT(...),USE(Pre:Photo) ...
! Store bytes into a blob: SIZE FIRST
Pre:Photo{PROP:Size} = LEN(bytes)
Pre:Photo[0 : LEN(bytes) - 1] = bytes
PUT(File)
! A wide TopSpeed file
WF FILE,DRIVER('TopSpeedW'),CREATE,PRE(W1) ! lands as .tpsw
UK KEY(W1:UName),DUP,NOCASE
Rec RECORD
Id LONG
UName USTRING(20)
END
END
,UNICODE added: preview, print-after-preview, direct print, band splits, LISTs. The same report un-opted should post error 546 on wide data and still complete; an all-ANSI report with the attribute should look pixel-identical to its classic twin. Export the UNICODE report to TEXT/HTML/XML and verify your language survives at rest (TEXT opens as UTF-8 with a BOM); a PDF export of the same report should show exactly one Report Export Notice. If you use a third-party preview/report tool, check how it reacts to a UNICODE report's .emf page files (expected: it will not read them — that is why the attribute is opt-in).
Most useful evidence: the exact text (as a U'...' literal or code points),
the declaration (field / control / driver string), what you expected, what happened —
and whether the same steps work with plain ANSI text.