UTF8 - The BRemove statement

Recent changes to the "[]" operators in OpenInsight 9.2 have resulted in substantial performance improvements to UTF8 mode string handling. This post highlights another such enhancement introduced in 9.2 to help bring UTF8 mode applications up to the standard of their ANSI counterparts.

Consider the Loop/Remove construct below:

0001  /* 0002     Example showing standard loop/remove construct used 0003     to parse dynamic arrays at high speed 0004  */ 0005   0006     mark = 1 0007     pos  = 1 ; * // This is the CHARACTER position 0008     Loop 0009        Remove nextVal From dynArray At pos Setting mark 0010         0011        // Process nextVal… 0012         0013     While mark 0014     Repeat 

This is a common way to efficiently parse dynamic arrays in Basic+, but just like the normal "[]" operators it suffers from a severe performance degradation in UTF8 mode due to the need to find the byte offset of a character when given the position.

To alleviate this Revelation have introduced the BRemove statement - this operates in exactly the same fashion as the normal Remove statement, but the index variable used in BRemove refers to a byte offset rather than a character position.

Here is the same example rewritten to use BRemove:

0001  /* 0002     Example showing UTF8-friendly loop/remove construct used 0003     to parse dynamic arrays at high speed 0004  */ 0005   0006     mark = 1 0007     pos  = 1 ; * // This is the BYTE offset 0008     Loop 0009        BRemove nextVal From dynArray At pos Setting mark 0010         0011        // Process nextVal… 0012         0013     While mark 0014     Repeat 

As you can see it's a simple change and one worth making - using BRemove in your UTF8 applications will ensure that your dynamic array parsing remains fast and efficient.

Thanks Carl! Glad you have such great influence at Revelation :) Cheers, M@

Hi M@,

Well it's not just me :) Mike and the guys made a serious decision to get UTF8 mode sorted out with this release, and we need folks like yourself to tell us what you need...

/c