Release of TeaScript 0.12.0 🌈

TeaScript 0.12.0 was published on the 18th January in 2024 and can be downloaded for free:
☛☛☛ Download is available here. ☚☚☚

(or browse the source code of the TeaScript C++ Library on Github)

This Release could also be published with the headline: “TeaScript – in living color”, or “TeaScript – colorized” – hence the rainbow 🌈 symbol for this release.

Are you new to TeaScript? Then you may read here first: Overview and Highlights of TeaScript.

What is new?

The main features of this release are:

• colorful output
• {format} string (like in C++ with libfmt/std::format)
• the forall loop (+ Sequences)
• first capability of interactive debugging
• (BREAKING) parameters of functions are const by default now.
• improved floating point handling

(… and don’t miss the little Teaser at the end of the article…)

More or less the following screenshot collage gives a good first impression of the biggest new features.

The scripts which produce the colorful test output for demonstration are included in the download packages.

Please, also watch out for the 2 topics “Breaking Changes” and “Deprecation” below.

Colorful output

(TeaScript language feature)

For the C++ Library and when compiling the Host Application from source this is an optional feature and will be automatically activated if the libfmt library is included during compilation. The (free) pre-compiled download packages of the Host Application have this feature always enabled.

This feature adds 3 new functions to the TesScript Core Library (Level Util):
★ cprint / cprintln
★ make_rgb

The usage is simple and straight forward. The cprint and cprintln functions are like its uncolored variants print and println but taking a 24bit RGB color as first argument. The color can be easily specified via the make_rgb function which expects a value for red, green and blue respectively – each in the range of [0..255].

As println also cprintln will add a new line to the end of the string input.

// Hello World in green

cprintln( make_rgb( 0, 255, 0 ), "Hello World!" )

The Host Application will also use some decent colorful output by default now. Especially the pretty error printing now highlights the marked section in the code for a better error readability. Printing the AST, or viewing sections of the source are looking also nice now and can be read and understood better.

Format String

(TeaScript language feature)

For the C++ Library and when compiling the Host Application from source this is an optional feature and will be automatically activated if the libfmt library is included during compilation. The (free) pre-compiled download packages of the Host Application have this feature always enabled.

This feature adds one new function to the TeaScript Core Library (Level Core):
★ format

You can use the format function to format strings with the well known curly bracket syntax {} as in libfmt and std::format (C++20) (which is based on libfmt as well).
Everything what is possible with the feature set from libfmt format.h for fmt::vformat is usable in TeaScript as well except ‘named arguments’ and that the usable types are limited to teascript::ValueObject for Bool, i64, f64 and String. Everything else will be converted to String. Non-printable types cannot be used.

// will produce the string "Hello World from TeaScript!"
format( "{2} {1} from {0}!", "TeaScript", "World", "Hello" )

// will produce a string with 0 as filler for a 3 column wide string.
format( "{:03}",  1 )  // "001"
format( "{:03}", 12 )  // "012"

// will produce a string with 3 digits after the dot.
format( "{:.3f}", PI )  // "3.142"

Please, read the docu of libfmt for all possible format options and features.

Forall Loop (+ Sequences)

(TeaScript language feature)

With the new forall loop you can do 2 things:

1. Iterate over all valid indexes of a Tuple
2. Iterate over all numbers of an Integer Sequence.

Examples for iterating over tuples:

// make some tuples to start with

def tup1 := _tuple_create()    // empty tuple
def tup2 := (1, 2, 3)          // 3 elements in tuple
def tup3 := ("Hello", 3.142, true, 9, "World" ) // 5 elements

// this loop body will not be executed b/c tuple is empty.
forall( idx in tup1 ) {
    println( "Not printed!" )
}

// prints 1 2 3
forall( idx in tup2 ) {
    print( tup2[ idx ] % " " )
}
println( "" ) // next line


// prints 5 lines in the form "element number: value"
forall( idx in tup3 ) {
    println( format( "{}: {}", idx + 1, tup3[ idx ] ) )
}


// of course it works also with Named Tuples
def person :=  _tuple_create()    // empty tuple
def person.first_name := "Peter"
def person.last_name  := "Meyer"
def person.email      := "meyer@some_mail.com"


// prints 3 lines in the form "key: value"
forall( idx in person ) {
    println( format( "{}: {}", _tuple_name_of( person, idx ), person[ idx ] ) )
}

Sequences consist of 3 values: start, end and step. If end is smaller start, step must be negative. The first valid value is always start. The next value is start + step and so on. All numbers from [start...end] which are reachable exactly via step are valid. If end is not reachable exactly by last current + step the last current value is also the last value. Thus, with start=1, end=10 and a step of 2 the sequence will stop at 9.
Actually, an Integer Sequence (a Sequence with values of type i64) can be created with the TeaScript Core Library function (Level CoreReduced) _seq( start, end, step )

Example code for iterate over sequences.

// prints 1 3 5 7 9
forall( n in _seq( 1, 10, 2 ) ) {
    print( n % " " )
}
println( "" ) // next line

// prints 10 9 8 7 6 5 4 3 2 1 0
forall( n in _seq( 10, 0, -1 ) ) {
    print( n % " " )
}
println( "" ) // next line


// prints 6, 1, -4, -9
forall( n in _seq( 6, -11, -5 ) ) {
    print( n % " " )
}
println( "" ) // next line

As for the repeat loop (docu) you can use loop (jump to loop head) and stop [with expr] (for stop the loop with an optional return value). “Labels” for naming/addressing the loop are also usable.

Interactive Debugging

(Host Application feature)

The interactive shell of the TeaScript Host Application now has a new set of internal commands for load, show, parse, and execute (parts of) script files. These commands might be useful for testing and debugging.

As a recap: The interactive shell will be launched when the Host Application is invoked without any arguments or when executing a script file with -i | --interactive after the script has been executed.

In the interactive shell you can use :? or :help for showing a list of internal commands.

The new commands are:

• :load=<file> – loads a new file for be used with the next commands. You can use this also for re-load the file if you want to start from the beginning again. Note: This command doesn’t clear the context, e.g., variables and functions in the top-level scope will remain unchanged. If you need a fresh environment use :reset or :reset=<config>.
  TIP: you can use the TeaScript function change_cwd( String ) for set the working directory prior loading a script. This could save a lot of typing, depending on the script file location. (cwd() will print the current working directory.)
• :show <N> – will show N lines of code from the current position including its line numbers. If N equals 0 it will show the current position with a marker.
• :goto line[,col] – changes the position to absolute given line and column number. column defaults to 1.
• :parse <N> – will parse N lines from the loaded script file starting at the current position. It will print the amount of fully parsed new top level statements.
• :stmt <N> – prints the AST of an already parsed top level statement. N is the number of an available statement, not an index.
• :exec <N> – executes / evaluates N already parsed statements and prints the result (if any).

During the session you can do everything in the interactive shell as before, e.g., use the debug operator for inspect variables and tuple elements, or use any valid TeaScript code and the build-in Core Library variables, types and functions for manipulate or test the environment.

The next screenshot shows an example:

Parameters are const by default now

Function parameters are const by default now. This effects only parameters which are passed by copy assignment. Shared parameters remain mutable by default, because they are often used as in-out parameters.

// a and b are const by default now!
func sum( a, b ) { a + b }


// This test function illustrates the new behavior.
func test( a, b @= )
{
    println( "a is const = " % to_string( a is Const ) )
    println( "b is const = " % to_string( b is Const ) )
    
    if( not b is Const ) {
        b := a
    }
}

def x  := 3
def y  := 5

println( "before call y is %(y)" )  // y is 5

test( x, y )

println( "after call y is %(y)" )   // y is 3

As before you can always explicit set the constness of the parameters with def and const.

// As before you can always explicit set the constness/mutability.
func test( const a_is_const @=, def b_is_mutable )
{
}

More about function parameters in the docu.

This release has several options available for activate the old behavior, for the case that you cannot or don’t want to change your script code yet. They are listed in the Deprecation section below. Note: The options will be removed in later versions, or – for those which will stay present – will form an unofficial legacy dialect of the TeaScript language. The details are described in the comments of the code.

Improved floating point handling

Floating point literals which containing a dot . and an e (for the exponent) the same time can be parsed now.
E.g., this floating point literal can be parsed now: 123.456e-10
This solves known issue number 1 from the Known_issues.txt.
Also, an optional + sign for the exponent is supported now.

Floating point values can now be converted to strings like expected and without loosing precision after the 6th digit by using either fmt::to_string() if libfmt is included, or std::format() as a fallback (the prior used C++ std::to_string() has a wrong (broken) behavior here).
This solves item number 2 from the Known_Issues.txt.

The trunc() function (and the new added underlying _trunc()) are supporting the full range of f64 now instead of only the range of i64.
This solves item number 12 from the Known_Issues.txt.

Misc

Added a features Tuple with the elements .color, .format, .toml
reflecting if the feature was available during compile and can be used in general. (This does not mean that the corresponding functions are actually loaded into the Core Library. It just tells if they could be loaded.) (TeaScript language)

The class teascript::ValueObject now has a Visit( Visitor ) method for dispatching the inner variant value with a given visitor.
A very great example usage which also demonstrates the using of lambdas/functors and ‘type agnostic’ programming in TeaScript is provided in test_code8() function of the demo project: Link to test function. (C++ Library)

Added a new protected constructor to the class teascript::Engine which allows to not bootstrap the Core Library. This is useful if you want use your own provided Library or use a derived Library. (Don’t forget to override ResetState() as well.) (C++Library)

Added inc( val @=, step := 1 ) and dec( val @=, step := 1 ) for increment / decrement a given variable by a specified amount (default 1). (TeaScript language)

Added a Deprecation_and_Breaking_Changes.txt for have a list of these things always on board.

Added a changelog.txt for have a list with all important changes and bugfixes.

Several bugfixes and improvements (see changelog.txt)

Breaking Changes

Default constness

The default constness of copy assigned function parameters changed from mutable (def) to const. Please, see the corresponding section above for an illustration.

This version has several ways to enable the old behavior:

1. For the Host Application launch your scripts with
   --old-mutable-parameters
   E.g.:
   TeaScript.exe --old-mutable-parameters my_script.tea arg1 arg2
2. For the C++ Library use the high level API of class Engine:
   void ActivateDeprecatedDefaultMutableParameters() noexcept;
   Please, read also the comments.
3. For C++ Library low level programming:
   Overwrite the TeaScript language dialect in the Parser:
   void OverwriteDialect( Dialect const dialect ) noexcept;
   Please, read also the comments there and in teascript/Dialect.hpp
4. Compile the Library and/or Host Application with
   #define TEASCRIPT_DEFAULT_CONST_PARAMETERS false
   You can do this from inside Dialect.hpp or define it outside (e.g. your project settings/before include)
   Please, read also the comments in teascript/Dialect.hpp

Point 1 and 2 will be deprecated and removed in later versions!
Point 3 has experimental state for now. (means API is not promoted to official and might be changed/removed.)
All of the points creating a legacy TeaScript language dialect which is not the official standard language!

Visibility of Core Library functions

The following Core Library functions moved up to Level CoreReduced:

The following Core Library functions moved up to Level Core:

eval_file

The following Core Library functions moved up to Level Util:

The following Core Library functions moved up to Level Full:

trunc               (check _trunc from LevelUtil)
ceil
floor
sqrt                (check _sqrt from LevelUtil)
file_copy
file_copy_newer
readtextfile
writetextfile
create_dir
path_delete
file_exists         (check path_exists and file_size from LevelUtil)
readtomlfile

libfmt preferred over C++23

If you compile in C++23 mode with the <print> header (for std::vprint_unicode()) but in your include path is a libfmt (#include “fmt/format.h”) then libfmt will be preferred over the C++23 variant (because more features).
You can disable libfmt with #define TEASCRIPT_DISABLE_FMTLIB 1 in teascript/Print.hpp

Deprecation

The following deprecated parts have been finally removed from this release:

virtual void CoreLibrary::Bootstrap( Context &, bool );
use the new one instead: virtual void Bootstrap( Context &rContext, config::eConfig const config )

The following parts are now deprecated and will be removed in some future release:

func eval( sth_unspecified ) from TeaScript Core Library (use the not confusing _eval( code_string ) variant instead)

bool Parser::Int( Content & )
use Integer( Content & ) or Num( Content &, bool ) instead.

More Infos

More information about the TeaScript language is available here:

✯ Overview and Highlights
✯ TeaScript language documentation
✯ Core Library documentation

☛☛☛ Try and download TeaScript here. ☚☚☚

Outlook

The next release 0.13 will come with an improved storage layout for the scopes. This will be only a first step of improvements and refactorings for the scope handling and variable / function storage and lookup. But this step will bring a huge performance improvement already, especially when undefining variables/functions and when using shared assignment with @= (docu).
I will also publish a benchmark for this comparing old VS new.
So, please, be aware of incompatible changes in the Context class as well as for the way of registering Core Library functions. (This will not effect the high-level API of the Engine and EngineBase class.)

Also, the following things might be coming with 0.13 – but they are not guaranteed and might be shifted to a future release.

Integrated JSON support (from / to Tuple, and … lets see…)

The as operator for explicit casting.

A class MTEngine (C++) which will make it possible to execute (the same or different) script code in more than one thread the same time while sharing a global set of variables and functions which might be also used for interchange data between the threads/scripts.

A Buffer type (based on std::vector<unsigned char>) for using and manipulating arbitrary binary data from within the script code.
This feature will need tons of utility functions for be usable in a good and safe way. So, I have to find a suitable subset for this feature can make it to 0.13.

Beside that there are also tons of smaller things which may made it to 0.13. Those will be mentioned in the next release article then.

Teaser

I want end this article with a little Teaser for the ‘next next’ release 0.14.
Maybe technical experts are seeing on the picture for what I have a working prototype already. 😉

Yes, you can see a functional and working prototype of an integrated virtual stack machine. This will bring a performance improvement for executing the script code. Of course, the ‘compilation’ will be done automatically under the hood (as in e.g., Python and Lua).
TeaScript will keep its AST evaluation also (especially for the interactive shell) but the new Tea StackVM will be used as the default for executing the script code.
Along with that release I will also publish a benchmark comparing eval VS compile VS ChaiScript VS Python VS Lua. (Of course, I won’t beat Lua.)

The Tea StackVM feature will be also the foundation for single step debugging and fine grained execution possibilities (pause/resume, etc).

Final notes

Personally I think, that this release makes TeaScript a way more handy and usable. Also, the outlook looks very promising.

How about you?

Do you have some feature wishes for TeaScript?
Do you like or dislike this current release? Don’t hesitiate to let me know your opinion. I would be happy about any feedback. 🙂

PS: If you are interested in to learn more about TeaScript, it will be also very helpful to read the prior release articles as they introduce the new features in a tutorial-like and/or easy-to-understand way.
The useful articles are:
Release of TeaScript 0.11.0 🎂
Release of TeaScript 0.10.0 🌞