authors/bcardiff.jpg Brian J. Cardiff 01 Nov 2018

Crystal 0.27.0 released!

Crystal 0.27.0 has been released!

This release includes changes in public APIs that enable new features, some polishing and important updates for upcoming changes in parallelism, platform support and safety. There were 173 commits since 0.26.1 by 36 contributors.

Let’s review some of the most relevant changes in this release. But don’t miss the rest of the release changelog with a lot of valuable information.

Language changes

From now on, if the arguments of a method call are to be splitted across multiple lines, the comma must be at the end of the line, before the line break.

So, instead of:

foo(x
, y)

You will need to write:

foo(x,
  y)

Probably it won’t affect a lot of code, but since it’s a language breaking change we mention it first. You can read more at #6514.

Detecting stack overflows

Sometimes a program will enter an infinite recursion or, due to another reason, run out of space in the stack memory. This situation known as stack overflow was reported as an Invalid memory access. This version ships with a boundary check that allows a better error message when a stack overflow happens.

This was a long standing issue and at the end of the day it was a collaboration that pushed an initial proof of concept implementation to a complete solution and closed one of the top 10 oldest issues.

Read more at #6928.

Concurrency and parallelism changes

We are aiming for great things to happen in this area, we are all very excited and looking forward to them. Next releases should enable some stories around parallelism. But for now we can talk about preparation steps that can offer some visibility of the progress.

The Boehm GC has an API that enables support for multithreading environment in 7.6.x. From now on the GC 7.6.8 or greater is used across the official builds. Since crystal 0.26.1 was shipped with 7.4.10, we first needed to update the dependency so the CI will be able to compile the compiler with the new GC API. This update of the GC includes a contribution from the crystal team to the GC related to mutexes and forks detected during the previous work regarding multithreading.

In parallel, a great refactor related to polishing API and separating responsibilities of Fiber, Event, Scheduler and EventLoop was done in #6897.

Again, these are preparation steps mostly, but we are shipping them already to deliver them as early as possible and avoid a havoc of the runtime that would be hard to review and be confident with.

Arithmetic changes

There are some missing pieces to make Crystal great for data science and more safe and sound regarding numbers. We are aiming to fix this during the next releases. One of the notable changes is that now == and != will work in a sound way when comparing signed with unsigned numbers. Previously a bit comparison was used but thanks to #6689 these operators have now the right semantic. Other comparison operators were working already since #2240.

In this version, a couple of arithmetic operators were added thanks to #6890: &+, &-, &*. They will perform additions, subtraction and multiplication with wrapping (as in, not overflowing). Some may notice that that is the exact same behaviour as +, -, *. In a future version, the regular operators are going to raise on overflow. This will allow you to trust the result of the operations when reaching the limits of the representable range. The ampersand operators might be useful to think of them as applying an & 0xFF bit mask to the result.

It will be up to you to start choosing the correct operators – with or without ampersand. This usually depends on how the data is used: business logic and data science usually should raise on overflow, while transport protocol implementation, checksums and low level programming might benefit from the ampersand operators.

From this version, thanks to #6891 you can start using // as floor division. For integers this matches the regular division, while for floating point numbers it is equivalent to (a / b).floor.

Whenever you want integer division, you should use //. In a future version, / will match the mathematical notion of division and return a float, despite the type of the operands.

So, in a future version, (a + b) / 2 will always be the average, no matter the type of a and b. It will no longer be required to write (a + b).to_f / 2 which has issues with precision and a generic implementation that could work with BitFloat. Soon the return types of integer division (//) and division (/) will be determined by the operator and not the operand.

These operator changes are introduced in stages to provide a migration path, and in some cases, because the compiler is bootstrapped.

One breaking change in this version is that it is no longer allowed to request a random value with a zero as argument. Read more at #6686.

Collections changes

There are breaking changes in Indexable module and in Hash. Indexable#at was dropped in favor of Indexable#fetch. The APIs of Indexable and Hash are more aligned now. Including how to deal with default values in case of a missing key. If no default value is needed, the #[] method must be used. Hash#fetch(key) was dropped.

items = ["a", "b"]
items[1]                              # => "b"
items.fetch(2, "c")                   # => "c"
items.fetch(3) { |k| ('a' + k).to_s } # => "d"

The #dig method is now a supported idiom to traverse through a hierarchy of collections and containers.

data = {foo: [0, {1, {"key" => 42}}]}
puts data.dig(:foo, 1, 1, "key") # => 42

The code is digging a named tuple, an array, a tuple, and a hash to get the answer.

If some of the keys are missing along the way, an exception as in #[] will be raised. Use #dig? to have a nilable result instead of raising.

It is thanks to the nature of the method typing and a recursive implementation that the type of the result is useful. Read more at #6719.

Another handy idiom introduced in this release is the possibility of chaining an arbitrary number of iterators thanks to Iterator.chain. Read more at #6570.

Time changes

Time keeps moving forward. In this release, there are breaking changes in favor of cleaner and more portable names. All references to “epoch” are be replaced by “unix”. Effectively: Time#epoch was renamed to Time#to_unix, #epoch_ms to #to_unix_ms, and #epoch_f to #to_unix_f. Read more at #6662.

Two notable features were added that will come handy. Thanks to #6681 there is now support for ISO calendar week numbers. And changing the timezone while keeping the wall clack is now easy thanks to #6572.

Files changes

Tools for working with temporal files and directories used to be provided by the Tempfile class. This class has been replaced by the methods File.tempfile and File.tempname. This change also tidies up the usage of arguments prefix, suffix and default temp dir. Read more at #6485.

Platform support

There was an issue detected in Boehm GC regarding its support in Google Cloud because someone wanted to run crystal compiled programs there. The fix will be released in the next version of the GC, but meanwhile we are including the patch in 0.27.0.

Some preparation for Windows support related to processes, forking, file handlers and arguments was done. Read more at #6744.

Other notable fixes in the realm of platform support are: #6426 to fix signals between forked processes, and #6660 to deal with how IO on a TTY behaves in different environments.

Networking changes

The breaking change in this area is pretty minimal: HTTP::Server#bind_ssl was dropped since in the previous version #bind_tls was introduced, but the former wasn’t removed to avoid a breaking change in a patch release. Read more at #6699.

The binding for OpenSSL were updated to support version 1.1.1. Read more at #6738.

Compiler fixes

In 0.25.0 due to literals auto casting a method call could lead to an ambiguous match. In this case the compiler will abort compilation since the programmer’s intention is not clear enough. A rule was missing though, if there is an exact match among all the candidates, then the intention is crystal clear. Read more at #6618.

Support for annotations inside enums was missing, but not anymore. Read more at #6713.

Calling super will forward by default all the method arguments. But if the call was expanded by macros that was not the case. It’s fixed in this release. Read more at #6638.

When using splats argument you can restrict the type of values, and also the whole Tuple or NamedTuple that is expected as splatted arguments. There was a bug when these kind of restrictions were used in initialize method arguments to initialize instance variables. Read more at #6648.

Next steps

Please update your Crystal and report any issues. We have scheduled some breaking changes for 0.28.0 but we will aim to include them in a 0.27.x version with an opt-in flag so you can try them earlier.

If there are regressions or blocking issues report them as soon as possible so 0.27.1 can consider them. We are eager to move forward with 0.28.0.

The development is possible thanks to the community’s effort, 84codes’ support, and every BountySource supporter.

Crystal 0.27.0 has been released!

This release includes changes in public APIs that enable new features, some polishing and important updates for upcoming changes in parallelism, platform support and safety. There were 173 commits since 0.26.1 by 36 contributors.

Let’s review some of the most relevant changes in this release. But don’t miss the rest of the release changelog with a lot of valuable information.

Language changes

From now on, if the arguments of a method call are to be splitted across multiple lines, the comma must be at the end of the line, before the line break.

So, instead of:

foo(x
, y)

You will need to write:

foo(x,
  y)

Probably it won’t affect a lot of code, but since it’s a language breaking change we mention it first. You can read more at #6514.

Detecting stack overflows

Sometimes a program will enter an infinite recursion or, due to another reason, run out of space in the stack memory. This situation known as stack overflow was reported as an Invalid memory access. This version ships with a boundary check that allows a better error message when a stack overflow happens.

This was a long standing issue and at the end of the day it was a collaboration that pushed an initial proof of concept implementation to a complete solution and closed one of the top 10 oldest issues.

Read more at #6928.

Concurrency and parallelism changes

We are aiming for great things to happen in this area, we are all very excited and looking forward to them. Next releases should enable some stories around parallelism. But for now we can talk about preparation steps that can offer some visibility of the progress.

The Boehm GC has an API that enables support for multithreading environment in 7.6.x. From now on the GC 7.6.8 or greater is used across the official builds. Since crystal 0.26.1 was shipped with 7.4.10, we first needed to update the dependency so the CI will be able to compile the compiler with the new GC API. This update of the GC includes a contribution from the crystal team to the GC related to mutexes and forks detected during the previous work regarding multithreading.

In parallel, a great refactor related to polishing API and separating responsibilities of Fiber, Event, Scheduler and EventLoop was done in #6897.

Again, these are preparation steps mostly, but we are shipping them already to deliver them as early as possible and avoid a havoc of the runtime that would be hard to review and be confident with.

Arithmetic changes

There are some missing pieces to make Crystal great for data science and more safe and sound regarding numbers. We are aiming to fix this during the next releases. One of the notable changes is that now == and != will work in a sound way when comparing signed with unsigned numbers. Previously a bit comparison was used but thanks to #6689 these operators have now the right semantic. Other comparison operators were working already since #2240.

In this version, a couple of arithmetic operators were added thanks to #6890: &+, &-, &*. They will perform additions, subtraction and multiplication with wrapping (as in, not overflowing). Some may notice that that is the exact same behaviour as +, -, *. In a future version, the regular operators are going to raise on overflow. This will allow you to trust the result of the operations when reaching the limits of the representable range. The ampersand operators might be useful to think of them as applying an & 0xFF bit mask to the result.

It will be up to you to start choosing the correct operators – with or without ampersand. This usually depends on how the data is used: business logic and data science usually should raise on overflow, while transport protocol implementation, checksums and low level programming might benefit from the ampersand operators.

From this version, thanks to #6891 you can start using // as floor division. For integers this matches the regular division, while for floating point numbers it is equivalent to (a / b).floor.

Whenever you want integer division, you should use //. In a future version, / will match the mathematical notion of division and return a float, despite the type of the operands.

So, in a future version, (a + b) / 2 will always be the average, no matter the type of a and b. It will no longer be required to write (a + b).to_f / 2 which has issues with precision and a generic implementation that could work with BitFloat. Soon the return types of integer division (//) and division (/) will be determined by the operator and not the operand.

These operator changes are introduced in stages to provide a migration path, and in some cases, because the compiler is bootstrapped.

One breaking change in this version is that it is no longer allowed to request a random value with a zero as argument. Read more at #6686.

Collections changes

There are breaking changes in Indexable module and in Hash. Indexable#at was dropped in favor of Indexable#fetch. The APIs of Indexable and Hash are more aligned now. Including how to deal with default values in case of a missing key. If no default value is needed, the #[] method must be used. Hash#fetch(key) was dropped.

items = ["a", "b"]
items[1]                              # => "b"
items.fetch(2, "c")                   # => "c"
items.fetch(3) { |k| ('a' + k).to_s } # => "d"

The #dig method is now a supported idiom to traverse through a hierarchy of collections and containers.

data = {foo: [0, {1, {"key" => 42}}]}
puts data.dig(:foo, 1, 1, "key") # => 42

The code is digging a named tuple, an array, a tuple, and a hash to get the answer.

If some of the keys are missing along the way, an exception as in #[] will be raised. Use #dig? to have a nilable result instead of raising.

It is thanks to the nature of the method typing and a recursive implementation that the type of the result is useful. Read more at #6719.

Another handy idiom introduced in this release is the possibility of chaining an arbitrary number of iterators thanks to Iterator.chain. Read more at #6570.

Time changes

Time keeps moving forward. In this release, there are breaking changes in favor of cleaner and more portable names. All references to “epoch” are be replaced by “unix”. Effectively: Time#epoch was renamed to Time#to_unix, #epoch_ms to #to_unix_ms, and #epoch_f to #to_unix_f. Read more at #6662.

Two notable features were added that will come handy. Thanks to #6681 there is now support for ISO calendar week numbers. And changing the timezone while keeping the wall clack is now easy thanks to #6572.

Files changes

Tools for working with temporal files and directories used to be provided by the Tempfile class. This class has been replaced by the methods File.tempfile and File.tempname. This change also tidies up the usage of arguments prefix, suffix and default temp dir. Read more at #6485.

Platform support

There was an issue detected in Boehm GC regarding its support in Google Cloud because someone wanted to run crystal compiled programs there. The fix will be released in the next version of the GC, but meanwhile we are including the patch in 0.27.0.

Some preparation for Windows support related to processes, forking, file handlers and arguments was done. Read more at #6744.

Other notable fixes in the realm of platform support are: #6426 to fix signals between forked processes, and #6660 to deal with how IO on a TTY behaves in different environments.

Networking changes

The breaking change in this area is pretty minimal: HTTP::Server#bind_ssl was dropped since in the previous version #bind_tls was introduced, but the former wasn’t removed to avoid a breaking change in a patch release. Read more at #6699.

The binding for OpenSSL were updated to support version 1.1.1. Read more at #6738.

Compiler fixes

In 0.25.0 due to literals auto casting a method call could lead to an ambiguous match. In this case the compiler will abort compilation since the programmer’s intention is not clear enough. A rule was missing though, if there is an exact match among all the candidates, then the intention is crystal clear. Read more at #6618.

Support for annotations inside enums was missing, but not anymore. Read more at #6713.

Calling super will forward by default all the method arguments. But if the call was expanded by macros that was not the case. It’s fixed in this release. Read more at #6638.

When using splats argument you can restrict the type of values, and also the whole Tuple or NamedTuple that is expected as splatted arguments. There was a bug when these kind of restrictions were used in initialize method arguments to initialize instance variables. Read more at #6648.

Next steps

Please update your Crystal and report any issues. We have scheduled some breaking changes for 0.28.0 but we will aim to include them in a 0.27.x version with an opt-in flag so you can try them earlier.

If there are regressions or blocking issues report them as soon as possible so 0.27.1 can consider them. We are eager to move forward with 0.28.0.

The development is possible thanks to the community’s effort, 84codes’ support, and every BountySource supporter.

comments powered by Disqus