Error handling

[sighoya]

@shelby3
@keean

How does your future language handle errors?
What do you prefer:

1.) Error handling over return types ala Rust/Haskell/SML/Ocaml
2.) Error handling over an Exception System ala C++/Java/C#/Python/Swift/Haskell/SML/Ocaml

[keean]

At the moment I am favouring error handling by exceptions using algebraic effects.

For me the argument is about keeping code clean to read.

[shelby3]

try-catch-finally which can be emulated with a Go transpiler.

At the moment I am favouring error handling by exceptions using algebraic effects.

I’ve soured on (even free) monads for modeling effects. Lots of tsuris of trying to arrange all ducks into a serial line.

For me the argument is about keeping code clean to read.

Why are they any cleaner in code than try-catch-finally?

Why make it abstruse for programmers who know try-catch-finally?

[sighoya]

I found exceptions to be superior to error return handling. Of course, one can create specific error return types instead of error return values but two main problems still exists:

1.) performance is decreased because of double bookkeeping errors, the one in the function itself which is also by exceptions, but the other outside the called function at the call site and this even in cases where no error happens

2.) You pollute your intended return types and more importantly you destroy the modern builder pattern aka left pipe pattern to write function applications more sequentially. And return tuples of errors and return type are not really a help.

Of course one should not overuse exceptions like in java but a balanced use of exceptions, optionals and eithers is all you need most of the time.

So I don't understand the critics from the go site

[shelby3]

@sighoya wrote:

I found exceptions to be superior to error return handling.

It’s definitely advantageous to not have to clutter the algorithm with checking the return value of every function call for errors. It’s more readable to group function calls by the commonality of the processing that will be required for any exceptions they throw, and handle that commonality in common catch and/or finally code block.

Exceptions are slower than checking every return value in the cases when there’s an error, but they’re faster in cases where there’s no an error. So for infrequent errors then exceptions are more performant.

The performance of throwing an exception can be quite high if there’s no stack cleanup required. For example if allocation of all objects that have a destructor are tracing GC, so just need to rewind the stack pointer (SP). Then just need walk backwards the list of stack activation frames which have an active try-catch-finally block. However, if there can be destructible objects stored directly on the stack or there are reference counting (RC) references stored on the stack which require decrementing refcounts on destruction of the stack activation frame, then the performance cost of rewinding the stack can be much higher. I elaborated on possible designs to avert these performance issues.

1.) performance is decreased because of double bookkeeping errors, the one in the function itself which is also by exceptions, but the other outside the called function at the call site and this even in cases where no error happens

I don’t understand what you mean about types versus values. In fact, I find that sentence very difficult to parse. What does “the one in the function itself which is also by exceptions” mean?

2.) You pollute your intended return types and more importantly you destroy the modern builder pattern aka left pipe pattern to write function applications more sequentially. And return tuples of errors and return type are not really a help.

I don’t know what this means, “modern builder pattern aka left pipe pattern to write function applications more sequentially”?

Are you referring to monads and applicatives which can seamlessly for example lift functions which accept types T to functions which accept Option[T]?

Or are you referring to the criticism that in a strongly typed PL, errors as return values have the same problem as checked exceptions in that they can be incongruent with interfaces and callbacks. A possible workaround would be if interface or callback signature would have a generic type for the possible errors that can be generated by the function that implements the interface or is supplied as the callback. Yet that would then be the analogous as punting from checked exception into unchecked exceptions because the type of the returned errors wouldn’t be enforced (by the compiler) on callers of the interface or callback.

Yet in many contexts we will know the type of the errors in the local context where the interface or callback is being deployed, so that criticism doesn’t also apply. In the cases of polymorphic recursion where it does apply (i.e. the callback requires additional typeclass bounds not supplied by caller due to its type for the callback), then typeclasses will also be incongruent.

For callbacks the alternatives are to specify on the accepted type for the callback the only checked exceptions that are allowed. And/or throw unchecked exceptions.

Of course one should not overuse exceptions like in java but a balanced use of exceptions, optionals and eithers is all you need most of the time.

A blog by Andrei Alexandrescu (one of the authors of the D language) which was linked from the bottom of one of the blogs which I cited, points out that we shouldn’t not use exceptions when their poor performance could be significant:

^{The exceptional path is slow (00:10:23). Facebook was using exceptions to signal parsing errors, which turned out to be too slow when dealing with loosely formatted input. Facebook found that using exceptions in this way increased the cost of parsing a file by 50x (00:10:42). No real surprise here, this is also a common pattern in the Java world and clearly the wrong way to do it. Exceptions are for the exceptional.}

---

Should exceptions be checked?

• checked exceptions are annoying boilerplate when the caller knows they can’t occur such as for CheckedInvalidRowNumberException.

  A possible solution is to have a suppress … exception declaration that the caller can make (perhaps far off to the right side so not cluttering the code) to assert the exception never occurs and so the system will halt if it occurs and report the broken assertion.

• exceptions that can’t be handled at the low-level caller must be explicitly bubbled up with annoying boilerplate. It can be argued that in some circumstances the low-level caller should catch the exception and merge it into an exception which adds more detail to what has failed for the each caller turtles up the call hierarchy, but in most cases the type of the exception is rather irrelevant and the higher-level logic will just undertake some generalized recovery.

  A possible solution is to have a bubble … exception declaration that the caller can make (perhaps far off to the right side so not cluttering the code) to indicate there’s nothing the caller can do about the exception. A bubble all and bubble all except … could be helpful so do not have to list every possible exception when the caller can do no recovery for any of them, which is important because of the combinatorial explosion of exceptions in a call hierarchy. Yet there’s still the problem of needing to annotate the function declaration site with all the throws for all those bubbled, unless we allow the throws to be inferred by the compiler so these are never explicitly annotated (which is necessary as aforementioned with interfaces or callbacks where the list of throws is variable until instantiated) which also reduces the need for the programmer to turn off checked exceptions with something like throws Exception and catch (Exception e) {}. I would like to throw exceptions to include some documentation, so that functions don’t have to separately document what each exception means. IDEs can be then make this documentation available in context. The bubble all should probably be some symbol to make it even more terse, because it’s argued that rarely will lower-level callers want to handle them.

  However given higher-level logic will just undertake some generalized recovery, then it can catch all exceptions and thus no need to check for specific exceptions. Yet this is what bubble all accomplishes. It still forces each caller in the turtles call hierarchy to specifically declare there’s no recovery they can do for an exception.

• It’s argued that some exceptions (i.e. the non-exogenous exceptions) need to be unchecked because they are faults which the program should not try to recover from. In the case, even the higher-level logic should notify the user and/or a log file that a general recovery was attempted for a fault that should not occur.

• There’s a problem of lack of specificity when the same type of checked exception is throw by different functions turtles down the call hierarchy from the caller, because the caller may want to handle one but bubble the other.

  The compiler could help out here by detecting duplicates and prefixing them with the function name that throws them (for anonymous function then use the hash of the function body?).

It seems to me that checked exceptions done correctly as per my proposals above are better for large-scale development than unchecked exceptions. The key seems to be minimizing the noise they create.

Thoughts?

[keean]

@shelby3 I think.we should think in terms of transactions with concurrent software. Transactions either complete successfully or roll-back (with an exception)

Exceptions can be implemented as an additional return value by writing code in the Error Monad. So there is no difference in speed between exceptions and error return values.

[sighoya]

@shelby3 wrote:

I don’t understand what you mean about types versus values. In fact, I find that sentence very difficult to parse. What does “the one in the function itself which is also by exceptions” mean?

The "one"/"other" means a check. You check at first for a condition to throw or return an error value. But for error return values you have to check for errors even in cases where no error has happened because there is no signaling mechanism for error values.

Sorry for the confusion of values vs types. I mean numbers which encode an error versus objects of some error type.

I don’t know what this means, “modern builder pattern aka left pipe pattern to write function applications more sequentially”?

I work a lot with c++ at work and they handle all errors by values of an error enum which are returned in each function requiring to pass the "normally" returned object by reference instead to return it which looks like this:

namespace::error function(OutPut& o) {...}

The drama with this is that you can't chain such expressions. Imagine you update some object with setters you could chain all those setters in one line:
o.set(a).set(b).set(c)
by returning the reference to itself (which is here o) which is called the "builder pattern" or otherwise known as the left pipe instead of the right pipe ($ in Haskell) but this is not possible because you return an error value.

Of course, one can argue to take the reference of the error value as input instead but this is in some kind awkward and I have never seen someone before who has done this.

I find checked exceptions better than unchecked exceptions, too.
Some Notes:

A possible solution is to have a suppress … exception declaration that the caller can make

I thought the same about it. Each function could throw except if it is treated not to do so for instance by the tag nothrow.

bubble all and bubble all except …

Or you solve this by subtyping with Error as union type or by composition with the enum/data type for all errors. Then you can check afterwards the specific exception you want by pattern matching over the union

unless we allow the throws to be inferred by the compiler so these are never explicitly annotated

Sounds like exception deduction :). This can become quite cumbersome if the compiler checks down all exceptions in the call hierarchy, which becomes especially problematic if functions are already compiled, sure you can put a list of possible exceptions into your program binary.
In my eyes, the user generally should annotate his functions with the exceptions important for him. And exceptions which are not important could be together mentioned by annotating the general supertype Exception.
The compiler should only look for exceptions at function signatures which are directly called in the current scope. This would be more lightweight.

(which is necessary as aforementioned with interfaces or callbacks where the list of throws is variable until instantiated)

Do you mean the number of exceptions is unbounded? Otherwise the worst case could be assumed.

Exceptions can be implemented as an additional return value by writing code in the Error Monad

Could you elaborate a bit more?
Afaics, additional return values incur a cost anyways even in non error cases.

[sighoya]

@keean, if you don't like try-catch-finally you could look onto the dlang solution with scopes. They could however not replace try-catch-finally in every case, at least in my mind.

[keean]

@sighoya My point is that the language syntax is independent of the implementation. Using the Error monad the source language syntax looks like 'throw/catch' but we translate this into normal functions with an error return value in the intermediate language in the compiler.

[sighoya]

@keean
I've understood you in this respect but it does however incur costs. Maybe you need this because of go :)

[keean]

@sighoya you mean it incurs costs on the success path (checking for the error condition), whereas exceptions implemented by stack-unwinding do not? This is true, but due to branch prediction and speculative execution, it is probably zero measurable cost on a modern CPU. You also need proper exceptions to deal with runtime problems like Out-Of-Memory, Division-By-Zero etc.

[shelby3]

@keean wrote:

This is true, but due to branch prediction and speculative execution, it is probably zero measurable cost on a modern CPU.

Have you not read my recent post that argues that speculative pipelines are going away and besides they are not cost-free. They waste CPU power.

You also need proper exceptions to deal with runtime problems like Out-Of-Memory, Division-By-Zero etc.

What is the distinction between “proper” exceptions and try-catch-finally-throw? Why can’t the latter handle those exceptions?

Using the Error monad the source language syntax looks like throw/catch but we translate this into normal functions with an error return value in the intermediate language in the compiler.

What is the advantage of that?

I think.we should think in terms of transactions with concurrent software. Transactions either complete successfully or roll-back (with an exception)

What does “transaction” mean in this context? I think of transactions as blocks of logic that must complete atomically and this leads to interleaving conflicts with other such transactions. Transactions must wait in queue for each other and this leads to livelocks and deadlocks.

[keean]

@shelby3

Have you not read my recent post that argues that speculative pipelines are going away and besides they are not cost-free. They waste CPU power.

Respectfully, I disagree. Unless they can recover the performance another way, speculative execution is not going anywhere on general purpose CPUs. They will fix the security without abandoning the concept. the gains are just too big. The Pentium was the first Intel speculative out-of-order processor, and the whole architecture relies on the RISC core being fed micro-instructions from the re-order-buffer. To abandon this architecture would mean going back to a 486 era design and re-engineering from there. Its just too expensive and we would lose too much from it. The next gen Intel chips already have fixes for Spectre and Meltdown without losing SOOO, and AMD have SOOO and may not be vulnerable.

What is the distinction between “proper” exceptions and try-catch-finally-throw? Why can’t the latter handle those exceptions?

The syntax is not the exception mechanism. We are discussing the way the compiler implements those exceptions in the generated code. It can generate extra return values and use the error monad, or it can use stack unwinding and jumps. CPU hardware exceptions will always require the unwind and jump approach as we cannot continue after a division by zero, or out of memory, for example.

What is the advantage of that?

Lower cost if there is a failure.

What does “transaction” mean in this context? I think of transactions as blocks of logic that must complete atomically and this leads to interleaving conflicts with other such transactions. Transactions must wait in queue for each other and this leads to livelocks and deadlocks.

Exactly, we can thing of a program as a state-machine. It has a state and a set of transitions. We want each transition to either complete or be as if it never happened. When you have partially completed transitions, that is one way software can go wrong. Basically we are saying that software should never be left in a partially complete state, it should either look as if the action never happened, or it has completed. In terms of the blockchain, I have either paid you a bitcoin, or I have not, any partial state would be a serious error.

[shelby3]

@keean wrote:

Unless they can recover the performance another way, speculative execution is not going anywhere on general purpose CPUs.

We can. That is one of the main points of the ACM article that I linked to. Have you read it?

And we will also stop wasting so much heat and battery power on heat.

And we will have more secure computers.

And faster, more scalable software.

And we need to design our PLs for this new reality.

And goodbye C.

I hope you take the time to read my post, Eric Raymond’s post, and the original ACM article.

They will fix the security without abandoning the concept.

They can’t.

The next gen Intel chips already have fixes for Spectre and Meltdown without losing SOOO

There are unbounded number of such leaks hiding like an iceberg because of the fundamental anti-pattern. All they can do is play Whac-A-Mole with the chaos of it.

To abandon this architecture would mean going back to a 486 era design and re-engineering from there. Its just too expensive and we would lose too much from it.

There have already been chips for the new paradigm. I suggest you read the ACM article. The major cost is the inertia in software and broken PLs such as C. Also we already have very performant GPUs, which is basically what we need.

Respectfully, I disagree.

Respectfully, I do not think you have looked at this from the correct perspective.

You’re wrong on this one. I promise you.

The upcoming rate of change in the world is going to shock you. The dam is about to burst in so many ways to disintermediate all this wrongly designed shit in the world, including 10 years of waste on shit PLs like JavaScript, 20 years of waste on C after Moore’s Law stopped scaling serially, the entire nation-state and debt paradigm is going bye-bye, the entire nation-state fiat monetary system is being disintermediated by decentralization as we speak, etc..

https://steemit.com/cryptocurrency/@anonymint/bitcoin-rises-because-land-is-becoming-worthless

https://steemit.com/money/@anonymint/countries-vulnerable-to-economic-devastation-soon

https://steemit.com/programming/@anonymint/c-is-not-a-low-level-programming-language

https://steemit.com/tax/@anonymint/taxes-destroy-all-of-us-rich-or-poor

[keean]

@shelby3

We can. That is one of the main points of the ACM article that I linked to. Have you read it?

Yes, and I disagree with it. Only certain operations are vectorisable, and we have had vector architectures since the CRAY. They have all died out in favour of parallel nodes running Intel x86 chips. Parallel architectures like GPUs are great at certain specific tasks, but are not great at general purpose computing. There is no 'new architecture', we either solve the problems, or go back to 486 (at a higher clockspeed) performance.

Intel's next generation includes fixes for these attacks. Expect to see more fixes, I really don't think the x86 juggernaught can be stopped. You have to sell huge quantities of these CPUs to afford the fabrication plants to make them. Intel invests a lot of its CPU revenue into building these billion dollar facilities, and it keeps itself ahead in the fab-technology business. In this way Intel's chips are nearly a whole generation ahead of the rest of the market. People buy CPUs on price/performance, so Intel are already at an advantage. Add to this that no other architecture has been able to come close to speculative-out-of-order (not even VLIW that was behind Intels Itanium and ment to be the next generation, only to be out-performed by AMD's 64 bit x86 variant, that forced Intel to release a 64bit x86 or risk being left behind).

https://threatpost.com/intel-details-cpu-virtual-fences-fix-as-safeguard-against-spectre-meltdown-flaws/130501/