`@stableABI` SIP. #654

DarkDimius · 2017-01-13T14:19:18Z

TLDR:
This is intended to be @tailrec - like annotation that will fail compilation in case emitted methods or their signatures are different from those written by users. In order for @stableabi compatible class to succeed compilation there should be: no methods synthesized and no signatures changed.

The intended use-case is providing binary compatibility across major Scala versions by ensuring that a class or a trait is compiled as written by developer.

UPD: changed name of the annotation and the SIP.

heathermiller · 2017-01-13T14:23:37Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+
+__Dmitry Petrashko__
+
+__first submitted 13 February 2017__


It's currently January, not February yet ;-)

densh · 2017-01-13T14:41:02Z

This seems to be a replacement for java-defined interfaces, why not name it @interface to make crystal clear what it means? Binary compatibility is quite a broad term.

smarter · 2017-01-13T14:51:20Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+will be compatible across major version of Scala. 
+
+## Use Cases
+Dotty currently uses java defined interfaces as public API for SBT in order to ensure binary compatibility. 


Actually dotty-interfaces is not used by sbt at all. dotty-sbt-bridge depends on dotty itself, and sbt will automatically compile it from source against the current version of dotty (this is nothing new, the same thing happens with the scalac sbt bridge).
However, dotty-interfaces is used by IntelliJ IDEA.

DarkDimius · 2017-01-13T15:42:49Z

@densh, it does not only apply to interfaces. It also applies to classes.

Given that there's an assumption that this feature is rarely used and only used by advanced users, I'd prefer not to give it a common name such as @interface.

milessabin · 2017-01-13T16:44:46Z

I'm not clear on the benefits of adding to the language specification for such an apparently narrow use case. Are there any concrete examples of where this might be useful other than the SBT bridge? If not then I'm not sure why a Java shim or some other ad hoc approach isn't adequate and appropriate.

FWIW, it seems that the criteria for @binaryCompatible more or less correspond to those for a direct unencoded mapping to Java language artefacts, so maybe @java would be a better name?

DarkDimius · 2017-01-13T17:04:49Z

@milessabin thanks for your comment.

FWIW, it seems that the criteria for @binaryCompatible more or less correspond to those for a direct unencoded mapping to Java language artefacts, so maybe @java would be a better name?

Inside the bodies of methods one can use all the features of Scala, as they won't be seen in the binary API observed from outside. I think I should be explicit about it and include examples that illustrate this. Thank you for pointing this out.

travisbrown · 2017-01-13T17:10:14Z

👍, but also 👍 to reconsidering the name. In the cases where I've personally wanted something like this, the goal has been ease of Java interop in a multi-language codebase, not primarily binary compatibility as I think most people would understand it.

dragos · 2017-01-14T09:09:51Z

I think this would be a very useful, but I agree with previous commenters that the name should be more specific. Something that suggests that only platform signatures are being used.

aalleexxeeii · 2017-01-15T21:11:55Z

Do you mean absolutely any synthetic construct? For instance, should bit-fields for lazy values be considered as those? Or stuff like MODULE$?

xeno-by · 2017-01-17T19:57:57Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+
+## Introduction ##
+
+Scala is a language which evolves fast and thus made a decision to only promise binary compatibility across minor releases.


This is a very apt description. I wonder whether there are publications/talks that provide more context on the matter.

What's a minor release and what's a major release? There are different conventions that people follow in our community.

I'll add a link defining both terms. Thank you.

xeno-by · 2017-01-17T20:23:45Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+
+Scala is a language which evolves fast and thus made a decision to only promise binary compatibility across minor releases.
+At the same time, there is a demand to develop APIs that live longer than a major release cycle of Scala.
+This SIP introduces an annotation `@binaryCompatible` that checks that `what you write is what you get`.


I agree with other commenters that a different name would be beneficial. To me, saying that something is binary compatible, immediately raises the question "with what?". This annotation doesn't answer that question.

While I agree that the @binaryCompatible name isn't perfect, I didn't yet find a better one.
As this annotation is intended to be rarely used and only by advanced users, I don't think we should concentrate that much on the name, but I welcome name suggestions.

xeno-by · 2017-01-17T20:25:16Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+that would be locally compiled.
+
+There's also a use-case of defining a class which is supposed to be also used from Java. 
+`@binaryCompatible` will ensure that there are no not-expected methods that would show up in members of a class or an interface.


s/not-expected/unexpected/

xeno-by · 2017-01-17T20:26:31Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+As long as signatures of methods in source is not changed, `@binaryCompatible` annotated class 
+will be compatible across major version of Scala. 
+
+## Use Cases


Can we number these use cases, i.e. have this section look as follows:

1) Use case 1 <explanation> 2) Use case 2 <explanation> etc

xeno-by · 2017-01-17T20:28:52Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+
+## Overview ##
+In order for a class or a trait to succeed compilation with the `@binaryCompatible` annotation it has to be:
+  - defined on the top level;


Why limit this to just top-level classes? If I recall correctly, MiMa also flags changes to nested classes and synthetic stuff generated for things like new {} and old-style lambdas.

Why limit this to just top-level classes?

Allowing to define @stableABI inner classes will most likely fail compilation due to second reason, as they will get successors synthesized to access the outer class;

there are several subtleties in how outer class can affect compilation of inner class, eg by converting private method to public;

use-case of an inner class can be easily handled by defining a top-level supper class and making inner class extend it;

In order to have a more user-friendly behavior than 1 & 2, and given presence of a simple workaround, I've decided to prohibit it.

xeno-by · 2017-01-17T20:29:12Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+  - defined on the top level;
+  - use a subset of Scala that during compilation does not require changes to public API of the class, including
+     - synthesizing new members, either concrete or abstract;
+     - changing binary signatures of existing members, either concrete or abstract;


Are changes to the public API of class's companion also prohibited?

xeno-by · 2017-01-17T20:29:23Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+check which is sufficient to promise binary compatibility. 
+
+## Overview ##
+In order for a class or a trait to succeed compilation with the `@binaryCompatible` annotation it has to be:


What about objects?

xeno-by · 2017-01-17T20:33:26Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+
+## Features that will fail compilation with `@binaryCompatible`
+The features listed below have complex encodings that may change in future versions. We prefer not to compromise on them.
+Most of those features can be simulated in a binary compatible way by writing a verbose re-impelemtation 


xeno-by · 2017-01-17T20:35:09Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+  - default arguments;
+  - default methods. See Addendum;
+  - constant types(both explicit and inferred);
+  - inline.


Why exclude inline?

The idea is that @stableABI classes don't need @scalaSignature or @tasty to compile against. In order for inline to work, we'll need to be able to read the trees from those sections and make assumptions about those trees, that can be dependent on compiler version.

I propose to go conservative route here, we'll have an opportunity to allow more features later, but we won't have an opportunity to remove them.

xeno-by · 2017-01-17T20:40:38Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+## Compilation scheme ##
+No modification of typer or any existing phase is planned. The current proposed scheme introduces a late phase that runs before the very bytecode emission that checks that:
+ - classes and traits annotated as  `@binaryCompatible` are on the top level; 
+ - no non-private members where introduced inside classes and traits annotated as  `@binaryCompatible` by compiler using phase travel;


What's phase travel?

I don't think Compilation scheme section needs explanation of common terms used inside both compilers.

xeno-by · 2017-01-17T21:00:24Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+At the same time, there is a demand to develop APIs that live longer than a major release cycle of Scala.
+This SIP introduces an annotation `@binaryCompatible` that checks that `what you write is what you get`.
+It will fail compilation in case emitted methods or their signatures 
+are different from those written by users. 


s/users/authors/

xeno-by · 2017-01-17T21:02:12Z

sips/pending/_posts/2017-01-13-binary-compatibility.md

+
+__first submitted 13 January 2017__
+
+## Introduction ##


I think that this document would benefit from the definitions of the following terms:

Binary compatibility (both the intransitive and transitive variations used in the text below)

Public API

Binary signature

The notion of signatures being different from those written by their authors.

I'd suggest having a small example that introduces and illustrates these notions.

DarkDimius · 2017-01-17T21:03:23Z

@xeno-by, thank you for your through review.

As most questions are asking about definitions of terms, I think I'll introduce a new section that defines terms used in this SIP. It's indeed harder to follow than most sips as current writing assumes knowledge of both compiler internals and JVM spec.

DarkDimius · 2017-01-17T21:06:41Z

Speaking of the name, I welcome suggestions. @interop is the best alternative I came up with.
My personal opinion is that the current name isn't that bad considering that this annotation is for very advanced users(framework authors) and should be rarely used.

DarkDimius · 2017-01-18T10:57:40Z

Just had a Dotty meeting, which was also joined by @retronym.
It seems that everyone present on the meeting has converged on the name @stableABI.

olafurpg · 2017-02-10T14:17:47Z

FWIW. With @stableABI, I imagine it might become popular to release small libraries without cross compilation. A hypothetical example if Function and Tuple were to get stable ABIs, libraries such as geny could take advantage of this.

DarkDimius · 2017-02-13T09:24:04Z

The CI seems to fail, without reporting any errors.
@heathermiller @jvican, could you please help with getting this merged?
Afaik it should be discussed tomorrow on the SIP meeting, so it would be nice to get it in.

heathermiller · 2017-02-13T09:34:23Z

It won't be displayed even if the CI succeeds. The GitHub pages builds are also timing out and failing. So until we can move this site to chara/alaska, or fix #607, this page can't be displayed on the site.

DarkDimius · 2017-02-13T09:35:53Z

@heathermiller, ouch. Thanks for having a look.

jvican · 2017-02-13T10:18:40Z

Hey @DarkDimius, a fix is on the way #684. When we merge it, can you rebase your changes so that Travis doesn't time out?

jvican · 2017-02-13T15:10:56Z

Can you please rebase on top of #685?

DarkDimius · 2017-02-13T15:12:33Z

@jvican thank you for the ping. Rebased.

xeno-by · 2017-02-13T23:16:54Z

I'm a bit confused about the current name for the annotation. Is it @stableABI or @binaryCompatible? From what I can see, the PR title has been updated, but the SIP text hasn't.

DarkDimius · 2017-02-14T13:30:43Z

@xeno-by thanks for pinging. Apperantly when fixing the repo we've rebased and merged the wrong version. #691 is the up-to-date one and the one that was reviewed in this PR.

DarkDimius force-pushed the bincompat branch from cd92f76 to a0b6542 Compare January 13, 2017 14:20

heathermiller reviewed Jan 13, 2017

View reviewed changes

jvican requested a review from xeno-by January 13, 2017 14:44

jvican added the Ongoing SIP label Jan 13, 2017

smarter reviewed Jan 13, 2017

View reviewed changes

xeno-by reviewed Jan 17, 2017

View reviewed changes

jvican assigned xeno-by Jan 23, 2017

DarkDimius changed the title ~~@binaryCompatible SIP.~~ @stableABI SIP. Feb 10, 2017

DarkDimius added 2 commits February 13, 2017 16:11

Inital draft of @binaryCompatible proposal.

0a8d6aa

@binaryComatible: fixes and clarifications.

810637b

DarkDimius force-pushed the bincompat branch from 4e13db2 to 810637b Compare February 13, 2017 15:11

jvican merged commit aa988a6 into scala:master Feb 13, 2017


		## Introduction ##

		Scala is a language which evolves fast and thus made a decision to only promise binary compatibility across minor releases.

@stableABI SIP. #654

@stableABI SIP. #654

Conversation

DarkDimius commented Jan 13, 2017 • edited Loading

Choose a reason for hiding this comment

densh commented Jan 13, 2017

Choose a reason for hiding this comment

DarkDimius commented Jan 13, 2017

milessabin commented Jan 13, 2017

DarkDimius commented Jan 13, 2017

travisbrown commented Jan 13, 2017

dragos commented Jan 14, 2017

aalleexxeeii commented Jan 15, 2017

xeno-by Jan 17, 2017 • edited Loading

Choose a reason for hiding this comment

xeno-by Jan 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xeno-by Jan 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xeno-by Jan 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xeno-by Jan 17, 2017 • edited Loading

Choose a reason for hiding this comment

DarkDimius commented Jan 17, 2017

DarkDimius commented Jan 17, 2017 • edited Loading

DarkDimius commented Jan 18, 2017

olafurpg commented Feb 10, 2017 • edited Loading

DarkDimius commented Feb 13, 2017

heathermiller commented Feb 13, 2017

DarkDimius commented Feb 13, 2017

jvican commented Feb 13, 2017

jvican commented Feb 13, 2017

DarkDimius commented Feb 13, 2017

xeno-by commented Feb 13, 2017

DarkDimius commented Feb 14, 2017

`@stableABI` SIP. #654

`@stableABI` SIP. #654

DarkDimius commented Jan 13, 2017 •

edited

Loading

xeno-by Jan 17, 2017 •

edited

Loading

xeno-by Jan 17, 2017 •

edited

Loading

xeno-by Jan 17, 2017 •

edited

Loading

xeno-by Jan 17, 2017 •

edited

Loading

xeno-by Jan 17, 2017 •

edited

Loading

DarkDimius commented Jan 17, 2017 •

edited

Loading

olafurpg commented Feb 10, 2017 •

edited

Loading