Audit which private/fork (-Y*) options should be lifted to one of the stable categories

[Gedochao]

Compiler version

nightly

Context

A number of -Y* options are considered private (unstable), while in reality they're widely used and could be considered production-ready.
We need to look at them case-by-case, potentially lift them to their appropriate categories and deprecate their -Y* aliases, where applicable.
Consider this an umbrella ticket to tidy this up.

[SethTisue]

Perhaps there are -Y options that really ought to only exist in nightly/experimental compilers? A tradition of using -Y for experimental stuff developed over the many years before the "experimental" concept arose and was codified, but we could reconsider that tradition now.

[Gedochao]

Perhaps there are -Y options that really ought to only exist in nightly/experimental compilers?

I'm not suggesting to get rid of -Y* options completely.

For example, options related to capture checking (like -Ycc-debug or -Ycc-no-abbrev) can probably stay the way they are. We do, however, have a bunch of older options which stayed in this category for a long while and their status could perhaps be reconsidered.

[odersky]

Most -Y options are debug options to get more insight into what the compiler does or they are options relating to experimental features. Another bunch is options that tweak the compiler output, but that need expert knowledge to understand what they do. I think these can stay. But the following could be reclassified as -X options since they have been used for a while and are compiler-independent.

  val YnoImports: Setting[Boolean] = BooleanSetting("-Yno-imports", "Compile without importing scala.*, java.lang.*, or Predef.")
  val Yimports: Setting[List[String]] = MultiStringSetting("-Yimports", helpArg="", "Custom root imports. If set, none of scala.*, java.lang.*, or Predef.* will be imported unless explicitly included.")
  val YnoGenericSig: Setting[Boolean] = BooleanSetting("-Yno-generic-signatures", "Suppress generation of generic signatures for Java.")
  val YnoPredef: Setting[Boolean] = BooleanSetting("-Yno-predef", "Compile without importing Predef.")

And the following are about features that are in train of stabilization. They need a decision when they are stable in which case they could be promoted to -X options or become default behavior.

  val YexplicitNulls: Setting[Boolean] = BooleanSetting("-Yexplicit-nulls", "Make reference types non-nullable. Nullable types can be expressed with unions: e.g. String|Null.")
  val YcheckInit: Setting[Boolean] = BooleanSetting("-Ysafe-init", "Ensure safe initialization of objects.")
  val YcheckInitGlobal: Setting[Boolean] = BooleanSetting("-Ysafe-init-global", "Check safe initialization of global objects.")

[odersky]

There are also options related to old Scala versions like -Ykind-projector. These will go away eventually. Since everybody is now used to their -Y status I don't think it's useful to upgrade them to -X before deleting them in some future version.

[odersky]

Actually, let's hold off on the imports settings.

 val YnoImports: Setting[Boolean] = BooleanSetting("-Yno-imports", "Compile without importing scala.*, java.lang.*, or Predef.")
  val Yimports: Setting[List[String]] = MultiStringSetting("-Yimports", helpArg="", "Custom root imports. If set, none of scala.*, java.lang.*, or Predef.* will be imported unless explicitly included.")
  val YnoPredef: Setting[Boolean] = BooleanSetting("-Yno-predef", "Compile without importing Predef.")

We need to have a (SIP) discussion about that. In my view it is a code smell to fiddle with imports via a compiler option and not in source. So -Yimports should definitely seen as experimental (and undesirable, maybe we should even remove it?). For replacing -YnoImports and -YnoPredef we need better ways to do that in source. Right now we can do it for -YnoPredef like this:

import Predef.{String as _}

object Test:
  summon[Int] // error: not found: summon

But that's very roundabout: (1) We need to know that the repeated import actually replaces the default one. (2) We need to import something and then rename it to nothing.

It would be great if this worked for at least all root imports with a better syntax. My preferred syntax would be:

 import Predef.{}
 import scala.{}
 import java.lang.{}

Or, as the note in Importinfo.scala suggests, allow the "import override" for all imports and demand an explicit "override". I.e.

 override import Predef.{}
 override import scala.{}
 override import java.lang.{}

In any case. I would not elevate -Y import options to officia;l status before this question is settled.

[odersky]

Here's the note:

  /** The root import symbol hidden by this symbol, or NoSymbol if no such symbol is hidden.
   *  Note: this computation needs to work even for un-initialized import infos, and
   *  is not allowed to force initialization.
   *
   *  TODO: Once we have fully bootstrapped, I would prefer if we expressed
   *  unimport with an `override` modifier, and generalized it to all imports.
   *  I believe this would be more transparent than the current set of conditions. E.g.
   *
   *      override import Predef.{any2stringAdd => _, StringAdd => _, _} // disables String +
   *      override import java.lang.{}                                   // disables all imports
   */
  def unimported(using Context): Symbol =

[odersky]

Maybe there are other options like -YnoGenericSig that suppress some output in the generated files. Maybe these could be moved to -X since the goal is presumably to cut down on generated file size, which easy to understand.

[SethTisue]

Perhaps there are -Y options that really ought to only exist in nightly/experimental compilers?

Let me expand on this a bit.

In the chat rooms, I've lost count of how many puzzled people we've had to inform that -Y is supposed to be a signal that a feature is experimental and/or internal. We ourselves know this, but normal Scala programmers cannot be expected to know it — especially newcomers. -X, -Y, it's just alphabet soup to them.

It's especially common that people try -Yexplicit-nulls, have problems all over the place even on simple examples, and are confused and/or disappointed.

I've just opened scala/docs.scala-lang#2976 about improving the Scala 3 reference documentation to try and head off this problem, but I'm also interested in discussing possible changes like making known-dubious features like -Yexplicit-nulls less accessible.

[lrytz]

About -Yimports, this feature can be very valuable for managing migrations / cross-building in large codebases.

It seems the most pressing issue is -Y-enabled language features that should instead be handled the same as other experimental language features. Since 3.4.0 has the -experimental flag, all experimental features are available on release compilers anyway, so there is probably no reason to use a -Y flag instead.

[odersky]

We have to strike a balance but putting things under more locks is not the right way to proceed. -Yexplicit-nulls is a good example. If it was so well hidden that nobody could use it, explicit nulls will never see the light of day. That turns Scala 3 into a Scala 2 where things never change and the language becomes stagnant.

If we were a big organization we could develop a new feature behind walls, test it thoroughly and release it when it is fully cooked. But we are not. We rely on volunteer work without fixed schedules. So we can discuss about names. I personally think that all -Y should off-limits for non-expert users who don't know what they are doing. Then we don't need to invent yet another name and confuse people.

And, -Yimports should stay a -Y option because it is really bad and would be a huge problem if used widely. If an expert user needs it for cross-building a large library, fine. -Y options are for that. But we should never promote this as good programming practice.

[lrytz]

-Yimports should stay a -Y

Sure, I was just pointing out that there's a valid use case. To me it's fine if -Y is a grab bag of debugging options and some advanced tools.

For experimental language features I think it's worthwhile having one mechanism.

[odersky]

For experimental language features I think it's worthwhile having one mechanism.

Generally I agree. There is a complication that the three experimental extension options we have, (explicit nulls and the two check-init options) need to be turned on globally. They don't work (yet) for individual compilation units. So that means we can't use a language import for them. But we could insist on better naming. Maybe

• Yexperimental-explicit-nulls

?

[lrytz]

Or require -experimental? That would also add the @experimental annotation to things compiled with these features, which would be desirable?

[odersky]

I am not sure it would be desirable. These are checking options that don't change the binary or tasty output in any way. So I don't see why they would require transitiveness.

[olhotak]

I discussed with @liufengyun . -Ysafe-init is ready to be renamed to -Wsafe-init. (Is there a policy/procedure to follow to do that, or just make a PR?) -Ysafe-init-global is not ready yet. Both these options just turn on checkers/linters. They don't affect generated code at all.

-Yexplicit-nulls does affect generated code and does set a Tasty attribute to indicate that code was generated with this option. My hope is that it is very close to ready to become stable and turned on by default, but we'd need to discuss with @noti0na1 .

[odersky]

@olhotak @liufengyun I think we can just make it a PR for -Ysafe-init.

[som-snytt]

-Yimports is really bad

The people who hate that it doesn't work the way they want would agree.

I see it as a REPL feature (where it also avoided compilations on startup). But maybe scala-cli using can improve on that UX.

The use case for "ordinary user Joe" is starting a REPL with everything plugged into the namespace, without experimenting with "what imports are needed". Also, an imports management command in REPL doesn't care about root imports, so that these project imports are hidden from Joe, who also doesn't care (at least as a beginner trying out some code).

The limitation of import syntax is order of typechecking, unless "imports that opt-out of root imports" are required to be located at the top of the text file.

[SethTisue]

I see it as a REPL feature (where it also avoided compilations on startup). But maybe scala-cli using can improve on that UX

(ticket on that is VirtusLab/scala-cli#2732)

[nicolasstucki]

We can probably remove -Yforce-inline-while-typing. This was a flag added to emulate pre-3.0.0 behavior to avoid breaking early adopters' code.

[odersky]

About the -Yimport options. I think it's completely OK to compile with some customized root imports. But we should not express it using external options, but instead support it in the source code itself. I worked out a proposal in #19740.

[nicolasstucki]

We could use a dotty.tools.dotc.Printers instead of -Ydebug-macros. This is an internal debugging tool.

[SethTisue]

Reminder that -V exists as a category for flags that only increase verbosity during compilation. Scala 2 examples include -Vstatistics, -Vdebug, -Vlog, -Vprint, -Vprint-pos, -Vprint-types, -Vprint-args, -Vshow-symowners...

These are flags that one would never leave enabled in one's build, so I don't think it's necessary to have any guarantees about -V flags changing or going away. It's a category, but not a "stable" category.

In Scala 3, we already have some -V flags: -Vprint, -Vshow-phases... but they tend to have -X aliases, and I think on the Scala 3 team there isn't a culture of using -V, probably because -V didn't exist for many years while Dotty was being developed. But I'm suggesting we change that culture — it's only a small mental shift.

I think awareness in the community of -V is not super high, but people don't know what -Y means either. We should have a doc page that explains the categories, a blog post that highlights the existence of the doc page, and we can link to the doc page from the release notes of the version where the -Y reorg lands.

[Gedochao]

After doing an initial analysis with @szymon-rd, here's an updated list of options to be lifted out of -Y:

• lift to -X
  • -Yresolve-term-conflict
  • -YnoGenericSig
  • -Ydumpclasses
  • -YjarCompressionLevel
  • -YkindProjector
  • -YdropComments
  • -YcookComments
  • -YreadComments
  • -YnoDecodeStacktraces
  • -YnoEnrichErrorMessages
  • -YdebugMacros
  • -YjavaTasty
  • -YjavaTastyOutput
  • -YallowOutlineFromTasty
• lift to -W
  • -Wsafe-init
• to be removed
  • --Yforce-inline-while-typing
• TBD
  • -YfromTastyIgnoreList
  • -YoutputOnlyTasty

Other than that, we should improve the help message for -Y to make it clear to users what they're for: they're unstable options with no support guarantees, aimed to be used by compiler maintainers.
The current wording of private options does not set up clear expectations.

[odersky]

This looks mostly good to me, except for

• -YcheckReentrant: this is really intended only for bootstrapping the compiler itself. We are not prepared at the moment to make that a generally available option with docs and so on. So, maybe leave it as a -Y option.

• -YfromTastyIgnoreList, -YallowOutlineFromTasty I don't really know what these do and whether they are stable options.

[Gedochao]

@odersky Ok, removed -YcheckReentrant from the list, and moved -YfromTastyIgnoreList & -YallowOutlineFromTasty.
The latter can be treated as out of scope for this ticket, in case we are not sure if they are ready to be lifted.

@rochala I updated the list in #19693 (comment), let's treat it as the requirements for closing this issue.