Stacktrace not trimmed or trimmed stacktrace ruined by additionnal full stacktrace

[johnsbrew]

Type of issue: bug report
Impact: no functional change
Development Phase: request

Other information

Although chisel does a great job at trimming stack trace, when throwing an exception during chisel elaboration, firrtl.options.StageError still does (additionally) throw the whole stack trace.
While I do understand it remains very convenient for internal debug purpose, it is quite a pain because the beautiful trimmed stack-trace is hidden from user.

If the current behavior is a bug, please provide the steps to reproduce the problem:
Note that the additional function & custom Exception are not required to demonstrate the behaviour.
There are here only to add some user-space call in the stacktrace.

import chisel3._
import chisel3.stage.ChiselStage

case class UserException(msg: String) extends Exception(msg)

class HwClass(doThrow: Boolean = false) extends MultiIOModule {
  val in = IO(Input(Bool()))
  val out = IO(Output(Bool()))
  
  def f(b: Bool): Bool = {
    if(doThrow) throw UserException("User error message")
    b
  }
  
  out := f(in)
}

object SimpleExceptionIssue extends App {
  (new ChiselStage).emitVerilog(new HwClass(doThrow = true))
}

What is the current behavior?

[info] [0,003] Elaborating design...
[error] <user error message>
[error] 	...
[error] 	at <user code call>
// <potentially many more lines of user calls> //
[error] 	at <user code call>
[error] 	... (Stack trace trimmed to user code only, rerun with --full-stacktrace if you wish to see the full stack trace)
[error] (run-main-4c) firrtl.options.StageError:
[error] firrtl.options.StageError:
[error] 	at chisel3.stage.ChiselStage.run(ChiselStage.scala:60)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Stage.$anonfun$transform$5(Stage.scala:47)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.Stage.$anonfun$transform$3(Stage.scala:47)
[error] 	at logger.Logger$.$anonfun$makeScope$2(Logger.scala:166)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at logger.Logger$.makeScope(Logger.scala:164)
[error] 	at firrtl.options.Stage.transform(Stage.scala:47)
[error] 	at firrtl.options.Stage.execute(Stage.scala:58)
[error] 	at chisel3.stage.ChiselStage.emitVerilog(ChiselStage.scala:117)
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:431)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[error] Caused by: chisel3.internal.ChiselException: Exception thrown when elaborating ChiselGeneratorAnnotation
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.elaborate(ChiselAnnotations.scala:65)
[error] 	at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:24)
[error] 	at scala.collection.immutable.List.flatMap(List.scala:366)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:23)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:16)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:278)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.DependencyManager.transform(DependencyManager.scala:269)
[error] 	at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255)
[error] 	at firrtl.options.PhaseManager.transform(DependencyManager.scala:436)
[error] 	at chisel3.stage.ChiselStage.run(ChiselStage.scala:46)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Stage.$anonfun$transform$5(Stage.scala:47)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.Stage.$anonfun$transform$3(Stage.scala:47)
[error] 	at logger.Logger$.$anonfun$makeScope$2(Logger.scala:166)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at logger.Logger$.makeScope(Logger.scala:164)
[error] 	at firrtl.options.Stage.transform(Stage.scala:47)
[error] 	at firrtl.options.Stage.execute(Stage.scala:58)
[error] 	at chisel3.stage.ChiselStage.emitVerilog(ChiselStage.scala:117)
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:431)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[error] Caused by: <user error message>
[error] 	at <user code call>
// <potentially many more lines of user calls> //
[error] 	at <user code call>
[error] 	at chisel3.Module$.do_apply(Module.scala:54)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.$anonfun$elaborate$1(ChiselAnnotations.scala:60)
[error] 	at chisel3.internal.Builder$.$anonfun$build$1(Builder.scala:642)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:639)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:635)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.elaborate(ChiselAnnotations.scala:60)
[error] 	at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:24)
[error] 	at scala.collection.immutable.List.flatMap(List.scala:366)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:23)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:16)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:278)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.DependencyManager.transform(DependencyManager.scala:269)
[error] 	at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255)
[error] 	at firrtl.options.PhaseManager.transform(DependencyManager.scala:436)
[error] 	at chisel3.stage.ChiselStage.run(ChiselStage.scala:46)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Stage.$anonfun$transform$5(Stage.scala:47)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.Stage.$anonfun$transform$3(Stage.scala:47)
[error] 	at logger.Logger$.$anonfun$makeScope$2(Logger.scala:166)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at logger.Logger$.makeScope(Logger.scala:164)
[error] 	at firrtl.options.Stage.transform(Stage.scala:47)
[error] 	at firrtl.options.Stage.execute(Stage.scala:58)
[error] 	at chisel3.stage.ChiselStage.emitVerilog(ChiselStage.scala:117)
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:431)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at <user main code for calling chisel call>
[error] 	at <user main code for calling chisel call>
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1
[error] Total time: 7 s, completed 27 janv. 2021 à 11:19:35

Note that this becomes even worse when using object ChiselStage rather than class ChiselStage such as

object SimpleExceptionIssue extends App {
  ChiselStage.emitVerilog(new HwClass(doThrow = true))
}

In such a case you don't even get the trimmed stacktrace and the user error message is hidden as a cause:

[info] running SimpleExceptionIssue
[info] [0,017] Elaborating design...
[error] (run-main-5f) chisel3.internal.ChiselException: Exception thrown when elaborating ChiselGeneratorAnnotation
[error] chisel3.internal.ChiselException: Exception thrown when elaborating ChiselGeneratorAnnotation
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.elaborate(ChiselAnnotations.scala:65)
[error] 	at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:24)
[error] 	at scala.collection.immutable.List.flatMap(List.scala:366)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:23)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:16)
[error] 	at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:278)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.DependencyManager.transform(DependencyManager.scala:269)
[error] 	at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255)
[error] 	at firrtl.options.PhaseManager.transform(DependencyManager.scala:436)
[error] 	at chisel3.stage.ChiselStage$.emitVerilog(ChiselStage.scala:236)
[error] 	at fpgaTests.utils.SimpleExceptionIssue$.delayedEndpoint$fpgaTests$utils$SimpleExceptionIssue$1(PlayGroundSpec.scala:202)
[error] 	at fpgaTests.utils.SimpleExceptionIssue$delayedInit$body.apply(PlayGroundSpec.scala:200)
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:431)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at fpgaTests.utils.SimpleExceptionIssue$.main(PlayGroundSpec.scala:200)
[error] 	at fpgaTests.utils.SimpleExceptionIssue.main(PlayGroundSpec.scala)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[error] Caused by: fpgaTests.utils.UserException: User error message
[error] 	at fpgaTests.utils.HwClass.f(PlayGroundSpec.scala:192)
[error] 	at fpgaTests.utils.HwClass.$anonfun$new$8(PlayGroundSpec.scala:196)
[error] 	at chisel3.Data.$anonfun$$colon$eq$1(Data.scala:524)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] 	at chisel3.internal.prefix$.apply(prefix.scala:32)
[error] 	at chisel3.Data.$colon$eq(Data.scala:524)
[error] 	at fpgaTests.utils.HwClass.<init>(PlayGroundSpec.scala:196)
[error] 	at fpgaTests.utils.SimpleExceptionIssue$.$anonfun$new$9(PlayGroundSpec.scala:202)
[error] 	at chisel3.Module$.do_apply(Module.scala:54)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.$anonfun$elaborate$1(ChiselAnnotations.scala:60)
[error] 	at chisel3.internal.Builder$.$anonfun$build$1(Builder.scala:642)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:639)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:635)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.elaborate(ChiselAnnotations.scala:60)
[error] 	at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:24)
[error] 	at scala.collection.immutable.List.flatMap(List.scala:366)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:23)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:16)
[error] 	at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:278)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.DependencyManager.transform(DependencyManager.scala:269)
[error] 	at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255)
[error] 	at firrtl.options.PhaseManager.transform(DependencyManager.scala:436)
[error] 	at chisel3.stage.ChiselStage$.emitVerilog(ChiselStage.scala:236)
[error] 	at fpgaTests.utils.SimpleExceptionIssue$.delayedEndpoint$fpgaTests$utils$SimpleExceptionIssue$1(PlayGroundSpec.scala:202)
[error] 	at fpgaTests.utils.SimpleExceptionIssue$delayedInit$body.apply(PlayGroundSpec.scala:200)
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:431)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at fpgaTests.utils.SimpleExceptionIssue$.main(PlayGroundSpec.scala:200)
[error] 	at fpgaTests.utils.SimpleExceptionIssue.main(PlayGroundSpec.scala)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[error] stack trace is suppressed; run last Test / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Test / runMain) Nonzero exit code: 1
[error] Total time: 26 s, completed 29 janv. 2021 à 10:18:31

What is the expected behavior?

[info] [0,003] Elaborating design...
[error] <user error message>
[error] 	...
[error] 	at <user code call>
// <potentially many more lines of user calls> //
[error] 	at <user code call>
[error] 	... (Stack trace trimmed to user code only, rerun with --full-stacktrace if you wish to see the full stack trace)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1
[error] Total time: 7 s, completed 27 janv. 2021 à 11:19:35

Please tell us about your environment:
- version: 3.4.1
- OS: Darwin 17.7.0 Darwin Kernel Version 17.7.0: Fri Oct 30 13:34:27 PDT 2020; root:xnu-4570.71.82.8~1/RELEASE_X86_64 x86_64

What is the use case for changing the behavior?
Nicer errors = happier users

Issue summary

• with ChiselStage.emitVerilog => stacktrace is not trimmed and is awful (user error hidden right in the middle)
• with (new ChiselStage).emitVerilog => stacktrace is trimmed but then the whole stacktrace is appended
• underlying issue: difference of behaviour between object & class ChiselStage
  • object ChiselStage relies on PhaseManager
  • class ChiselStage relies on firrtl.options.Stage (who is actually the culprit for throwing the additional stacktrace)

For me ChiselStage (and the entire Stage/Phase API) is one of the most obscure piece of code of the chisel/firrtl ecosystem although, several times, I spend hours (if not days) digging into it just to be able to customize our build flow a little bit...
And here we have... a companion object not even using its associated class but both somehow relying on the same underlying Stage/Phase API, with roughly the same result... but not quite!

There is a really serious underlying concern about trust here:

• how can we expect the generated verilog to be same in both case?
• Can we expect custom transforms to be applied in both case? In the same order?
• It's a real nightmare.

When I first posted about this issue in chipsalliance/firrtl#974 I thought I was looking for a quick fix... but as soon as Stage/Phase API gets involved, suprises occur and everything gets cumbersome...

So I would be happy to help about the stacktrace issue, but this entire file https://github.com/chipsalliance/chisel3/blob/master/src/main/scala/chisel3/stage/ChiselStage.scala needs some uniformization first... and on that one I would like to request help from @seldridge or @jackkoenig ?

[seldridge]

Sorry that you're having issues with this!

If you want stack trace trimming, you can use ChiselMain which can be an entry point to generators you have in your library.

E.g., you can do:

import chisel3._
import chisel3.stage.ChiselMain

class Foo(param: Boolean) extends Module {
  val io = IO(new Bundle {
    val in = Input(Bool())
    val out = Output(Bool())
  })

  io.out := ~io.in
  throw new Exception("barf")
}

object Main extends App {
  
  class Bar extends Foo(true) {
    override def desiredName = "Foo"
  }

  ChiselMain.main(Array("--module", classOf[Bar].getName) ++ args)
  
}

Scastie link for above.

There was never support for passing parameters directly into your module (thought I've prototyped this)... You should be able to add this in in your own Main object, though.

@johnsbrew wrote:

There is a really serious underlying concern about trust here:

• how can we expect the generated verilog to be same in both case?
• Can we expect custom transforms to be applied in both case? In the same order?
• It's a real nightmare.

😬 😬 😬
I understand that these can be real concerns...

The guarantee is that you'll get the same Verilog for the same set of transforms in the same order. A DependencyManager[A] is (supposed to be...) guaranteed to come up with the same solution for the same inputs. As long as you're running the same compiler phase with the same inputs I wouldn't worry about this.

(Have you experienced issues here?)

The split between ChiselStage and ChiselStage$ grew out of the request that there was a way to run Chisel without doing any file IO. This is why the utility methods in ChiselStage$ were created which contrasts with the behavior of the class.

There's also some organic growth here where the stack trace pruning is jankily swallowing the stack trace and printing a pruned version that pretends it's an actual stack trace, but is actually println'd. It would be better if the stack trace was actually pruned instead of doing this weird hack.

[johnsbrew]

If you want stack trace trimming, you can use ChiselMain which can be an entry point to generators you have in your library.

So now you are suggesting a third method that is behaving like neither of both previous ones ...
Not even to mention the poor API here... have you ever considered user-experience when designing such awful API: (Array("--module", classOf[Bar].getName)? Looks like users & usability are at the very bottom of your concerns.

And, looking further into it:
https://github.com/chipsalliance/firrtl/blob/v1.4.1/src/main/scala/firrtl/options/Stage.scala#L65:L80

Is this really your "solution", catching the culprit I identified earlier?

The split between ChiselStage and ChiselStage$ grew out of the request that there was a way to run Chisel without doing any file IO. This is why the utility methods in ChiselStage$ were created which contrasts with the behavior of the class.

As far as I digged into this issue, I found no way to avoid file IO since the API change with version 3.4.0 (in particular getting the verilog string without getting the verilog file written) so I don't really get your point here. The only acceptable solution here would be a global (or category-based: anno, fir, v) annotation preventing the actual file IO wherever required.

As long as you're running the same compiler phase with the same inputs I wouldn't worry about this.

Well, that's precisely the point, we don't really require the same phases (in ChiselStage and ChiselStage$) here so we don't really keep the control about what is done, and unless we activate the verbose stage/phase compilation output, we have no clues and no guarantees about what is happening in backstage.

Needless to say that I can't accept your answer here: no solutions, just an ugly workaround and no willingness to fix what has definitely been an issue for quite some time: chisel generation API is a wonderful undocumented mess with not only 2 but now 3 different ways to produce apparently the same output but with at least different runtime behaviors...

I am sorry about my dramatic tone here, and while I do understand some of the underlying reasons in your answer I haven't seen any improvement regarding this basic but crucial API, and I really want to emphasize how painful it is and insecure it feels working with it.

[edwardcwang]

Hi Jean, I understand that you are frustrated with the current API and its associated inconsistencies, and I too would be frustrated and upset in your case. I think we can all agree that we are all in the business of improving Chisel, and I did want to point out that Schuyler has made many great contributions to the Chisel ecosystem and I am not sure that we should say that he maliciously disregards users and usability - after all, we have all made mistakes in API design and there is certainly always more to learn. Perhaps we could all work together to find a way to better understand the current state of the art and figure out the right path to move forward on. Just 2 cents.

[johnsbrew]

Hi @edwardcwang , I cannot agree more with you :)
Let's be clear:

• the contribution of @seldridge to chisel/firrtl is quite incredible: I am quite aware of his huge work on the stage/phase API (and not only of course), with both numerous lines of code, and many schematics to explain the algorithms & principles of dependencies invalidation. I am also grateful for the time he spent reviewing/improving my contribution on preset.
• chisel3 itself is a great piece of work thanks to you: despite some inconsistencies as you said, it works! In my company we are about to achieve a shift from SystemVerilog to Chisel for all new module developments, given a team of (mainly) hardware developers quite used to their proprietary tools.
• and ... yes you're right I sometimes (often?) end up quite frustrated, and while I express myself noisily, please take no offense of it!

So what should be the next steps to work on here?

There's also some organic growth here where the stack trace pruning is jankily swallowing the stack trace and printing a pruned version that pretends it's an actual stack trace, but is actually println'd. It would be better if the stack trace was actually pruned instead of doing this weird hack.

Is this something I could help with as a first step?
Is there a global strategy for Exception handling in the chisel/firrtl stack? To be defined? To be upgraded/standardized?

Does the idea of disabling file IO with some annotations sounds reasonable? I think I could submit a PR for that. Would this be enough to drop the duplicated implementation of ChiselStage vs ChiselStage$?

I have unfortunately more ideas than time and API changes might (by design) be quite controversial so let me know if there is something useful I can work on.

And once again thank you all for the good work on chisel, I am sorry if anyone felt insulted, it was definitely not my intention at all.

[seldridge]

And, looking further into it:
https://github.com/chipsalliance/firrtl/blob/v1.4.1/src/main/scala/firrtl/options/Stage.scala#L65:L80

Is this really your "solution", catching the culprit I identified earlier?

Annoyingly, this was done to not have to touch the original stack trace trimming implementation (#931). Stack trace trimming works in the following way (summarizing #931):

1. Grab the stack trace
2. Trim the stack trace
3. Print the trimmed stack trace to stdout and fake the look of the stack trace format
4. Throw an exception

This defines the stack trace trimming API as, "If you see a magical exception, exit as fast as possible and, by the way, no need to show the user any info because I already printed whatever information they need." StageError is the Stage-equivalent of this that was added to preserve this API.

I'd like to rework this to something cleaner: actually show a trimmed stack trace.

As far as I digged into this issue, I found no way to avoid file IO since the API change with version 3.4.0 (in particular getting the verilog string without getting the verilog file written) so I don't really get your point here. The only acceptable solution here would be a global (or category-based: anno, fir, v) annotation preventing the actual file IO wherever required.

Clarifying:

/* This will cause File IO to happen because it's a Stage */
(new ChiselStage).emitVerilog(new Foo)

/* This will not cause any File IO, it just returns a string. This just a bag of phases. */
ChiselStage.emitVerilog(new Foo)

The ChiselStage$ object is intended for simple usage, like on Scastie or whatever.

Needless to say that I can't accept your answer here: no solutions, just an ugly workaround and no willingness to fix what has definitely been an issue for quite some time: chisel generation API is a wonderful undocumented mess with not only 2 but now 3 different ways to produce apparently the same output but with at least different runtime behaviors...

I think what the real problem here is that there's no good documentation on how to build standalone tools using the Stage/Phase stuff. ChiselStage is closer to a standalone program than something that is supposed to compose with other tools (like usage inside a main app).

I'm willing to help either refactor things or to get some documentation here.

I am sorry about my dramatic tone here, and while I do understand some of the underlying reasons in your answer I haven't seen any improvement regarding this basic but crucial API, and I really want to emphasize how painful it is and insecure it feels working with it.

All good man. I'm here to help and want to make things better!

(I'm a bit distracted today with other things, but I'm trying to follow the discussion here.)

[seldridge]

Using #1771, this behavior is unified:

1. Exceptions during elaboration have their stack traces trimmed to user code with a stdout message indicating to use --full-stacktrace if you don't want trimming
2. If you use --full-stacktrace, you get the full stack trace and no stdout message
3. Exceptions are let through and not wrapped

Consider:

package example

object SimpleExceptionIssue extends App {
  (new ChiselStage).emitVerilog(new HwClass(doThrow = true), args)
}

object SimpleExceptionIssue2 extends App {
  ChiselStage.emitVerilog(new HwClass(doThrow = true))
}

class HwClassNoParameters extends HwClass(true)

Running all permutations of this using #1771 locally published:

sbt:johnsbrew> runMain example.SimpleExceptionIssue
[info] running example.SimpleExceptionIssue 
Elaborating design...
Error during elaboration!
Stack trace will be trimmed to user code only. Rerun with --full-stacktrace to see the full stack trace
[error] (run-main-13) example.UserException: User error message
[error] example.UserException: User error message
[error] 	at example.HwClass.f(SimpleExceptionIssue.scala:16)
[error] 	at example.HwClass.$anonfun$new$1(SimpleExceptionIssue.scala:20)
[error] 	at chisel3.Data.$anonfun$$colon$eq$1(Data.scala:540)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] 	at chisel3.internal.prefix$.apply(prefix.scala:32)
[error] 	at chisel3.Data.$colon$eq(Data.scala:540)
[error] 	at example.HwClass.<init>(SimpleExceptionIssue.scala:20)
[error] 	at example.SimpleExceptionIssue$.$anonfun$new$2(SimpleExceptionIssue.scala:24)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1

sbt:johnsbrew> runMain example.SimpleExceptionIssue --full-stacktrace
[info] running example.SimpleExceptionIssue --full-stacktrace
Elaborating design...
Error during elaboration!
[error] (run-main-15) example.UserException: User error message
[error] example.UserException: User error message
[error] 	at example.HwClass.f(SimpleExceptionIssue.scala:16)
[error] 	at example.HwClass.$anonfun$new$1(SimpleExceptionIssue.scala:20)
[error] 	at chisel3.Data.$anonfun$$colon$eq$1(Data.scala:540)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] 	at chisel3.internal.prefix$.apply(prefix.scala:32)
[error] 	at chisel3.Data.$colon$eq(Data.scala:540)
[error] 	at example.HwClass.<init>(SimpleExceptionIssue.scala:20)
[error] 	at example.SimpleExceptionIssue$.$anonfun$new$2(SimpleExceptionIssue.scala:24)
[error] 	at chisel3.Module$.do_apply(Module.scala:53)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.$anonfun$elaborate$1(ChiselAnnotations.scala:60)
[error] 	at chisel3.internal.Builder$.$anonfun$build$1(Builder.scala:645)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:642)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:638)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.elaborate(ChiselAnnotations.scala:60)
[error] 	at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:26)
[error] 	at scala.collection.immutable.List.flatMap(List.scala:366)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:24)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:17)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:278)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.DependencyManager.transform(DependencyManager.scala:269)
[error] 	at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255)
[error] 	at firrtl.options.PhaseManager.transform(DependencyManager.scala:436)
[error] 	at chisel3.stage.ChiselStage.run(ChiselStage.scala:45)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Stage.$anonfun$transform$5(Stage.scala:47)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.Stage.$anonfun$transform$3(Stage.scala:47)
[error] 	at logger.Logger$.$anonfun$makeScope$2(Logger.scala:166)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at logger.Logger$.makeScope(Logger.scala:164)
[error] 	at firrtl.options.Stage.transform(Stage.scala:47)
[error] 	at firrtl.options.Stage.execute(Stage.scala:58)
[error] 	at chisel3.stage.ChiselStage.emitVerilog(ChiselStage.scala:101)
[error] 	at example.SimpleExceptionIssue$.delayedEndpoint$example$SimpleExceptionIssue$1(SimpleExceptionIssue.scala:24)
[error] 	at example.SimpleExceptionIssue$delayedInit$body.apply(SimpleExceptionIssue.scala:23)
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:431)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at example.SimpleExceptionIssue$.main(SimpleExceptionIssue.scala:23)
[error] 	at example.SimpleExceptionIssue.main(SimpleExceptionIssue.scala)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1

**sbt:johnsbrew> runMain example.SimpleExceptionIssue2
[info] running example.SimpleExceptionIssue2 
Elaborating design...
Error during elaboration!
Stack trace will be trimmed to user code only. Rerun with --full-stacktrace to see the full stack trace
[error] (run-main-14) example.UserException: User error message
[error] example.UserException: User error message
[error] 	at example.HwClass.f(SimpleExceptionIssue.scala:16)
[error] 	at example.HwClass.$anonfun$new$1(SimpleExceptionIssue.scala:20)
[error] 	at chisel3.Data.$anonfun$$colon$eq$1(Data.scala:540)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] 	at chisel3.internal.prefix$.apply(prefix.scala:32)
[error] 	at chisel3.Data.$colon$eq(Data.scala:540)
[error] 	at example.HwClass.<init>(SimpleExceptionIssue.scala:20)
[error] 	at example.SimpleExceptionIssue2$.$anonfun$new$3(SimpleExceptionIssue.scala:28)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1

You can also not use your own main and then just use ChiselMain directly:

runMain chisel3.stage.ChiselMain --module "example.HwClassNoParameters"
[info] compiling 1 Scala source to /Users/schuylere/repos/local/johnsbrew/target/scala-2.12/classes ...
[warn] there was one deprecation warning (since Chisel 3.5); re-run with -deprecation for details
[warn] one warning found
[info] running chisel3.stage.ChiselMain --module example.HwClassNoParameters
Elaborating design...
Error during elaboration!
Stack trace will be trimmed to user code only. Rerun with --full-stacktrace to see the full stack trace
[error] (run-main-18) example.UserException: User error message
[error] example.UserException: User error message
[error] 	at example.HwClass.f(SimpleExceptionIssue.scala:16)
[error] 	at example.HwClass.$anonfun$new$1(SimpleExceptionIssue.scala:20)
[error] 	at chisel3.Data.$anonfun$$colon$eq$1(Data.scala:540)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] 	at chisel3.internal.prefix$.apply(prefix.scala:32)
[error] 	at chisel3.Data.$colon$eq(Data.scala:540)
[error] 	at example.HwClass.<init>(SimpleExceptionIssue.scala:20)
[error] 	at example.HwClassNoParameters.<init>(SimpleExceptionIssue.scala:31)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1

sbt:johnsbrew> runMain chisel3.stage.ChiselMain --module "example.HwClassNoParameters" --full-stacktrace
[info] running chisel3.stage.ChiselMain --module example.HwClassNoParameters --full-stacktrace
Elaborating design...
Error during elaboration!
[error] (run-main-1a) example.UserException: User error message
[error] example.UserException: User error message
[error] 	at example.HwClass.f(SimpleExceptionIssue.scala:16)
[error] 	at example.HwClass.$anonfun$new$1(SimpleExceptionIssue.scala:20)
[error] 	at chisel3.Data.$anonfun$$colon$eq$1(Data.scala:540)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] 	at chisel3.internal.prefix$.apply(prefix.scala:32)
[error] 	at chisel3.Data.$colon$eq(Data.scala:540)
[error] 	at example.HwClass.<init>(SimpleExceptionIssue.scala:20)
[error] 	at example.HwClassNoParameters.<init>(SimpleExceptionIssue.scala:31)
[error] 	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[error] 	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
[error] 	at java.base/java.lang.Class.newInstance(Class.java:584)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation$.$anonfun$apply$1(ChiselAnnotations.scala:76)
[error] 	at chisel3.Module$.do_apply(Module.scala:53)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.$anonfun$elaborate$1(ChiselAnnotations.scala:60)
[error] 	at chisel3.internal.Builder$.$anonfun$build$1(Builder.scala:645)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:642)
[error] 	at chisel3.internal.Builder$.build(Builder.scala:638)
[error] 	at chisel3.stage.ChiselGeneratorAnnotation.elaborate(ChiselAnnotations.scala:60)
[error] 	at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:26)
[error] 	at scala.collection.immutable.List.flatMap(List.scala:366)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:24)
[error] 	at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:17)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:278)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.DependencyManager.transform(DependencyManager.scala:269)
[error] 	at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255)
[error] 	at firrtl.options.PhaseManager.transform(DependencyManager.scala:436)
[error] 	at chisel3.stage.ChiselStage.run(ChiselStage.scala:45)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.Stage$$anon$1.transform(Stage.scala:43)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38)
[error] 	at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Translator.transform(Phase.scala:248)
[error] 	at firrtl.options.Translator.transform$(Phase.scala:248)
[error] 	at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15)
[error] 	at firrtl.options.Stage.$anonfun$transform$5(Stage.scala:47)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
[error] 	at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
[error] 	at scala.collection.immutable.List.foldLeft(List.scala:91)
[error] 	at firrtl.options.Stage.$anonfun$transform$3(Stage.scala:47)
[error] 	at logger.Logger$.$anonfun$makeScope$2(Logger.scala:166)
[error] 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] 	at logger.Logger$.makeScope(Logger.scala:164)
[error] 	at firrtl.options.Stage.transform(Stage.scala:47)
[error] 	at firrtl.options.Stage.execute(Stage.scala:58)
[error] 	at firrtl.options.StageMain.main(Stage.scala:71)
[error] 	at chisel3.stage.ChiselMain.main(ChiselStage.scala)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[error] stack trace is suppressed; run last Compile / bgRunMain for the full output
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1

[seldridge]

@johnsbrew wrote:

Is this something I could help with as a first step?

Sorry, missed this. No need as I went ahead and fixed this in #1771/#1773. This should show up in 3.4.3 (the next minor release).

Does this fix your issue?

@johnsbrew wrote:

Is there a global strategy for Exception handling in the chisel/firrtl stack? To be defined? To be upgraded/standardized?

There is now. The policy is roughly that exceptions should not be swallowed or wrapped. They should be trimmed to user code close to the point of origin if possible.

The StageError API should be removed. This was a stop-gap to deal with different behavior between ChiselStage and the Driver.

What do you think?

@johnsbrew wrote:

Does the idea of disabling file IO with some annotations sounds reasonable? I think I could submit a PR for that. Would this be enough to drop the duplicated implementation of ChiselStage vs ChiselStage$?

Possibly. I think that can be done with an annotation that controls firrtl.options.phases.WriteOutputAnnotations. I'm not sure if this is exactly the right approach, though. I'd probably prefer to have easy-to-reuse collections of phases that users compose into applications that they want.

Overall, the rationale is roughly the following:

• Phases are re-usable units of work that aren't supposed to do file IO.

• Stages are Phases that add a command line interface and do file IO when they finish (using the CustomFileEmission API).

• A StageMain is a ready-to-use main method that wraps a Stage, e.g., if you have a Chisel project you can directly use sbt runMain chisel3.stage.ChiselMain --module foo.Foo

The split between ChiselStage vs. ChiselStage$ should really be presented as ChiselStage vs. ChiselPhase. The ChiselStage$ object is a wrapper around running just the phases necessary to produce the requested output. It's roughly analogous to specifying different build targets with different rules.

This overall factoring results in the underlying phases being reusable, but the stages being closer to command line apps that may not compose well together (or will do file IO).

Anyway, I think this all warrants discussion. My general thinking is that Phase is the right API and the opinionated packaging of Stages may not be right or may not be right for all use cases.

@johnsbrew wrote:

I have unfortunately more ideas than time and API changes might (by design) be quite controversial so let me know if there is something useful I can work on.

Suggestions and feedback are always welcome. If you could take the time to write up what you're thinking that would help seed a discussion.

@johnsbrew wrote:

And once again thank you all for the good work on chisel, I am sorry if anyone felt insulted, it was definitely not my intention at all.

Thanks for stating this.

[johnsbrew]

Hi @seldridge

Does this fix your issue?

Yes it does, thank you very much for your reactivity on this, you pretty much nailed it with #1771 !

There is now. The policy is roughly that exceptions should not be swallowed or wrapped. They should be trimmed to user code close to the point of origin if possible.

The StageError API should be removed. This was a stop-gap to deal with different behavior between ChiselStage and the Driver.

What do you think?

This is great, I really like your ExceptionHelpers$ I'm thinking about reusing it, or what do you think about making it possible for an advanced user to provide additional entries to packageTrimlist? Typically thinking about library developer use-case: hide the part of the stacktrace which belong to library code in order to point the user directly on the user-triggering line. #UserInception

Clarifying:

/* This will cause File IO to happen because it's a Stage */
(new ChiselStage).emitVerilog(new Foo)

/* This will not cause any File IO, it just returns a string. This just a bag of phases. */
ChiselStage.emitVerilog(new Foo)

Sorry I get confused here as I am still using some now outdated phase in my flow, I should fix that. (But as you said, the phase system is intended to be integrated in a main where it works well)

Anyway, I think this all warrants discussion. My general thinking is that Phase is the right API and the opinionated packaging of Stages may not be right or may not be right for all use cases.

I get your point, but as you said earlier

I think what the real problem here is that there's no good documentation on how to build standalone tools using the Stage/Phase stuff. ChiselStage is closer to a standalone program than something that is supposed to compose with other tools (like usage inside a main app).

Although I'm willing to help, I don't think I qualify for that one...

Suggestions and feedback are always welcome. If you could take the time to write up what you're thinking that would help seed a discussion.

Well as you fixed both the initial issue and (by the way!) the entire Exception handling system I can only point out some potential improvements:

• update StageMain which is still catching a StageError? https://github.com/chipsalliance/firrtl/blob/v1.4.1/src/main/scala/firrtl/options/Stage.scala#L73
• add equivalent options for functions of ChiselStage$, at least the annotations (it's really useful for small tests to be able to do val result = (new ChiselStage).emitVerilog(new DUT(...), annotations = Seq(RunFirrtlTransformAnnotation(...))), and most of the time you don't need the emitted files. If you think it is worth it I can submit a PR.

Thank you again for your time here!

[seldridge]

@johnsbrew wrote:

This is great, I really like your ExceptionHelpers$ I'm thinking about reusing it, or what do you think about making it possible for an advanced user to provide additional entries to packageTrimlist? Typically thinking about library developer use-case: hide the part of the stacktrace which belong to library code in order to point the user directly on the user-triggering line. #UserInception

Good thought! The packageTrimList: Set[String] is now parameter with a default argument of the original trim list. The anchor is also optional so there should be some flexibility with that implicit class.

@johnsbrew wrote:

Although I'm willing to help, I don't think I qualify for that one...

I just need to stop sandbagging and write the documentation... There's some high-level motivation-type stuff that I wrote up over a year ago here: freechipsproject/www.chisel-lang.org#39. I also started to work up a Scastie example that uses stage/phase stuff out of transforms that operate on Strings. (This is nowhere near ready, but this was how I was thinking to demonstrate the idea.)

@johnsbrew wrote:

update StageMain which is still catching a StageError? https://github.com/chipsalliance/firrtl/blob/v1.4.1/src/main/scala/firrtl/options/Stage.scala#L73

This should live for a while as it's a public API. However, the first step would be to deprecate StageError and plan to remove it in 3.6.

@johnsbrew wrote:

add equivalent options for functions of ChiselStage$, at least the annotations (it's really useful for small tests to be able to do val result = (new ChiselStage).emitVerilog(new DUT(...), annotations = Seq(RunFirrtlTransformAnnotation(...))), and most of the time you don't need the emitted files. If you think it is worth it I can submit a PR.

This sounds great. Just adding the annotations parameter would be great. 👍