Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add New Rule Builder: Tactic Generator #70

Closed
wants to merge 37 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
35dd673
Finish a RuleTac and a RuleBuilder to run neural network under the ho…
Peiyang-Song Sep 12, 2023
5919da4
Bump to LeanInfer 2.0.0, simplify additional installation steps
Peiyang-Song Sep 13, 2023
03fd2aa
Bump to LeanInfer v0.0.4
Peiyang-Song Sep 14, 2023
8d1adba
Add script builder for neural
Peiyang-Song Sep 20, 2023
e34adc4
Bump to LeanInfer v0.0.7
Peiyang-Song Sep 20, 2023
3bc8350
Add Aesop-LeanInfer readme
Peiyang-Song Sep 23, 2023
ad2b3e8
Separate Aesop-LeanInfer readme from original Aesop
Peiyang-Song Sep 23, 2023
9d410b5
Create Aesop-README.md
Peiyang-Song Sep 23, 2023
2cdbed4
Add build check example
Peiyang-Song Sep 23, 2023
1804f21
Link update
Peiyang-Song Sep 25, 2023
2671f5e
Add exception handling and fix specific-tactic bug
Peiyang-Song Sep 27, 2023
abe4b7b
Rename package
Peiyang-Song Sep 27, 2023
2e9134f
Remove or suggested by the model: fake proofs
Peiyang-Song Sep 28, 2023
1ef6a4a
Detect when part of tactic is sorry or admit
Peiyang-Song Sep 28, 2023
100fcd3
Increase number of candidate sequences to 64
Peiyang-Song Sep 28, 2023
de4d756
Add reminder of LeanInfer cloud build not working
Peiyang-Song Sep 28, 2023
002ceca
detect sorries
Sep 28, 2023
a86552e
lake update
Sep 28, 2023
1abe960
restore states
Sep 28, 2023
5ea54ad
bump to leanprover/lean4:v4.2.0-rc1
Sep 28, 2023
5ddb1a3
rollback to v4.0.0
Sep 28, 2023
874d274
Merge pull request #1 from Peiyang-Song/Kaiyu-LeanInfer-dev
Peiyang-Song Sep 28, 2023
a25763f
Disable timeout
Peiyang-Song Sep 28, 2023
4dfb047
Merge remote-tracking branch 'origin/peiyang-LeanInfer-dev' into peiy…
Peiyang-Song Sep 28, 2023
0dc1f4a
minor fix
Sep 28, 2023
4fb28bc
Minor fix to get rid of PANIC
Peiyang-Song Sep 28, 2023
3442b8e
Merge pull request #2 from Peiyang-Song/Kaiyu-LeanInfer-dev
Peiyang-Song Sep 28, 2023
6fce70e
Merge pull request #3 from Peiyang-Song/peiyang-LeanInfer-dev
Peiyang-Song Sep 28, 2023
73186b1
Merge branch 'peiyang-LeanInfer'
Oct 6, 2023
ee63cbe
simplify
Oct 6, 2023
f1977a4
use LLM-generated scores
Oct 7, 2023
7669ebc
use the tactic rule builder
Oct 7, 2023
744bae6
further simplify
Oct 7, 2023
76375b4
clean
Oct 7, 2023
c238fd6
leanprover/lean4:v4.2.0-rc1
Oct 7, 2023
76de841
clean
Oct 7, 2023
4bc99ff
Merge pull request #4 from Peiyang-Song/refactor
Oct 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Aesop/Builder/Tactic.lean
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,11 @@ def RuleBuilder.tactic (opts : RegularBuilderOptions) : RuleBuilder :=
mkResult $ .singleRuleTac decl
else if ← isDefEq (mkConst ``RuleTac) type then
mkResult $ .ruleTac decl
else if ← isDefEq (← mkArrow (mkConst ``MVarId) (mkApp (mkConst ``MetaM) (← mkAppM ``Array #[(← mkAppM ``Prod #[(mkConst ``String), (mkConst ``Float)])]))) type then
-- MVarId → MetaM (Array (String × Float))
mkResult $ .tacGen decl
else
throwError "aesop: {decl} was expected to be a tactic, i.e. to have one of these types:\n TacticM Unit\n SimpleRuleTac\n RuleTac\nHowever, it has type{indentExpr type}"
throwError "aesop: {decl} was expected to be a tactic, i.e. to have one of these types:\n TacticM Unit\n SimpleRuleTac\n RuleTac\n MVarId → MetaM (Array (String × Float))\nHowever, it has type{indentExpr type}"
where
builderName : BuilderName :=
.tactic
Expand Down
1 change: 1 addition & 0 deletions Aesop/RuleTac.lean
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ protected def run : RuleTacDescr → RuleTacInput → MetaM RuleTacOutput
| tacticM decl => RuleTac.tacticM decl
| singleRuleTac decl => RuleTac.singleRuleTac decl
| ruleTac decl => RuleTac.ruleTac decl
| tacGen decl => RuleTac.tacGen decl
| preprocess => RuleTac.preprocess

end RuleTacDescr
3 changes: 3 additions & 0 deletions Aesop/RuleTac/Basic.lean
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ structure RuleApplication where
goals : Array MVarId
postState : Meta.SavedState
scriptBuilder? : Option RuleTacScriptBuilder
probabilityModifier : Float := 1.0

namespace RuleApplication

Expand Down Expand Up @@ -114,6 +115,7 @@ inductive RuleTacDescr
(isRecursiveType : Bool)
| tacticM (decl : Name)
| ruleTac (decl : Name)
| tacGen (decl : Name)
| singleRuleTac (decl : Name)
| preprocess
deriving Inhabited
Expand All @@ -129,6 +131,7 @@ def isGlobal : RuleTacDescr → Bool
| cases .. => true
| tacticM .. => true
| ruleTac .. => true
| tacGen .. => true
| singleRuleTac .. => true
| preprocess => true

Expand Down
1 change: 1 addition & 0 deletions Aesop/RuleTac/RuleApplicationWithMVarInfo.lean
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ structure RuleApplicationWithMVarInfo where
mvars : UnorderedArraySet MVarId
introducedMVars : UnorderedArraySet MVarId
assignedMVars : UnorderedArraySet MVarId
probabilityModifier : Float
scriptBuilder? : Option (ScriptBuilder MetaM)

namespace RuleApplicationWithMVarInfo
Expand Down
45 changes: 44 additions & 1 deletion Aesop/RuleTac/Tactic.lean
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import Aesop.RuleTac.Basic

open Lean
open Lean.Meta
open Lean.Elab.Tactic (TacticM run)
open Lean.Elab.Tactic (TacticM evalTactic run)

namespace Aesop.RuleTac

Expand Down Expand Up @@ -42,4 +42,47 @@ unsafe def singleRuleTacImpl (decl : Name) : RuleTac :=
@[implemented_by singleRuleTacImpl]
opaque singleRuleTac (decl : Name) : RuleTac

-- Precondition: `decl` has type `MVarId → MetaM (Array (String × Float))`.
unsafe def tacGenImpl (decl : Name) : RuleTac := λ input => do
let tac ← evalConst (MVarId → MetaM (Array (String × Float))) decl
let initialState ← saveState
let suggestions ← tac input.goal
let apps ← suggestions.filterMapM fun (tacticStr, score) => do
assert! 0 ≤ score ∧ score ≤ 1
match Parser.runParserCategory (← getEnv) `tactic tacticStr (fileName := "<stdin>") with
| .error _ => return none
| .ok stx =>
try
initialState.restore
let tac := commitIfNoEx $ evalTactic stx
-- let tstx : TSyntax `tactic := {raw := stx}
let goals ← run input.goal tac |>.run'
let pf? ← getExprMVarAssignment? input.goal
if pf?.isSome then
if (← instantiateMVars pf?.get!) |>.hasSorry then
initialState.restore
return none
-- let scriptBuilder? :=
-- mkScriptBuilder? generateScript $
-- .ofTactic goals.toArray.size do
-- withAllTransparencySyntax md tstx
let postState ← saveState
let thisApp : RuleApplication := {
postState := postState
goals := goals.toArray
probabilityModifier := score
-- scriptBuilder? := scriptBuilder?
scriptBuilder? := none
}
return thisApp
catch _ => pure none
restoreState initialState
if apps.isEmpty then throwError
"failed to apply any tactics generated"
return ⟨apps⟩

-- Precondition: `decl` has type `MVarId → MetaM (Array (String × Float))`.
@[implemented_by tacGenImpl]
opaque tacGen (decl : Name) : RuleTac

end Aesop.RuleTac
4 changes: 3 additions & 1 deletion Aesop/Search/Expansion.lean
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,14 @@ def addRapps (parentRef : GoalRef) (rule : RegularRule)
(rapps : Array RuleApplicationWithMVarInfo) :
SearchM Q RuleResult := do
let parent ← parentRef.get
let successProbability := parent.successProbability * rule.successProbability

let mut rrefs := Array.mkEmpty rapps.size
let mut subgoals := Array.mkEmpty $ rapps.size * 3
for h : i in [:rapps.size] do
let rapp := rapps[i]'(by simp_all [Membership.mem])
let some probabilityModifier := Percent.ofFloat rapp.probabilityModifier | throwError
"aesop: internal error: rule {rule.name} returned an invalid probability modifier {rapp.probabilityModifier}"
let successProbability := parent.successProbability * rule.successProbability * probabilityModifier
let rref ← addRapp {
rapp with
parent := parentRef
Expand Down
21 changes: 21 additions & 0 deletions AesopTest/TacGen.lean
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import Aesop

open Lean
open Lean.Meta

theorem foo (a b c : Nat) : a + b + c = c + b + a := by
rw [Nat.add_assoc]
rw [Nat.add_comm]
rw [Nat.add_comm b]

-- @[aesop unsafe 50% tactic]
-- def x (_ : MVarId) : MetaM (Array (String × Float)) := do
-- return #[("apply foo", 1.0)]

-- It wouldn't work without this.
@[aesop unsafe 80% tactic]
def x : MVarId → MetaM (Array (String × Float)) := fun _ => do
return #[("rw [Nat.add_comm b]", 0.5), ("rw [Nat.add_assoc]", 0.9), ("rw [Nat.add_comm]", 0.8)]

example (a b c : Nat) : a + b + c = c + b + a := by
aesop
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -943,4 +943,4 @@ usually perfectly safe rule `∀ n, n < n + 1` would, if treated as safe, force
to commit to one particular instantiation of the metavariable `?m`.

For more details on the handling of metavariables, see the [Aesop
paper](https://zenodo.org/record/7430233).
paper](https://zenodo.org/record/7430233).
Loading