Lupi #151
Open
Lupi #151
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JulietteMoreau
requested review from
ThierryJudge,
PierreRouge,
RomainDeleat-besson,
robintrmbtt,
EmileSaillard and
sophie-carneiro
sophie-carneiro
requested changes
Oct 30, 2023
|
|
||
| # Highlights | ||
|
|
||
| * This paper introduces to a new leanrning paradigm called LUPI. |
There was a problem hiding this comment.
Suggested change
| * This paper introduces to a new leanrning paradigm called LUPI. | |
| * This paper introduces a new learning paradigm called LUPI. |
| # Highlights | ||
|
|
||
| * This paper introduces to a new leanrning paradigm called LUPI. | ||
| * The addition of privileged information during the training even if it is not during the inference improves the performances. |
There was a problem hiding this comment.
Suggested change
| * The addition of privileged information during the training even if it is not during the inference improves the performances. | |
| * The addition of privileged information during the training even if it is not present during the inference improves the performances. |
|
|
||
| $$(x_1,y_1),(x_2, y_2),...,(x_n,y_n) \quad where \quad x_i \in X, \quad y_i \in \{-1,1\} $$ | ||
|
|
||
| following an unknown probability measure $$P(x,y)$$ and the goal is to find among a collection of functions $$f(x, \alpha), \alpha \in \Lambda$$ the function $$y=f(x, \alpha*)$$ that minimizes the number of incorrect classifications. |
There was a problem hiding this comment.
I'm perturbed by $$\Lambda$$ and y in $$y=f(x, \alpha*)$$
|
|
||
| $$(x_1, x^*_1, y_1),(x_2, x^*_2, y_2),...,(x_n, x^*_n, y_n) \quad where \quad x_i \in X, \quad x^*_i \in X^*, \quad y_i \in \{-1,1\} $$ | ||
|
|
||
| following the unknow probabilty measure $$P(x, x^*, y)$$ and the goal is to find among a collection of functions $$f(x, \alpha), \alpha \in \Lambda$$ the function $$y=f(x, \alpha*)$$ that minimizes the number of incorrect classifications. |
There was a problem hiding this comment.
Suggested change
| following the unknow probabilty measure $$P(x, x^*, y)$$ and the goal is to find among a collection of functions $$f(x, \alpha), \alpha \in \Lambda$$ the function $$y=f(x, \alpha*)$$ that minimizes the number of incorrect classifications. | |
| following the unknown probability measure $$P(x, x^*, y)$$ and the goal is to find among a collection of functions $$f(x, \alpha), \alpha \in \Lambda$$ the function $$y=f(x, \alpha*)$$ that minimizes the number of incorrect classifications. |
| > | ||
| > $$ P(y[(w_l,x)+b_l]<0) \leq P(1-\xi^0 <0) + A \frac{h ln\frac{l}{h}-ln (\eta)}{l}$$ | ||
| > | ||
| > where $$P(y[(w_l,x)+b_l]<0)$$ is the probability of error for the Oracle SVM solution on the training set of size $$l$$, $$P(1-\xi^0 <0)$$ is the probability of error for the best solution in the asmissible set of functions, $$h$$ is the VC dimension of the admissible set of hyperplanes, ans $$A$$ is a constant. |
There was a problem hiding this comment.
Suggested change
| > where $$P(y[(w_l,x)+b_l]<0)$$ is the probability of error for the Oracle SVM solution on the training set of size $$l$$, $$P(1-\xi^0 <0)$$ is the probability of error for the best solution in the asmissible set of functions, $$h$$ is the VC dimension of the admissible set of hyperplanes, ans $$A$$ is a constant. | |
| > where $$P(y[(w_l,x)+b_l]<0)$$ is the probability of error for the Oracle SVM solution on the training set of size $$l$$, $$P(1-\xi^0 <0)$$ is the probability of error for the best solution in the admissible set of functions, $$h$$ is the VC dimension of the admissible set of hyperplanes, ans $$A$$ is a constant. |
|
|
||
| Some propositions are demontrated introducing oracle SVM. | ||
|
|
||
| > Proposition 1: If any vector $$x \in X$$ belongs to one and only one of the classes and there exists an Oracle function with respect to the best decision rule in the asmissible set of hyperparameters, then with the probablity $$-\eta$$ the following bound holds true |
There was a problem hiding this comment.
Suggested change
| > Proposition 1: If any vector $$x \in X$$ belongs to one and only one of the classes and there exists an Oracle function with respect to the best decision rule in the asmissible set of hyperparameters, then with the probablity $$-\eta$$ the following bound holds true | |
| > Proposition 1: If any vector $$x \in X$$ belongs to one and only one of the classes and there exists an Oracle function with respect to the best decision rule in the admissible set of hyperparameters, then with the probablity $$-\eta$$ the following bound holds true |
| > | ||
| > That is the Oracle solution converges to the best possible solution in the admissible set of solutions with the rate $$O(h/l)$$. | ||
|
|
||
| But in reality a teacher does not know the oracle function, but can supply privileged information instead with the admissible set of the correcting functions $$\phi(x^*, \delta), \delta \in \Delta$$ whiche defines that values of the oracle function $$ \xi^0(x_i) = \phi(x^*_i, \delta_0), \forall (x_i, x^*_i, y_i)$$ |
There was a problem hiding this comment.
Suggested change
| But in reality a teacher does not know the oracle function, but can supply privileged information instead with the admissible set of the correcting functions $$\phi(x^*, \delta), \delta \in \Delta$$ whiche defines that values of the oracle function $$ \xi^0(x_i) = \phi(x^*_i, \delta_0), \forall (x_i, x^*_i, y_i)$$ | |
| But in reality a teacher does not know the oracle function, but can supply privileged information instead with the admissible set of the correcting functions $$\phi(x^*, \delta), \delta \in \Delta$$ which defines that values of the oracle function $$ \xi^0(x_i) = \phi(x^*_i, \delta_0), \forall (x_i, x^*_i, y_i)$$ |
|
|
||
| ## Future events as privileged information: Mackey-Glass series | ||
|
|
||
| There are two parametrisations of such problem: quantitatively when with the value at time $$t$$ we predict the value at $$t+Dt$$ or qualitatively if with the value at time $$t$$ we estimate if it is greater of lower at time $$t+Dt$$. |
There was a problem hiding this comment.
Suggested change
| There are two parametrisations of such problem: quantitatively when with the value at time $$t$$ we predict the value at $$t+Dt$$ or qualitatively if with the value at time $$t$$ we estimate if it is greater of lower at time $$t+Dt$$. | |
| There are two parametrisations of such problem: quantitatively when with the value at time $$t$$ we predict the value at $$t+Dt$$ or qualitatively, when with the value at time $$t$$ we estimate if it is greater of lower at time $$t+Dt$$. |
|
|
||
| Some propositions are demontrated introducing oracle SVM. | ||
|
|
||
| > Proposition 1: If any vector $$x \in X$$ belongs to one and only one of the classes and there exists an Oracle function with respect to the best decision rule in the admissible set of hyperparameters, then with the probablity $$-\eta$$ the following bound holds true |
There was a problem hiding this comment.
A negative probability? Should this be 1-\eta?
ThierryJudge
requested changes
Nov 1, 2023
Co-authored-by: ThierryJudge <[email protected]>
Co-authored-by: ThierryJudge <[email protected]>
Co-authored-by: ThierryJudge <[email protected]>
Co-authored-by: ThierryJudge <[email protected]>
Co-authored-by: ThierryJudge <[email protected]>
Co-authored-by: ThierryJudge <[email protected]>
Co-authored-by: ThierryJudge <[email protected]>
Co-authored-by: ThierryJudge <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The paper is quite complicated and i didn't understand a great part of it so i limited the maths part. Still i copied 2 propositions that are demonstrated in the paper, but it might not be relevant without the rest that i surely not present, let me know if i keep it or not.
Also it will never be merged to main in the end because the paper is not available even if it is old.